Machine Learning Model Layer Compression via Hardware-Aware Iterative Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for accelerating machine learning models, such as reducing precision of parameter values or using hardware acceleration, either compromise accuracy or limit model execution to specific hardware, making them ineffective for devices with limited resources.
Innovation Solution
A computer-implemented method that identifies suitable compression techniques for each layer of a machine learning model, applies iterative operations to select and optimize these techniques based on target hardware, and generates a compressed model, while preserving accuracy through fine-tuning and quantization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the precision of parameter values is reduced to accelerate model execution, then computational requirements and storage space are reduced, but model accuracy deteriorates
Solution Approach 1:
The patent segments the model compression process into distinct phases: structure search (identifying which parameters to compress), quantization (reducing precision of selected parameters), and fine-tuning (restoring accuracy). This segmentation allows selective application of compression techniques to different parameter types, preserving accuracy-critical parameters while compressing others, thus resolving the contradiction between speed and accuracy.
Solution Approach 2:
The patent dynamically changes parameter precision levels based on their importance to model accuracy. Through automated structure search and quantization, it identifies which parameters can be reduced in precision without significant accuracy loss, and applies different quantization bits to different parameters. This selective parameter change resolves the contradiction by optimizing the balance between compression and accuracy maintenance.
2Productivity
If hardware acceleration is used to speed up model execution, then processing capability is improved, but device compatibility is reduced
Solution Approach 1:
The patent creates a universal compression framework that produces models optimized for deployment on diverse hardware platforms without requiring specialized acceleration units. The automated structure search and quantization process generates hardware-agnostic compressed models that can run efficiently on general-purpose processors, mobile devices, and edge computing platforms, thus achieving both speed improvement and broad compatibility.
3Quantity of substance
If the size of machine learning models is reduced through compression, then storage requirements are reduced, but model accuracy deteriorates
Solution Approach 1:
The patent performs preliminary structure search and quantization before final model deployment, identifying the optimal compression configuration that minimizes size while maintaining accuracy. The fine-tuning phase further refines the compressed model to recover any accuracy loss. This preliminary optimization action resolves the contradiction by establishing the right balance between size reduction and accuracy preservation before the model is put into production.
Data Source
AI summary
One embodiment of a method for accelerating a trained machine learning model includes parsing the trained machine learning model to identify one or more layers of the trained machine learning model and, for each layer included in the one or more layers, one or more corresponding compression techniques that can be applied to compress the layer, performing, based on a hardware device on which the trained machine learning model is intended to execute, one or more iterative operations to select, for each layer included in the one or more layers, a compression technique and values of one or more parameters associated with the compression technique, and compressing each layer included in the one or more layers using the compression technique that is selected for the layer and the values of the one or more parameters associated with the compression technique to generate a compressed trained machine learning model.