ML Model Compression Using ONNX and Pruning Ratio Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning model compression techniques are expensive, require expert knowledge, and are not user-friendly, lacking a common method applicable to all frameworks, and rely on manual experimentation and reinforcement learning.
Innovation Solution
A method that converts machine learning models into a standard format using ONNX, prunes them with various pruning ratios, evaluates their size-to-error ratios, and selects an optimal pruning ratio for compression, facilitating deployment without manual programming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing machine learning model compression techniques are used, then model size is reduced, but the process becomes expensive and requires expert knowledge
Solution Approach 1:
The system performs automatic model compression by evaluating multiple pruning ratios and selecting the optimal one based on size-to-error ratio, eliminating the need for expert intervention. The automated evaluation process compares different compression levels and automatically chooses the best configuration, making the process self-service rather than requiring manual expert analysis.
Solution Approach 2:
The system converts models to a standard format and systematically varies pruning ratios as parameters to explore different compression levels. By changing the pruning ratio parameter across multiple candidate values and evaluating each, the system finds the optimal parameter setting that balances model size reduction with performance maintenance.
2Quantity of substance
If manual experimentation and reinforcement learning are used for compression, then compression effectiveness is improved, but time consumption and cost increase
Solution Approach 1:
The system performs preliminary actions by converting the input model to a standard format and pre-evaluating multiple pruning ratios before final deployment. This preliminary evaluation of different compression levels allows the system to select the optimal configuration in advance, avoiding time-consuming manual experimentation during deployment and reducing overall compression time.
3Quantity of substance
If model compression is performed to reduce hardware requirements, then deployment cost is reduced, but model accuracy may deteriorate
Solution Approach 1:
The system uses feedback by evaluating each pruned model's performance and calculating size-to-error ratios. This feedback mechanism allows the system to monitor how compression affects accuracy and adjust the selection process accordingly, choosing compression levels that maintain acceptable accuracy thresholds while reducing model size and hardware requirements.
Data Source
AI summary
A computer-implemented method for compressing a machine learning model includes converting an input machine learning model into a standard machine learning model. The method further includes converting the standard machine learning model into a plurality of pruned machine learning models, each of the pruned machine learning models converted using a corresponding pruning ratio from a pruning ratio candidate list. The method further includes determining, for each of the pruned machine learning models, a size-to-error ratio. The method further includes selecting, based on the size-to-error ratio of the pruned machine learning models, a first pruning ratio from the pruning ratio candidate list. The method further includes generating a compressed machine learning model by compressing the input machine learning model using the first pruning ratio that is selected. The method further includes deploying the compressed machine learning model for production.