AI Model Compression Rate Optimization for Edge Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
AI models require more resources and computation power for accuracy, but limited CPU time and memory in industrial environments pose a challenge, where higher compression rates improve speed but decrease accuracy, and lower rates have the opposite effect.
Innovation Solution
A computerized method using mathematical optimization techniques to determine an optimal compression rate for AI models, considering hardware resources and runtime properties, by compressing models multiple times, recording results, and training a machine learning model to predict the best compression rate for new tasks while maximizing accuracy within resource limits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If AI model compression is applied to reduce parameters, then memory space and computation time are reduced, but prediction accuracy decreases
Solution Approach 1:
The patent applies parameter changes by systematically varying compression rates (e.g., from 0.2 to 0.8 in steps of 0.1) to transform the AI model into different versions with varying degrees of compression. This allows finding an optimal compression rate that balances inference speed and prediction accuracy for specific industrial tasks and hardware constraints.
Solution Approach 2:
The patent introduces dynamics by making the compression rate adaptive rather than fixed. The system dynamically selects the optimal compression rate based on runtime properties, hardware resources, and task requirements, allowing the model to be optimally compressed for different deployment scenarios while maintaining the necessary balance between speed and accuracy.
2Use of energy by moving object
If higher compression rates are used, then memory space and computation time are reduced, but AI model accuracy decreases
Solution Approach 1:
The patent uses parameter changes by adjusting the compression rate parameter to control the trade-off between power consumption and accuracy. By evaluating multiple compression rates and selecting the optimal one based on power constraints and accuracy requirements, the system achieves energy-efficient deployment without excessive accuracy loss.
Solution Approach 2:
The patent applies partial action by compressing the AI model to a degree that is sufficient to meet power and resource constraints but not excessive to the point of unacceptable accuracy loss. The optimal compression rate is determined to be just enough to satisfy the power consumption limit while maintaining adequate prediction accuracy for the industrial task.
3Adaptability or versatility
If AI models are compressed to meet resource limits, then deployment on edge devices becomes feasible, but prediction accuracy is reduced
Solution Approach 1:
The patent introduces dynamics by making the compression rate adaptable to different deployment environments and task requirements. The system can dynamically determine the optimal compression rate based on the specific edge device capabilities, resource constraints, and industrial task characteristics, enabling flexible deployment while minimizing accuracy loss for each specific scenario.
Solution Approach 2:
The patent applies local quality by tailoring the compression rate to the specific requirements of each industrial task and deployment environment. Rather than using a uniform compression approach, the system determines task-specific optimal compression rates that maintain adequate accuracy for local task requirements while enabling deployment on resource-constrained edge devices.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention regards a recommendation system and a method for determining a compression rate r* for an AI model of an industrial task (Task 1, …, k; Task k+1, …, k+n) according to a set of requirements for the runtime properties of the AI model, wherein, for the AI model, the original number of pa- rameters (n) is reduced to a reduced number of parameters (n*). In a first stage each AI model is compressed with a number of different compression rates ri, wherein each of the compressed AI models is executed in an execution environment and as first results during the executions the runtime prop- erties are recorded, and wherein an optimal compression rate r* for each of the AI models is calculated by an analytical analysis of the first results and stored in a database or similar system. In a second stage the data from the database is used to train an additional machine learning model. In a third stage for a new AI model of a new task (Task k+1, …, k+n) a new set of desired runtime properties is defined and the additional machine learning model is employed for deter- mining the optimal compression rate r* for that new AI model in respect to the desired runtime properties.