Inference Model Quantization Training for IoT Accuracy Retention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods fail to consider the loss in inference performance due to quantization when reducing the size of neural network models for integration into IoT devices, leading to decreased accuracy and disagreement between reference and integrated environment networks.
Innovation Solution
An information processing method that involves obtaining a larger second inference model based on a first reference model, quantizing it, and training it using machine learning to find a third model that reduces loss caused by quantization, while maintaining performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of moving object
If a neural network model is quantized to reduce model size for IoT device integration, then the model can be deployed on resource-constrained devices, but the inference performance and accuracy deteriorate due to quantization loss
Solution Approach 1:
The patent applies preliminary action by computing a second inference model that is larger than the first reference model before quantization. This enlarged model is prepared in advance with additional capacity to compensate for the expected quantization loss, ensuring that after quantization the model still maintains acceptable accuracy on IoT devices.
Solution Approach 2:
The patent changes the model size parameter by computing a second inference model with larger dimensions than the reference model. This parameter adjustment is specifically designed to offset the degradation caused by quantization, allowing the model to retain sufficient precision after being compressed for deployment on resource-constrained IoT devices.
2Device complexity
If a smaller inference model is used for IoT device integration, then device resource requirements are reduced, but the model performance and agreement with reference model deteriorate
Solution Approach 1:
The patent prepares a second inference model with enlarged size before the quantization process. This preliminary enlargement ensures that even after quantization reduces the model size for IoT deployment, the model maintains sufficient performance and agreement with the reference model, thus preemptively compensating for expected performance loss.
Solution Approach 2:
The patent applies preliminary anti-action by intentionally creating an oversized model that anticipates and counteracts the harmful effects of quantization. The enlarged model serves as a buffer against the performance degradation and disagreement with the reference model that would otherwise occur during quantization for IoT device integration.
3Ease of operation
If quantization is applied to reduce model size, then the model becomes suitable for edge deployment, but loss of information and performance degradation occur
Solution Approach 1:
The patent computes a second inference model with larger size before quantization to preemptively compensate for the information loss that will occur during quantization. This preliminary enlargement ensures that the model retains sufficient performance information even after being quantized for edge deployment on IoT devices.
Solution Approach 2:
The patent adjusts the model size parameter by creating an enlarged second inference model that compensates for the information loss expected from quantization. This parameter change ensures that the quantized model remains suitable for edge deployment while minimizing performance information loss.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
An information processing method includes: obtaining a first inference model serving as a reference (S701); computing a second inference model that is larger than the first inference model in model size, based on the first inference model (S702); quantizing the second inference model computed to generate a third inference model (S703); training the third inference model, using machine learning (S704); determining whether a performance of the third inference model trained satisfies a condition (S705); and outputting the third inference model trained, when the performance satisfies the condition (S706).