Hybrid Neural Network Quantization Error Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for hybrid accuracy optimization in neural network model quantization are inefficient due to indirect error gradient propagation, slow training speeds, and risks of overfitting, making it difficult to identify and correct quantized layers affecting the final result.
Innovation Solution
A method that involves obtaining a floating-point model, calculating cumulative outputs, selectively quantizing and sorting layers, and restoring quantization layers to floating-point layers based on error values until a preset loss threshold is reached, using batch normalization and exponential function layers as needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If error back propagation method is used to determine error gradient of each quantization layer, then the effect of quantization error on final result can be identified, but training speed is slow and multiple batches of data inference are required
Solution Approach 1:
The patent creates multiple copies of the neural network model, each with a different quantization layer restored to floating-point precision. By evaluating these copied models directly without iterative training, the error gradient identification is achieved rapidly while maintaining accuracy.
Solution Approach 2:
The patent performs preliminary restoration of quantization layers to floating-point precision before final evaluation. This preliminary action allows direct calculation of error gradients without requiring slow iterative training processes.
2Measurement precision
If error back propagation method is used with multiple trainings on small amount of data, then quantization layers with greater effect on error can be found, but risks of overfitting occur and convergence to optimum is not guaranteed
Solution Approach 1:
Instead of repeatedly training the same model (which causes overfitting), the patent creates multiple model copies with different quantization layers restored. Each copy is evaluated once directly, eliminating overfitting risks while reliably identifying error gradients.
Solution Approach 2:
The patent segments the error analysis task by evaluating each quantization layer's effect separately through individual model copies. This segmentation allows direct measurement of each layer's error contribution without the confounding effects of iterative training and overfitting.
3Productivity
If all floating-point layers are quantized to reduce model size and improve efficiency, then quantization errors are introduced that significantly affect final result accuracy
Solution Approach 1:
The patent applies different precision levels to different parts of the model based on their error impact. Critical layers identified through error gradient analysis are restored to floating-point precision, while non-critical layers remain quantized, achieving local optimization of accuracy versus efficiency.
Solution Approach 2:
The patent dynamically changes the precision parameter (floating-point vs. quantized) for different layers based on their calculated error gradients. Layers with high error impact are assigned floating-point precision, while layers with low impact use quantized precision, optimizing the trade-off between accuracy and efficiency.
Data Source
AI summary
A method and a system for improving accuracy of model quantification includes obtaining a floating-point model with multiple floating-point layers, and calculating a cumulative original output of all floating-point layers of the floating-point model; selecting one floating-point layer from floating-point model separately each time for quantization to form multiple hybrid models each containing one quantization layer, and separately calculating error value of cumulative output of all layers of each hybrid model relative to the cumulative original output to obtain multiple calculated error values; sorting the calculated error values; and quantizing all floating-point layers of the floating-point model, and restoring corresponding quantization layer(s) to floating-point layer(s) one by one in descending order of the error values and calculating a difference between the cumulative output of all layers of a corresponding restored model and the cumulative original output until the difference is less than preset loss threshold to obtain target hybrid model.