Mixed Precision Neural Network Quantization via Layer Sensitivity Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training neural networks for image compression are time-consuming and processor-intensive, especially when transitioning between high and low precision formats, lacking an efficient system for generating mixed precision models that balance accuracy and performance.
Innovation Solution
A method and system for quantization-aware training of neural networks that involves receiving a validation dataset, generating a union sensitivity list, selecting layers based on sensitivity values, and iteratively quantizing them into high precision format to achieve target accuracy, thereby reducing training time and improving image compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the neural network model is quantized to a high precision format (e.g., FP 16) to provide better accuracy in compression, then the compression accuracy is improved, but the time required to compress images increases
Solution Approach 1:
The patent applies local quality by differentiating precision requirements across different layers of the neural network. Instead of uniformly quantizing all layers to high precision, the system identifies and quantizes only specific layers (e.g., later layers with higher sensitivity) to high precision while keeping other layers in low precision format. This selective approach maintains compression accuracy where it matters most while reducing overall computation time.
Solution Approach 2:
The patent segments the neural network into multiple layers and applies different quantization strategies to different segments. The system divides the network into groups of layers, evaluates their sensitivity to quantization, and assigns appropriate precision levels to each segment, thereby achieving a balance between accuracy and speed.
2Productivity
If the neural network model is quantized to a low precision format (e.g., int8 or int4) to reduce processing time, then the image compression time is reduced, but the compression accuracy deteriorates
Solution Approach 1:
The patent applies local quality by assigning different precision levels to different layers based on their sensitivity characteristics. Layers that are less sensitive to quantization errors are kept in low precision format to maintain fast processing, while only critical layers are quantized to high precision to preserve accuracy.
Solution Approach 2:
The patent applies partial action by quantizing only a subset of layers to high precision rather than the entire network. This selective quantization approach applies precision where necessary and maintains speed where possible, achieving a practical compromise between accuracy and performance.
3Measurement precision
If the neural network model is trained to a mixed precision model layer by layer to provide both accuracy and performance, then the model accuracy is improved, but the training process becomes time consuming and processor intensive
Solution Approach 1:
The patent applies preliminary action by pre-evaluating the sensitivity of each layer to quantization before actual training. The system performs a sensitivity analysis in advance to identify which layers require high precision, allowing the training process to focus computational resources on those specific layers rather than uniformly processing all layers.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting the precision parameter (quantization level) for different layers based on their sensitivity characteristics. The system modifies the precision parameter selectively across layers, transitioning from a uniform precision approach to a differentiated precision strategy that optimizes both accuracy and training efficiency.
Data Source
AI summary
Disclosed herein is a method and a system for generating a mixed precision quantization model for performing image processing. The method comprises receiving a validation dataset of images to train a neural network model. The method comprises for each image of the validation dataset, generating a union sensitivity list, selecting a group of layers, generating a mixed precision quantization model by quantizing the selected group of layers into a high precision format; computing accuracy of the mixed precision quantization model for comparison with a target accuracy; in response to determining the accuracy is less than the target accuracy, generating another mixed precision model by selecting a next group of layers and computing the accuracy. In response to determining the accuracy is greater than or equal to the target accuracy, storing the mixed precision quantization model as a final mixed precision quantization model for image processing.