Neural Network Quantization Range Adjustment via Saturation Ratio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for determining a quantization range in artificial neural networks have high computational complexity, making it difficult to adjust the range during inference, especially on low-performance devices, which leads to performance degradation and fixed quantization ranges.
Innovation Solution
A method and device that observe the saturation ratio of tensors and adjust the quantization range to follow a target saturation ratio, reducing computational complexity and enabling dynamic calibration during both training and inference stages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional methods are used to determine quantization range, then quantization can be performed, but computational complexity is high making it difficult to adjust during inference
Solution Approach 1:
The patent pre-calculates saturation ratios for different quantization ranges during the training phase and stores them in a lookup table. During inference, the system simply queries this pre-computed table rather than performing complex calculations, thus resolving the contradiction between needing accurate quantization range adjustment and avoiding high computational complexity.
Solution Approach 2:
The system uses the saturation ratio information that is naturally available during neural network operation to automatically determine optimal quantization ranges without requiring external calibration tools or complex algorithms. The neural network itself provides the information needed for its own quantization optimization.
2Adaptability or versatility
If quantization range is fixed before deployment, then computational complexity is reduced, but adaptability to different inference conditions is lost
Solution Approach 1:
The patent enables dynamic quantization range adjustment during inference by utilizing saturation ratio observations from actual neural network operations. The system can adaptively select or adjust quantization ranges based on real-time saturation ratio measurements, transforming the static quantization approach into a dynamic one that responds to actual inference conditions.
Solution Approach 2:
The system observes saturation ratios during neural network operation and uses this feedback information to adjust quantization ranges. By monitoring whether tensors are saturating at current quantization ranges and adjusting accordingly, the system creates a feedback loop that continuously optimizes quantization performance during inference.
3Adaptability or versatility
If saturation ratio is observed and used to adjust quantization range, then adaptability improves, but additional computational overhead is introduced
Solution Approach 1:
The patent uses simple saturation ratio observations and basic threshold comparisons rather than complex optimization algorithms. The approach uses lightweight operations that consume minimal computational energy, making adaptive calibration feasible even on resource-constrained devices where every computational operation counts.
Data Source
AI summary
A method and a device for determining a quantization range based on a saturation ratio for quantization of an artificial neural network are disclosed.According to one aspect of the present invention, there is provided a computer-implemented method and a device for determining a quantization range for tensors of an artificial neural network, comprising observing a saturation ratio at a current iteration from the tensors of the artificial neural network and the quantization range; and adjusting the quantization range so that the observed saturation ratio follows a predetermined target saturation ratio.