Dynamic Quantization Adjustment for Neural Network Fixed-Point Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional quantization techniques in neural networks face challenges in balancing accuracy and computational resource usage, as fixed quantization levels lead to either low accuracy with limited bits or high resource consumption with more bits, and are inefficient in handling hardware constraints like accumulator overflow.
Innovation Solution
A dynamic quantization level adjustment scheme that adjusts quantization levels based on hardware constraints, such as hardware cost, performance, and accuracy, by determining a peak value of interest and applying different quantization scales for each channel, allowing for optimized fixed-point representation of neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If conventional fixed quantization levels are used, then computational resources are reduced, but quantization accuracy degrades
Solution Approach 1:
The patent applies dynamic quantization level adjustment by modifying the quantization function to include a dynamic level parameter that adapts based on input characteristics. The quantization level is no longer fixed but dynamically adjusted during operation to optimize the balance between resource consumption and accuracy for different input scenarios.
Solution Approach 2:
The patent changes the quantization parameter (quantization level) from a fixed value to a dynamically adjustable parameter. By modifying the quantization function to accept and respond to changing quantization levels, the system can adapt its precision and resource usage based on the specific characteristics of the input data and operational requirements.
2Measurement precision
If asymmetric quantization is used, then quantization error is reduced, but computational cost increases
Solution Approach 1:
The patent enables dynamic switching or adjustment between symmetric and asymmetric quantization modes. The quantization function is designed to adapt its symmetry characteristics based on input data properties, allowing the system to use asymmetric quantization when it provides better accuracy while falling back to symmetric quantization when computational efficiency is prioritized.
Solution Approach 2:
The patent introduces dynamic parameters that control the quantization characteristics, allowing the system to adjust the degree of asymmetry in quantization. By making the quantization parameters adaptive rather than fixed, the system can optimize the trade-off between quantization error reduction and computational cost based on real-time conditions.
3Quantity of substance
If small number of bits is used in quantization, then memory and processing resources are reduced, but quantization saturation error increases
Solution Approach 1:
The patent applies dynamic quantization level adjustment that responds to the distribution and range of input data. When input values are small or clustered, the system uses coarser quantization levels that require fewer bits. When input values span a wider range or require higher precision, the system dynamically increases the quantization level, thereby adapting bit usage to actual needs rather than using a fixed conservative allocation.
Data Source
AI summary
Aspects of the present disclosure address systems and methods for fixed-point quantization using a dynamic quantization level adjustment scheme. Consistent with some embodiments, a method comprises accessing a neural network comprising floating-point representations of filter weights corresponding to one or more convolution layers. The method further includes determining a peak value of interest from the filter weights and determining a quantization level for the filter weights based on a number of bits in a quantization scheme. The method further includes dynamically adjusting the quantization level based on one or more constraints. The method further includes determining a quantization scale of the filter weights based on the peak value of interest and the adjusted quantization level. The method further includes quantizing the floating-point representations of the filter weights using the quantization scale to generate fixed-point representations of the filter weights.