Non-uniform Quantization of Pre-trained Deep Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural network (DNN) quantization techniques assign uniform bit-width values to layers or channels without considering their varying contributions to accuracy, leading to suboptimal performance in resource-limited devices.
Innovation Solution
A system that determines optimal layer-wise or channel-wise bit-width values for weights and activations by analyzing the range and statistical information of weights and activations, using a range determiner, multiplier, and comparator to select the minimum quantization noise-based bit-width values, allowing for efficient quantization without retraining the DNN.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If uniform bit-width quantization is applied to all layers or channels, then device complexity is reduced and ease of operation is improved, but manufacturing precision (quantization accuracy) deteriorates because each layer/channel has different contributions to overall DNN accuracy
Solution Approach 1:
The patent applies different bit-width quantization values to different layers or channels based on their individual characteristics (weight ranges and contribution to accuracy). Instead of using a uniform quantization scheme across the entire network, the system determines optimal bit-width values locally for each layer/channel, thereby achieving both reduced complexity and improved accuracy.
Solution Approach 2:
The patent changes the quantization parameter (bit-width value) for each layer or channel based on statistical analysis of weight ranges and accuracy contributions. By dynamically adjusting these parameters rather than using fixed uniform values, the system optimizes the trade-off between device complexity and quantization precision.
2Manufacturing precision
If higher bit-width values are used for all layers, then quantization accuracy is improved, but loss of energy and computational overhead increase
Solution Approach 1:
The patent determines appropriate bit-width values locally for each layer or channel based on their specific weight ranges and accuracy contributions. This prevents unnecessary use of high bit-widths in layers that contribute less to overall accuracy, thereby reducing computational energy consumption while maintaining necessary precision where it matters most.
Solution Approach 2:
The patent applies higher precision (higher bit-width) only partially to layers that require it for maintaining accuracy, rather than applying high precision universally. By identifying and focusing computational resources on critical layers, the system avoids excessive energy consumption in non-critical portions of the network.
3Use of energy by moving object
If lower bit-width values are used to reduce energy consumption, then use of energy is reduced, but quantization noise increases and accuracy deteriorates
Solution Approach 1:
The patent dynamically adjusts bit-width parameters for each layer or channel based on statistical analysis of weight distributions and accuracy contributions. This optimized parameter selection achieves lower energy consumption compared to uniform high-bit-width approaches while preventing excessive quantization noise by maintaining adequate precision in critical layers.
Data Source
AI summary
A system and a method of quantizing a pre-trained neural network, includes determining by a layer/channel bit-width determiner for each layer or channel of the pre-trained neural network a minimum quantization noise for the layer or the channel for each master bit-width value in a predetermined set of master bit-width values; and selecting by a bit-width selector for the layer or the channel the master bit-width value having the minimum quantization noise for the layer or the channel. In one embodiment, the minimum quantization noise for the layer or the channel is based on a square of a range of weights for the layer or the channel that is multiplied by a constant to a negative power of a current master bit-width value.