Hardware-Aware Mixed-Precision Quantization for Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network computing faces challenges in balancing low power consumption and memory usage while maintaining accuracy, particularly in tasks like image segmentation and super-resolution, where low bit-width networks fall short in performance.
Innovation Solution
A method and system for determining bit-widths in mixed-precision neural networks by calculating activation quantization sensitivity (AQS) values for convolution layers, forming quantization groups, and selecting bit-widths supported by the target hardware to optimize sensitivity metrics under given constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If low bit-width neural networks are used, then power consumption and memory usage are reduced, but performance deteriorates for vision construction and quality problems
Solution Approach 1:
The patent applies different bit-width precisions to different parts of the neural network based on their quantization sensitivity. Convolution layers with high AQS values maintain higher precision (e.g., 16-bit or 32-bit) to preserve performance, while layers with low AQS values use lower precision (e.g., 8-bit or 4-bit) to reduce power consumption and memory usage. This localized precision allocation resolves the contradiction by optimizing power efficiency without sacrificing critical performance.
Solution Approach 2:
The patent dynamically changes the precision parameter (bit-width) of neural network operations based on calculated AQS values. By adjusting the precision parameter adaptively for different convolution layers rather than using a fixed precision throughout, the system achieves both reduced power consumption and maintained performance where needed.
2Quantity of substance
If low bit-width neural networks are used, then memory usage is reduced, but performance deteriorates for vision construction and quality problems
Solution Approach 1:
The patent allocates memory resources with different precision levels to different convolution layers based on their AQS values. Critical layers with high sensitivity receive adequate memory allocation with higher precision to maintain performance, while less sensitive layers use lower precision with reduced memory usage, achieving an optimal balance between memory efficiency and performance.
3Measurement precision
If high bit-width neural networks are used, then accuracy is improved, but power consumption and memory usage increase
Solution Approach 1:
The patent identifies which specific convolution layers require high precision for accuracy and which can tolerate lower precision. By applying high bit-width only to critical layers with high AQS values and using lower bit-width for other layers, the system achieves acceptable overall accuracy while significantly reducing power consumption compared to uniform high-precision networks.
4Measurement precision
If high bit-width neural networks are used, then accuracy is improved, but device complexity increases
Solution Approach 1:
The patent reduces device complexity by implementing mixed-precision architecture where different hardware units handle different precision levels. Critical operations requiring high accuracy use high-bit-width hardware, while less critical operations use low-bit-width hardware, thereby reducing overall hardware complexity and resource requirements while maintaining necessary accuracy.
Data Source
AI summary
A bit-widths determination method selects bit-widths for mixed-precision neural network computing on a target hardware platform. An activation quantization sensitivity (AQS) value is calculated for each convolution layer in a neural network. The AQS value indicates the sensitivity of convolution output to quantized convolution input. One or more convolution layers are grouped into a quantization group, which is to be executed by a corresponding set of target hardware. A group AQS value is calculated for each quantization group based on the AQS values of the convolution layers in the quantization group. Then bit-widths supported by the target hardware platform are selected for the corresponding quantization groups. The bit-widths are selected to optimize, under a given constraint, a sensitivity metric that is calculated based on each quantization group's group AQS value.