Dynamic Data Quantization for Low Power CNN Hardware
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) consume high power due to intensive computational processing, necessitating a reduction in power consumption while minimizing the loss in numerical precision caused by data quantization.
Innovation Solution
A dynamic data quantization scheme is employed, where each local block of activation data is quantized independently based on its constrained dynamic range, using a shared exponent and reduced bit width for mantissa values, and applied to both activation and kernel data, with precomputed or on-the-fly quantization during training and inference phases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If data quantization is applied to reduce bit width, then power consumption is reduced, but numerical precision is degraded
Solution Approach 1:
The patent segments the quantization process by applying different quantization schemes to different data types: dynamic quantization for activation data and static quantization for kernel data. This segmentation allows optimization of power consumption while maintaining numerical precision for each data type according to its specific characteristics and usage patterns in the CNN computation.
Solution Approach 2:
The patent applies local quality by using dynamic quantization specifically for activation data where the dynamic range varies across different layers and operations, while using static quantization for kernel data where the values are fixed. This localized approach ensures that quantization is adapted to the specific requirements of each data type, reducing overall power consumption without uniformly degrading precision across all data.
2Device complexity
If dynamic range is reduced for quantization, then bit width is reduced, but representation accuracy is lost
Solution Approach 1:
The patent employs dynamic quantization for activation data where the quantization parameters (such as scale factors) are adjusted dynamically based on the actual dynamic range of the data in each layer or operation. This dynamic adaptation allows the system to use fewer bits while maintaining representation accuracy by optimizing the quantization range to match the actual data distribution, rather than using a fixed conservative range.
3Measurement precision
If independent block quantization is applied, then local precision is improved, but computational overhead increases
Solution Approach 1:
The patent segments the activation data into local blocks and applies independent dynamic quantization to each block. This segmentation improves local precision by adapting the quantization parameters to the specific characteristics of each block, capturing local variations in the data distribution more accurately than a global quantization approach would allow.
Solution Approach 2:
The patent applies partial dynamic quantization only to the activation data while keeping the kernel data statically quantized. This partial application of dynamic quantization achieves the benefit of improved local precision where it matters most (in the varying activation data) without incurring the full computational overhead of dynamic quantization across all data types, thus providing a practical compromise.
Data Source
AI summary
Dynamic data quantization may be applied to minimize the power consumption of a system that implements a convolutional neural network (CNN). Under such a quantization scheme, a quantized representation of a 3×3 array of m-bit activation values may include 9 n-bit mantissa values and one exponent shared between the n-bit mantissa values (n<m); and a quantized representation of a 3×3 kernel with p-bit parameter values may include 9 q-bit mantissa values and one exponent shared between the q-bit mantissa values (q<p). Convolution of the kernel with the activation data may include computing a dot product of the 9 n-bit mantissa values with the 9 q-bit mantissa values, and summing the shared exponents. In a CNN with multiple kernels, multiple computing units (each corresponding to one of the kernels) may receive the quantized representation of the 3×3 array of m-bit activation values from the same quantization-alignment module.