Dynamic Quantization Using Look-Up Tables for Neural Network Feature Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural network (DNN) feature maps represented as single precision floating point values are costly in terms of hardware computational and storage resources, and conventional quantization methods using uniform distribution fail to accurately reflect the non-uniform distribution of feature maps, leading to quantization errors and loss of performance.
Innovation Solution
The use of look-up tables (LUTs) between layers of deep neural networks to store non-uniform distributions of quantization levels, allowing for dynamic quantization of feature maps, which reduces storage needs and minimizes quantization errors by representing feature data with index values that correspond to optimal quantization levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single precision floating point values are used to represent DNN feature maps, then measurement precision is maintained, but hardware computational and storage costs increase significantly
Solution Approach 1:
The patent implements dynamic quantization where the quantization parameters (number of bits and distribution characteristics) are adapted based on the actual data distribution of each feature map. This allows the system to adjust the precision level dynamically - using fewer bits when possible to reduce storage costs while maintaining adequate precision when needed, thereby resolving the contradiction between precision and storage cost
Solution Approach 2:
The patent changes the parameter representation from fixed single precision floating point to variable precision quantized formats. By transforming the data representation parameters (bit depth, quantization levels, distribution modeling), the system achieves significant storage cost reduction while preserving essential feature information through intelligent parameter selection
2Quantity of substance
If conventional uniform quantization is applied to reduce storage costs, then hardware storage cost decreases, but quantization errors increase due to mismatch with non-uniform feature map distribution
Solution Approach 1:
The patent applies local quality by modeling the non-uniform distribution characteristics of different regions within feature maps. Instead of applying uniform quantization across all values, the system adapts quantization levels to match the local data density and importance, placing more quantization levels in regions where feature values are more concentrated or significant, thereby reducing quantization errors while maintaining storage efficiency
Solution Approach 2:
The patent implements dynamic adaptation of quantization parameters based on the actual feature map distribution. The quantization scheme is not fixed but dynamically adjusted to match the non-uniform distribution characteristics of each layer's feature maps, optimizing the balance between storage cost and accuracy for each specific case rather than applying a one-size-fits-all uniform approach
Data Source
AI summary
A method includes processing, using at least one processor of an electronic device, input data using a first layer of a neural network to generate a feature map. The method also includes representing, using the at least one processor, feature data of the feature map using index values. The index values correspond to multiple records of a look up table (LUT), and the records of the LUT represent a non-uniform distribution of quantization levels of the feature map. The method further includes storing, using the at least one processor, the index values in a memory of the electronic device. The method also includes regenerating, using the at least one processor, the feature data of the feature map by cross-referencing the index values with the LUT. In addition, the method includes processing, using the at least one processor, the feature data using a second layer of the neural network.