Histogram-Based Fixed Point Format Selection for DNN Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in efficient hardware implementation due to high power consumption and silicon area requirements, particularly when processing power or silicon area is limited, and existing fixed point number formats are inefficient as they may include redundant bits, leading to increased complexity and resource usage.
Innovation Solution
A histogram-based method for selecting a fixed point number format by obtaining a histogram representing the expected distribution of values, quantizing representative values for each bin, estimating total quantization errors for different formats, and selecting the format with the smallest estimated error to minimize resource usage while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If fixed point number formats with more bits are used to represent DNN values, then precision and accuracy are improved, but silicon area and device complexity increase
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the number of mantissa bits and exponent bits based on the actual data distribution characteristics of each layer. Instead of using a fixed precision format throughout the network, the system analyzes the histogram of values and adapts the fixed point format parameters (mantissa bits, exponent bits) to match the actual precision requirements, thereby reducing silicon area while maintaining necessary accuracy.
Solution Approach 2:
The patent implements local quality by allowing different layers or even different values within the same layer to use different fixed point number formats with varying precision. Each layer's data format is independently optimized based on its specific data distribution, so that only the necessary precision is allocated where needed, rather than uniformly high precision across the entire network.
2Measurement precision
If fixed point number formats with more bits are used, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The system dynamically changes the parameters of the fixed point number format (number of mantissa bits, number of exponent bits) based on the actual data distribution. This adaptive approach reduces device complexity by avoiding the use of excessive precision bits that would increase hardware complexity without providing corresponding benefits to the computation accuracy.
Solution Approach 2:
The patent introduces dynamics by making the fixed point format configuration adaptive rather than static. The system analyzes the data distribution characteristics and dynamically selects or adjusts the format parameters during network deployment or inference, allowing the hardware to operate with optimized precision levels that reduce complexity while maintaining accuracy.
3Device complexity
If a single fixed point number format is used for all layers, then device complexity is reduced, but adaptability worsens due to inability to handle different data distributions
Solution Approach 1:
The patent applies local quality by allowing each layer to have its own optimized fixed point number format based on its specific data distribution characteristics. Different layers can use different combinations of mantissa bits and exponent bits, enabling the system to adapt to local precision requirements without significantly increasing overall device complexity, as the format selection is done through systematic analysis rather than complex runtime adjustments.
Solution Approach 2:
The system changes the parameters of the fixed point format (mantissa bits, exponent bits) to match the data distribution of each layer. This allows the hardware to be adaptable to different data characteristics across layers while maintaining relatively simple fixed point arithmetic operations, achieving a balance between adaptability and device complexity.
Data Source
AI summary
A histogram-based method of selecting a fixed point number format for representing a set of values input to, or output from, a layer of a Deep Neural Network (DNN). The method comprises obtaining a histogram that represents an expected distribution of the set of values of the layer, each bin of the histogram is associated with a frequency value and a representative value in a floating point number format; quantising the representative values according to each of a plurality of potential fixed point number formats; estimating, for each of the plurality of potential fixed point number formats, the total quantisation error based on the frequency values of the histogram and a distance value for each bin that is based on the quantisation of the representative value for that bin; and selecting the fixed point number format associated with the smallest estimated total quantisation error as the optimum fixed point number format for representing the set of values of the layer.