DNN Fixed-Point Format Allocation for Accuracy-Resource Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware implementations of Deep Neural Networks (DNNs) face challenges in terms of power consumption, processing capabilities, and silicon area, particularly when flexibility is required to support various DNN configurations, and there is a need for efficient and adaptable hardware configurations.
Innovation Solution
A method is described for determining fixed point number formats for DNN layers based on the output error attributed to the quantization of weights and input data values, using a Taylor approximation to adjust mantissa bit lengths iteratively, allowing for optimized hardware implementations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If hardware is configured to implement DNN with reduced silicon area and processing power, then resource efficiency is improved, but flexibility to support various DNN configurations deteriorates
Solution Approach 1:
The patent implements dynamic fixed point number format configuration where the hardware can adaptively adjust the number format (precision, integer bits, fractional bits) for different DNN layers and operations. This dynamic reconfigurability allows the same hardware to efficiently support various DNN configurations without requiring dedicated hardware for each configuration, thus maintaining flexibility while optimizing resource usage.
Solution Approach 2:
The patent changes the numerical parameters (fixed point format, precision, bit allocation) of the hardware implementation to optimize performance for different DNN workloads. By allowing parameter adjustment of the number format based on the specific DNN configuration being executed, the hardware achieves both resource efficiency and adaptability to various applications.
2Measurement precision
If fixed point number format precision is increased, then output accuracy is improved, but resource usage deteriorates
Solution Approach 1:
The patent applies different fixed point number formats (different precision levels, integer bits, fractional bits) to different layers and operations within the DNN based on their specific requirements. Critical layers that require high accuracy use higher precision formats, while less critical layers use lower precision formats, thereby optimizing the balance between output accuracy and resource usage across the entire network.
Solution Approach 2:
The patent dynamically adjusts the precision parameters (number of bits, integer/fractional allocation) of the fixed point number format based on the error attribution analysis. By changing these parameters adaptively rather than using a fixed high-precision format throughout, the system achieves high output accuracy where needed while minimizing resource consumption overall.
3Manufacturing precision
If error attribution analysis is performed for each layer, then optimization precision is improved, but computational overhead deteriorates
Solution Approach 1:
The patent performs error attribution analysis and determines the optimal fixed point number formats for each DNN layer in advance, before the actual DNN execution. This preliminary optimization phase allows the system to pre-calculate the precision requirements for each layer based on error propagation analysis, and then use these pre-determined formats during runtime, thereby reducing the computational overhead during actual operation while maintaining high optimization precision.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods for determining a fixed point format for one or more layers of a DNN based on the portion of the output error of the DNN attributed to the fixed point formats of the different layers. Specifically, in the methods described herein the output error of a DNN attributable to the quantisation of the weights or input data values of each layer is determined using a Taylor approximation and the fixed point number format of one or more layers is adjusted based on the attribution. For example, where the fixed point number formats used by a DNN comprises an exponent and a mantissa bit length, the mantissa bit length of the layer allocated the lowest portion of the output error may be reduced, or the mantissa bit length of the layer allocated the highest portion of the output error may be increased. Such a method may be iteratively repeated to determine an optimum set of fixed point number formats for the layers of a DNN. To be accompanied, when published, by FIG. 9 of the accompanying drawings.