DNN Number Format Selection via Sensitivity and Quantization Error
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face challenges in efficiently representing network parameters in hardware due to the trade-off between accuracy and resource consumption, particularly when using floating point numbers, which increase silicon area, power consumption, and complexity.
Innovation Solution
A method is developed to determine the optimal number format for representing network parameters by calculating sensitivity and quantization error for each parameter, allowing for the selection of the candidate number format with the minimum local error, thereby balancing accuracy and resource efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating point numbers are used to represent network parameters, then accuracy is improved, but silicon area, power consumption, and complexity increase
Solution Approach 1:
The patent changes the numerical representation parameter from floating point to fixed point format, fundamentally altering how network parameters are stored and processed. This parameter change reduces hardware complexity while maintaining acceptable accuracy through careful selection of fixed point precision levels.
Solution Approach 2:
The patent applies different fixed point number formats to different layers and parameters within the neural network based on their specific sensitivity requirements. By tailoring the precision and range of fixed point representation locally to each layer's needs, the system achieves optimal balance between accuracy and resource efficiency for each specific component.
2Productivity
If quantisation is applied to reduce resource consumption, then resource efficiency is improved, but error in DNN output increases
Solution Approach 1:
The patent performs sensitivity analysis and quantisation error calculation before deploying the quantised model, allowing preliminary identification of which parameters can be safely quantised and which require higher precision. This preliminary action enables optimal allocation of precision resources across different parameters.
Solution Approach 2:
The patent uses sensitivity analysis as a feedback mechanism to guide the quantisation process. By calculating how sensitive each parameter is to quantisation errors, the system can adjust the quantisation strategy for different parameters, applying higher precision where needed and lower precision where acceptable, thereby minimizing overall output error while maximizing resource efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of determining a number format for representing a set of two or more network parameters of a Deep Neural Network "DNN" for use in configuring hardware logic to implement the DNN. The method includes: determining a sensitivity of the DNN with respect to each network parameter in the set of network parameters; for each candidate number format of a plurality of candidate number formats: determining a quantisation error associated with quantising each network parameter in the set of network parameters in accordance with the candidate number format; generating an estimate of an error in an output of the DNN caused by quantisation of the set of network parameters based on the sensitivities and the quantisation errors; generating a local error based on the estimated error; and selecting the candidate number format of the plurality of candidate number formats with the minimum local error as the number format for the set of network parameters.