DNN Weight Quantization for Fixed-Point Hardware Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Neural Networks (DNNs) face accuracy reduction when converting from floating-point to fixed-point number formats due to quantization errors, especially when retraining is not practical or possible.
Innovation Solution
A method is developed to convert DNN weights from a floating-point to a fixed-point number format by determining quantization errors, identifying a subset of weights to adjust, and generating quantized weights to minimize total quantization error, allowing the DNN to maintain accuracy without retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If weights are quantized from floating-point to fixed-point number format, then silicon area and power consumption are reduced, but accuracy deteriorates due to quantization errors
Solution Approach 1:
The patent applies preliminary action by calculating quantization errors for all weights before final quantization, then using these pre-calculated errors to selectively adjust weights that would otherwise cause excessive total error. This advance preparation allows the system to achieve accurate fixed-point conversion without retraining.
Solution Approach 2:
The patent changes parameters by selectively modifying specific weight values based on their individual quantization errors. Instead of uniformly quantizing all weights, the system identifies weights where adjusting the quantized value (even beyond standard rounding) reduces total quantization error, thereby optimizing accuracy for the given bit-width constraint.
2Device complexity
If weights are quantized from floating-point to fixed-point number format, then device complexity is reduced, but accuracy deteriorates due to quantization errors
Solution Approach 1:
The patent performs preliminary calculation of quantization errors and identifies which weights need adjustment before the actual quantization process. This pre-computation step enables the system to achieve high accuracy with simple fixed-point hardware, as the optimal weights are determined offline without requiring complex runtime decision-making in the hardware.
Solution Approach 2:
The patent optimizes weight parameters by selectively adjusting individual weights based on their quantization characteristics. This parameter optimization allows the use of simpler fixed-point arithmetic hardware while maintaining accuracy comparable to more complex floating-point implementations.
3Ease of manufacture
If standard quantization method is used for all weights, then conversion process is simplified, but total quantization error increases
Solution Approach 1:
The patent applies local quality by treating different weights differently based on their individual quantization characteristics. Instead of applying a uniform quantization method to all weights, the system identifies specific weights that benefit from adjusted quantization and applies different quantization strategies to different weights, optimizing overall accuracy.
Solution Approach 2:
The patent changes the quantization parameter (the quantized weight value) selectively for specific weights rather than applying a fixed quantization rule to all weights. This allows the system to improve quantization precision for critical weights while maintaining simplicity for others, achieving a balance between accuracy and process simplicity.
Data Source
AI summary
Methods and system for converting a plurality of weights of a filter of a Deep Neural Network (DNN) in a first number format to a second number format, the second number format having less precision than the first number format, to enable the DNN to be implemented in hardware logic. The method comprising: determining, for each of the plurality of weights, a quantisation error associated with quantising that weight to the second number format in accordance with a first quantisation method; determining a total quantisation error for the plurality of weights based on the quantisation errors for the plurality of weights; identifying a subset of the plurality of weights to be quantised to the second number format in accordance with a second quantisation method based on the total quantisation error for the plurality of weights; and generating a set of quantised weights representing the plurality of weights in the second number format, the quantised weight for each weight in the subset of the plurality of weights based on quantising that weight to the second number format in accordance with the second quantisation method and the quantised weight for each of the remaining weights of the plurality of weights based on quantising that weight to the second number format in accordance with the first quantisation method.