DNN Weight Quantization With Selective Rounding Error Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for converting floating point Deep Neural Networks (DNNs) to fixed point DNNs for hardware implementation suffer from accuracy loss due to quantization errors, which are often addressed by retraining, but this is not always practical or possible.
Innovation Solution
A method for converting DNN weights from a first to a second, lower precision number format by determining quantization errors, selecting a subset of weights to change rounding directions, and generating quantized weights to minimize total quantization error without retraining, using a first and second quantization method.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If weights are quantized from floating point to fixed point format, then silicon area and hardware complexity are reduced, but quantization errors cause accuracy loss
Solution Approach 1:
The patent applies preliminary action by adjusting weights before quantization to compensate for expected quantization errors. The method calculates adjustment values based on the difference between floating-point and fixed-point representations, then modifies weights beforehand so that after quantization, the overall accuracy is maintained without requiring retraining
Solution Approach 2:
The patent changes parameters by modifying weight values and quantization strategies. It dynamically selects which weights to adjust based on their quantization error characteristics, and uses different adjustment magnitudes for different weights, thereby optimizing the balance between hardware efficiency and accuracy
2Productivity
If standard quantization methods are used to convert weights to fixed point format, then conversion is simple and fast, but total quantization error increases
Solution Approach 1:
The patent applies local quality by treating different weights differently based on their individual quantization error characteristics. Instead of applying a uniform quantization approach to all weights, it selectively adjusts specific weights that benefit most from correction, thereby minimizing total quantization error while maintaining computational efficiency
Solution Approach 2:
The patent creates adjusted weight values as corrected copies of the original quantized weights. These adjusted weights incorporate compensation for quantization errors and are used in place of standard quantized weights, achieving higher accuracy without significantly increasing computational overhead
3Measurement precision
If retraining is performed after quantization to maintain accuracy, then DNN accuracy is preserved, but training time and computational resources are consumed
Solution Approach 1:
The patent performs preliminary weight adjustment before quantization to pre-compensate for accuracy loss, eliminating the need for time-consuming retraining. By calculating and applying adjustment values based on quantization error analysis, the method achieves accuracy preservation in a single pass without iterative training
Solution Approach 2:
The patent extracts and corrects quantization errors through a separate adjustment step that operates on quantized weights. This extraction of error correction from the training process allows accuracy maintenance without requiring full retraining, significantly reducing time and computational resource requirements
Data Source
AI summary
Methods and system for converting a plurality of weights of a filter of a Deep Neural Network (DNN) in a first number format to a second number format, the second number format having less precision than the first number format, to enable the DNN to be implemented in hardware logic. The method comprising: determining, for each of the plurality of weights, a quantisation error associated with quantising that weight to the second number format in accordance with a first quantisation method; determining a total quantisation error for the plurality of weights based on the quantisation errors for the plurality of weights; identifying a subset of the plurality of weights to be quantised to the second number format in accordance with a second quantisation method based on the total quantisation error for the plurality of weights; and generating a set of quantised weights representing the plurality of weights in the second number format, the quantised weight for each weight in the subset of the plurality of weights based on quantising that weight to the second number format in accordance with the second quantisation method and the quantised weight for each of the remaining weights of the plurality of weights based on quantising that weight to the second number format in accordance with the first quantisation method.