Post-Training Quantization Range Expansion Neural Network Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for quantizing floating-point deep neural networks (DNNs) to fixed-point numbers are prone to errors such as quantization error and clipping error, leading to accuracy degradation of the DNN.
Innovation Solution
A computer-implemented method for post-training quantization that involves obtaining fixed-point output values from a layer of an artificial neural network, converting them to floating-point, expanding their range, calculating a new output scaling factor, and storing it for improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional asymmetric quantization is used to convert floating-point DNN weights to fixed-point numbers, then the DNN operations can be accelerated via fixed-point hardware, but quantization error and clipping error occur leading to accuracy degradation
Solution Approach 1:
The patent applies preliminary action by performing range expansion on floating-point output values before quantization to fixed-point representation. This preliminary expansion ensures that extreme values are adequately represented in the fixed-point domain, preventing clipping errors that would otherwise occur during the quantization process. The method expands the range of floating-point values, then quantizes them to fixed-point with appropriate scaling factors, thereby maintaining accuracy while enabling fixed-point hardware acceleration.
2Measurement precision
If quantization is performed to reduce the difference between floating-point and fixed-point representations, then quantization error is minimized, but extreme values are clipped leading to accuracy degradation
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the output scaling factor based on the expanded range of floating-point values. Instead of using a fixed scaling factor, the method calculates a new output scaling factor that accounts for the expanded value range, ensuring that both typical and extreme values are properly represented in the fixed-point domain. This adaptive parameter adjustment prevents clipping while maintaining quantization accuracy.
3Device complexity
If fixed-point weights are used in neural network layers, then hardware implementation becomes feasible with reduced computational complexity, but quantization errors are introduced affecting network performance
Solution Approach 1:
The patent applies feedback by using the range of floating-point output values to inform and adjust the quantization parameters. The method calculates the range of floating-point outputs, uses this information to determine appropriate scaling factors for fixed-point representation, and thereby optimizes the quantization process. This feedback loop ensures that the fixed-point weights are quantized with parameters that maintain accuracy while enabling simplified hardware implementation.
Data Source
AI summary
Disclosed herein are systems and methods for performing post training quantization. A processor obtains fixed-point output values from a layer of an artificial neural network (ANN) wherein the layer includes fixed-point weights determined based on floating-point weights and a weight scaling factor determined based on an output scaling factor. Next, the processor converts the fixed-point output values to floating-point output values based on the output scaling factor. Then, the processor expands a range of floating-point values. Next, the processor calculates a new output scaling factor based on the expanded range of floating-point output values. Finally, the processor stores the new output scaling factor in an associated memory.