Neural Network Quantization Using Segmented Density Approximation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial neural networks face a trade-off between accuracy and computational complexity, with quantization methods often resulting in decreased accuracy and requiring significant computational resources, especially when transitioning from floating-point to fixed-point neural networks.
Innovation Solution
A method and apparatus that divide the input distribution of an artificial neural network into segments, generate an approximated density function, calculate quantization errors based on this function, and determine a final step size to minimize accuracy loss while reducing computational complexity, using an approximator and quantizer to achieve this.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If quantization is applied to reduce computational complexity, then computational complexity decreases, but accuracy deteriorates
Solution Approach 1:
The input distribution is divided into multiple segments, and an approximated density function is generated for each segment separately. This segmentation allows the quantization process to adapt to local distribution characteristics, reducing quantization error and maintaining accuracy while enabling efficient fixed-point implementation.
Solution Approach 2:
The method optimizes quantization parameters (step sizes) based on the approximated density function and calculated quantization errors. By dynamically adjusting these parameters according to the actual data distribution, the system achieves high accuracy in fixed-point representation without requiring excessive computational resources.
2Device complexity
If traditional quantization methods are used, then computational complexity is reduced, but significant computational resources are still required for accuracy maintenance
Solution Approach 1:
The approximated density function is generated in advance during the quantization design phase, capturing the input distribution characteristics before actual inference. This preliminary analysis enables the system to determine optimal step sizes beforehand, eliminating the need for complex runtime computations and reducing computational resource requirements during deployment.
Solution Approach 2:
The method uses an approximated density function that copies the essential characteristics of the input distribution without requiring the full complexity of the original floating-point computations. This approximation captures the necessary statistical properties while enabling efficient fixed-point quantization with minimal computational overhead.
3Measurement precision
If floating-point neural networks are used, then accuracy is maintained, but computational complexity increases
Solution Approach 1:
The method transforms the neural network from floating-point to fixed-point representation by optimizing quantization parameters based on the approximated density function. This parameter transformation maintains numerical accuracy while reducing computational complexity, as fixed-point operations are more efficient and require fewer computational resources than floating-point operations.
Data Source
AI summary
A method of quantizing an artificial neural network includes dividing an input distribution of the artificial neural network into a plurality of segments, generating an approximated density function by approximating each of the plurality of segments, calculating at least one quantization error corresponding to at least one step size for quantizing the artificial neural network, based on the approximated density function, and determining a final step size for quantizing the artificial neural network based on the at least one quantization error.