Fixed-Point Neural Network Quantization via Statistical Fraction Length
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks with floating-point type parameters require significant computations and resources, making them inefficient for low-power devices like smartphones, which need to minimize accuracy loss while reducing computations for real-time data analysis.
Innovation Solution
A method to generate a fixed-point type neural network by quantizing a floating-point type neural network, involving obtaining post-activation values, deriving statistical characteristics, determining a step size, and calculating a fraction length to minimize accuracy loss and reduce computations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point type neural network is used, then processing accuracy is maintained, but computation resources and power consumption increase significantly
Solution Approach 1:
The patent applies parameter changes by converting neural network parameters from floating-point format to fixed-point format with specific fraction lengths. This changes the numerical representation parameters to reduce computation complexity and power consumption while maintaining acceptable processing accuracy through statistical analysis of activation value distributions.
Solution Approach 2:
The patent applies partial action by selectively quantizing only certain layers of the neural network to fixed-point format while keeping other layers in floating-point format. This partial quantization approach reduces overall power consumption and computation resources while maintaining processing accuracy in critical layers.
2Measurement precision
If floating-point type neural network is used, then processing accuracy is maintained, but device complexity increases
Solution Approach 1:
The patent changes the numerical format parameter from floating-point to fixed-point representation, which simplifies the computational hardware requirements and reduces device complexity. The fixed-point format with determined fraction lengths enables simpler arithmetic operations while maintaining sufficient processing accuracy through statistical optimization.
Solution Approach 2:
The patent applies partial quantization to specific layers where it provides the most benefit, reducing device complexity in those layers while maintaining floating-point precision in layers where it is critical for processing accuracy.
3Use of energy by moving object
If quantization is applied to reduce computations, then power consumption decreases, but accuracy loss increases
Solution Approach 1:
The patent employs feedback mechanisms by performing statistical analysis on activation value distributions from the trained neural network to determine optimal fixed-point fraction lengths. This feedback loop ensures that the quantization parameters are optimized to minimize accuracy loss while achieving power consumption reduction through appropriate bit-width selection.
Solution Approach 2:
The patent applies partial quantization with different fraction lengths for different layers based on their specific requirements. This selective approach ensures that layers requiring higher precision maintain better accuracy while still achieving overall power consumption reduction through quantization in less critical layers.
Data Source
AI summary
A method of generating a fixed-point type neural network by quantizing a floating-point type neural network, includes obtaining, by a device, a plurality of post-activation values by applying an activation function to a plurality of activation values that are received from a layer included in the floating-point type neural network, and deriving, by the device, a plurality of statistical characteristics for at least some of the plurality of post-activation values. The method further includes determining, by the device, a step size for the quantizing of the floating-point type neural network, based on the plurality of statistical characteristics, and determining, by the device, a final fraction length for the fixed-point type neural network, based on the step size.