Fixed-Point Neural Network Quantization for Real-Time Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face challenges in processing large amounts of data in real-time due to the complexity of floating-point operations, which hinders efficient analysis and information extraction, especially in resource-constrained devices like mobile devices.
Innovation Solution
A method is introduced to generate a fixed-point quantized neural network by analyzing the statistical distribution of floating-point parameter values, determining fractional lengths for biases and weights, and bit-shifting to optimize operations, thereby reducing the number of operations required for processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point operations are used in neural networks, then calculation precision is maintained, but processing speed and computational efficiency deteriorate
Solution Approach 1:
The patent applies parameter changes by converting the data representation format from floating-point to fixed-point. This involves changing the numerical precision parameters and quantization levels, transforming the neural network operations to use fixed-point arithmetic which is computationally more efficient while maintaining adequate precision for the application
2Reliability
If floating-point neural networks are used, then accuracy is maintained, but computational burden and resource consumption increase
Solution Approach 1:
The patent changes the numerical representation parameters from floating-point to fixed-point format, which reduces the computational complexity and resource consumption. The fixed-point quantization parameters are carefully selected to maintain the necessary accuracy while significantly reducing the computational burden on resource-constrained devices
Solution Approach 2:
The patent employs fixed-point arithmetic which can be implemented using simpler, less resource-intensive hardware operations compared to floating-point arithmetic. This approach uses computationally cheaper operations that are sufficient for the application requirements, reducing overall resource consumption
3Ease of operation
If floating-point operations are performed, then processing capability is sufficient, but device complexity and operational difficulty increase
Solution Approach 1:
The patent transforms the operational parameters from floating-point to fixed-point representation, which simplifies the hardware requirements and makes the system easier to operate. Fixed-point operations can be performed using simpler digital logic circuits that are more accessible and easier to implement in resource-constrained environments
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of generating a fixed-point quantized neural network includes analyzing a statistical distribution for each channel of floating-point parameter values of feature maps and a kernel for each channel from data of a pre-trained floating-point neural network, determining a fixed-point expression of each of the parameters for each channel statistically covering a distribution range of the floating-point parameter values based on the statistical distribution for each channel, determining fractional lengths of a bias and a weight for each channel among the parameters of the fixed-point expression for each channel based on a result of performing a convolution operation, and generating a fixed-point quantized neural network in which the bias and the weight for each channel have the determined fractional lengths.