Channel-Wise Fixed-Point Quantization for Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks face challenges in processing large amounts of data in real-time due to the complexity of floating-point operations, making it difficult to analyze and extract desired information efficiently.
Innovation Solution
A method for generating a fixed-point quantized neural network by analyzing the statistical distribution of floating-point parameter values, determining fixed-point expressions, and adjusting fractional lengths of biases and weights for each channel to optimize operations, followed by retraining the network to maintain accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point operations are used in neural networks, then calculation precision is maintained, but computational complexity and resource requirements increase
Solution Approach 1:
The patent changes the numerical representation parameter from floating-point to fixed-point format. By analyzing the statistical distribution of parameter values and determining appropriate fractional lengths for each channel, the system converts complex floating-point operations into simpler fixed-point operations while maintaining sufficient precision for neural network calculations
Solution Approach 2:
The patent applies different fractional lengths to different channels based on their specific statistical distributions. Instead of using a uniform precision level across all parameters, the system determines optimal fixed-point expressions individually for each channel's weights, biases, and activations, thereby achieving precision efficiency tailored to local characteristics
2Measurement precision
If floating-point operations are used in neural networks, then accuracy is maintained, but processing speed for real-time data decreases
Solution Approach 1:
The patent transforms the numerical parameter representation from floating-point to fixed-point, enabling faster computational operations. By determining optimal fixed-point expressions based on statistical analysis of each channel's data distribution, the system achieves both improved processing speed and maintained accuracy in real-time neural network inference
3Measurement precision
If floating-point operations are used in neural networks, then calculation accuracy is preserved, but resource requirements increase
Solution Approach 1:
The patent changes the numerical format parameter from floating-point to fixed-point representation. This transformation reduces memory bandwidth requirements, decreases computational resource consumption, and lowers energy usage while maintaining calculation accuracy through statistically-determined optimal fixed-point expressions for each channel
Solution Approach 2:
The patent optimizes resource usage by applying channel-specific fractional lengths based on local statistical characteristics. Each channel receives a tailored fixed-point expression that uses the minimum necessary precision, thereby reducing overall resource requirements while preserving accuracy where needed
Data Source
AI summary
A method of generating a fixed-point quantized neural network includes analyzing a statistical distribution for each channel of floating-point parameter values of feature maps and a kernel for each channel from data of a pre-trained floating-point neural network, determining a fixed-point expression of each of the parameters for each channel statistically covering a distribution range of the floating-point parameter values based on the statistical distribution for each channel, determining fractional lengths of a bias and a weight for each channel among the parameters of the fixed-point expression for each channel based on a result of performing a convolution operation, and generating a fixed-point quantized neural network in which the bias and the weight for each channel have the determined fractional lengths.