Neural Network Quantization Using Dynamic Range Fixed-Point Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face challenges in efficiently processing floating-point parameters due to high computational requirements, which are not suitable for devices with limited processing power, leading to accuracy issues in applications like smartphones and wearable devices.
Innovation Solution
A method and apparatus for quantizing neural network parameters from a floating-point format to a fixed-point format using Lp norm calculations to determine dynamic ranges, allowing for efficient conversion and operation in devices with lower computational capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If floating-point format is used for neural network parameters, then computational accuracy is improved, but computational requirements and processing power demands increase
Solution Approach 1:
The patent changes the numerical format parameter from floating-point to fixed-point representation. This parameter change allows the system to maintain sufficient computational accuracy while dramatically reducing the computational power and memory requirements, enabling deployment on resource-constrained devices like smartphones and wearables
2Reliability
If floating-point format is used for neural network parameters, then accuracy is maintained, but device complexity and processing power requirements increase
Solution Approach 1:
The patent applies fixed-point quantization to change the numerical representation parameter, which simplifies the computational operations and reduces device complexity. The fixed-point format enables standard arithmetic operations without requiring complex floating-point units, thereby reducing processing power requirements while maintaining acceptable accuracy levels
Solution Approach 2:
The patent adopts a simpler fixed-point numerical representation that can be processed by basic arithmetic logic units rather than complex floating-point processors. This approach uses computationally 'cheaper' operations that are faster and require less hardware complexity, suitable for mobile and embedded devices
3Productivity
If quantization is applied to convert floating-point to fixed-point format, then computational efficiency is improved, but potential accuracy loss occurs
Solution Approach 1:
The patent performs preliminary analysis of the neural network parameters to determine optimal quantization ranges and scales before actual quantization. By pre-calculating the distribution statistics and determining appropriate fixed-point representations in advance, the system minimizes accuracy loss while achieving computational efficiency
Solution Approach 2:
The patent employs feedback mechanisms where the quantization process is optimized based on the actual performance and accuracy requirements. The system adjusts quantization parameters iteratively to find the optimal balance between computational efficiency and accuracy preservation
Data Source
AI summary
A method of quantizing parameters of a neural network includes acquiring a parameter of a floating-point format used in a process of inferring by the neural network, quantizing, based on statistics of a weight included in the parameter, the weight into a fixed-point format, determining, based on statistics of an activation of one or more layers configuring the neural network included in the parameter, a dynamic range of the activation, and quantizing, based on statistics of input data of the neural network, the input data into a fixed-point format.