Neural Network Quantization Using Reference Layer Statistics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Typical neural network quantization methods result in large quantization errors due to determining the target quantization range based solely on statistical information from the target layer, leading to inefficient memory consumption and computation complexity.
Innovation Solution
The method involves retrieving statistical information from a reference layer and determining a quantization range for the target layer, excluding regions outside the frequency distribution range of adjacent layers to reduce quantization errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If quantization range is determined based solely on statistical information from the target layer, then the quantization process is simple and fast, but large quantization errors occur
Solution Approach 1:
The patent introduces statistical information from reference layers as an intermediary to improve quantization precision. By using frequency distribution data from adjacent layers as a mediator, the quantization range determination becomes more accurate without requiring complex iterative optimization, thus resolving the contradiction between precision and complexity.
Solution Approach 2:
The patent applies preliminary action by pre-calculating and storing frequency distribution statistics of reference layers before quantization. This preliminary preparation of statistical information enables more accurate quantization range determination for target layers, improving precision while maintaining computational efficiency during the actual quantization process.
2Productivity
If quantization range includes regions outside frequency distribution of adjacent layers, then the quantization covers more potential values, but ineffective regions are created increasing computation complexity
Solution Approach 1:
The patent applies local quality by making the quantization range adaptive to local frequency distribution characteristics of each layer. Instead of using a uniform quantization approach, the method tailors the quantization range to the specific statistical properties of reference layers, eliminating ineffective regions and improving both precision and inference speed.
Solution Approach 2:
The patent changes the parameters of quantization range determination by incorporating frequency distribution statistics from reference layers. This parameter change allows the quantization process to adapt to the actual data characteristics, removing ineffective regions and optimizing the balance between quantization accuracy and computational efficiency.
3Measurement precision
If quantization interval is large, then the quantization process is computationally efficient, but quantization errors increase
Solution Approach 1:
The patent applies dynamics by making the quantization interval adaptive rather than fixed. The quantization interval dynamically adjusts based on the frequency distribution statistics of reference layers, allowing smaller intervals in regions where precision is critical and larger intervals where computation efficiency is prioritized, thus resolving the contradiction between error reduction and computational efficiency.
Data Source
AI summary
A neural-network quantization method includes retrieving, from a reference layer, statistical information on layer parameters related to the reference layer. The layer parameters include features of the reference layer. The neural-network quantization method includes determining, based on the statistical information, a quantization range for the layer parameters related to a quantization target layer. The neural-network quantization method quantizes selected layer parameters in the layer parameters related to the quantization target layer. The selected layer parameters are within the quantization range.