Neural Network Parameter Compression With Adaptive Quantization Steps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face challenges in achieving high compression of parameter data due to varying optimal weight values, leading to inconsistent compression sizes.
Innovation Solution
A data processing device and method that includes a compression controlling unit to determine quantization steps on a node-by-node or kernel-by-kernel basis, encoding network configuration information and quantization information to generate compressed data, allowing for high compression of parameter data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If scalar-quantizing and encoding edge weights is used, then data size of edge weights is compressed, but high compression of parameter data cannot be achieved due to variations in compression size
Solution Approach 1:
The patent applies local quality by determining quantization steps on a node-by-node or kernel-by-kernel basis rather than using a uniform quantization step for all parameters. This allows different regions of the neural network to have different quantization characteristics optimized for their specific properties, thereby achieving consistent high compression while maintaining accuracy.
Solution Approach 2:
The patent implements dynamics by making the quantization steps adaptive and variable rather than fixed. The compression controlling unit dynamically determines appropriate quantization steps for each node or kernel based on local characteristics, enabling the compression scheme to adapt to varying parameter distributions and achieve reliable high compression ratios.
2Measurement precision
If optimal weight values vary depending on network training results, then neural network performance is optimized, but compression size becomes inconsistent
Solution Approach 1:
The patent changes the parameter of quantization step from a fixed global value to variable local values determined for each node or kernel. This parameter change allows the compression scheme to accommodate variations in optimal weight values while maintaining both accuracy and consistent compression size by adapting the quantization granularity to local needs.
3Quantity of substance
If quantization steps are changed on a node-by-node or kernel-by-kernel basis, then high compression is achieved, but quantization information to be encoded increases
Solution Approach 1:
The patent applies merging by combining multiple quantization step values into a single quantization information structure that can be efficiently encoded. The encoding unit encodes both the network configuration information and the quantization information together, reducing overall complexity while maintaining the ability to achieve high compression through localized quantization.
Data Source
Figure 1~2
Figure 3A~4
Figure 5
AI summary
A data processing unit (101) processes input data using a neural network. A compression controlling unit (102) generates quantization information that defines quantization steps. An encoding unit (103) encodes network configuration information including parameter data which is quantized using the quantization steps determined by the compression controlling unit (102), and the quantization information, to generate compressed data.