Neural Network Quantization via Convolution Kernel Splitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models face significant computational costs and power consumption due to large parameter sizes, and prior quantization methods often result in excessive quantization errors, limiting their deployment on edge devices.
Innovation Solution
The method involves identifying convolution kernels with anomalous coefficient distributions, splitting them into sub-kernels, and quantizing these sub-kernels separately to reduce quantization errors, thereby improving the computation performance of neural network models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If quantization is applied to reduce computational cost and power consumption, then energy efficiency improves, but quantization errors increase
Solution Approach 1:
The patent divides the quantization process into multiple stages: first identifying convolution kernels with anomalous coefficient distributions, then splitting them into sub-kernels, and finally quantizing each sub-kernel separately. This segmentation allows different quantization strategies to be applied to different parts of the model, reducing overall quantization error while maintaining energy efficiency benefits.
Solution Approach 2:
The patent applies different quantization approaches to different convolution kernels based on their coefficient distribution characteristics. Kernels with anomalous distributions are handled differently from normal kernels, with the former being split into sub-kernels and quantized separately. This local quality approach ensures that each part of the model receives appropriate quantization treatment, minimizing overall precision loss.
2Measurement precision
If convolution kernels with anomalous coefficient distribution are split into sub-kernels, then quantization error reduces, but computational complexity increases
Solution Approach 1:
The patent segments only the problematic convolution kernels with anomalous coefficient distributions into sub-kernels, rather than dividing the entire model. This selective segmentation reduces quantization errors for problematic kernels while avoiding the computational overhead of splitting all kernels, thus balancing precision improvement with complexity management.
Solution Approach 2:
The patent changes the structural parameter of selected convolution kernels by splitting them into sub-kernels with specific size ratios. This parameter change is applied selectively based on coefficient distribution analysis, transforming the kernel structure only where necessary to reduce quantization errors, thereby managing the trade-off between precision and complexity.
3Reliability
If selective quantization of sub-kernels is performed, then model accuracy is maintained, but processing time increases
Solution Approach 1:
The patent performs preliminary identification and splitting of convolution kernels with anomalous coefficient distributions before the quantization process. By pre-processing and organizing the kernels that require special handling, the actual quantization phase can proceed more efficiently with clearly defined sub-kernels, reducing overall processing time while maintaining accuracy.
Solution Approach 2:
The patent segments the quantization processing into distinct phases: identification of anomalous kernels, splitting into sub-kernels, and quantization execution. This segmentation of the processing workflow allows for optimized execution of each phase, reducing total processing time while ensuring model accuracy is maintained through targeted quantization of problematic kernels.
Data Source
AI summary
A quantization method and device for a neural network model, and a computer-readable storage medium are provided. The method includes determining, from a neural network model, a target convolution kernel having an abnormal coefficient distribution, splitting the target convolution kernel so as to obtain a plurality of sub-convolution kernels, quantizing the plurality of sub-convolution kernels respectively to obtain a plurality of quantized convolution kernels, and replacing the target convolution kernel with the plurality of quantized convolution kernels. The method can reduce quantization errors.