Neural Network Quantization Removing Outlier Parameters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network quantization often results in distorted values and performance degradation due to the inclusion of outlier parameters, which are not effectively addressed by existing methods that rely solely on parameter distribution for quantization.
Innovation Solution
A method and apparatus that remove parameters based on output values or batch normalization parameters to prevent distortion and performance degradation, allowing for accurate quantization by focusing on surviving parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If quantization is applied to all parameters including outliers, then computational complexity and power consumption are reduced, but parameter value distortion occurs and neural network performance degrades
Solution Approach 1:
The patent extracts and removes outlier parameters from the parameter set before quantization. By identifying parameters with extreme values that deviate significantly from the normal distribution and removing them, the patent prevents these outliers from causing value distortion during quantization, thereby resolving the contradiction between energy reduction and precision maintenance
Solution Approach 2:
The patent changes the parameter set by removing outliers and then applies quantization to the modified parameter distribution. This parameter change approach allows the quantization process to operate on a more concentrated parameter distribution, reducing value distortion while maintaining the energy efficiency benefits of quantization
2Productivity
If quantization is applied to all parameters, then computational speed is improved, but neural network accuracy deteriorates due to outlier parameters
Solution Approach 1:
The patent extracts outlier parameters that would otherwise deteriorate neural network accuracy during quantization. By removing these problematic parameters before the quantization process, the patent enables accurate quantization of the remaining parameters, thus achieving both computational speed improvement and accuracy preservation
Solution Approach 2:
The patent converts the harmful effect of outlier parameters into a benefit by identifying and removing them. This process transforms the potential source of accuracy deterioration into an opportunity to improve overall quantization quality by focusing on the well-behaved parameter distribution
3Device complexity
If existing quantization methods are used without parameter removal, then the quantization process is simple, but value distortion occurs due to outlier parameters
Solution Approach 1:
The patent performs preliminary action by removing outlier parameters before the quantization process. This pre-processing step prepares the parameter set for quantization by eliminating elements that would cause value distortion, thereby achieving accurate quantization with only moderate increases in process complexity
Data Source
AI summary
A method and a device for quantizing parameters of a neural network are disclosed. According to one aspect of the present invention, a computer-implemented method for quantizing parameters of a neural network including batch normalization parameters, the method comprising obtaining parameters in a second layer connected to a first layer; removing at least one parameter among the parameters based on either any one of output values of the first layer or batch normalization parameters applied to the parameters; and quantizing the parameters in the second layer based on parameters that have survived the removing, is provided.