Depthwise Convolution Bias Correction for Accurate Model Pruning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for making learning models more lightweight through pruning result in a trade-off between compressibility and accuracy, with conventional approaches leading to degradation in model performance.
Innovation Solution
A support method and device that corrects bias terms in subsequent convolutional layers of a neural network by calculating correction values based on the bias values of depthwise convolutional layers, thereby maintaining model accuracy during pruning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Weight of stationary object
If pruning is applied to make the learning model more lightweight, then model size is reduced, but model accuracy deteriorates
Solution Approach 1:
The patent changes the parameter values of bias terms in subsequent convolutional layers by calculating correction values based on the pruned depthwise convolutional layer's bias term. This parameter adjustment compensates for the accuracy loss caused by pruning while maintaining the reduced model size.
Solution Approach 2:
The patent performs preliminary calculation of correction values for bias terms before finalizing the pruned model. By pre-calculating the correction values based on the relationship between depthwise and subsequent convolutional layers, the method prepares compensation mechanisms in advance to prevent accuracy deterioration.
2Weight of stationary object
If quantization is used to make the learning model more lightweight, then model size is reduced, but model precision deteriorates
Solution Approach 1:
The patent applies parameter changes to bias terms by calculating and applying correction values that compensate for the precision loss from quantization. This allows the model to maintain higher precision while still benefiting from the size reduction achieved through quantization.
Data Source
AI summary
A support method supports pruning of a learning model using a neural network including a depthwise convolutional layer and a subsequent convolutional layer subsequent to the depthwise convolutional layer. The learning model includes a target neuron to be pruned in the depthwise convolutional layer. The support method includes: determining whether the depthwise convolutional layer has a bias term; obtaining a bias value based on the bias term, when the depthwise convolutional layer has the bias term; and calculating a correction value for correcting a bias term of the subsequent convolutional layer, using a value based on the bias value.