Convolutional neural network pruning method based on feature map sparsification

Abstract

The invention discloses a convolutional neural network pruning method based on feature map sparsification, and relates to how to compress a convolutional neural network to reduce the parameter quantity and calculation quantity of the convolutional neural network so as to facilitate actual deployment. In the TRAINING PROCESS, regularization is added on a feature map L1 or L2 behind an active layerin a loss function; corresponding feature map channels are enabled to have different sparseness; a convolution kernel corresponding to a corresponding channel is cut off according to the sparsity of afeature map channel under a certain pruning rate, a new accuracy is obtained by finely tuning the pruned network, the pruning rate is adjusted according to the accuracy change before and after pruning, a near-optimal pruning rate is found through multiple iterations, and maximum pruning is realized under the condition that the accuracy is not reduced. According to the method, the parameter quantity and the calculation quantity of the convolutional neural network are greatly reduced.

Description

technical field [0001] The invention relates to a convolutional neural network compression technology in the field of deep learning, in particular to a convolutional neural network pruning method based on feature map sparseness. Background technique [0002] Convolutional neural network is a common deep learning network architecture that performs well on many problems such as image classification, object detection, and image style transfer. At the same time, the operation mode of the convolutional neural network is still a black box. There is no solid theoretical basis for how to set the number of layers of the network and the number of channels of each layer for specific problems. This causes redundancy of network parameters, which in turn makes the convolutional neural network have a large amount of calculation and takes up too much memory, which is not conducive to the application of the network. Therefore, it is necessary to perform model compression of convolutional ne...

AI Technical Summary

Problems solved by technology

[0004] Non-Patent Document 1 (Jaderberg M, et. al. "Speeding up convolutional neural networks with low rank expansions." Proc. arxiv, 2014.) uses the method of low-rank decomposition to split the convolution operation of the convolutional layer, achieving 4.5 times acceleration effect, but causes a 1% decrease in accuracy

[0005] Non-Patent Document 2 (Gupta S, et. al. "Deep learning with limited numerical precision." Proc. International Conference on Machine Learning, 2015.) uses 16-bit fixed-point numbers to train convolutional neural networks, and at the same time realizes the MNIST data The accuracy on the set does not decrease, but the article does not give the test results on a larger data set

Images