A Convolutional Neural Network Pruning Method Based on Feature Map Sparsification

Abstract

The invention discloses a convolutional neural network pruning method based on feature map sparseness, and relates to how to compress the convolutional neural network to reduce its parameter amount and calculation amount so as to facilitate actual deployment; in the training process, by adding Regularize the feature map L1 or L2 after the activation layer, so that the corresponding feature map channels have different sparsity. Under a certain pruning rate, the convolution kernel corresponding to the corresponding channel is cut according to the sparsity of the feature map channel, and the pruning is fine-tuned. The post-branch network obtains new accuracy, and adjusts the pruning rate according to the change in accuracy before and after pruning. After multiple iterations, the near-optimal pruning rate is found, and the maximum pruning rate is achieved under the condition that the accuracy does not decrease. branch. This method greatly reduces the amount of parameters and computation of convolutional neural networks.

Description

technical field [0001] The invention relates to a convolutional neural network compression technology in the field of deep learning, in particular to a convolutional neural network pruning method based on feature map sparseness. Background technique [0002] Convolutional neural network is a common deep learning network architecture that performs well on many problems such as image classification, object detection, and image style transfer. At the same time, the operation mode of the convolutional neural network is still a black box. There is no solid theoretical basis for how to set the number of layers of the network and the number of channels of each layer for specific problems. This causes redundancy of network parameters, which in turn makes the convolutional neural network have a large amount of calculation and takes up too much memory, which is not conducive to the application of the network. Therefore, it is necessary to perform model compression of convolutional ne...

AI Technical Summary

Problems solved by technology

[0004] Non-Patent Document 1 (Jaderberg M, et.al. "Speeding up convolutional neural networks with low rank expansions." Proc.arxiv, 2014.) uses the method of low-rank decomposition to split the convolution operation of the convolutional layer, and achieved 4.5 times acceleration effect, but causes a 1% decrease in accuracy

[0005] Non-Patent Document 2 (Gupta S, et.al. "Deep learning with limited numerical precision." Proc. International Conference on Machine Learning, 2015.) uses 16-bit fixed-point numbers to train convolutional neural networks, and at the same time realizes the MNIST data The accuracy on the set does not decrease, but the article does not give the test results on a larger data set

Images