Pruning method and device for convolutional neural network, equipment and medium

Abstract

The invention provides a convolutional neural network pruning method, apparatus and device, and a medium. The method comprises the steps of obtaining a data set of an image and dividing the data set into a training set and a verification set; after a to-be-pruned convolutional neural network model is initialized, multiple rounds of pre-training being carried out on images in the training set, and determining a sensitive layer of the network; multiple rounds of automatic pruning operation being carried out through reinforcement learning, the model accuracy of each round of pruning operation being obtained, and therefore obtaining the model pruning strategy with the highest model accuracy. According to a deterministic strategy of reinforcement learning, different compression strategies are carried out on a sensitive layer and a non-sensitive layer of a network, so that the number of filters of each network layer is pruned; and performing fine tuning operation to obtain a final convolutional neural network model. According to the method, reinforcement learning is involved from the beginning of model pruning, the pruning strategy of the model is optimized according to the environment, the pruning object is a complete filter, irregularity of the model cannot be caused, and generalization is greatly improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a pruning method, device, equipment and medium of a convolutional neural network. Background technique [0002] In recent years, on the one hand, the research results of deeper neural networks are getting better and better. On the other hand, with the continuous development and innovation of unmanned driving and smart mobile devices and other related fields, it is suitable for computing power. The requirements for deep neural network models on weaker edge devices are also gradually increasing. Due to the characteristics of the deep neural network, when it is deployed on a mobile device, the amount of parameters and floating-point calculations it contains are extremely large. For example, when using a 107-layer deep YOLO v3 network to detect an image with a resolution of 416×416, it will generate 240MB of parameters and perform up to 65 billion multiplication and accumu...

AI Technical Summary

Problems solved by technology

[0004] (1) Most of the existing neural network pruning methods require manual experts to continuously adjust the parameters to achieve the best pruning effect, which is very time-consuming in terms of actual effect, and the effect achieved at the same time is prone to local optimum or suboptimal Condition

[0005] (2) Existing model pruning methods focus on weight pruning in the model, which requires specific hardware support and poor generalization

After repeating so many times, record the previous pruning method and the corresponding accuracy, so as to train reinforcement learning, and finally predict through reinforcement learning, which is actually a random strategy that obeys the normal distribution, which leads to reinforcement learning. Practical application is not enough

At the same time, the invention is a model pruning operation for weights rather than complete filters, which will inevitably lead to irregular models, poor generalization in actual use, and specific hardware support is required

Images