Heuristic filter pruning method and system in convolutional neural network

Abstract

The invention discloses a heuristic filter pruning method and system in a convolutional neural network, and the method comprises the steps: obtaining the adjustment cosine similarity of each filter ofeach convolution layer between two periods, and carrying out the soft pruning of the filters according to the adjustment cosine similarity; reconstructing the convolutional neural network after the soft pruning updating; repeating the processes of soft pruning and reconstruction until the convolutional neural network with stable precision is obtained; obtaining the adjustment cosine similarity ofeach filter of each convolution layer of the convolution neural network after precision stabilization between two periods, and carrying out hard pruning on the filters according to the adjustment cosine similarity; and carrying out fine tuning on the convolutional neural network after hard pruning updating until the network precision of the convolutional neural network reaches a stable value. According to the invention, the technical problem that the precision loss of the pruned convolutional neural network is serious due to the fact that the dynamic changes of the distance and the directionof the filter in the convolutional neural network training process are not considered can be solved.

Description

technical field [0001] The invention belongs to the field of computer technology, and more specifically relates to a heuristic filter pruning method and system in a convolutional neural network. Background technique [0002] Convolutional neural networks are widely used in the field of computer vision, but due to huge computational costs and storage overhead, they cannot be deployed on resource-constrained devices (such as mobile devices). In view of this, the pruning method is currently widely used to reduce the network complexity of the convolutional neural network and achieve the purpose of compressing the network. [0003] Nowadays, the pruning process of filters in convolutional neural networks usually uses a heuristic algorithm, that is, by judging the importance of each filter in each convolutional layer, removing unimportant filters in convolutional neural networks, which will Brings many advantages: (1) The pruned network model has no difference in network structur...

AI Technical Summary

Problems solved by technology

[0005] In view of the above defects or improvement needs of the prior art, the present invention provides a heuristic filter pruning method and system in a convolutional neural network. The technical problem of the serious loss of accuracy of the pruned convolutional neural network caused by the dynamic change of the pruning and direction

Images