Convolutional neural network filter pruning technology based on similarity learning

Abstract

The invention provides a convolutional neural network filter pruning technology based on similarity learning. The method comprises the steps of obtaining the similarity between different filters through similarity survey; performing clustering on the filters according to the similarity; enabling the filters in the same cluster to become more and more similar through convergent training; and finally pruning the filters after the convergent training. The technology solves the problem that the model representation capability is reduced and needs to be re-trained in a convolutional neural networkfilter pruning process, so that the representation capability and precision of a network can be better retained; and in addition, the precision loss of the convolutional neural network is hardly caused, the precision of a model does not need to be restored through re-training, and the better balance of model precision and efficiency is achieved.

Description

Technical field [0001] The invention belongs to the technical field of neural networks, and particularly relates to a convolutional neural network filter pruning technology based on similarity learning. Background technique [0002] In recent years, with the explosive growth of the amount of information in the human society and the enormous abundance of computing resources, data-driven deep learning methods have been widely and successfully applied in many fields. In particular, in the fields of computer vision and natural language processing, the success of Convolutional Neural Networks (CNN) is particularly prominent, and it has become an indispensable tool. [0003] However, as CNN becomes deeper and deeper, the number of parameters, energy consumption, the number of floating-point operations (FLOPs) required, and memory usage continue to increase, making it more and more difficult to deploy on platforms with limited computing resources. Such as mobile devices. Therefore, in r...

AI Technical Summary

Problems solved by technology

Some techniques try to define the importance of filters in a certain way, remove some unimportant filters, and then retrain the network to restore its damaged accuracy. The limitation of this is that removing the entire filter is a negative A large damage to the network structure will cause a significant loss of accuracy and require a long period of retraining to restore its accuracy

Some techniques are pruning and retraining layer by layer, or even filter by filter, which is very inefficient on very deep networks

Other techniques try to introduce certain constraints and train the network in order to set some filters to zero, so that the accuracy loss of the network can be greatly reduced when removing this part of the filter; In addition, this method often cannot really set all the parameters of the filter to zero, but only reduces its magnitude to a certain limit, so the accuracy will still be caused when pruning. loss, still requires retraining to restore its accuracy

Images