Model compression method based on pruning sequence active learning

Abstract

The invention provides a model compression method based on pruning sequence active learning. an end-to-end pruning framework based on sequential active learning is provided; The method has the advantages that the importance of all layers of the network can be actively learned, the pruning priority is generated, a reasonable pruning decision is made, the problem that an existing simple sequential pruning method is unreasonable is solved, pruning is preferentially carried out on the network layer with the minimum influence, pruning is carried out step by step from simplification to difficulty, and the model precision loss in the pruning process is minimized; And meanwhile, the final loss of the model is taken as a guide, and the importance of the convolution kernel is evaluated in a multi-angle, efficient, flexible and rapid manner, so that the compression correctness and effectiveness of the whole-process model are ensured, and technical support is provided for subsequent transplantation of a large model to portable equipment. Experimental results show that the model compression method based on pruning sequence active learning provided by the invention is leading under the conditions of multiple data sets and multiple model structures, can greatly compress the model volume under the condition of ensuring the model precision, and has a very strong practical application prospect.

Description

technical field [0001] The invention belongs to the technical field of neural network models, in particular to a model compression method based on active learning of pruning order. Background technique [0002] In recent years, with the vigorous development of deep neural networks, the academic and industrial circles have witnessed major breakthroughs in deep learning in many fields such as computer vision and natural language processing. The expressive power of convolutional neural network (CNN) in some visual fields even exceeds the visual processing ability of human beings. [0003] Although deep networks have made significant breakthroughs in the field of vision, the size and computation of the models have become the bottlenecks in their practical applications. In-depth network applications in real-world scenarios require fast computing power, a large amount of storage space, and battery capacity attached to hardware. Large-scale neural networks can be efficiently run ...

AI Technical Summary

Problems solved by technology

Existing methods focus on evaluating the importance of convolution kernels, but the pruning strategy is too simple, resulting in unsatisfactory results

However, there is an important phenomenon that has been overlooked, and each convolutional layer has different importance: if a very small number of convolutional kernels are subtracted from an important convolutional layer, the accuracy of the overall model may also be greatly reduced; conversely, If on an unimportant convolutional layer, even if a large number of convolution kernels are cut off, the accuracy will hardly be affected. The method of sequential pruning or global pruning obviously does not take into account the importance of each convolutional layer to the pruning effect. The effect of branching

Images