Convolution kernel pruning model compression method and device

Abstract

The invention discloses a convolution kernel pruning model compression method and device, which are used in the field of convolutional neural networks, and the processing flow of the convolution kernel pruning model compression method comprises the following steps: firstly, for a use scene, training a convolutional neural network by using a common model training method until convergence; calculating two-dimensional image entropies corresponding to all feature maps in the convolution layer to form a convolution kernel importance array; then pre-cutting the convolution kernel, further observing the performance change of the model, and finally determining whether the convolution kernel is determined to be cut; after all convolutional layers in the model are traversed, ending the process; according to the improved convolution kernel pruning method, the convolution layer of a convolutional neural network is pruned, and a two-dimensional image entropy is put forward to be used as a criterion for importance evaluation of a convolution kernel; according to the method, on the basis of ensuring the performance of the convolutional neural network model, compared with traditional convolution kernel pruning, the convolutional layer is cut to a greater extent, and a greater degree of model compression effect is achieved.

Description

technical field [0001] The invention relates to a convolutional neural network model compression technology, in particular to a convolution kernel pruning model compression method and device. Background technique [0002] With the development of neural network, it has achieved excellent results in many aspects such as computer vision, target detection, natural language processing, etc., but the problems that follow are also obvious. The model size, parameter amount and calculation amount of deep neural network are all different Showing geometric growth, and due to problems in storage capacity, computing power and energy supply, etc., the deep neural network model cannot be directly deployed on resource-constrained devices such as mobile terminals and edge terminals, so now for deep neural network models The compression is of practical significance. The pruning method is a commonly used model compression method, including structured pruning methods and unstructured pruning m...

AI Technical Summary

Problems solved by technology

In the traditional convolution kernel pruning process, there are two main problems. The first is the convolution kernel contribution evaluation criterion; currently, the more commonly used evaluation criteria include L1 norm, ApoZ, and methods based on absolute value. Each method has a certain theoretical basis and experimental proof, but there is no theoretically optimal and experimentally verified criterion.

Second: The pruning strength is not enough; in traditional convolution kernel pruning, the first n pruning units with the lowest importance are generally selected for pruning (n is determined by pruning sensitivity), but since there is no Unify the optimal importance evaluation criteria, so it will cause such a phenomenon: the pruning unit to be deleted is determined by pruning sensitivity, generally speaking, it is the least important, but the rest is considered important

In fact, due to the non-optimality of the importance evaluation criteria, this phenomenon is not necessarily correct, and the remaining convolution kernels after pruning may still be unimportant

Images