Global rank perception neural network model compression method based on filter feature map

Abstract

The invention discloses a global rank perception neural network model compression method based on a filter feature map, and solves the problems of high labor cost, low pruning efficiency and poor stability of a filter pruning model compression method. The method comprises the steps of obtaining and preprocessing image data; constructing a universal convolutional neural network, and setting hyper-parameters; selecting a loss function and an optimization algorithm; training and storing the pre-training model; acquiring an average rank of the feature map; adaptively learning a parameter matrix; representing and sequencing the importance of the filters; pruning and storing the pre-training model; and finely adjusting the network model to realize the global rank perception neural network model compression based on the filter feature map. According to the method, one-time learning and multi-time pruning are adopted, the filters are globally sequenced, pruning is unified, and ill-conditioned selection does not exist. The pruning effect and stability are better, the pruning efficiency of large-scale different-complexity networks is high, and the adaptability of different edge devices is good. The method is used for computing and storing resource-limited edge equipment.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence, and mainly relates to neural network model compression, in particular to a global rank-aware neural network model compression method based on filter feature maps. It can be used for neural network deployment on edge devices with limited computing and storage resources. Background technique [0002] With the continuous development of artificial intelligence in the era of big data, the performance of deep convolutional neural networks is getting stronger and stronger, and has achieved great success in computer vision applications such as image classification, object detection, anomaly detection, semantic segmentation, and instance segmentation. However, along with it, the depth of the network model is getting deeper and larger, and the computing requirements are getting higher and higher. The deployment of most advanced convolutional neural networks in limited application environme...

AI Technical Summary

Problems solved by technology

Based on the assumption that each layer of the convolutional neural network is equally important, the weight of the unimportant filter can be minimized through the regularization operation, that is, the unified threshold can be applied to each convolutional layer of the network to achieve pruning, but the regularization operation exists The ill-conditioned problem of filter selection

He Y et al. in the document "He Y, Lin J, Liu Z, et al.Amc: Automl for model compression and acceleration on mobile devices. In: Proceedings of the European Conference on Computer Vision (ECCV), 2018.784-800." The proposed automatic model compression pruning model compression method AMC (Automated Model Compression), the filter pruning model compression method uses reinforcement learning to search for a convolutional neural network that meets the complexity specified by the user, but the learning time cost of the entire neural network is relatively high. Big

[0006] To sum up, most of the existing filter pruning model compression methods based on predefined structure or adaptive structure represent and sort the importance of local filters in the neural network model, but do not The importance of the filter is evaluated globally and uniformly. At the same time, it also faces the problems of low training efficiency and high labor cost when pruning general-purpose neural networks, and cannot achieve satisfactory results.

Images