Flexible deep learning network model compression method based on channel gradient pruning

Abstract

The invention discloses a flexible deep learning network model compression method based on channel gradient pruning, and the method comprises the steps: 1, adding a masking layer constraint to an original network, and obtaining a to-be-pruned deep convolutional neural network model; wherein the absolute value of the product of the channel gradient and the weight serves as an importance standard toupdate the masking layer constraint of the channel to obtain a mask and a sparse model, 3, carrying out pruning operation on the sparse model based on the mask, and 4, retraining a compact deep convolutional neural network model. The invention further provides an application effect of the flexible deep learning network model compression method based on channel gradient pruning on an actual objectrecognition APP, the recognition speed of the model to the object after pruning is greatly improved, and the problem that the deep neural network model cannot be applied to the actual object recognition APP due to high storage space occupation and high memory occupation, high computing resources are occupied and cannot be deployed to embedded devices, smart phones and other devices are solved, and the application range of the deep neural network is expanded.

Description

technical field [0001] The invention relates to a flexible deep learning network model compression method, in particular to the practical application of realizing object recognition on smart phone APP. Background technique [0002] Deep learning has been applied in multiple tasks such as image recognition, target detection, image segmentation, speech recognition, QA question answering, etc., and has achieved better results than traditional methods. However, the excellent performance of deep neural network is closely related to its complex structure and its own huge amount of parameters. And with the advancement of technology, the field of deep learning has a development trend that the structure of neural network models tends to be more complex. Therefore, high-performance deep neural network models have very high requirements for hard disk storage space, memory bandwidth and platform computing resources. The contradiction between the high resource requirements of the deep ...

AI Technical Summary

Problems solved by technology

However, after this method transforms the structure of the convolutional layer + BN layer into a fully connected layer structure, the introduction of hyperparameters to control the compression rate cannot directly reflect the actual compression rate, that is, it is impossible to predict the final number of floating-point operations per second before the pruning ends. (Floating-point Operations Per second, FLOPs) compression ratio

In addition, the pruned channel no longer extracts features, which means that the pruned model will inevitably experience a certain degree of performance degradation

Images