Stepwise convolution neural network pruning compression method

Abstract

The invention discloses a stepwise convolution neural network pruning compression method, which utilizes GPU to calculate the L1 norm and variance of each layer neuron in a convolution neural networkmodel; the importance of each neuron at the moment is estimated according to the L1 norm and variance of each layer neuron; as for the unimportant neuron at the moment, it is compulsorily set to 0, and then the backpropagation error item in the same location is also set to 0; the convolution neural network is recalculated and evaluated after being trained by m iterations training; all the active neurons after being evaluated enter into the next round of the m iterations training; the above process is repeated n times, so that the learning rate will be attenuated step by step. The unimportant neurons in the convolution neural network are pruned at the moment, and then are judged after iterating again and again. The important neurons in the current moment are restored, and the unimportant neurons at present are pruned. The operation is repeated to achieve the purpose of effective compression and fast calculation of the convolution neural network.

Description

technical field [0001] The invention relates to the technical field of convolutional neural network pruning compression, in particular to a step-by-step convolutional neural network pruning compression method. Background technique [0002] Convolutional Neural Networks (CNN) is an efficient image recognition and classification method deep learning architecture that has been developed rapidly since 2012 and has attracted widespread attention. This architecture effectively improves the ability of image classification and object recognition. Compared with the traditional method of manually extracting features, CNN can directly input the original image, avoiding complicated preprocessing, and has higher recognition accuracy; compared with the traditional Back Propagation (BP) neural network, CNN Due to the use of the sharing strategy, the number of neurons is greatly reduced, thus avoiding the phenomenon of over-fitting to a certain extent. [0003] However, for embedded termin...

AI Technical Summary

Problems solved by technology

[0003] However, for embedded terminal devices, due to their limited computing resources and storage resources, most CNN networks such as VGG19 / VGG16, ResNet, GoogLeNet, etc. can only Deployed on high-end cloud platforms supported by GPU or FPGA

In 2016, ICLR's bestpaper proposed a compression pruning scheme (Deep Compression), but the algorithm's estimation of invalid neurons simply set a threshold for the absolute value of neurons, and did not take into account the training process. Complex changes in neurons, so the compression efficiency is still not satisfactory

Images