CNN model compression method based on activation-entropy weight pruning

Abstract

The invention discloses a CNN model compression method based on activation-entropy weight pruning. Evaluation of importance based on activation-entropy weight and layer-oriented iterative pruning arecarried out. At the stage of evaluation of the importance based on the activation-entropy weight, a weight evaluation method based on the activation-entropy is adopted, neuron activation values and information entropies are combined to calculate weight importance scores to measure the weight importance. In the layer-oriented iterative pruning stage, layer-by-layer pruning is adopted, according toweight importance scores, by a screening-clipping-fine tuning step, iterative pruning is performed on each layer of neurons, and the volume of the CNN model is effectively compressed under the condition that the model accuracy is ensured. The pruning method based on the activation-entropy weight is adopted for screening model parameters with low importance, so that the size of a CNN model is reduced. And meanwhile, the pruned model is finely adjusted to ensure the calculation precision of the model.

Description

technical field [0001] The invention belongs to the field of CNN neural network model compression, in particular to a CNN model compression method based on activation-entropy weight pruning. Background technique [0002] With the advent of the era of big data and artificial intelligence, deep learning has become a research hotspot in academia and industry. As an important branch in the field of deep learning research, CNN has been widely used in image recognition, target detection, natural language processing, speech recognition, and bioinformatics with its powerful feature learning and classification capabilities. In the field of mobile cloud computing, CNN is getting more and more attention. At the same time, with the continuous enhancement of computing power, the number of CNN model layers is also increasing, which makes most CNN models have a huge amount of parameters. Taking several common CNN models as examples, the LeNet-5 model includes nearly 60,000 parameters, th...

AI Technical Summary

Problems solved by technology

The present invention aims at the shortcomings of existing pruning methods that have poor effect in model compression, and proposes a CNN model compression method based on activation-entropy weight pruning

Images