CNN model compression method based on activation-entropy weight pruning

A compression method and model technology, applied in neural learning methods, biological neural network models, neural architectures, etc., can solve problems such as poor results, achieve the effect of reducing volume and ensuring calculation accuracy

Inactive Publication Date: 2019-08-06
HOHAI UNIV
View PDF0 Cites 12 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The present invention aims at the shortcomings of existing pruning methods that have poor effect in mo

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • CNN model compression method based on activation-entropy weight pruning
  • CNN model compression method based on activation-entropy weight pruning
  • CNN model compression method based on activation-entropy weight pruning

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0051] The present invention will be further explained below in conjunction with the drawings.

[0052] In the CNN model compression method based on activation-entropy weight pruning in the present invention, since the CNN model parameters are mainly concentrated in the fully connected layer, the method pruning is mainly used in the fully connected layer. In the pruning process, each layer is pruned separately, and the activation-entropy criterion is used to judge the importance of each weight. Each layer of pruning process carries out multiple iterations. Retrain the model after each round of pruning to compensate for the loss of accuracy. When all the specified layers are pruned, the compressed CNN model is obtained.

[0053] ① Judgment of weight importance based on activation-entropy:

[0054] Aiming at activation-based and importance-based pruning methods, a CNN model pruning method based on activation-entropy weight is proposed.

[0055] Activation is used as the input of a ne...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a CNN model compression method based on activation-entropy weight pruning. Evaluation of importance based on activation-entropy weight and layer-oriented iterative pruning arecarried out. At the stage of evaluation of the importance based on the activation-entropy weight, a weight evaluation method based on the activation-entropy is adopted, neuron activation values and information entropies are combined to calculate weight importance scores to measure the weight importance. In the layer-oriented iterative pruning stage, layer-by-layer pruning is adopted, according toweight importance scores, by a screening-clipping-fine tuning step, iterative pruning is performed on each layer of neurons, and the volume of the CNN model is effectively compressed under the condition that the model accuracy is ensured. The pruning method based on the activation-entropy weight is adopted for screening model parameters with low importance, so that the size of a CNN model is reduced. And meanwhile, the pruned model is finely adjusted to ensure the calculation precision of the model.

Description

technical field [0001] The invention belongs to the field of CNN neural network model compression, in particular to a CNN model compression method based on activation-entropy weight pruning. Background technique [0002] With the advent of the era of big data and artificial intelligence, deep learning has become a research hotspot in academia and industry. As an important branch in the field of deep learning research, CNN has been widely used in image recognition, target detection, natural language processing, speech recognition, and bioinformatics with its powerful feature learning and classification capabilities. In the field of mobile cloud computing, CNN is getting more and more attention. At the same time, with the continuous enhancement of computing power, the number of CNN model layers is also increasing, which makes most CNN models have a huge amount of parameters. Taking several common CNN models as examples, the LeNet-5 model includes nearly 60,000 parameters, th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/08G06N3/04G06N3/06
CPCG06N3/082G06N3/061G06N3/044G06N3/045
Inventor 毛莺池王庆永王绎超戚荣志黄倩平萍
Owner HOHAI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap