Unlock instant, AI-driven research and patent intelligence for your innovation.

Coarse and fine granularity combined neural network pruning method

A neural network, coarse and fine-grained technology, applied in neural learning methods, biological neural network models, neural architectures, etc., to achieve the effects of high accuracy, recovery accuracy, and high hardware friendliness

Pending Publication Date: 2021-12-28
BEIHANG UNIV
View PDF0 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, most of the current methods combining structured and unstructured pruning stack different pruning schemes in series, lacking the analysis of the pruning sequence and the interaction of different pruning modes

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Coarse and fine granularity combined neural network pruning method
  • Coarse and fine granularity combined neural network pruning method
  • Coarse and fine granularity combined neural network pruning method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0049] Such as figure 1 As shown, the embodiment of the present invention discloses a coarse-fine-grained combined neural network pruning method, including the following steps:

[0050] S1, filter pruning:

[0051] Select the candidate filters that need to be pruned in the original model according to the two-norm and layer by layer, and perform group sparse training on the candidate filters, and prune the candidate filters that are smaller than the threshold ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a coarse and fine granularity combined neural network pruning method, which comprises the following steps of: performing group sparse training on screened candidate filters, and pruning the candidate filters smaller than a threshold value after a certain number of rounds; sequencing the importance of the convolution kernels layer by layer, and obtaining the convolution kernels needing to be pruned layer by layer according to a predefined pruning rate; taking the weight as a unit, carrying out regularization compression on the convolution kernel, and dynamically generating a mode set meeting a pre-constructed mode discrimination function in the compression process; matching each convolution kernel to the optimal mode of the convolution kernel in the mode set; performing convolution kernel pruning and mode pruning; setting parameters needing to be pruned to be zero, and carrying out hard pruning on the model; and performing retraining and fine adjustment on the model after hard pruning in combination with a knowledge distillation method to obtain a final model after pruning. According to the method, the advantages of structured pruning and unstructured pruning can be brought into full play, the model storage and reasoning efficiency is improved, and higher hardware friendliness is achieved.

Description

technical field [0001] The present invention relates to the technical field of embedded AI, and more specifically relates to a neural network pruning method combining thickness and granularity. Background technique [0002] With the advent of a range of embedded devices, performing inference on deep neural networks is very challenging given the high computational and storage requirements. Pruning is a method widely used in model compression at present. It finds an effective evaluation method to judge the importance of parameters, and cuts unimportant parameters to reduce the redundancy of the model. Existing pruning methods mainly include structured and unstructured pruning. Unstructured pruning has the characteristics of refinement and high precision, but it is not friendly to hardware. Structured pruning has the characteristics of coarse granularity and high hardware efficiency, but the accuracy Big loss. [0003] For unstructured pruning, directly resetting the weights ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/08G06N3/04
CPCG06N3/082G06N3/045Y02D10/00
Inventor 姜宏旭朱雨婷李波张永华东东胡宗琦从容子
Owner BEIHANG UNIV