A neural network structured pruning compression optimization method for a convolutional layer

Abstract

The invention discloses a neural network structured pruning compression optimization method for convolutional layers, and the method comprises the steps: (1), carrying out the sparse value distribution of each convolutional layer: (1.1) training an original model, obtaining the weight parameter of each convolutional layer capable of being pruned, and carrying out the calculation, and obtaining theimportance score of each convolutional layer; (1.2) according to the sequence of importance scores from small to large, carrying out average scale segmentation by referring to the maximum value and the minimum value, carrying out sparse value configuration from small to large on the convolution layers of all the sections in sequence, and through model training adjustment, obtaining sparse valueconfiguration of all the convolution layers capable of being pruned; (2) structured pruning: selecting a convolution filter according to the sparse value determined in the step (1.2), and carrying outstructured pruning training; Wherein only one convolution filter is used for each convolution layer. According to the optimization method provided by the invention, the deep neural network can be more conveniently operated on a resource-limited platform, so that the parameter storage space can be saved, and the model operation can be accelerated.

Description

Technical field [0001] The invention belongs to the field of computer artificial intelligence, deep neural network optimization technology and picture recognition technology, and particularly relates to a neural network structured pruning compression optimization method for convolutional layers. Background technique [0002] In the field of artificial intelligence, deep neural networks are one of the cornerstones, and its complexity and portability directly affect the application of artificial intelligence in life. Research on the acceleration and compression optimization of deep networks can make artificial intelligence more convenient to realize and more convenient to serve life. [0003] At present, common deep network acceleration and compression methods are as follows: 1. Low-Rank: low-rank decomposition; 2. Pruning: pruning, pruning methods are divided into: structured pruning, nuclear pruning, gradient Pruning, a wide range of applications; 3.Quantization: quantization, qua...

AI Technical Summary

Problems solved by technology

[0004] The present invention is mainly aimed at further improving the pruning of the second compression method. The idea of ​​structured pruning is also used in the prior art scheme. Its method is to use multiple convolution filters for each convolutional layer, and these The type of convolutional filter is obtained through training; the existing method not only has a long training cycle and consumes huge computing resources (resulting in the inability to use large training data sets smoothly), but also such structured pruning cannot be performed in the forward direction of the model. The calculation process saves more computing and storage resources

Images