Structured network model compression acceleration method based on multistage pruning

Abstract

The invention discloses a structured network model compression acceleration method based on multistage pruning, and belongs to the technical field of model compression acceleration. The method comprises the following steps: obtaining a pre-training model, and training to obtain an initial complete network model; measuring the sensitivity of the convolution layers, and obtaining a sensitivity-pruning rate curve of each convolution layer through controlling variables; carrying out single-layer pruning from low to high according to a sensitivity sequence, and finely tuning and re-training a network model; selecting a sample as a verification set, and measuring the information entropy of the filter output feature map; performing iterative flexible pruning according to the size sequence of theoutput entropy, and finely tuning and re-training the network model; and hard pruning: carrying out retraining on the network model to recover the network performance, and obtaining and storing a lightweight model. According to the method, the large-scale convolutional neural network can be compressed on the premise of maintaining the original network performance, the local memory occupation of the network can be reduced, the floating point operation and the video memory occupation during operation are reduced, and the lightweight of the network is realized.

Description

technical field [0001] The invention relates to the technical field of model compression acceleration, in particular to a multi-level pruning-based structured network model compression acceleration method. Background technique [0002] Deep convolutional neural networks are widely used in related fields such as computer vision and natural language processing, and have achieved great success. As people pay more and more attention to convolutional neural networks, more and more layers and more structured Complex networks have sprung up like mushrooms after rain, applied to more and more research fields, and put forward higher requirements for the development of hardware devices. [0003] With the rapid development of deep learning, the improvement of hardware conditions is not so rapid. The development of convolutional neural networks relies on the improvement of computing power and storage space of today's computer equipment, especially the improvement of parallel computing c...

AI Technical Summary

Problems solved by technology

Model pruning has been widely studied as an efficient and universal model compression method, but the compression effect achieved by existing pruning methods is very limited, and many parameter-level pruning algorithms cannot obtain actual compressed storage and Reduce operations, many filter-level pruning algorithms are often difficult to balance parameter reduction and actual network acceleration

Images