Compression and acceleration method of structured network model based on multi-level pruning

Abstract

The invention discloses a method for compression and acceleration of a structured network model based on multi-level pruning, and belongs to the technical field of model compression and acceleration. The invention includes the following steps: obtaining a pre-training model, and training to obtain an initial complete network model; measuring the sensitivity of the convolution layer, and obtaining the sensitivity-pruning rate curve of each convolution layer through a control variable; according to the sensitivity order from low to high Perform single-layer pruning, fine-tune and retrain the network model; select samples as the validation set, measure the information entropy of the filter output feature map; perform iterative flexible pruning according to the order of output entropy, fine-tune and retrain the network model; hard pruning, correct The network model is retrained to restore the network performance, and the lightweight model is obtained and saved. The invention can compress the large-scale convolutional neural network on the premise of maintaining the original network performance, can reduce the local memory occupation of the network, reduce the floating-point operation and the display memory occupation during operation, and realize the lightweight of the network.

Description

technical field [0001] The invention relates to the technical field of model compression and acceleration, in particular to a method for compression and acceleration of structured network models based on multi-level pruning. Background technique [0002] Deep convolutional neural networks are widely used in computer vision and natural language processing and other related fields, and have achieved great success. As people pay more and more attention to convolutional neural networks, there are more and more layers and structures. Complex networks have sprung up like mushrooms after a spring rain, and they have been applied to more and more research fields, and have also put forward higher requirements for the development of hardware devices. [0003] With the rapid development of deep learning, the improvement of hardware conditions is not so rapid. The development of convolutional neural networks depends on the improvement of computing power and storage space of today's comp...

AI Technical Summary

Problems solved by technology

Model pruning has been widely studied as an efficient and universal model compression method, but the compression effect achieved by existing pruning methods is very limited, and many parameter-level pruning algorithms cannot obtain actual compressed storage and Reduce operations, many filter-level pruning algorithms are often difficult to balance parameter reduction and actual network acceleration

Images