Convolutional neural network compression method, system and device based on decomposition and pruning

Abstract

The invention belongs to the field of artificial intelligence, particularly relates to a convolutional neural network compression method, system and device based on decomposition and pruning, and aimsto solve the problem of relatively small compression strength caused by adopting low-rank approximate decomposition or structured sparse pruning to perform convolutional neural network compression. The method of the system comprises the following steps: adding a coefficient matrix representation layer behind each convolution layer to be compressed; carrying out sparse processing on the coefficient matrix representation layer through a low-rank approximate decomposition algorithm, and carrying out pruning processing on a filter of the corresponding convolution layer according to the sparse position of the coefficient matrix representation layer; carrying out sparse processing on the decomposed coefficient matrix representation layer by adopting a structured sparse pruning method, and carrying out pruning processing on a filter of the coefficient matrix representation layer according to the sparse position of the coefficient matrix representation layer; and training the convolutional neural network after the sparse pruning processing. According to the method, two methods of low-rank approximate decomposition and structured sparse pruning are fused, so that the defects caused by a single method are overcome, and the compression strength is improved.

Description

technical field [0001] The invention belongs to the field of artificial intelligence, and in particular relates to a convolutional neural network compression method, system and device based on decomposition and pruning. Background technique [0002] With the absolute advantages of deep learning in the field of artificial intelligence, such as computer vision, speech recognition and natural language processing, artificial intelligence research has ushered in a new round of climax. In recent years, deep learning algorithms have been gradually applied to the industry. However, due to the large parameters of deep learning models and long calculation time, the application of deep learning in mobile devices and embedded devices is limited. The neural network model compression and acceleration method can reduce model storage, reduce computing energy consumption, reduce real-time memory consumption, and shorten inference delay time. [0003] Model compression can be roughly divided...

AI Technical Summary

Problems solved by technology

[0006] In order to solve the above-mentioned problems in the prior art, that is, to solve the problem that the convolutional neural network is compressed by using the low-rank approximate decomposition method or the structured sparse pruning method, resulting in a relatively small compression force, the first aspect of the present invention proposes a A convolutional neural network compression method based on decomposition and pruning, the method includes:

Images