Convolution neural network compression method and decompression method based on compressed sensing principle

Abstract

The invention discloses a convolution neural network compression method and a decompression method based on a compressed sensing principle. The compression method comprises a preprocessing step in which weights of layers in the convolution neural network are preprocessed to a series of matrix, a compression step in which the preprocessing result obtained in the preprocessing step is compressed toobtain a compressed weight, a training step in which the compressed weight is trained, a coding step in which the compressed weight trained in the training step is coded, and a model generation step in which a compressed convolution neural network model file is generated according to the coding result obtained in the coding step. Compared with other method, the convolution neural network compression method based on the compressed sensing principle has a higher compression ratio than the current popular direct pruning quantization method, and excessive information loss can be prevented throughkeeping low-frequency information in a frequency domain.

Description

technical field [0001] The invention relates to a method for compressing a convolutional neural network model, which belongs to the technical field of deep learning. Background technique [0002] Since the Alexnet convolutional neural network won the first place in the ImageNet image classification competition in 2012, and its accuracy far exceeded the second place, deep learning technology ushered in a boom. In the past 5 years, deep learning technology can be said to have entered all walks of life, including automatic driving, recommendation systems, medical imaging, game AI and other technical fields. Deep learning technology has been used and achieved better performance than before. However, since the calculation of the deep neural network requires high-performance GPU and a large amount of memory, and the current embedded devices cannot provide such high-performance computing power and memory capacity, there is an urgent need for neural network compression technology. ...

AI Technical Summary

Problems solved by technology

However, since the calculation of the deep neural network requires the use of high-performance GPU and a large amount of memory, and the current embedded devices cannot provide such high-performance computing power and memory capacity, the demand for neural network compression technology is urgent.

[0003] Existing convolutional neural network compression techniques, such as the Deep Compression (DeepCompression) algorithm published in ICLR2016, do not consider converting weights to sparse domains in the pruning process before pruning, so often after pruning will lose a lot of information

Images