Compression method and device of convolutional neural network and electronic equipment

Abstract

The embodiment of the invention provides a compression method and device of a convolutional neural network and electronic equipment. For each convolution layer, the method comprises the following steps: deforming a weight tensor of the convolution layer into a first two-dimensional matrix; determining a plurality of first singular value decomposition items of the first two-dimensional matrix; selecting at least one first singular value decomposition item from the plurality of first singular value decomposition items according to a first selection rule; performing approximate decomposition on the first two-dimensional matrix by utilizing each first singular value decomposition item to obtain a first decomposition matrix and a second decomposition matrix; respectively deforming the first decomposition matrix and the second decomposition matrix into a first sub-weight tensor and a second sub-weight tensor as compressed weight tensors; and after obtaining the compressed weight tensor of each convolutional layer of the target convolutional neural network, obtaining the compressed target convolutional neural network. Compared with the prior art, by applying the scheme provided by the embodiment of the invention, the accuracy difference between the convolutional neural networks before and after compression can be reduced.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a convolutional neural network compression method, device and electronic equipment. Background technique [0002] At present, with the continuous development of artificial intelligence technology, convolutional neural networks have made major breakthroughs in tasks such as image recognition, target detection, and image semantic segmentation, and are widely used in the fields of the Internet, security, and smart homes. [0003] With the advent of the mobile Internet era, people's demand for the application of convolutional neural networks on mobile devices to complete tasks such as target recognition, detection, and tracking is becoming more and more urgent. However, due to the limited storage and computing capabilities of mobile devices, and the current excellent convolutional neural network usually has a large amount of parameters and calculations, it is ...

AI Technical Summary

Problems solved by technology

[0005] However, in the above-mentioned related technologies, the connection mode of neurons and the shape of the convolution kernel in the network structure of the convolutional neural network will change before and after compression, resulting in a large difference, which is not conducive to the application process of the compressed convolutional neural network. In the feature learning, the accuracy of the compressed convolutional neural network is greatly reduced compared with that of the uncompressed convolutional neural network.

Images