A compression method of a deep convolution neural network based on a small amount of unlabeled data

Abstract

The invention belongs to the technical field of deep neural network, in particular to a compression method of a depth convolution neural network based on a small amount of unlabeled data. The purposeof the method of the present invention is to solve the problem of performance loss when all layers of the deep convolution neural network are compressed based on the existing full-precision network model without retraining based on the original annotated training data, and the method includes: acquiring the original deep convolution neural network; thinning the weight tensors of each layer in theoriginal depth convolution neural network to obtain a plurality of weight tensors with more 0 elements; based on the compressed weight tensor, obtaining a new deep convolution neural network by updating the statistics in the normalized layer of the batch data in the compressed deep convolution neural network with a small amount of unlabeled data. Through the embodiment of the invention, the compression of the large-scale depth convolution neural network relying on only the small amount of unlabeled data is realized, and the performance loss is reduced.

Description

technical field [0001] The invention belongs to the technical field of deep neural networks, and in particular relates to a compression method of a deep convolutional neural network based on a small amount of unlabeled data. Background technique [0002] In recent years, convolutional neural networks have made great progress in target detection and recognition tasks, and their detection accuracy has reached the commercial level. At the same time, the rapid development of mobile terminals and smart devices has enabled researchers to see an opportunity to combine convolutional neural networks with portable devices. However, object recognition based on convolutional neural networks relies on large memory consumption and strong computing performance. They often rely on high-performance GPU devices, which are difficult to work on such as smartphones and embedded devices. Running a convolutional neural network model will quickly use up limited memory resources, hard disk storage...

AI Technical Summary

Problems solved by technology

[0004] In order to solve the above problems in the prior art, that is, to solve the performance loss when compressing all layers of the deep convolutional neural network on the basis of the existing full-precision network model, and not performing retraining based on the original labeled training data The larger problem, one aspect of the present invention, provides a method for compressing a deep convolutional neural network based on a small amount of unlabeled data, comprising the following steps:

Images