Non-zero filling training method for binary convolutional neural network

Abstract

The invention discloses a non-zero filling training method for a binary convolutional neural network, and the method comprises the steps: firstly improving a loss function in the training of a commonneural network, constructing a joint loss function through employing a knowledge distillation theory, and achieving the guide training of a zero filling binary network for the non-zero filling binarynetwork. And then, a progressive training method is adopted to train the non-zero filling binary network, and on the basis of the zero filling binary network, the number of activated non-zero fillingbinary values is gradually increased, so that the training difficulty of the non-zero filling binary network is reduced. According to the method, the pseudo binary activation problem in the zero-filling binary network is corrected, the training difficulty of the non-zero-filling binary network is effectively reduced by combining the joint loss function and the progressive training method, and theproblem that the performance of the non-zero-filling binary network is reduced after the filling value is corrected is greatly reduced.

Description

technical field [0001] The invention belongs to the field of image processing, and specifically refers to a non-zero padding training method for a binary convolutional neural network. Background technique [0002] In the past ten years, due to its great advantages over shallow models in feature extraction and model construction, deep learning has attracted more and more researchers' attention, and has achieved rapid development in fields such as computer vision and text recognition. Deep learning takes deep neural network as its main presentation form, and convolutional neural network (CNN) is a pioneering research result inspired by biological neuroscience. Compared with the traditional method, the convolutional neural network has the characteristics of weight sharing, local connection, pooling operation, etc., so it can effectively reduce the global optimization training parameters, reduce the complexity of the model, and make the network model more effective for input sca...

AI Technical Summary

Problems solved by technology

Since the current binary network research field has not paid attention to this problem, in fact, various results reported in various studies are based on the {-1, +1, 0} mixed activation "pseudo-binary network ", cannot enjoy the extremely high compression ratio of the binary network in practical applications

It has been verified by experiments that when the padding is 1, it is a non-zero-filled binary network whose activation is {-1, +1} after quantization. Compared with the zero-filled binary network with the same structure, its performance is very obvious. Decline

How to make up for the performance degradation of the non-zero padding binary network brought about by correcting the padding value, so that it can run with high precision and high compression ratio in actual deployment in the future, has become a problem that needs to be further studied.

Images