Deep binary neural network training method and system

Abstract

The invention relates to a deep binary neural network training method and system. The deep binary neural network training method comprises the steps: initializing a floating point type deep neural network, and obtaining an initialized network model; based on an alternating direction multiplier method, according to the initialized network model, adopting a target propagation algorithm to obtain anoptimized deep neural network with binary activation and floating point type weight; and based on an alternating direction multiplier method, obtaining a deep binary neural network according to the optimized deep neural network. According to the deep binary neural network training method, the framework is optimized through the alternating direction multiplier method; weight and activation are binarized respectively; the coupling effect caused by simultaneous binarization can be reduced; the training effect of the deep binary neural network is improved; and the target propagation algorithm is adopted to optimize the deep neural network with binary activation, so that the problem of difficulty in quantitative optimization of the deep neural network due to infinitesimal quantization process can be reduced.

Description

technical field [0001] The invention relates to the technical field of deep learning, in particular to a deep binary neural network training method and system. Background technique [0002] Deep neural networks have achieved great success in the fields of computer vision, speech recognition, etc., and the application of deep neural networks is becoming more and more extensive. The structure of deep neural networks has been greatly developed in recent years, and the accuracy rate in performing many tasks has surpassed that of humans. However, it is accompanied by a huge amount of parameters and calculations, which greatly limit the application of deep neural networks, such as the application deployment of mobile devices. [0003] In recent years, the compression and acceleration of deep neural networks has become a research hotspot in academia and industry, and a large number of methods have emerged, such as model quantization, network pruning, low-rank decomposition, knowle...

AI Technical Summary

Problems solved by technology

However, the accuracy loss of deep binary neural network is large, and the training method needs further research

[0005] Simultaneously binarizing weights and activations will intensify the coupling effect and increase the difficulty of training deep binary neural networks.

There are existing methods to decompose the quantization of weight and activation into two stages, which can reduce the coupling effect caused by simultaneous quantization, but it is not suitable for deep binary neural networks.

Images