Step-by-step training method for nonlinear quantitative deep neural network

Abstract

The invention discloses a step-by-step training method for a nonlinear quantization deep neural network, and the method enables a quantization deep convolutional neural network training process to bedivided into a plurality of steps, achieves the gradual quantification of weights and feature maps, and reduces the quantization network training difficulty. Meanwhile, nonlinear quantization is carried out on network parameters in the training process, quantization errors are reduced, and the network computing performance is improved.

Description

technical field [0001] The invention belongs to the field of compression and acceleration of deep convolutional neural networks, and in particular relates to a step-by-step training method for nonlinear quantization deep neural networks. Background technique [0002] In recent years, deep convolutional neural networks have made breakthroughs in various image intelligent processing such as target detection and classification recognition, and have been widely used in many fields such as autonomous driving, mobile phone image processing, and remote sensing image processing. There are a large number of network weight parameters and calculations in the deep convolutional neural network, so high-performance devices such as CPUs and GPUs are often used as implementation platforms for algorithm deployment. However, the high power consumption overhead of these devices makes it difficult to apply to application scenarios where computing resources and power consumption are strictly lim...

AI Technical Summary

Problems solved by technology

However, most current quantization methods adopt linear quantization, which is very sensitive to outliers in the data and easily causes large quantization errors; in addition, during training, these methods quantize feature maps and weight parameters at the same time, and training is prone to fall into local optimum. optimal solution rather than the global optimal solution

These two problems will cause a certain loss of precision in the quantized network, especially the low bit width quantized network.

Images