Neural network quantification method

Abstract

The invention discloses a neural network quantization method, belongs to the technical field of deep learning training algorithms, and particularly comprises a forward propagation stage of neural network training and a back propagation stage of neural network training, and the above processes are repeated until a neural network converges to complete deep neural network quantization. Compared witha non-quantized network, the precision loss of the quantized deep neural network can be ignored, the storage capacity and the operation complexity of the model can be reduced, and the quantized deep neural network can be conveniently transplanted to embedded hardware equipment to play the calculation advantage of the quantized neural network on hardware. According to the neural network quantized by the quantization method, a high-precision quantization value can be expressed by using low-bit coding; according to the method, the model memory space and the operation complexity can be effectivelyreduced, the high precision of the network cannot be reduced, the method is more suitable for a hardware shift calculation mode, and the method can be deployed on corresponding hardware to fully exert the calculation advantages of the quantitative neural network.

Description

technical field [0001] The present invention relates to the technical field of deep learning training algorithms, and more specifically, relates to a neural network quantization method. Background technique [0002] Artificial neural network, also referred to as neural network or connection model for short, is an algorithmic mathematical model that imitates the behavior characteristics of animal neural networks and performs distributed parallel information processing. This kind of network depends on the complexity of the system, and achieves the purpose of processing information by adjusting the interconnection relationship between a large number of internal nodes. Artificial neural network is a mathematical model that uses a structure similar to that of the brain's synaptic connections for information processing. In engineering and academia, it is often referred to directly as "neural network" or neural network. [0003] The existing method of quantizing the neural networ...

AI Technical Summary

Problems solved by technology

Since the weight values ​​of the neural network are mostly floating-point numbers, there will be a large error with the actual full-precision weight values ​​in the network when quantized with a value of ±1. Although the neural network has certain robustness, it will also be different from the full-precision network. A large error is generated, resulting in a loss of the accuracy of the quantized neural network relative to the unquantized neural network