Fixed-point neural network model quantification device and method

Abstract

The invention relates to a fixed-point neural network model quantification method and device. The method comprises the following steps: a verification stage: a verification graph model is a directed acyclic graph, and a multi-input graph model is converted into a single-input model; a preparation stage: the graph model is converted equivalently to facilitate subsequent quantification; a scaling stage: all samples are input and executed in the floating point model, statistics is conducted on output of all operators in the model, and possible output domain values of the operators in all the samples are predicted according to characteristics of output data; and a quantization stage: fixed-point conversion is performed on the operator according to topological sorting by the model. According tothe method, the storage and calculation overhead of the model can be effectively reduced, the uncertainty caused by rounding errors in floating point operation is eliminated, and the high efficiency,transparency and safety of the deep neural network model are improved.

Description

technical field [0001] The present application relates to a fixed-point neural network model quantification device and method, which are suitable for the technical field of artificial intelligence. Background technique [0002] Deep neural network models are widely used in machine vision tasks such as image classification and object detection, and have achieved great success. However, due to the limitations of storage space and power consumption, the storage and computation of neural network models on embedded chips and specially designed neural network chips is still a great challenge. At the same time, the design of existing neural network models usually only considers the accuracy rate, but does not consider the reproducibility and consistency of the operation, and the operation results may be inconsistent under different architectures or even the same computing environment. Affected by this, the application of neural network algorithms in fields with higher security req...

AI Technical Summary

Problems solved by technology

However, on the one hand, the existing model quantization technology is not mature enough. While the performance of the model is improved, the accuracy of the model cannot be effectively guaranteed.

On the other hand, most of the existing model quantization devices only accelerate some specific operators (such as convolution operators, matrix multiplication operators), but there are still a large number of floating-point intermediate values ​​in the calculation process, which This kind of semi-integer quantization still cannot completely avoid the rounding error in the model operation

Images