Precision improving method of offline quantification tool

Abstract

The invention relates to the field of deep learning, in particular to a precision improvement method of an offline quantization tool. The precision improvement method comprises the steps that S1, a processing unit carries out linear processing on each quantization conversion factor; s2, the processing unit obtains first processing data output by all convolution layers; s3, the processing unit obtains second processing data output by the convolution layer; s4, determining a quantization parameter related to the current update according to the first processing data and the second processing data, and updating the corresponding quantization conversion factor by adopting the determined quantization parameter; s5, judging whether a quantization conversion factor which is not updated yet existsin the quantization conversion factor set or not, and if yes, returning to the step S3; and if not, outputting the updated quantization conversion factor set as a precision improvement result, and then ending. The technical scheme has the beneficial effects that the processing precision of the network model is further improved, the processing speed is increased, and the economic cost is reduced.

Description

technical field [0001] The invention relates to the field of deep learning, in particular to a method for improving the accuracy of an offline quantization tool. Background technique [0002] With the continuous development of AI technology, the neural network algorithm based on deep learning has become the mainstream method of AI research. Due to considerations of cost, power consumption, privacy and other issues, more and more application scenarios migrate the calculation of AI algorithms from the cloud to mobile embedded terminal devices. [0003] The current embedded terminal equipment has limited computing power and storage resources. When the neural network algorithm model is deployed on the embedded terminal, it is necessary to adopt the network model compression technology, and the most widely used in the industrial level is to quantize the floating point (Float32) network model into Integer (Int8) network model. Reduce storage requirements and improve network mode...

AI Technical Summary

Problems solved by technology

[0004] Int8 network model quantization is divided into two implementation schemes: one is the perceptual quantization training that requires training framework support, directly outputs the Int8 network model, and the quantized Int8 network model The accuracy is high, but the existing Float32 network model needs to be retrained, which requires a large amount of data sets and long-term training support; the other is based on the existing Float32 network model, through model quantification tools and a small amount of calibration Image offline output Int8 network model, the operation is relatively simple, no retraining is required, but the accuracy is low

Images