Model quantification method and device and terminal equipment

Abstract

The invention is suitable for the technical field of model quantification, and provides a model quantification method and device, terminal equipment and a storage medium, and the model quantification method comprises the steps: processing input data through a floating point model, and obtaining target output; for each to-be-quantized layer, performing quantization processing on the corresponding to-be-quantized layer according to the quantization function of the to-be-quantized node in the corresponding to-be-quantized layer to obtain a quantization layer corresponding to the corresponding to-be-quantized layer; processing the first input of the corresponding to-be-quantized layer through the quantization layer corresponding to the corresponding to-be-quantized layer to obtain a second output; according to the second output and the first output, making a quantization function corresponding to the corresponding to-be-quantized layer optimized, and obtaining a target quantization function corresponding to the corresponding to-be-quantized layer; and quantizing the floating point model according to the target quantization function corresponding to each to-be-quantized layer to obtain a target quantization model. Through the method, the precision of the quantization model can be improved.

Description

technical field [0001] The present application belongs to the technical field of model quantization, and in particular relates to a model quantization method, device, terminal equipment, and computer-readable storage medium. Background technique [0002] Artificial intelligence technology has developed rapidly in recent years and has continuously penetrated into various application fields represented by computer vision, natural language processing, and speech recognition. However, in practical application scenarios, the huge amount of data and computational complexity of the deep learning model pose a huge challenge to the computing power of the hardware. Therefore, quantization methods for deep learning models have also been derived. Quantization technology can reduce the memory usage of neural network models, improve data throughput, and reduce inference latency. However, compared with the deep learning model before quantization, the quantization model usually introduces...

AI Technical Summary

Problems solved by technology

However, compared with the deep learning model before quantization, the quantization model usually introduces a large precision loss and increases the calculation error

Images