Quantization method, device and equipment for model reasoning intermediate data and medium

Abstract

The invention discloses a quantification method, device and equipment for model reasoning intermediate data and a medium, which are applied to a text input sub-block of any layer of reasoning module of a model, and comprise the following steps: inputting all to-be-quantized statements for performing block processing on an input text into a text input sub-block of a current layer, determining the sub-network with the maximum probability of each to-be-quantized statement, quantizing the corresponding to-be-quantized statement to obtain a quantized result, reasoning all the quantized results by using the remaining sub-blocks of the current layer to obtain a reasoning result of the current layer, and taking the reasoning result of the current layer as an input text, and taking the inference module of the next layer as the inference module of the current layer, and returning to execute word segmentation processing on the input text until the inference module of the last layer outputs an inference result. The method can be applied to a medical data processing scene, the calculation overhead and video memory occupation of the model in the medical data processing process are reduced, the quantification efficiency is improved, and then the overall reasoning efficiency of the model is improved.

Images