Pre-training model reasoning processing method and device, electronic equipment and storage medium

Abstract

The invention provides a pre-training model reasoning processing method and device, electronic equipment and a storage medium, the method is applied to a server for carrying out reasoning processing on a to-be-processed model, the method comprises the steps that the to-be-processed model is determined, the to-be-processed model is expressed by adopting a high-bit floating-point number and is obtained through pre-training, and the to-be-processed model is used for carrying out reasoning processing on the to-be-processed model; the digit of the high-bit floating-point number is greater than or equal to a first digit threshold; based on a model quantification technology, model parameters of the to-be-processed model are converted from high-bit floating-point number representation to low-ratio specific point number representation so as to achieve accelerated reasoning processing on the to-be-processed model, and the digit of the low-ratio specific point number is smaller than or equal to a second digit threshold value. By means of the pre-training model reasoning processing method, low cost and high processing speed of the large-scale to-be-processed model in the reasoning process are achieved.

Description

technical field [0001] The present invention relates to the field of model processing, in particular to a pre-training model inference processing method, device, electronic device and storage medium. Background technique [0002] In recent years, large-scale pre-trained models have become a research hotspot. For example, large-scale pre-trained language models have become a research hotspot in the field of natural language processing. The related technologies of pre-training language models make it possible to train large-scale models including tens of billions or even hundreds of billions of parameters (such as Open AI GPT3, Zhiyuan Enlightenment 2.0 model, etc.). These large-scale models have achieved amazing results in many natural language processing tasks and have attracted the continuous attention of many researchers. [0003] Although large-scale pre-trained language models perform amazingly on multiple tasks, their large parameter size also brings great challenges t...

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a pre-training model reasoning processing method, device, electronic equipment and storage medium, which are used to solve the high-cost defects existing in the reasoning process of large-scale pre-training models in the prior art, and realize large-scale pre-training models Low cost and high processing speed during inference

Images