Processing method and device for language model, text generation method and device and medium

Abstract

The embodiment of the invention relates to a processing method and device for a language model, a text generation method and device and a medium. The language model is deployed in the electronic equipment, and a plurality of calculation operations between calculation of a target type in calculation of the same feature layer of the language model are combined into one fusion calculation operation, and the processing method for the language model comprises the following steps: when it is determined that the fusion calculation operation is about to be executed, the CPU of the electronic equipment sends an operation instruction containing the plurality of calculation operations to a GPU (Graphics Processing Unit); in response to receiving the operation instruction, the GPU processes the plurality of computing operations. Therefore, the scheduling overhead between the CPU and the GPU and the repeated read-write overhead of the GPU on the video memory in the processing process of the language model can be effectively reduced, so that the calculation efficiency of the GPU can be effectively improved, the calculation efficiency of the language model is further improved, and the delay of text processing based on the language model is effectively reduced.

Description

technical field [0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular, relate to a processing method for a language model, a text generation method, a device, and a medium. Background technique [0002] In the natural language generation task, the language model is usually used for text prediction and generation, and the calculation amount of the language model is usually relatively large, which brings difficulties to the actual deployment and execution. [0003] In related technologies, the calculation graph is usually used. However, in the above process, a large number of GPU (Graphics Processing Unit, graphics processing unit) operators need to be used, resulting in additional overhead such as operator scheduling and video memory transmission. After the deployment of the language model , the execution delay is high and the availability is low when text processing is performed based on this language model. Contents of ...

AI Technical Summary

Problems solved by technology

[0002] In the natural language generation task, the language model is usually used for text prediction and generation. The calculation amount of the language model is usually relatively large, which brings difficulties to the actual deployment and execution.

[0003] In related technologies, the calculation graph is usually used. However, in the above process, a large number of GPU (Graphics Processing Unit, graphics processing unit) operators need to be used, resulting in additional overhead such as operator scheduling and video memory transmission. After the deployment of the language model , the execution delay is high and the availability is low when text processing based on this language model

Images