Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Processing method and device for language model, text generation method and device and medium

A language model and processing method technology, applied in the computer field, can solve the problems of huge calculation amount of language model, difficulty in deployment and execution, and high delay in language model execution

Pending Publication Date: 2021-05-18
BEIJING YOUZHUJU NETWORK TECH CO LTD
View PDF5 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] In the natural language generation task, the language model is usually used for text prediction and generation. The calculation amount of the language model is usually relatively large, which brings difficulties to the actual deployment and execution.
[0003] In related technologies, the calculation graph is usually used. However, in the above process, a large number of GPU (Graphics Processing Unit, graphics processing unit) operators need to be used, resulting in additional overhead such as operator scheduling and video memory transmission. After the deployment of the language model , the execution delay is high and the availability is low when text processing based on this language model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Processing method and device for language model, text generation method and device and medium
  • Processing method and device for language model, text generation method and device and medium
  • Processing method and device for language model, text generation method and device and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein; A more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the protection scope of the present disclosure.

[0038] It should be understood that the various steps described in the method implementations of the present disclosure may be executed in different orders, and / or executed in parallel. Additionally, method embodiments may include additional steps and / or omit performing illustrated steps. The scope of the present disclosure is not limited in this regard. ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention relates to a processing method and device for a language model, a text generation method and device and a medium. The language model is deployed in the electronic equipment, and a plurality of calculation operations between calculation of a target type in calculation of the same feature layer of the language model are combined into one fusion calculation operation, and the processing method for the language model comprises the following steps: when it is determined that the fusion calculation operation is about to be executed, the CPU of the electronic equipment sends an operation instruction containing the plurality of calculation operations to a GPU (Graphics Processing Unit); in response to receiving the operation instruction, the GPU processes the plurality of computing operations. Therefore, the scheduling overhead between the CPU and the GPU and the repeated read-write overhead of the GPU on the video memory in the processing process of the language model can be effectively reduced, so that the calculation efficiency of the GPU can be effectively improved, the calculation efficiency of the language model is further improved, and the delay of text processing based on the language model is effectively reduced.

Description

technical field [0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular, relate to a processing method for a language model, a text generation method, a device, and a medium. Background technique [0002] In the natural language generation task, the language model is usually used for text prediction and generation, and the calculation amount of the language model is usually relatively large, which brings difficulties to the actual deployment and execution. [0003] In related technologies, the calculation graph is usually used. However, in the above process, a large number of GPU (Graphics Processing Unit, graphics processing unit) operators need to be used, resulting in additional overhead such as operator scheduling and video memory transmission. After the deployment of the language model , the execution delay is high and the availability is low when text processing is performed based on this language model. Contents of ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F40/216G06F40/279
CPCG06F40/216G06F40/279
Inventor 熊鹰王晓晖陈家泽李磊
Owner BEIJING YOUZHUJU NETWORK TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com