Web GPU-based Web large language model reasoning acceleration optimization method and device

Abstract

The present application provides a WebGPU-based Web large language model reasoning acceleration optimization method and device, including: based on a first text input by a target user, generating multiple first reasoning tokens, determining first reasoning operators corresponding to each first reasoning token, generating first computing pipelines corresponding to each first reasoning operator, and when at least one first computing pipeline is obtained, continuously inputting the first computing pipeline into a WebGPU application to obtain first reasoning texts continuously output by the WebGPU application, combining the first reasoning texts based on the order in which the WebGPU application outputs the first reasoning texts to obtain a target reasoning text, enabling the process of generating the computing pipeline and the process of obtaining the first reasoning text to be executed in parallel, which can improve the utilization rate of the Web large language model of WebGPU computing resources to a certain extent, shorten the execution time of the reasoning task, and thus improve the reasoning efficiency of the large language model.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a method and device for accelerating optimization of Web large language model reasoning based on WebGPU. Background Art

[0002] Generative large-scale language model (LLM) refers to an artificial intelligence model in the field of natural language processing (NLP) that infers the subsequent word elements of a text sequence by learning the joint probability distribution of word sequences. Large language models can handle a variety of natural language tasks, such as text classification, question-answering, and dialogue, and are an important path to artificial intelligence. However, large language models are usually deployed in cloud platforms. When users use services based on large language models, they may send private files that need to be kept confidential to the model, and the user's privacy data is at risk of being leaked.

[0003] In the related art, the Web large language model reasoning method can be used to perform model reasoning tasks in a Web browser. Web large language model reasoning refers to deploying a large language model to a browser environment, and all calculations of model reasoning are completed in the Web browser of the user's local device. Web large language model reasoning uses WebGPU (Web Graphics Processing Unit, network graphics processor), a new generation of Web graphics application programming interface (Application Programming Interface, API) to implement graphics processing unit (Graphics Processing Unit, GPU) hardware acceleration. Under this method, user data is no longer uploaded to the cloud platform, thereby protecting user data privacy.

[0004] However, in the above methods, the Web large language model often takes a long time to perform reasoning tasks, and the reasoning efficiency of the Web large language model is poor. Summary of the invention

[0005] In view of the above problems, the embodiments of the present application provide a Web large language model reasoning acceleration optimization method, device, electronic device and readable storage medium based on WebGPU to overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, an embodiment of the present application provides a Web large language model reasoning acceleration optimization method based on WebGPU, the method comprising:

[0007] Based on the first text input by the target user, generate a plurality of first inference tokens;

[0008] Determining first reasoning operators corresponding to each of the first reasoning tokens;

[0009] Generate first computing pipelines corresponding to the first inference operators respectively;

[0010] When at least one first computing pipeline is obtained, continuously inputting the first computing pipeline into the WebGPU application to obtain a first reasoning text continuously output by the WebGPU application;

[0011] Based on the order in which the WebGPU application program outputs the first reasoning texts, the first reasoning texts are combined to obtain a target reasoning text.

[0012] Optionally, generating first computing pipelines corresponding to the first inference operators respectively includes:

[0013] generating a first compute shader corresponding to the first inference operator;

[0014] generating a first resource binding group based on the first compute shader and the first inference operator;

[0015] Based on the first resource binding group, a first computing pipeline is generated.

[0016] Optionally, the first resource binding group is stored in a cache space, and the generating of first computing pipelines corresponding to the first inference operators includes:

[0017] If there is a first resource binding group matching the first inference operator in the cache space, read the first resource binding group from the cache space;

[0018] Based on the first resource binding group, first computing pipelines corresponding to the first inference operators are generated.

[0019] Optionally, determining the first reasoning operators corresponding to the first reasoning tokens respectively includes:

[0020] When at least one first inference word-gram is obtained, first inference operators corresponding to each of the first inference word-grams are continuously determined.

[0021] Optionally, the first inference operator is stored in a cache space, and determining the first inference operator corresponding to each of the first inference tokens includes:

[0022] In a case where the cache space contains the first inference operator corresponding to the first inference word-element, the first inference operator corresponding to the first inference word-element is read from the cache space.

[0023] Optionally, the first computing pipeline is stored in a cache space, and the method further includes:

[0024] In a case where a first computing pipeline matching the first inference operator exists in the cache space, the first computing pipeline matching the first inference operator is read from the cache space.

[0025] Optionally, combining the first reasoning texts based on the order in which the WebGPU application program outputs the first reasoning texts to obtain a target reasoning text includes:

[0026] Counting a first number of the first inference texts;

[0027] When the first number is greater than or equal to a first threshold, the first inference texts are combined based on the order in which the WebGPU application outputs the first inference texts to obtain a target inference text.

[0028] Optionally, the continuously inputting the first computing pipeline into a WebGPU application to obtain a first reasoning text continuously output by the WebGPU application includes:

[0029] Continuously inputting the first computing pipeline into a WebGPU application to obtain a first machine code continuously output by the WebGPU application;

[0030] The first machine code is decoded to obtain a first inference text.

[0031] Optionally, combining the first reasoning texts based on the order in which the WebGPU application program outputs the first reasoning texts to obtain a target reasoning text includes:

[0032] Based on the first inference text, generate a second inference word-gram;

[0033] Determining a second inference operator corresponding to the second inference word;

[0034] generating a second computing pipeline corresponding to the second inference operator;

[0035] When at least one second computing pipeline is obtained, continuously inputting the second computing pipeline into the WebGPU application to obtain a second reasoning text continuously output by the WebGPU application;

[0036] Based on the order in which the first reasoning text and the second reasoning text are output by the WebGPU application, the first reasoning text and the second reasoning text are combined to obtain a target reasoning text.

[0037] In a second aspect, an embodiment of the present application provides a Web large language model reasoning acceleration optimization device based on WebGPU, the device comprising:

[0038] A first generating module, configured to generate a plurality of first inference tokens based on a first text input by a target user;

[0039] A first determination module, used to determine the first reasoning operators corresponding to the first reasoning words;

[0040] A second generating module, used to generate first computing pipelines corresponding to each of the first inference operators;

[0041] A continuous input module, configured to continuously input the first computing pipeline into the WebGPU application program when at least one first computing pipeline is obtained, so as to obtain a first reasoning text continuously output by the WebGPU application program;

[0042] The combination module is used to combine the first reasoning texts based on the order in which the WebGPU application outputs the first reasoning texts to obtain a target reasoning text.

[0043] Optionally, the second generating module includes:

[0044] A first generating submodule, configured to generate a first computing shader corresponding to the first inference operator;

[0045] A second generation submodule, configured to generate a first resource binding group based on the first compute shader and the first inference operator;

[0046] The third generating submodule is used to generate a first computing pipeline based on the first resource binding group.

[0047] Optionally, the first resource binding group is stored in a cache space, and the second generating module includes:

[0048] A first reading submodule, configured to read the first resource binding group from the cache space if a first resource binding group matching the first inference operator exists in the cache space;

[0049] The fourth generating submodule is used to generate first computing pipelines corresponding to each of the first inference operators based on the first resource binding group.

[0050] Optionally, the first determining module includes:

[0051] The first determination submodule is used to continuously determine a first reasoning operator corresponding to at least one first reasoning word-unit when at least one first reasoning word-unit is obtained.

[0052] Optionally, the first inference operator is stored in a cache space, and the first determination module includes:

[0053] A second reading submodule is configured to read the first inference operator corresponding to the first inference word from the cache space when the cache space contains the first inference operator corresponding to the first inference word.

[0054] Optionally, the first computing pipeline is stored in a cache space, and the device further comprises:

[0055] The reading module is used to read the first computing pipeline matching the first inference operator from the cache space when there is a first computing pipeline matching the first inference operator in the cache space.

[0056] Optionally, the combined module includes:

[0057] A statistical submodule, used for counting a first quantity of the first inference texts;

[0058] The first combining submodule is used to combine the first inference texts based on the order in which the first inference texts are output by the WebGPU application program to obtain a target inference text when the first number is greater than or equal to a first threshold.

[0059] Optionally, the continuous input module includes:

[0060] A first continuous input submodule, configured to continuously input the first computing pipeline into a WebGPU application to obtain a first machine code continuously output by the WebGPU application;

[0061] A decoding submodule is used to decode the first machine code to obtain a first inference text.

[0062] Optionally, the combined module includes:

[0063] A fifth generating submodule, configured to generate a second inference word based on the first inference text;

[0064] A second determination submodule, configured to determine a second reasoning operator corresponding to the second reasoning word;

[0065] a sixth generation submodule, configured to generate a second computing pipeline corresponding to the second inference operator;

[0066] A second continuous input submodule is used for continuously inputting the second computing pipeline into the WebGPU application program when at least one second computing pipeline is obtained, so as to obtain a second reasoning text continuously output by the WebGPU application program;

[0067] The second combining submodule is used to combine the first reasoning text and the second reasoning text based on the sequence in which the first reasoning text and the second reasoning text are output by the WebGPU application program to obtain a target reasoning text.

[0068] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the WebGPU-based Web large language model reasoning acceleration optimization method as described in any one of the above.

[0069] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the WebGPU-based Web large language model reasoning acceleration optimization method as described in any one of the above is implemented.

[0070] The specific beneficial effects are:

[0071] The embodiment of the present application generates multiple first reasoning tokens based on the first text input by the target user, determines the first reasoning operators corresponding to each first reasoning token, generates the first computing pipelines corresponding to each first reasoning operator, and continuously inputs the first computing pipeline into the WebGPU application to obtain the first reasoning text continuously output by the WebGPU application when at least one first computing pipeline is obtained, and combines the first reasoning texts based on the order in which the WebGPU application outputs the first reasoning texts to obtain the target reasoning text. In the process of the Web large language model performing the reasoning task based on the first text input by the target user, when at least one computing pipeline is obtained, the first reasoning text can be output according to the computing pipeline and the WebGPU application without waiting for all computing pipelines to be generated before starting to output the first reasoning text. The process of generating the computing pipeline and the process of obtaining the first reasoning text can be executed in parallel, which can improve the utilization rate of the Web large language model of WebGPU computing resources to a certain extent, shorten the execution time of the reasoning task, and thus improve the reasoning efficiency of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0073] Figure 1 It is a flowchart of a WebGPU-based Web large language model reasoning acceleration optimization method provided in an embodiment of the present application;

[0074] Figure 2 It is a flowchart of another WebGPU-based Web large language model reasoning acceleration optimization method provided in an embodiment of the present application;

[0075] Figure 3 It is a flowchart of a specific implementation method of a WebGPU-based Web large language model reasoning acceleration optimization method provided in an embodiment of the present application;

[0076] Figure 4 It is a logical block diagram of a Web large language model reasoning acceleration optimization device based on WebGPU provided in an embodiment of the present application;

[0077] Figure 5 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.

[0079] Reference Figure 1 , Figure 1 A flowchart of a Web GPU-based Web large language model reasoning acceleration optimization method provided in an embodiment of the present application is applied to a Web large language model, and the control center is a central processing unit (CPU). The method includes:

[0080] Step 101: Generate a plurality of first inference word-grams based on a first text input by a target user.

[0081] In the embodiments of the present application, tokens are a basic concept in natural language processing and can play a key role in processing text data. Simply put, tokens divide text data into smaller units, which may be words, characters, or subwords. Tokens are the most basic units of operation of language models during training and reasoning. The Web large language model can receive a first text input by a target user and perform preliminary reasoning based on the first text, so that multiple first reasoning tokens can be generated and obtained. In this process, the first reasoning token can be generated serially, that is, after obtaining the first reasoning token, a second reasoning can be performed on the basis of the reasoning token to obtain a second reasoning token. The second reasoning token is located after the first reasoning token, and it can have coherent semantics at this time. Repeat the above process to obtain multiple first reasoning tokens.

[0082] Step 102: determine the first inference operators corresponding to the first inference words.

[0083] In the embodiments of the present application, the inference operator refers to the inference logic from the upper level semantics (the text input by the user or the previous inference word) to the first inference word, also called context information. In the program, the inference operator can be encapsulated into a sub-function, and each time a first inference word is obtained, the inference logic between the first inference word and the upper level semantics can be obtained in the Web big language model, so that the first inference operator corresponding to each first inference word can be determined according to the inference logic.

[0084] Step 103: Generate first computing pipelines corresponding to each of the first inference operators.

[0085] In an embodiment of the present application, a compute pipeline is a pipeline in WebGPU that controls the compute shader stage. Compute shaders are programs that run on a graphics card and outside the normal rendering pipeline. They can be used for massively parallel GPU algorithms, or for accelerating certain parts of game rendering. An inference operator can be combined with a compute shader and encapsulated twice, so that a first compute pipeline can be generated and obtained. In a program, a compute pipeline can be an encapsulated subfunction that can be executed in a GPU.

[0086] Step 104: When at least one first computing pipeline is obtained, the first computing pipeline is continuously input into the WebGPU application to obtain a first reasoning text continuously output by the WebGPU application.

[0087] In an embodiment of the present application, when at least one first computing pipeline is obtained, the generated first computing pipeline can be continuously input into a WebGPU application, and the WebGPU application executes the computing process corresponding to the first computing pipeline, thereby obtaining a first inference text continuously output by the WebGPU application.

[0088] Step 105: Based on the order in which the WebGPU application program outputs the first reasoning texts, the first reasoning texts are combined to obtain a target reasoning text.

[0089] In an embodiment of the present application, the Web Big Language Model can combine the first reasoning texts according to the order in which the WebGPU application program outputs the first reasoning texts to obtain a target reasoning text. The combination process can be started when at least two first reasoning texts are obtained, so that the combination of the first reasoning texts can be completed within a short period of time after the WebGPU outputs all the first reasoning texts, thereby shortening the time to obtain the target reasoning text.

[0090] In an embodiment of the present application, by generating multiple first reasoning tokens based on a first text input by a target user, determining first reasoning operators corresponding to each first reasoning token, and generating first computing pipelines corresponding to each first reasoning operator, when at least one first computing pipeline is obtained, the first computing pipeline is continuously input into a WebGPU application to obtain first reasoning texts continuously output by the WebGPU application, and the first reasoning texts are combined based on the order in which the WebGPU application outputs the first reasoning texts to obtain a target reasoning text. In the process of the Web large language model performing a reasoning task based on the first text input by the target user, when at least one computing pipeline is obtained, the first reasoning text can be output according to the computing pipeline and the WebGPU application without waiting for all first computing pipelines to be generated before starting to output the first reasoning text. The process of generating the computing pipeline and the process of obtaining the first reasoning text can be executed in parallel, which can improve the utilization rate of the Web large language model for WebGPU computing resources to a certain extent, shorten the execution time of the reasoning task, and thus improve the reasoning efficiency of the large language model.

[0091] Reference Figure 2 , Figure 2 A flowchart of another WebGPU-based Web large language model reasoning acceleration optimization method provided in an embodiment of the present application, the method may include:

[0092] Step 201: Generate a plurality of first inference word-grams based on a first text input by a target user.

[0093] In the embodiments of the present application, the implementation content of this step can refer to the embodiment content of step 101, which will not be repeated here.

[0094] Step 202: determine the first reasoning operators corresponding to the first reasoning words.

[0095] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 102, which will not be repeated here.

[0096] Optionally, step 202 may include the following sub-steps:

[0097] Sub-step 2021, when the cache space contains the first inference operator corresponding to the first inference word-gram, read the first inference operator corresponding to the first inference word-gram from the cache space.

[0098] In an embodiment of the present application, the first inference operator can be stored in a cache space. After obtaining the first inference word, the cache space can be searched to see whether it contains the first inference operator corresponding to the first inference word. If the cache space contains the first inference operator corresponding to the first inference word, the first inference operator corresponding to the first inference word can be read in the cache space without generating it separately.

[0099] In an embodiment of the present application, by reading the first inference operator corresponding to the first inference word from the cache space when the first inference operator corresponding to the first inference word is included in the cache space, the first inference operator corresponding to the first inference word can be read from the cache space, thereby avoiding obtaining a duplicate first inference operator. At the same time, since the data reading efficiency of the cache space is much higher than the efficiency of generating the first inference operator, the time for obtaining the inference operator can be shortened to a certain extent, thereby improving the inference efficiency of the inference task.

[0100] Sub-step 2022, when at least one first inference word-gram is obtained, continue to determine the first inference operators corresponding to each of the first inference word-grams.

[0101] In an embodiment of the present application, when at least one first reasoning word-gram is obtained, the first reasoning operators corresponding to the first reasoning dimensions can be continuously determined without waiting for all the first reasoning word-grams to be generated.

[0102] In an embodiment of the present application, by continuously determining the first reasoning operators corresponding to each first reasoning word after obtaining at least one first reasoning word, the steps of generating the first reasoning word and determining the first reasoning operator can be in an asynchronous parallel state, which can improve the CPU computing resource utilization to a certain extent and improve the reasoning efficiency of the reasoning task.

[0103] Step 203: Generate first computing pipelines corresponding to each of the first inference operators.

[0104] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 103, which will not be repeated here.

[0105] Optionally, step 203 may include the following sub-steps:

[0106] Sub-step 2031: Generate a first compute shader corresponding to the first inference operator.

[0107] In an embodiment of the present application, the first compute shader can be used to convert the first reasoning operator into a machine code recognizable by WebGPU. Since each first reasoning operator can have different reasoning logic (context information), the first compute shader can also adaptively have different input types. In addition, the first compute shader can also design the input type according to the language type of the text. On this basis, after obtaining the first reasoning operator, a first compute shader corresponding to the first reasoning operator can be generated so that the first compute shader can convert the first reasoning operator into a machine code recognizable by WebGPU.

[0108] Sub-step 2032: generating a first resource binding group based on the first compute shader and the first inference operator.

[0109] In an embodiment of the present application, a resource binding group (GPUBindGroup) is a data container for packaging data resources and passing them to a shader program. In the architecture of WebGPU, the resource binding group plays a key role. It allows developers to organize different types of resources (such as vertex attributes, shader variables, textures, samplers, etc.) together and send them to the GPU by command submission. This form of data organization reduces the number of communications between the CPU and the GPU, thereby improving performance. Creating a resource binding group requires a corresponding descriptor, whose structure includes information such as a resource binding group layout (GPUBindGroupLayout), a binding point (binding), and a shader program (visibility) for a specific stage. This information describes the type of resources, the group to which they belong, and how they are used in the shader. In an embodiment of the present application, the first inference operator and the first compute shader can be repackaged to obtain a first resource binding group.

[0110] Sub-step 2033: generating a first computing pipeline based on the first resource binding group.

[0111] In an embodiment of the present application, the first resource binding group can be called in the manner of a computing pipeline. In the computing pipeline, a binding group layout (BindGroupLayout), a module address, an entry point, and a label can be included. Usually, when the first resource binding group is generated, a default binding group layout is already available, and no additional definition is required. The entry point of the computing pipeline needs to be able to access each resource in the resource binding group. According to this standard, the first computing pipeline can be generated based on the first resource binding group.

[0112] In an embodiment of the present application, by generating a first compute shader corresponding to a first inference operator, generating a first resource binding group based on the first compute shader and the first inference operator, and generating a first compute pipeline based on the first resource binding group, a compute pipeline can be generated through the inference operator and the compute shader, thereby improving the accuracy of the compute pipeline to a certain extent.

[0113] Sub-step 2034: if a first resource binding group matching the first inference operator exists in the cache space, read the first resource binding group from the cache space.

[0114] In an embodiment of the present application, the first resource binding group may be stored in a cache space. After obtaining the first inference operator, it is possible to query in the cache space whether there is a first resource binding group corresponding to the first inference operator (i.e., the first resource binding group including the first inference operator), and if so, the first resource binding group corresponding to the first inference operator may be read from the cache space.

[0115] Sub-step 2035: generating first computing pipelines corresponding to each of the first inference operators based on the first resource binding group.

[0116] In an embodiment of the present application, after the first resource binding group is read, first computing pipelines corresponding to respective first inference operators may be generated according to the first resource binding group.

[0117] In an embodiment of the present application, by reading the first resource binding group from the cache space when there is a first resource binding group matching the first reasoning operator in the cache space, and generating first computing pipelines corresponding to each first reasoning operator based on the first resource binding group, the first resource binding group can be read from the cache space when there is a first resource binding group matching the first reasoning operator in the cache space, thereby avoiding the generation of duplicate resource binding groups. Since the reading efficiency of the cache space is much higher than the efficiency of generating the resource binding group, the efficiency of obtaining the first resource binding group can be improved to a certain extent, the resource reuse rate can be improved, and the CPU computing resources can be saved at the same time.

[0118] Step 204 : if a first computing pipeline matching the first inference operator exists in the cache space, read the first computing pipeline matching the first inference operator from the cache space.

[0119] In an embodiment of the present application, the first computing pipeline can be stored in a cache space, and it can be queried in the cache space whether there is a first computing pipeline that matches the first inference operator (i.e., a first computing pipeline that includes the first inference operator). If so, the first computing pipeline that matches the first inference operator can be read in the cache space.

[0120] In a possible embodiment, if the first computing pipeline can be stored in the cache space, the cache space can be queried first to see if there is a first computing pipeline matching the first reasoning operator. If so, the first computing pipeline matching the first reasoning operator can be directly read without querying the first resource binding group. If not, the cache space is queried to see if there is a first resource binding group corresponding to the first reasoning operator. If there is a first resource binding group corresponding to the first reasoning operator, the first resource binding group is read from the cache space. If there is no first resource binding group corresponding to the first reasoning operator, the first resource binding group corresponding to the first reasoning operator can be generated. In this way, additional querying and generating steps of the first resource binding group can be avoided to a certain extent, waste of computing resources can be avoided, and the reasoning time of the large Web language model can be shortened.

[0121] In an embodiment of the present application, by reading the first computing pipeline that matches the first inference operator from the cache space when there is a first computing pipeline that matches the first inference operator in the cache space, it is possible to avoid generating a duplicate first computing pipeline. Since the reading efficiency of the cache space is much higher than the efficiency of generating the first computing pipeline, the efficiency of obtaining the first computing pipeline can be improved to a certain extent, the resource reuse rate can be improved, and the computing resources of the CPU can be saved.

[0122] Step 205: When at least one first computing pipeline is obtained, the first computing pipeline is continuously input into the WebGPU application to obtain a first reasoning text continuously output by the WebGPU application.

[0123] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 104, which will not be repeated here.

[0124] Optionally, step 205 may include the following sub-steps:

[0125] Sub-step 2051: continuously input the first computing pipeline into a WebGPU application to obtain a first machine code continuously output by the WebGPU application.

[0126] In an embodiment of the present application, each time a first computing pipeline is obtained, the first computing pipeline can be input into a WebGPU application so that WebGPU can execute the first computing pipeline. After execution, a first machine code can be obtained and can be output by the WebGPU application.

[0127] Sub-step 2052: decode the first machine code to obtain a first inference text.

[0128] In an embodiment of the present application, the first machine code can be decoded to obtain a first inference text. The decoding method can be specified by the Web Large Language Model and can correspond to the encoding method in the Web Large Language Model. That is, for a source text, the encoding method in the Web Large Language Model is used, and then the decoding method therein (excluding the inference process) can still be used to obtain the source text without any change. Among them, the first inference code output by the WebGPU can be stored in the result buffer, and when the first inference code is decoded, the first inference text obtained after decoding can be stored in the staging buffer (that is, the decoding process can be regarded as mapping the first inference code in the result buffer to the staging buffer).

[0129] In an embodiment of the present application, by continuously inputting the first computing pipeline into the WebGPU application, a first machine code continuously output by the WebGPU application is obtained, and the first machine code is decoded to obtain a first inference text. The machine code output by the WebGPU can be processed by decoding to obtain the first inference text, thereby improving the accuracy of the first inference text to a certain extent.

[0130] Step 206: Based on the order in which the WebGPU application program outputs the first reasoning texts, the first reasoning texts are combined to obtain a target reasoning text.

[0131] In the embodiments of the present application, the implementation content of this step can refer to the implementation content of step 105, which will not be repeated here.

[0132] Optionally, step 206 may include the following sub-steps:

[0133] Sub-step 2061: counting a first quantity of the first inference texts.

[0134] In an embodiment of the present application, a first number of the first reasoning texts may be counted, wherein in a complete reasoning task, when the reasoning task is completed, the total number of the first reasoning texts may be the same as the number of the first reasoning tokens.

[0135] Sub-step 2062, when the first number is greater than or equal to a first threshold, combining the first inference texts based on the order in which the WebGPU application outputs the first inference texts to obtain a target inference text.

[0136] In an embodiment of the present application, a first threshold value can be set manually. When the first number is greater than or equal to the first threshold value, the first reasoning texts can be combined according to the order in which the WebGPU application outputs the first reasoning texts, so as to obtain the target reasoning text. Among them, the first reasoning text can be stored in a staging buffer. When the first number is greater than or equal to the first threshold value, the first reasoning text can be read from the staging buffer and combined, so as to obtain the target reasoning text. It should be noted that before obtaining the target reasoning text, the above-mentioned reading and combining process can be repeated multiple times, and the first number is cleared each time it is executed. When the total number of the first reasoning texts is the same as the number of the first reasoning word units, it can be confirmed that all the first reasoning word units have been used up. At this time, after reading the first reasoning text and combining it, the final target reasoning text can be obtained.

[0137] In an embodiment of the present application, by counting a first number of the first reasoning texts, when the first number is greater than or equal to a first threshold, the first reasoning texts are combined based on the order in which the WebGPU application outputs the first reasoning texts to obtain a target reasoning text, and the first reasoning texts can be batch processed in batches to generate a target reasoning text, which can avoid the process of generating the target reasoning text occupying computing resources for a long time, and to a certain extent improve the performance overhead caused by generating the target reasoning text.

[0138] Sub-step 2063: generating a second inference word-gram based on the first inference word-gram.

[0139] In the embodiment of the present application, after obtaining the first inference word, in the Web large language model, the next inference word can be generated according to the language logic learned by the large language model itself and combined with the first inference word, which is called the second inference word. After obtaining the second inference word, the second word can be used as the next first inference word to further generate the next second inference word. In this way, autoregressive reasoning can be performed to generate multiple inference words.

[0140] Sub-step 2064: determine a second reasoning operator corresponding to the second reasoning word.

[0141] In the embodiment of the present application, the implementation content of this step is similar to the implementation content of step 102 for obtaining the first reasoning operator, and reference may be made to the implementation content of step 102, which will not be repeated here.

[0142] Sub-step 2065: Generate a second computing pipeline corresponding to the second inference operator.

[0143] In the embodiment of the present application, the implementation content of this step is similar to the implementation content of generating the first computing pipeline in step 103. The implementation content of each sub-step in step 103 and step 203 may be referred to and will not be repeated here.

[0144] Sub-step 2066, when at least one second computing pipeline is obtained, continuously input the second computing pipeline into the WebGPU application to obtain a second inference text continuously output by the WebGPU application.

[0145] In the embodiment of the present application, the implementation content of obtaining the second reasoning text in this step is similar to the implementation content of obtaining the first reasoning text in step 104. Please refer to the implementation content of step 104 and will not be repeated here.

[0146] Sub-step 2067, based on the order in which the first reasoning text and the second reasoning text are output by the WebGPU application, the first reasoning text and the second reasoning text are combined to obtain a target reasoning text.

[0147] In an embodiment of the present application, the first reasoning text and the second reasoning text may be combined according to the order in which the WebGPU application program outputs the first reasoning text and the second reasoning text, thereby obtaining a target reasoning text. When combining, only the order in which the first reasoning text and the second reasoning text are output may be considered, without considering the difference between the first reasoning text and the second reasoning text.

[0148] In an embodiment of the present application, a second reasoning word is generated based on a first reasoning word, a second reasoning operator corresponding to the second reasoning word is determined, and a second computing pipeline corresponding to the second reasoning operator is generated. When at least one second computing pipeline is obtained, the second computing pipeline is continuously input into a WebGPU application to obtain a second reasoning text continuously output by the WebGPU application. Based on the sequence in which the WebGPU application outputs the first reasoning text and the second reasoning text, the first reasoning text and the second reasoning text are combined to obtain a target reasoning text. Multiple reasoning words can be obtained in an autoregressive manner, and then multiple first reasoning texts and second reasoning texts can be obtained. Finally, the target reasoning text is obtained through the first reasoning text and the second reasoning text, which can improve the accuracy of the target reasoning text to a certain extent.

[0149] Reference Figure 3 , Figure 3It is a flowchart of a specific implementation method of a Web large language model reasoning acceleration optimization method based on WebGPU provided by an embodiment of the present application. In the figure, three modules may be included, namely a scheduling module, a task distribution module and a result reading module. The user text and the Web large language model can be used as the input of the scheduling module together (that is, a specific Web large language model is used to process the user text, and the processed result is used as the input of the scheduling module), and the obtained reasoning word element is sent to the task distribution module through the distribution management function of the scheduling module. In the task distribution module, multiple buffers can be created when the reasoning word element is received, which are used to temporarily store the reasoning word element, resource binding group and computing pipeline. After the buffer is created, a resource binding group can be created based on the reasoning word element in combination with the computing shader, and a computing pipeline can be created based on the resource binding group. After that, the computing pipeline can be sent to the WebGPU application, and the WebGPU executes the computing task corresponding to the computing pipeline, thereby obtaining the computing result output by the WebGPU application and sending it to the result buffer. The result reading management function in the scheduling module can read the calculation results in the result buffer and forward the calculation results to the result reading module. In the result reading module, after receiving the calculation results, a staging buffer can be created. After the creation, the results in the result buffer can be mapped to the staging buffer by means of buffer mapping to decode the calculation results and obtain the reasoning text. After that, the number of reasoning texts can be counted by the feedback management function of the scheduling module. If the number of reasoning texts reaches a preset threshold, the reasoning texts are combined in the order of generation, so that the target reasoning text can be obtained. In the task distribution module, a resource cache function can also be included, which is used to store the calculation shader, resource binding group and calculation pipeline in the buffer space when power is turned on. And when generating the resource binding group and the calculation pipeline, the calculation shader, resource binding group and calculation pipeline corresponding to the reasoning word can be queried from the cache space first to avoid repeated generation of the same shader, resource binding group and calculation pipeline.

[0150] Reference Figure 4 , Figure 4 A logical block diagram of a Web large language model reasoning acceleration optimization device based on WebGPU provided in an embodiment of the present application, the device 400 may include:

[0151] A first generating module 401, configured to generate a plurality of first inference tokens based on a first text input by a target user;

[0152] A first determining module 402, configured to determine first reasoning operators corresponding to the first reasoning words;

[0153] A second generating module 403, configured to generate first computing pipelines corresponding to the first inference operators;

[0154] A continuous input module 404 is used for continuously inputting the first computing pipeline into the WebGPU application program when at least one first computing pipeline is obtained, so as to obtain a first reasoning text continuously output by the WebGPU application program;

[0155] The combining module 405 is used to combine the first reasoning texts based on the order in which the WebGPU application outputs the first reasoning texts to obtain a target reasoning text.

[0156] Optionally, the second generating module 403 includes:

[0157] A first generating submodule, configured to generate a first computing shader corresponding to the first inference operator;

[0158] A second generation submodule, configured to generate a first resource binding group based on the first compute shader and the first inference operator;

[0159] The third generating submodule is used to generate a first computing pipeline based on the first resource binding group.

[0160] Optionally, the first resource binding group is stored in a cache space, and the second generating module 403 includes:

[0161] A first reading submodule, configured to read the first resource binding group from the cache space if a first resource binding group matching the first inference operator exists in the cache space;

[0162] The fourth generating submodule is used to generate first computing pipelines corresponding to each of the first inference operators based on the first resource binding group.

[0163] Optionally, the first determining module 402 includes:

[0164] The first determination submodule is used to continuously determine a first reasoning operator corresponding to at least one first reasoning word-unit when at least one first reasoning word-unit is obtained.

[0165] Optionally, the first inference operator is stored in a cache space, and the first determination module 402 includes:

[0166] A second reading submodule is configured to read the first inference operator corresponding to the first inference word from the cache space when the cache space contains the first inference operator corresponding to the first inference word.

[0167] Optionally, the first computing pipeline is stored in a cache space, and the apparatus 400 further includes:

[0168] The reading module is used to read the first computing pipeline matching the first inference operator from the cache space when there is a first computing pipeline matching the first inference operator in the cache space.

[0169] Optionally, the combination module 405 includes:

[0170] A statistical submodule, used for counting a first quantity of the first inference texts;

[0171] The first combining submodule is used to combine the first inference texts based on the order in which the first inference texts are output by the WebGPU application program to obtain a target inference text when the first number is greater than or equal to a first threshold.

[0172] Optionally, the continuous input module 404 includes:

[0173] A first continuous input submodule, configured to continuously input the first computing pipeline into a WebGPU application to obtain a first machine code continuously output by the WebGPU application;

[0174] A decoding submodule is used to decode the first machine code to obtain a first inference text.

[0175] Optionally, the combination module 405 includes:

[0176] A fifth generating submodule, configured to generate a second inference word based on the first inference text;

[0177] A second determination submodule, configured to determine a second reasoning operator corresponding to the second reasoning word;

[0178] a sixth generation submodule, configured to generate a second computing pipeline corresponding to the second inference operator;

[0179] A second continuous input submodule is used for continuously inputting the second computing pipeline into the WebGPU application program when at least one second computing pipeline is obtained, so as to obtain a second reasoning text continuously output by the WebGPU application program;

[0180] The second combining submodule is used to combine the first reasoning text and the second reasoning text based on the sequence in which the first reasoning text and the second reasoning text are output by the WebGPU application program to obtain a target reasoning text.

[0181] The Web GPU-based Web large language model reasoning acceleration optimization device in the embodiment of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. Exemplarily, the electronic device can be a GPUBOX, a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0182] The WebGPU-based Web large language model reasoning acceleration optimization device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, a Linux, a Windows operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0183] The Web GPU-based Web large language model reasoning acceleration optimization device provided in the embodiment of the present application can achieve Figures 1 to 3 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0184] The present application embodiment provides an electronic device, see Figure 5 The electronic device 50 includes: a processor 501, a memory 502, and a computer program 5021 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the program, the WebGPU-based Web large language model reasoning acceleration optimization method of the aforementioned embodiment is implemented.

[0185] The embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps in the WebGPU-based Web large language model reasoning acceleration optimization method disclosed in the embodiment of the present application are implemented.

[0186] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the processor is enabled to implement the steps in the WebGPU-based Web large language model reasoning acceleration optimization method disclosed in the embodiment of the present application.

[0187] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0188] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0189] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0190] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0191] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0192] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0193] The above is a detailed introduction to a Web large language model reasoning acceleration optimization method based on WebGPU provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A Web large language model reasoning acceleration optimization method based on WebGPU, characterized in that: The method comprises: Based on the first text input by the target user, generate a plurality of first inference tokens; Determine a first inference operator corresponding to each of the first inference tokens; wherein the first inference operator represents the inference logic from the previous inference token of the first inference token to the first inference token; Generate first computing pipelines corresponding to the first inference operators respectively; When at least one first computing pipeline is obtained, continuously inputting the first computing pipeline into the WebGPU application to obtain a first reasoning text continuously output by the WebGPU application; Based on the order in which the WebGPU application program outputs the first reasoning texts, the first reasoning texts are combined to obtain a target reasoning text.

2. The method according to claim 1, characterized in that The generating of first computing pipelines corresponding to the first inference operators includes: generating a first compute shader corresponding to the first inference operator; generating a first resource binding group based on the first compute shader and the first inference operator; Based on the first resource binding group, a first computing pipeline is generated.

3. The method according to claim 2, characterized in that The first resource binding group is stored in a cache space, and the generating of first computing pipelines corresponding to the first inference operators includes: If there is a first resource binding group matching the first inference operator in the cache space, read the first resource binding group from the cache space; Based on the first resource binding group, first computing pipelines corresponding to the first inference operators are generated.

4. The method according to claim 1, characterized in that: The determining the first reasoning operators corresponding to the first reasoning tokens respectively includes: When at least one first inference word-gram is obtained, first inference operators corresponding to each of the first inference word-grams are continuously determined.

5. The method according to claim 1, characterized in that The first inference operator is stored in a cache space, and the determining the first inference operator corresponding to each of the first inference tokens includes: In a case where the cache space contains the first inference operator corresponding to the first inference word-element, the first inference operator corresponding to the first inference word-element is read from the cache space.

6. The method according to claim 1, characterized in that The first computing pipeline is stored in a cache space, and the method further includes: In a case where a first computing pipeline matching the first inference operator exists in the cache space, the first computing pipeline matching the first inference operator is read from the cache space.

7. The method according to claim 1, characterized in that The combining the first reasoning texts based on the order in which the WebGPU application program outputs the first reasoning texts to obtain a target reasoning text includes: Counting a first number of the first inference texts; When the first number is greater than or equal to a first threshold, the first inference texts are combined based on the order in which the WebGPU application outputs the first inference texts to obtain a target inference text.

8. The method according to claim 1, characterized in that The continuously inputting the first computing pipeline into the WebGPU application to obtain the first reasoning text continuously output by the WebGPU application includes: Continuously inputting the first computing pipeline into a WebGPU application to obtain a first machine code continuously output by the WebGPU application; The first machine code is decoded to obtain a first inference text.

9. The method according to claim 1, characterized in that: The combining the first reasoning texts based on the order in which the WebGPU application program outputs the first reasoning texts to obtain a target reasoning text includes: Based on the first inference word-gram, generating a second inference word-gram; Determining a second inference operator corresponding to the second inference word; generating a second computing pipeline corresponding to the second inference operator; When at least one second computing pipeline is obtained, continuously inputting the second computing pipeline into the WebGPU application to obtain a second reasoning text continuously output by the WebGPU application; Based on the order in which the first reasoning text and the second reasoning text are output by the WebGPU application, the first reasoning text and the second reasoning text are combined to obtain a target reasoning text.

10. A Web large language model reasoning acceleration optimization device based on WebGPU, characterized in that: The device comprises: A first generating module, configured to generate a plurality of first inference tokens based on a first text input by a target user; A first determination module, configured to determine a first inference operator corresponding to each of the first inference tokens; wherein the first inference operator represents the inference logic from the previous inference token of the first inference token to the first inference token; A second generating module, used to generate first computing pipelines corresponding to each of the first inference operators; A continuous input module, configured to continuously input the first computing pipeline into the WebGPU application program when at least one first computing pipeline is obtained, so as to obtain a first reasoning text continuously output by the WebGPU application program; The combination module is used to combine the first reasoning texts based on the order in which the WebGPU application outputs the first reasoning texts to obtain a target reasoning text.

Images