Large language model processing method and device, computer equipment and storage medium

Abstract

The invention relates to a large language model processing method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining a mark sequence sample obtained by reasoning based on a mark sequence of an instruction sample; determining an initial mark sequence, and obtaining an initial prediction result; sequentially performing iterative mark prediction based on the mark sequence sample and the initial prediction result through the at least one mark prediction model to obtain respective mark prediction results of the at least one mark prediction model; according to the respective marked prediction result and the target reasoning result of each marked prediction model, updating the shared weight parameters included in each marked prediction model to obtain a trained large language model; and the trained large language model is used for reasoning the mark sequence of the input instruction so as to output the mark sequence of the corresponding response content of the input instruction. By adopting the method, the reasoning processing efficiency of the large language model can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to a large language model processing method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the development of computer technology, generative artificial intelligence technology represented by large language models (LLM) has made significant breakthroughs and is widely used in natural language processing. Currently, the traditional large language model inference process generally adopts a self-recurrent generation method, that is, each element in the output text is generated in sequence from left to right, and the generation of each element depends on all previously generated elements. Although this step-by-step recursive inference mechanism can ensure the coherence and logic of the generated results, when dealing with complex or long sequence inference tasks, the character-by-character generation feature makes the inference processing time-consuming, resulting in low inference processing efficiency of the large language model. SUMMARY

[0003] Therefore, it is necessary to provide a large language model processing method and device, computer equipment, computer readable storage medium and computer program product capable of improving the inference processing efficiency of the large language model, as well as an instruction response method and device based on the large language model, computer equipment, computer readable storage medium and computer program product.

[0004] In a first aspect, the present application provides a large language model processing method. The method comprises:

[0005] Obtaining a token sequence sample, the token sequence sample being obtained by inference based on a token sequence of an instruction sample;

[0006] Determining an initial token sequence from the token sequence sample, and obtaining an initial prediction result obtained by inference of a main model in the large language model on the initial token sequence;

[0007] Iteratively performing token prediction based on the token sequence sample and the initial prediction result through at least one token prediction model in the large language model to obtain token prediction results of the at least one token prediction model respectively; wherein in each iteration of token prediction, a target token prediction model of the current iteration of token prediction performs token prediction on an input token sequence of the current iteration of token prediction and a previous prediction result to obtain a token prediction result of the current iteration of token prediction, the input token sequence being obtained by offsetting the initial token sequence in the token sequence sample according to a sequence number of the current iteration of token prediction;

[0008] According to the respective mark prediction result of the at least one mark prediction model and the target reasoning result, the shared weight parameter included in each mark prediction model is updated, and a trained large language model is obtained; the target reasoning result is obtained by iteratively performing mark prediction on the mark sequence sample and the initial prediction result by the main model; and the trained large language model is used to perform reasoning on the mark sequence of an input instruction to output the mark sequence of the corresponding response content of the input instruction.

[0009] In a second aspect, the present application further provides a large language model processing device. The device comprises:

[0010] A mark sequence sample acquisition module is configured to acquire a mark sequence sample, the mark sequence sample being obtained by performing reasoning on a mark sequence of an instruction sample;

[0011] An initial processing data acquisition module is configured to determine an initial mark sequence from the mark sequence sample, and obtain an initial prediction result obtained by performing reasoning on the initial mark sequence by a main model in a large language model;

[0012] An iterative mark prediction module is configured to perform iterative mark prediction on the mark sequence sample and the initial prediction result by at least one mark prediction model in the large language model in sequence to obtain a respective mark prediction result of each mark prediction model; in each iterative mark prediction, a target mark prediction model of the current iterative mark prediction performs mark prediction on an input mark sequence of the current iterative mark prediction and a previous prediction result to obtain a mark prediction result of the current iterative mark prediction, the input mark sequence being obtained by offsetting the initial mark sequence in the mark sequence sample according to a sequence number of the current iterative mark prediction;

[0013] A shared weight parameter updating module is configured to update the shared weight parameter included in each mark prediction model according to the respective mark prediction result of the at least one mark prediction model and a target reasoning result, and obtain a trained large language model; the target reasoning result is obtained by iteratively performing mark prediction on the mark sequence sample and the initial prediction result by the main model; and the trained large language model is used to perform reasoning on the mark sequence of an input instruction to output the mark sequence of the corresponding response content of the input instruction.

[0014] In a third aspect, the present application further provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above large language model processing method when executing the computer program.

[0015] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium has a computer program stored thereon, and the computer program, when executed by a processor, implements the steps of the above large language model processing method.

[0016] In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program, and the computer program, when executed by a processor, implements the steps of the above large language model processing method.

[0017] The above large language model processing method, device, computer device, storage medium, and computer program product determine an initial token sequence from a token sequence sample obtained by reasoning based on a token sequence of an instruction sample, obtain an initial prediction result obtained by reasoning on the initial token sequence by a main model in a large language model, iteratively perform token prediction based on the token sequence sample and the initial prediction result by at least one token prediction model in the large language model, obtain token prediction results of the at least one token prediction model, and update shared weight parameters included in each token prediction model based on the token prediction results of the at least one token prediction model and a target reasoning result obtained by iteratively performing token prediction by the main model based on the token sequence sample and the initial prediction result, to obtain a trained large language model. In the processing of the large language model, the shared weight parameters included in each token prediction model are updated based on the token prediction results of the at least one token prediction model and the target reasoning result obtained by iteratively performing token prediction by the main model based on the token sequence sample and the initial prediction result, and the token prediction is performed by reusing the shared weight parameters under the premise that the token prediction results of the token prediction models match the reasoning result of the main model, which can reduce redundant computation and improve the efficiency of reasoning processing of the large language model.

[0018] In a sixth aspect, the present application provides a large language model-based instruction response method. The method comprises:

[0019] Obtaining a token sequence of an input instruction, determining a large language model, and determining a large language model based on the above large language model processing method;

[0020] Performing reasoning on the token sequence of the input instruction by a main model in the large language model to obtain an initial prediction result;

[0021] Iteratively performing token prediction based on the initial prediction result and the token sequence of the input instruction by at least one token prediction model in the large language model to obtain token prediction results of the at least one token prediction model;

[0022] Obtaining a token sequence of response content corresponding to the input instruction by the main model based on the token prediction results of the at least one token prediction model.

[0023] In a seventh aspect, the present application also provides a large language model-based instruction response apparatus. The apparatus comprises:

[0024] An instruction token sequence acquisition module is configured to acquire a token sequence of an input instruction, and determine a large language model, wherein the large language model is obtained based on the large language model processing method described above.

[0025] An initial prediction module is configured to perform reasoning on the token sequence of the input instruction by a main model in the large language model, and obtain an initial prediction result.

[0026] An iterative token prediction module is configured to perform iterative token prediction based on the initial prediction result and the token sequence of the input instruction by at least one token prediction model in the large language model, and obtain a token prediction result of each token prediction model.

[0027] A response token sequence acquisition module is configured to obtain a token sequence of a corresponding response content of the input instruction by the main model based on the token prediction result of each token prediction model.

[0028] In an eighth aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above large language model-based instruction response method when executing the computer program.

[0029] In a ninth aspect, the present application also provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program implements the steps of the above large language model-based instruction response method when executed by a processor.

[0030] In a tenth aspect, the present application also provides a computer program product. The computer program product comprises a computer program, and the computer program implements the steps of the above large language model-based instruction response method when executed by a processor.

[0031] The instruction response method, device, computer device, storage medium and computer program product based on the large language model, through the main model in the large language model, infers the token sequence of the input instruction to obtain an initial prediction result, iteratively performs token prediction based on the initial prediction result and the token sequence of the input instruction through at least one token prediction model in the large language model, obtains the token prediction result of each token prediction model, and obtains the token sequence of the corresponding response content of the input instruction through the main model based on the token prediction result of each token prediction model. In the instruction response processing based on the large language model, the token prediction is performed by reusing the shared weight parameters, the redundant calculation amount can be reduced, and the efficiency of the inference processing of the large language model is improved. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or the related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0033] Figure 1 An application environment diagram of the large language model processing method in an embodiment;

[0034] Figure 2 A flowchart of the large language model processing method in an embodiment;

[0035] Figure 3 A flowchart of token sequence sample construction in an embodiment;

[0036] Figure 4 A structure diagram of the large language model in an embodiment;

[0037] Figure 5 A diagram of obtaining different input token sequences by offset in an embodiment;

[0038] Figure 6 A flowchart of the large language model processing method in another embodiment;

[0039] Figure 7 A flowchart of shared weight parameter updating in an embodiment;

[0040] Figure 8 A flowchart of the instruction response method based on the large language model in an embodiment;

[0041] Figure 9 A comparison diagram of inference speed of the large language model processing method in an embodiment;

[0042] Figure 10 Inference speed comparison diagram of large language model processing method in another embodiment;

[0043] Figure 11 Flowchart of large language model processing method in another embodiment;

[0044] Figure 12 Flowchart of large language model processing method in another embodiment;

[0045] Figure 13 Structural block diagram of large language model processing device in an embodiment;

[0046] Figure 14 Structural block diagram of instruction response device based on large language model in an embodiment;

[0047] Figure 15 Internal structure diagram of computer device in an embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0049] The following explains the terms involved in the present application:

[0050] Large language model (LLM, Large Language Model): a deep learning model with strong language understanding and generation capabilities built through pre-training and fine-tuning of massive text data. Large language models usually have parameter sizes of tens of billions or even hundreds of billions, which can capture the complex semantics, syntax and context relationships of language, and thus exhibit human-level performance in text generation, machine translation, question answering systems, content summarization and other tasks.

[0051] Token: a basic processing unit used to represent text. Token is the result of tokenizing mapping continuous text (such as sentences, paragraphs) into discrete units, which can be a word, a subword, a character, or even a punctuation mark. LLM receives token sequences as input to understand text semantics and generate output.

[0052] Instruction: represents the input of the user to the large language model, the instruction can be an explicit request or task description issued by the user to the LLM, used to guide the model to perform a specific operation, the instruction can be a question, a command, a prompt, or a context containing examples.

[0053] Main Model: the model responsible for predicting the next token in the large language model, is the core inference unit of the large language model, such as the main model can include multiple layers of transformer structure. The main model can perform inference prediction based on the input token sequence, output the token distribution probability at each position, and determine the predicted token at each position.

[0054] MTP (Multi-Token Prediction Module): belongs to the lightweight draft model, assists in generating future tokens in training or specific inference scenarios, and is an extension component of the main model. A plurality of candidate tokens are generated in advance by the MTP, and then verified by the main model in batches, so that multiple tokens are output in a single forward propagation, which can be used as a token prediction model in the large language model.

[0055] The large language model processing method provided by the embodiments of the present application can be applied to, for example Figure 1The application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required by the server 104 to process. The data storage system can be set up separately, or integrated on the server 104, or placed on the cloud or other devices. The large language model can include a main model and K (K≥1) mark prediction models, and each mark prediction model can include shared weight parameters; the terminal 102 can send a mark sequence sample obtained by reasoning based on the instruction sample to the server 104, the server 104 can determine an initial mark sequence from the mark sequence sample, and obtain an initial prediction result obtained by reasoning based on the initial mark sequence by the main model in the large language model. The server 104 can perform iterative mark prediction based on the mark sequence sample and the initial prediction result through the K mark prediction models in the large language model, and obtain the mark prediction result of each mark prediction model. For example, mark prediction model 1 can obtain mark prediction result 1, mark prediction model 2 can obtain mark prediction result 2, and mark prediction model K can obtain mark prediction result K. The server 104 can also obtain each target reasoning result obtained by the main model based on the mark sequence sample and the initial prediction result through iterative mark prediction, and each target reasoning result corresponds to each mark prediction model. The server 104 can update the shared weight parameters included in each mark prediction model based on each mark prediction result and the target reasoning result, thereby obtaining the large language model after training.

[0056] In some embodiments, the large language model processing method can also be implemented by the terminal 102 or the server 104 alone, for example, the server 104 can directly obtain the mark sequence sample from the data storage system and process the large language model based on the mark sequence sample. For another example, the terminal 102 can directly process the large language model based on the mark sequence sample.

[0057] The large language model-based instruction response method provided by the embodiments of the present application can be applied to, for example Figure 1The application environment shown. The large language model can include a main model and K (K≥1) mark prediction models, and each mark prediction model can include shared weight parameters; the terminal 102 can send a mark sequence of an input instruction to the server 104, and the input instruction can be input by a user through the terminal 102; the server 104 can infer the mark sequence of the input instruction through the main model in the large language model to obtain an initial prediction result, and iteratively predict marks based on the initial prediction result and the mark sequence of the input instruction through the K mark prediction models in the large language model to obtain the mark prediction result of each of the K mark prediction models. The server 104 can obtain the mark sequence of the corresponding response content of the input instruction based on the mark prediction result of each of the K mark prediction models through the main model. The server 104 can also return the mark sequence of the response content to the terminal 102, so that the terminal 102 restores the natural language based on the mark sequence of the response content after processing, and displays it to the user on the terminal 102.

[0058] In some embodiments, the instruction response method based on the large language model can also be implemented by the terminal 102 or the server 104 alone, for example, the server 104 can directly obtain the mark sequence of the input instruction from the data storage system and infer and process based on the mark sequence of the input instruction to obtain the mark sequence of the corresponding response content of the input instruction. For another example, the terminal 102 can be deployed with a trained large language model, and the terminal 102 can directly infer and process based on the mark sequence of the input instruction to obtain the mark sequence of the corresponding response content of the input instruction.

[0059] The terminal 102 can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0060] In an exemplary embodiment, as Figure 2 shown, a large language model processing method is provided, which is executed by a computer device, specifically by a terminal or a server, or by both a terminal and a server. In the embodiments of the present application, the method is applied to the server in Figure 1 for example, and includes the following steps 202 to 208. Wherein:

[0061] Step 202, obtaining a mark sequence sample, the mark sequence sample is obtained by inferring based on a mark sequence of an instruction sample.

[0062] The token sequence sample is sample data used for training the large language model, and the token sequence sample can be inferred based on the token sequence of the instruction sample. For example, the token sequence sample can be obtained by inferring the token sequence of the instruction sample based on a main model (MainModel) in the large language model. The token sequence sample can include a plurality of token samples arranged in sequence. Each token sample in the token sequence sample can be decoded to restore the token sample to a character string. The instruction sample can include a task description sentence input to the large language model. The instruction sample can be tokenized (Tokenization) to obtain a token sequence of the instruction sample. For example, the token sequence of the instruction sample can include a token ID (token ID) obtained by Tokenization processing of a task description sentence input by a user. Each token ID can correspond to a unique identifier in a vocabulary.

[0063] Optionally, the server can obtain a token sequence sample inferred based on a token sequence of an instruction sample. In some embodiments, a user can input an instruction sample to a large language model. The server can map the instruction sample to obtain a token sequence of the instruction sample. The token sequence can include a plurality of token samples arranged in sequence, each corresponding to a character arranged in sequence in the instruction sample. For example, the server can perform text normalization (TextNormalization) processing on the instruction sample. The text normalization processing can include case processing, whitespace specification, character encoding standardization, special symbol processing, and the like, to eliminate non-semantic differences in the text and ensure consistency of the input. The server can perform tokenization mapping on the text after the text normalization processing to split the continuous text into the smallest semantic unit that can be processed by the large language model, i.e., into tokens. For example, the server can perform word segmentation processing on the text after the text normalization processing, and map each word segment to a unique token through a vocabulary, such as a token ID. Each token in the vocabulary has a unique index number without direct semantics, and only serves as a query key for an embedding layer (Embedding Layer).

[0064] The server can perform inference based on the token sequence of the instruction sample to obtain a token sequence sample of the response. For example, the server can perform inference on the token sequence of the instruction sample based on a main model in the large language model, output a token sequence sample obtained by inference from the main model, and perform training processing on the large language model based on the token sequence sample, so that the self-distillation technology can be used to train the large language model, that is, the large language model itself is used to generate training data, and the large language model generates "soft labels" in the training process, that is, the self-data obtained by self-distillation is used to guide the training of the large language model.

[0065] For example, as shown in Figure 3 The instruction sample input by the user can include "today the weather is" with a total of 5 characters, each character can correspond to a token, and the token sequence of the instruction sample can include "t1 t2 t3 t4 t5"; the server can perform inference on the token sequence based on the main model to obtain an inference sequence, which can include "t6 t7 t8 t9 t10 t11", and the inference sequence can correspond to the response content "good, suitable for going out" with a total of 6 characters. The server can combine the token sequence and the inference sequence output by the main model in sequence to obtain a token sequence sample obtained by performing inference on the token sequence of the instruction sample, which can specifically include "t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11" a total of 11 tokens arranged in sequence.

[0066] Step 204, determining an initial token sequence from the token sequence sample, and obtaining an initial prediction result obtained by performing inference on the initial token sequence by a main model in the large language model.

[0067] The initial token sequence is a token sequence predicted by performing inference on the main model in the large language model, and the initial token sequence can include a token sequence formed by T token samples in sequence from the first token sample in the token sequence sample. At this time, the sequence length of the initial token sequence can be T, indicating that the number of token samples included in the initial token sequence is T. The main model in the large language model is responsible for the next token prediction, which is the core inference unit of the large language model. After the main model generates the first token, a plurality of token prediction models can be used to generate subsequent token drafts in series, which are verified by the main model to determine whether to accept. The initial prediction result is a prediction result obtained by performing inference on the initial token sequence by the main model, which can include the next token obtained by performing inference on the initial token sequence by the main model, or a sequence length covering the next token. The sequence length of the initial token sequence is the same as the sequence length of the inference sequence.

[0068] Exemplarily, the large language model can include a main model, and the server can determine an initial token sequence from the token sequence sample for the main model to perform inference, such as the server can determine the initial token sequence as the token sequence composed of the first T token samples in the token sequence sample, and at this time, the sequence length of the initial token sequence is T, indicating that the initial token sequence includes T token samples. The server can perform inference on the initial token sequence by the main model to obtain an initial prediction result for the initial token sequence. For example, as shown in Figure 3 the server can determine the initial token sequence as "t1 t2 t3 t4" from the token sequence sample, that is, the length of the initial token sequence is 4, which includes 4 token samples; the server can perform inference on the initial token sequence by the main model to obtain the corresponding initial prediction result, such as the initial prediction result can include the next predicted token t5', the initial prediction result can also include the initial prediction sequence "t2' t3' t4' t5'" covering the next predicted token t5', and the sequence length of the initial prediction sequence can be the same as the sequence length of the initial token sequence, both of which are 4, that is, including 4 predicted tokens arranged in sequence.

[0069] In step 206, at least one token prediction model in the large language model is used to perform iterative token prediction based on the token sequence sample and the initial prediction result to obtain the token prediction result of each token prediction model; wherein in each iteration token prediction, the target token prediction model of this iteration token prediction is used to perform token prediction on the input token sequence and the previous prediction result of this iteration token prediction to obtain the token prediction result of this iteration token prediction, and the input token sequence is obtained by offsetting the initial token sequence in the token sequence sample according to the number sequence of this iteration token prediction.

[0070] In the large language model, at least one token prediction model is also included, and the token prediction model is used to perform iterative token prediction based on the initial prediction result of the main model to obtain the corresponding token prediction result, and the token prediction result of the token prediction model can be used as a draft token to be verified by the main model to determine whether to accept, if the main model selects to accept at least part of the draft token, the main model can directly obtain the inference result based on the accepted draft token and perform the next step of inference processing. The number of token prediction models can include at least one, and each token prediction model can correspond to one iteration token prediction and output one token prediction result, that is, the number of token prediction models included in the large language model matches the number of draft tokens.

[0071] For each label prediction model, the current iteration label prediction processing can be performed, and the label prediction model can perform label prediction on the input label sequence and the previous prediction result of the current iteration label prediction to obtain the label prediction result of the current iteration label prediction. The input label sequence can be obtained by offsetting the initial label sequence in the label sequence sample according to the number sequence of the current iteration label prediction. The number sequence is used to represent the number of the iteration label prediction processing. For example, the number sequence of the current iteration label prediction is i, that is, the current iteration label prediction belongs to the i-th iteration label prediction, and the initial label sequence can be offset by i positions in the label sequence sample to obtain the input label sequence of the current iteration label prediction. The previous prediction result can include the label prediction result corresponding to the historical iteration label prediction before the current iteration label prediction, such as the label prediction result corresponding to the last iteration label prediction, or the label prediction result corresponding to the previous m iteration label predictions, that is, the previous prediction result can include the label prediction result corresponding to one or more historical iteration label predictions before the current iteration label prediction.

[0072] Optionally, the large language model can further include at least one label prediction model, and each label prediction model can be connected in series with the main model in the large language model. As shown in Figure 4 The large language model can include a main model and K label prediction models, the K label prediction models can include shared weight parameters, the main model can be connected with the first label prediction model, such as the main model can be connected with the label prediction model 1, and the K label prediction models are connected in series. The server can perform iteration label prediction based on the label sequence sample and the initial prediction result through at least one label prediction model in the large language model, each label prediction model can perform one iteration label prediction, and thus obtain the respective label prediction result of each label prediction model. The number of label prediction results matches the number of iteration label predictions, and the number of iteration label predictions matches the number of label prediction models. For example, the label prediction model includes K, K times of iteration label prediction can be performed, and K label prediction results are obtained, and each label prediction model can output one label prediction result.

[0073] In each iteration of the label prediction, the target label prediction model of the current iteration of the label prediction can be implemented to perform label prediction, e.g., the target label prediction model can perform label prediction on the input label sequence of the current iteration of the label prediction and the previous prediction result to obtain the label prediction result of the current iteration of the label prediction. The input label sequence of the current iteration of the label prediction can be obtained by shifting the initial label sequence in the label sequence sample according to the sequence number of the current iteration of the label prediction. For example, if the sequence number of the current iteration of the label prediction is i, the input label sequence of the current iteration of the label prediction can be obtained by shifting the initial label sequence in the label sequence sample by i label samples. The previous prediction result of the current iteration of the label prediction can include the label prediction results obtained by the historical iterations of the label prediction before the current iteration of the label prediction, e.g., the previous prediction result can include m label prediction results obtained by m historical iterations of the label prediction before the current iteration of the label prediction.

[0074] In some embodiments, the previous prediction result can include the label prediction result of the previous iteration of the label prediction, e.g., Figure 4 As shown, for the label prediction model 1, the label prediction model 1 can perform label prediction according to the initial prediction result obtained by the main model inference and the input label sequence of the first iteration of the label prediction, and obtain the label prediction result 1, the input label sequence of the first iteration of the label prediction can be obtained by shifting the initial label sequence in the label sequence sample by 1 label sample; for the label prediction model 2, the label prediction model 2 can take the label prediction result 1 of the label prediction model 1 as the previous prediction result of the current iteration of the label prediction, the input label sequence of the current iteration of the label prediction can be obtained by shifting the initial label sequence in the label sequence sample by 2 label samples, and the label prediction model 2 can perform label prediction based on the input label sequence of the current iteration of the label prediction and the label prediction result 1 to obtain the label prediction result 2. After the iterations of the label prediction by the label prediction models in sequence, K label prediction results can be obtained.

[0075] In some embodiments, as shown in FIG. 6, the label prediction model 1 can perform label prediction according to the initial prediction result obtained by the main model inference and the input label sequence of the first iteration of the label prediction, and obtain the label prediction result 1, the input label sequence of the first iteration of the label prediction can be obtained by shifting the initial label sequence in the label sequence sample by 1 label sample; the label prediction model 2 can take the label prediction result 1 of the label prediction model 1 as the previous prediction result of the current iteration of the label prediction, the input label sequence of the current iteration of the label prediction can be obtained by shifting the initial label sequence in the label sequence sample by 2 label samples, and the label prediction model 2 can perform label prediction based on the input label sequence of the current iteration of the label prediction and the label prediction result 1 to obtain the label prediction result 2. Figure 5As shown, the mark sequence sample can include a total of 11 sequentially arranged mark samples, i.e., "t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 t11", and for the initial mark sequence, it can be "t1 t2 t3 t4"; for the input mark sequence of the first iteration mark prediction, it can be input mark sequence 1, specifically "t2 t3 t4 t5", which can be obtained by offsetting 1 position in the mark sequence sample based on the initial mark sequence; for the input mark sequence of the second iteration mark prediction, it can be input mark sequence 2, specifically "t3 t4 t5 t6", which can be obtained by offsetting 2 positions in the mark sequence sample based on the initial mark sequence; for the input mark sequence of the third iteration mark prediction, it can be input mark sequence 3, specifically "t4 t5 t6 t7", which can be obtained by offsetting 3 positions in the mark sequence sample based on the initial mark sequence.

[0076] In step 208, the shared weight parameters included in each mark prediction model are updated according to the mark prediction results of each mark prediction model and the target inference result, to obtain a trained large language model; the target inference result is obtained by the main model based on the mark sequence sample and the initial prediction result; the trained large language model is used to infer the mark sequence of the input instruction to output the mark sequence of the corresponding response content of the input instruction.

[0077] In each mark prediction model, there can be shared weight parameters, for example, the mark prediction model can include an MTP module, and the shared weight parameters can include parameters in the MTP header in the MTP module. The MTP header can include a lightweight output layer attached to the main model, which is used to iteratively predict the initial prediction result of the main model. The target inference result is obtained by the main model based on the mark sequence sample and the initial prediction result, i.e., the main model can iteratively predict at least once based on the mark sequence sample and the initial prediction result, and obtain each target inference result. The obtained target inference result can be used as a reference result of the mark prediction model, and the shared weight parameters included in the mark prediction model can be updated based on the target inference result, so that the mark prediction result output by each mark prediction model can match the mark prediction result output by the main model. The input instruction can be an instruction input by a user to the large language model, and the response content can be content fed back by the large language model after inferring the input instruction. For example, the input instruction can include "1+1", and the response content can include "2".

[0078] Exemplarily, the server can also perform iterative token prediction based on the token sequence sample and the initial prediction result by the main model to obtain a corresponding target reasoning result. In some embodiments, the number of times of performing iterative token prediction by the main model based on the token sequence sample and the initial prediction result can match the number of times of iterative token prediction of the token prediction model. For example, the token prediction model can include K, and the number of times of performing iterative token prediction based on the token prediction model can be K, and the main model can also perform K times of iterative token prediction based on the token sequence sample and the initial prediction result to obtain K target reasoning results. The server can update the shared weight parameters included in each token prediction model according to the token prediction result of each token prediction model and the target reasoning result, to obtain the large language model after training.

[0079] In some embodiments, each token prediction model can have a corresponding token prediction result and a target reasoning result with the same number of times. For example, for the i-th token prediction model, which is used to perform the i-th iterative token prediction, the i-th token prediction result can be output. The server can determine the target reasoning result obtained by the main model based on the token sequence sample and the initial prediction result, and the server can take the i-th target reasoning result as the target reasoning result associated with the i-th token prediction model, so that for each token prediction model, a corresponding result pair can be constructed. For example, for the i-th token prediction model, a result pair i (i-th token prediction result, i-th target reasoning result) can be constructed, the i-th token prediction result is the token prediction result output by the i-th token prediction model, and the i-th target reasoning result is the target reasoning result obtained by the main model based on the token sequence sample and the initial prediction result by the i-th time of iterative token prediction. The server can update the shared weight parameters included in the token prediction model based on each result pair to obtain the large language model after training, for example, the server can construct a loss based on K structure pairs (iterative token prediction includes K times), and update the shared weight parameters included in the token prediction model through the loss to obtain the large language model after training.

[0080] In some embodiments, as shown in Figure 6 The server can construct a result pair 1 based on the token prediction result 1 output by the token prediction model 1 and the target reasoning result 1 obtained by the main model by the first time of iterative token prediction. The server can construct a result pair 2 based on the token prediction result 2 output by the token prediction model 2 and the target reasoning result 2 obtained by the main model by the second time of iterative token prediction. For K times of iterative token prediction of the K token prediction models, the server can construct K result pairs, and update the shared weight parameters included in each token prediction model based on the K result pairs, to obtain the large language model after training.

[0081] In some embodiments, when the shared weight parameters included in each of the mark prediction models are updated according to the respective mark prediction results of the at least one mark prediction model and the target inference result, the main model can remain fixed, i.e., not updated, and the server can update the shared weight parameters included in each of the mark prediction models, thereby updating each of the mark prediction models to obtain each updated mark prediction model. Based on each updated mark prediction model and the main model, the server can obtain the trained large language model. In the case where the training does not satisfy the end condition, such as when the number of training times does not reach the number threshold or the loss function does not converge, the server can perform the next training based on the trained large language model, such as obtaining the next mark sequence sample and performing the next training on the trained large language model based on the next mark sequence sample, until the end condition is satisfied to end the training and obtain the final trained large language model.

[0082] In the above large language model processing method, the initial mark sequence is determined from the mark sequence sample obtained by inference based on the instruction sample, and the initial prediction result obtained by the main model in the large language model performing inference on the initial mark sequence. The at least one mark prediction model in the large language model iteratively performs mark prediction based on the mark sequence sample and the initial prediction result to obtain the respective mark prediction results of the at least one mark prediction model. According to the respective mark prediction results of the at least one mark prediction model and the target inference result obtained by the main model iteratively performing mark prediction based on the mark sequence sample and the initial prediction result, the shared weight parameters included in each of the mark prediction models are updated to obtain the trained large language model. In the processing of the large language model, according to the respective mark prediction results of the at least one mark prediction model and the target inference result obtained by the main model iteratively performing mark prediction based on the mark sequence sample and the initial prediction result, the shared weight parameters included in each of the mark prediction models are updated. Under the premise that the mark prediction results of the mark prediction models match the inference results of the main model, mark prediction is performed by reusing the shared weight parameters, which can reduce redundant computational complexity and improve the efficiency of large language model inference processing.

[0083] In one exemplary embodiment, as shown in Figure 7 The shared weight parameter updating process, i.e., updating the shared weight parameters included in each of the mark prediction models according to the respective mark prediction results of the at least one mark prediction model and the target inference result, to obtain the trained large language model, includes:

[0084] Step 702, obtaining the prediction result difference of each of the at least one mark prediction model according to the respective mark prediction result of the at least one mark prediction model and the target inference result.

[0085] The prediction result difference can be determined by comparing the label prediction result output by the label prediction model and the target inference result associated with the label prediction model. The target inference result associated with the label prediction model can be a target inference result whose iteration number of iterative label prediction is the same as the iteration number of iterative label prediction performed by the label prediction model. For example, if the iteration number of iterative label prediction performed by the label prediction model is i, the target inference result associated with the label prediction model can be a target inference result obtained by performing the i-th iteration of label prediction by the main model.

[0086] For each label prediction model, the server can obtain a respective prediction result difference based on the respective label prediction result and the target inference result of the label prediction model. In some embodiments, the server can traverse the respective label prediction result and the target inference result of each label prediction model, compare the label prediction result of each label prediction model with the associated target inference result, and obtain the prediction result difference of each label prediction model.

[0087] At step 704, a label prediction loss is obtained based on the respective prediction result difference of each label prediction model.

[0088] The label prediction loss can represent the loss of iterative label prediction by the at least one label prediction model in the large language model. Alternatively, the server can obtain the label prediction loss based on the respective prediction result difference of each label prediction model. In some embodiments, the server can construct a respective sub-loss of each label prediction model based on the respective prediction result difference of each label prediction model, and obtain the label prediction loss by summing the respective sub-loss of each label prediction model. In some embodiments, the server can weight and sum the respective sub-loss of each label prediction model to obtain the label prediction loss.

[0089] At step 706, the shared weight parameters included in each label prediction model are updated based on the label prediction loss, and a trained large language model is obtained.

[0090] For example, the server can determine the loss gradient of the label prediction loss, and update the shared weight parameters based on the loss gradient to obtain the trained large language model.

[0091] In this embodiment, the server can obtain the prediction result difference of each label prediction model according to the label prediction result of each label prediction model and the target inference result, and then obtain the label prediction loss, and update the shared weight parameter based on the label prediction loss, so as to train the label prediction result of each label prediction model in the large language model to align with the target inference result of the main model, so that the label prediction result of each label prediction model has a higher acceptance rate in the inference process, thereby improving the efficiency of the large language model inference processing.

[0092] In an exemplary embodiment, the label prediction result includes a label prediction distribution, and the target inference result includes a target inference distribution; obtaining the prediction result difference of each label prediction model according to the label prediction result of each label prediction model and the target inference result includes: obtaining the distribution difference of each label prediction model according to the label prediction distribution of each label prediction model and the target inference distribution; obtaining the label prediction loss according to the prediction result difference of each label prediction model includes: obtaining the label prediction loss according to the distribution difference of each label prediction model.

[0093] The label prediction result includes a label prediction distribution, and the label prediction distribution is used to represent the probability of assigning all possible labels in the entire vocabulary table when the label prediction model performs this iteration of label prediction. Based on the label prediction distribution, the next predicted label corresponding to the label prediction model performing this iteration of label prediction can be determined. The target inference result includes a target inference distribution, and the target inference distribution is used to represent the probability of assigning all possible labels in the entire vocabulary table when the main model performs this iteration of label prediction. Based on the target inference distribution, the next predicted label corresponding to the main model performing this iteration of label prediction can be determined. The distribution difference can be obtained by comparing the label prediction distribution and the target inference distribution, such as based on the KL divergence (Kullback-Leibler Divergence) algorithm to obtain the distribution difference between the label prediction distribution and the target inference distribution.

[0094] For each label prediction model, the corresponding label prediction distribution and the associated target inference distribution can be included. The server can compare the label prediction distribution and the target inference distribution of each label prediction model to obtain the distribution difference of each label prediction model. The server can obtain the label prediction loss based on the distribution difference of each label prediction model.

[0095] In the embodiment, the server can obtain the respective distribution differences of the respective label prediction models according to the respective label prediction distributions of the respective label prediction models and the target inference distribution, and then obtain a label prediction loss, and update the shared weight parameters based on the label prediction loss, so as to train the respective label prediction models in the large language model to align the respective label prediction distributions of the respective label prediction models with the target inference distribution of the main model, so that the label prediction results of the respective label prediction models have a higher acceptance rate in the inference process, thereby facilitating the improvement of the inference processing efficiency of the large language model.

[0096] In an example embodiment, the label prediction result includes a predicted label, and the target inference result includes a target label; the respective prediction result difference of the at least one label prediction model is obtained according to the respective label prediction result of the at least one label prediction model and the target inference result, including: obtaining the respective sub-loss of the at least one label prediction model according to the respective predicted label of the at least one label prediction model and the target label; the label prediction loss is obtained according to the respective prediction result difference of the at least one label prediction model, including: obtaining the label prediction loss according to the respective sub-loss of the at least one label prediction model.

[0097] The predicted label can include a predicted next label corresponding to the label prediction model performing this iteration of label prediction, and the target label can include a predicted next label corresponding to the main model performing this iteration of label prediction. The sub-loss is used to represent the loss of each label prediction model performing the corresponding iteration of label prediction.

[0098] Optionally, for each label prediction model, the corresponding predicted label and the associated target label can be included, and the server can compare the respective predicted label and the target label of each label prediction model to obtain the respective sub-loss of each label prediction model. For example, the server can calculate a cross-entropy loss based on the respective predicted label and the target label of each label prediction model to obtain the respective sub-loss of each label prediction model. The server can obtain the label prediction loss based on the respective sub-loss of each label prediction model. For example, the server can perform weighted summation on the respective sub-loss of each label prediction model to obtain the label prediction loss.

[0099] In the embodiment, the server can obtain the respective sub-loss of the respective label prediction models according to the respective predicted label and the target label of the respective label prediction models, and then obtain a label prediction loss, and update the shared weight parameters based on the label prediction loss, so as to train the respective label prediction models in the large language model to align the respective predicted label of the respective label prediction models with the target label of the main model, so that the label prediction results of the respective label prediction models have a higher acceptance rate in the inference process, thereby facilitating the improvement of the inference processing efficiency of the large language model.

[0100] In an example embodiment, the label prediction loss is obtained according to the respective prediction result difference of the at least one label prediction model, including: determining a respective prediction weight of the at least one label prediction model, the value of the prediction weight being negatively correlated with the serial number of the number of times of iterative label prediction performed by the label prediction model; and obtaining the label prediction loss according to the respective prediction result difference of the at least one label prediction model and the prediction weight.

[0101] The prediction weight is used to adjust the importance of each label prediction model, and the value of the prediction weight is negatively correlated with the serial number of the number of times of iterative label prediction performed by the label prediction model, that is, the greater the value of the serial number of the number of times of iterative label prediction performed by the label prediction model, the farther the distance of the iterative label prediction performed by the label prediction model, and the smaller the value of the prediction weight.

[0102] Optionally, for each label prediction model, the server can determine a respective prediction weight of the respective label prediction model, the value of the prediction weight being negatively correlated with the serial number of the number of times of iterative label prediction performed by the label prediction model. The server can obtain the label prediction loss according to the respective prediction result difference of the respective label prediction model and the prediction weight. For example, the server can perform weighted fusion on the respective prediction result difference of the respective label prediction model according to the respective prediction weight of the respective label prediction model to obtain the label prediction loss.

[0103] In this embodiment, the label prediction loss is obtained according to the respective prediction result difference of the respective label prediction model and the prediction weight, and the value of the prediction weight is negatively correlated with the serial number of the number of times of iterative label prediction performed by the label prediction model, so that the prediction distance is dynamically adjusted according to the prediction weight, so that the large language model can focus on short-term prediction accuracy and retain attention to long-term prediction, which is beneficial to ensure the reasoning accuracy of the large language model.

[0104] In an example embodiment, the label prediction loss includes a respective sub-loss of each iteration of label prediction; and the shared weight parameter included in each label prediction model is updated based on the label prediction loss to obtain the trained large language model, including: determining a loss gradient corresponding to each sub-loss in the label prediction loss; determining a historical gradient corresponding to each sub-loss of a previous label prediction, the previous label prediction being an iterative label prediction performed before the iteration of label prediction corresponding to the sub-loss; obtaining a cumulative gradient of the sub-loss according to the loss gradient and the historical gradient; obtaining a target gradient of the label prediction loss based on the cumulative gradient of each sub-loss in the label prediction loss; and updating the shared weight parameter included in each label prediction model based on the target gradient to obtain the trained large language model.

[0105] The label prediction loss includes respective sub-losses of each iteration of label prediction, i.e., the label prediction loss can be obtained based on respective sub-losses of each iteration of label prediction. The loss gradient is a gradient directly calculated for a sub-loss. The historical gradient is a gradient calculated for a sub-loss of a previous label prediction, which is an iteration of label prediction performed before the iteration of label prediction corresponding to the sub-loss. For example, for the i-th sub-loss Li, which can be a loss obtained for the i-th iteration of label prediction, the server can directly perform gradient calculation based on Li to obtain a loss gradient of the i-th sub-loss Li. The server can determine sub-losses of previous label predictions, which can include the 1st iteration of label prediction to the (i-1)th iteration of label prediction. For each previous label prediction, there can be a corresponding sub-loss, and for each sub-loss of a previous label prediction, a respective historical gradient can be calculated. For example, gradient calculation can be performed based on each sub-loss of a previous label prediction to obtain a respective historical gradient corresponding to the sub-loss of the previous label prediction. The accumulated gradient can be obtained by summing the loss gradient directly determined for a sub-loss and the historical gradients of previous label predictions. For example, for the i-th sub-loss obtained for the i-th iteration of label prediction, the accumulated gradient of the i-th sub-loss can be obtained according to the loss gradient of the sub-loss and i-1 historical gradients of i-1 previous label predictions. The target gradient can be a gradient of the label prediction loss, which can be obtained by summing respective accumulated gradients of sub-losses.

[0106] For example, the label prediction loss includes respective sub-losses of each iteration of label prediction. For example, the label prediction model includes K, and the iteration of label prediction includes K times. The label prediction loss can include K sub-losses of K iterations of label prediction. The server can traverse each sub-loss in the label prediction loss. For a traversed sub-loss, the server can determine a loss gradient of the sub-loss, which can be directly calculated based on the sub-loss. The server can determine previous label predictions performed before the iteration of label prediction corresponding to the sub-loss. For example, the iteration of label prediction corresponding to the sub-loss is the i-th iteration of label prediction, and the previous label predictions can include i-1 iterations of label prediction performed before the i-th iteration of label prediction. Each previous label prediction can have a corresponding sub-loss. The server can perform gradient calculation for each sub-loss of each previous label prediction to obtain a respective historical gradient corresponding to the sub-loss of the previous label prediction. For example, for the (i-1)th previous label prediction, the server can perform gradient calculation based on the sub-loss of the (i-1)th previous label prediction to obtain a historical gradient corresponding to the sub-loss of the (i-1)th previous label prediction. After traversing each sub-loss of each previous label prediction, i-1 historical gradients can be obtained.

[0107] The server can obtain the accumulated gradient for the sub-loss based on the loss gradient and the historical gradient. For example, the server can sum the loss gradient and the historical gradient to obtain the accumulated gradient for the sub-loss. In some embodiments, the number of gradients of the loss gradient and the historical gradient matches the sequence number of the iteration label prediction corresponding to the sub-loss. For example, if the sequence number of the iteration label prediction corresponding to the sub-loss is i, i.e., the sub-loss is the i-th iteration label prediction, the loss gradient of the sub-loss includes 1, the historical gradient of the sub-loss includes i-1, and the number of gradients of the loss gradient and the historical gradient is also i, which is the same as the sequence number of the iteration label prediction corresponding to the sub-loss. The server can traverse each sub-loss in the label prediction loss to obtain the accumulated gradient of each sub-loss. The server can obtain the target gradient of the label prediction loss based on the accumulated gradient of each sub-loss in the label prediction loss. For example, the server can sum the accumulated gradient of each sub-loss in the label prediction loss to obtain the target gradient of the label prediction loss. In some embodiments, the server can weight-sum the accumulated gradient of each sub-loss in the label prediction loss according to the prediction weight to obtain the target gradient of the label prediction loss. The server can update the shared weight parameters included in each label prediction model based on the target gradient, such as calculating the parameter adjustment amount based on the target gradient and the learning rate, and updating the shared weight parameters included in each label prediction model through the parameter adjustment amount, thereby obtaining the trained large language model.

[0108] In this embodiment, the server can use the recursive gradient propagation mechanism to enable the gradient to recursively adjust each historical prediction step, effectively improve the processing capability of long-term dependence, and thus improve the inference accuracy of the large language model.

[0109] In one exemplary embodiment, at least one label prediction model in the large language model is used to sequentially perform iteration label prediction based on the label sequence sample and the initial prediction result to obtain the respective label prediction result of the at least one label prediction model, including: sequentially performing iteration label prediction based on the label sequence sample, the initial prediction result, and the initial hidden state obtained by the main model performing inference on the initial label sequence, to obtain the respective label prediction result of the at least one label prediction model; and performing label prediction on the input label sequence and the previous prediction result of the current iteration label prediction by the target label prediction model of the current iteration label prediction to obtain the label prediction result of the current iteration label prediction, including: performing label prediction on the input label sequence, the previous prediction result, and the previous hidden state of the current iteration label prediction by the target label prediction model of the current iteration label prediction to obtain the label prediction result of the current iteration label prediction.

[0110] The hidden state is an intermediate representation inside the model (main model or token prediction model), which can include an intermediate feature vector calculated by a Transformer block in the model, and carries the context semantic information of the input sequence. The initial hidden state is an intermediate representation inside the main model when the main model reasons for the initial token sequence. The previous hidden state can include the intermediate representation of the historical iteration token prediction before the current iteration token prediction, which can include the intermediate representation of the last iteration token prediction, or the intermediate representation of the last n iteration token predictions, that is, the previous hidden state can include the intermediate representation of one or more historical iteration token predictions before the current iteration token prediction.

[0111] Exemplarily, the server can obtain the initial hidden state obtained by the main model when reasoning for the initial token sequence, and perform iteration token prediction based on the token sequence sample, the initial prediction result and the initial hidden state through at least one token prediction model in the large language model to obtain the token prediction result of each token prediction model. In each iteration token prediction, for the current iteration token prediction processing, the server can determine the input token sequence of the current iteration token prediction, the previous prediction result and the previous hidden state, and perform token prediction on the input token sequence of the current iteration token prediction, the previous prediction result and the previous hidden state through the target token prediction model of the current iteration token prediction to obtain the token prediction result of the current iteration token prediction. For example, the server can perform token prediction on the input token sequence of the current iteration token prediction, the token prediction result of the last iteration token prediction (previous prediction result) and the hidden state in the last iteration token prediction (previous hidden state) through the target token prediction model of the current iteration token prediction to obtain the token prediction result of the current iteration token prediction.

[0112] In this embodiment, the previous hidden state is also introduced when performing token prediction, and the token prediction is performed by comprehensively considering the input token sequence of the current iteration token prediction, the previous prediction result and the previous hidden state, which can improve the accuracy of token prediction, and the token prediction result of each token prediction model has a higher acceptance rate in the reasoning process, thereby facilitating the improvement of the efficiency of the large language model reasoning process.

[0113] In an example embodiment, the target mark prediction model for the current iteration mark prediction is used to perform mark prediction on the input mark sequence, the previous prediction result and the previous hidden state of the current iteration mark prediction, to obtain the mark prediction result of the current iteration mark prediction, including: determining the input mark sequence, the previous prediction result and the previous hidden state of the current iteration mark prediction; using the target mark prediction model for the current iteration mark prediction to perform mark prediction on the input mark sequence, the previous prediction result and the previous hidden state, to obtain the target hidden state; and performing mapping processing on the target hidden state to obtain the mark prediction result of the current iteration mark prediction.

[0114] The target hidden state is an intermediate representation directly output by the target mark prediction model in performing the current iteration mark prediction, and the mapping processing based on the target hidden state can obtain the mark prediction result of the current iteration mark prediction.

[0115] For example, in each iteration mark prediction, for the current iteration mark prediction processing, the server can determine the input mark sequence, the previous prediction result and the previous hidden state of the current iteration mark prediction, and the target mark prediction model for the current iteration mark prediction. The server can perform mark prediction on the input mark sequence, the previous prediction result and the previous hidden state based on the target mark prediction model, to obtain the target hidden state of the current iteration mark prediction. The server can perform mapping processing on the target hidden state, such as performing mapping processing on the target hidden state through a mapping model, to obtain the mark prediction result of the current iteration mark prediction. In some embodiments, the mark prediction model can include an MTP module, and for the current iteration mark prediction, the server can perform mark prediction on the input mark sequence, the previous prediction result and the previous hidden state of the current iteration mark prediction based on the MTP header in the target MTP module of the current iteration mark prediction, to obtain the target hidden state, and the weight parameters in the MTP header are shared weight parameters of each MTP module; the server can perform mapping processing on the target hidden state through the mapping structure in the target MTP module, such as performing mapping processing on the target hidden state based on the RNN (Recurrent Neural Network) in the target MTP module, to obtain the mark prediction result of the current iteration mark prediction.

[0116] In this embodiment, the server can perform mark prediction on the input mark sequence, the previous prediction result and the previous hidden state of the current iteration mark prediction to obtain the target hidden state of the current iteration mark prediction, and perform mapping processing on the target hidden state to obtain the mark prediction result of the current iteration mark prediction, so as to pass the hidden state in the iteration mark prediction, which can improve the accuracy of mark prediction.

[0117] In an example embodiment, the target label prediction model for the current iteration of label prediction is used to perform label prediction on the input label sequence for the current iteration of label prediction, the previous prediction results and the previous hidden state to obtain the label prediction results for the current iteration of label prediction, including: determining the input label sequence for the current iteration of label prediction, the previous hidden state and at least one previous prediction result, the previous prediction result being the label prediction result obtained by performing the iteration of label prediction before the current iteration of label prediction; using the target label prediction model for the current iteration of label prediction to perform label prediction on the input label sequence, the previous hidden state and the at least one previous prediction result to obtain the label prediction results for the current iteration of label prediction.

[0118] The previous prediction result can include at least one, for example, the previous prediction result can include the label prediction results corresponding to the previous m iterations of label prediction. Alternatively, in each iteration of label prediction, for the current iteration of label prediction, the server can determine the input label sequence for the current iteration of label prediction, the previous hidden state and at least one previous prediction result, the previous prediction result including at least one, for example, for the i-th iteration of label prediction, the previous prediction result can include the label prediction results of the previous i-1 iterations of label prediction, i.e. the previous prediction result can include i-1. The server can use the target label prediction model for the current iteration of label prediction to perform label prediction on the input label sequence, the previous hidden state and the at least one previous prediction result to obtain the label prediction results for the current iteration of label prediction.

[0119] In the example embodiment, the server can perform label prediction by comprehensively considering the input label sequence for the current iteration of label prediction, the previous hidden state and the at least one previous prediction result, can perform label prediction by comprehensively considering the historical previous prediction results, and can improve the accuracy of label prediction.

[0120] In an example embodiment, the token prediction model includes at least two; through at least one token prediction model in the large language model, the initial prediction result is obtained by sequentially performing iterative token prediction based on the token sequence sample, including: based on the initial token sequence, the first offset is performed in the token sequence sample to obtain the input token sequence of the first iteration token prediction, and the initial prediction result is determined as the previous prediction result of the first iteration token prediction; through the first token prediction model, the token prediction is performed on the input token sequence of the first iteration token prediction and the previous prediction result to obtain the token prediction result of the first iteration token prediction; from the second iteration token prediction, according to the number sequence of the current iteration token prediction, the initial token sequence is offset in the token sequence sample to obtain the input token sequence of the current iteration token prediction, and the token prediction result of the last iteration token prediction is determined as the previous prediction result of the current iteration token prediction; through the target token prediction model of the current iteration token prediction, the token prediction is performed on the input token sequence of the current iteration token prediction and the previous prediction result to obtain the token prediction result of the current iteration token prediction.

[0121] When the token prediction model includes at least two, at least two iteration token predictions can be performed. For example, for the first iteration token prediction, the server can perform the first offset in the token sequence sample based on the initial token sequence, such as offsetting the initial token sequence by 1 token sample in the token sequence sample to obtain the input token sequence of the first iteration token prediction. The server can determine the initial prediction result as the previous prediction result of the first iteration token prediction. The server can perform token prediction on the input token sequence of the first iteration token prediction and the previous prediction result through the first token prediction model connected with the main model to obtain the token prediction result of the first iteration token prediction. From the second iteration token prediction, the server can offset the initial token sequence in the token sequence sample according to the number sequence of the current iteration token prediction, such as offsetting the initial token sequence by i token samples in the token sequence sample to obtain the input token sequence of the i iteration token prediction. The server can determine the token prediction result of the last iteration token prediction as the previous prediction result of the current iteration token prediction. The server can perform token prediction on the input token sequence of the current iteration token prediction and the previous prediction result through the target token prediction model of the current iteration token prediction to obtain the token prediction result of the current iteration token prediction.

[0122] In some embodiments, the label prediction model includes one, and only one iteration of label prediction is performed based on the label sequence sample and the initial prediction result. In a specific application, the server can offset the initial label sequence by 1 label sample in the label sequence sample to obtain an input label sequence of the iteration of label prediction, and determine the initial prediction result as a previous prediction result of the iteration of label prediction. The server can perform label prediction on the input label sequence of the iteration of label prediction and the previous prediction result through the label prediction model to obtain a label prediction result of the iteration of label prediction.

[0123] In this embodiment, the server can update the shared weight parameters included in each label prediction model according to the label prediction result of each label prediction model and the target inference result of the iteration of label prediction of the main model based on the label sequence sample and the initial prediction result. By reusing the shared weight parameters for label prediction under the premise that the label prediction result of each label prediction model matches the inference result of the main model, the redundant calculation amount can be reduced, thereby improving the efficiency of the inference processing of the large language model.

[0124] In one exemplary embodiment, as shown in Figure 8 A large language model-based instruction response method is provided, which is executed by a computer device, specifically by a terminal or a server, or by a terminal and a server together. In the embodiments of the present application, the method is applied to the server in Figure 1 for example, and includes the following steps 802 to 808. Among them:

[0125] Step 802, obtaining the token sequence of the input instruction, determining the large language model, the large language model is obtained based on the large language model processing method.

[0126] The large language model is obtained based on the large language model processing method described above. The large language model can perform inference on the token sequence of the input instruction to output the token sequence of the corresponding response content of the input instruction.

[0127] Optionally, the server can determine the input instruction of the user and obtain the token sequence of the input instruction. In some embodiments, the server can obtain the input instruction sent by the user through the terminal, and perform token mapping processing on the input instruction to obtain the token sequence of the input instruction. In some embodiments, the user can edit the input instruction through the terminal, the terminal can perform token mapping processing on the input instruction, and send the obtained token sequence of the input instruction to the server, and the server can directly receive the token sequence of the input instruction sent by the terminal. The server can determine the large language model, which can be trained in advance through the large language model processing method and deployed in the server.

[0128] At step 804, an initial prediction result is obtained by the main model in the large language model based on the token sequence of the input instruction.

[0129] In some embodiments, the server can obtain the initial prediction result by inputting the token sequence of the input instruction into the main model in the large language model, and outputting the initial prediction result from the main model. For example, the initial prediction result can include a next token predicted by the main model.

[0130] At step 806, at least one token prediction result is obtained by at least one token prediction model in the large language model based on the initial prediction result and the token sequence of the input instruction.

[0131] In some embodiments, the server can obtain the at least one token prediction result by inputting the initial prediction result and the token sequence of the input instruction into the at least one token prediction model in the large language model, and outputting the at least one token prediction result from the at least one token prediction model. In each iteration of the token prediction, the server can determine an input token sequence and a previous prediction result for the iteration of the token prediction, and obtain the input token sequence by concatenating the previous prediction result and a token sequence based on a sequence number of the iteration of the token prediction. The sequence length of the input token sequence in each iteration of the token prediction can remain the same.

[0132] At step 808, a token sequence of a corresponding response content of the input instruction is obtained by the main model based on the at least one token prediction result.

[0133] In some embodiments, the server can obtain the token sequence of the corresponding response content of the input instruction by inputting the at least one token prediction result into the main model, and outputting the token sequence of the corresponding response content of the input instruction from the main model. In some embodiments, the server can verify the at least one token prediction result, and obtain a verification result for each token prediction result. The server can determine an accepted token prediction result from the at least one token prediction result based on the verification result for each token prediction result. The server can determine the accepted token prediction result as a target token for the inference, and perform a next inference based on the target token until the inference is completed. The server can sort the token sequence according to a generation order to obtain the token sequence of the corresponding response content of the input instruction.

[0134] In the instruction response method based on the large language model, the main model in the large language model is used to infer the token sequence of the input instruction to obtain an initial prediction result, at least one token prediction model in the large language model is used to iteratively predict tokens based on the initial prediction result and the token sequence of the input instruction to obtain token prediction results of the at least one token prediction model, and the main model is used to obtain the token sequence of the response content corresponding to the input instruction based on the token prediction results of the at least one token prediction model. In the instruction response process based on the large language model, the token prediction is performed by sharing the weight parameters, the redundant calculation amount is reduced, and the efficiency of the inference process of the large language model is improved.

[0135] In one exemplary embodiment, the at least one token prediction model in the large language model is used to iteratively predict tokens based on the initial prediction result and the token sequence of the input instruction to obtain token prediction results of the at least one token prediction model, including: the at least one token prediction model in the large language model is used to iteratively predict tokens based on the initial prediction result, the token sequence of the input instruction, and a first token set to obtain token prediction results of the at least one token prediction model; and the main model is used to obtain the token sequence of the response content corresponding to the input instruction based on the token prediction results of the at least one token prediction model, including: the main model is used to verify the token prediction results of the at least one token prediction model based on a second token set to obtain verification results of the at least one token prediction model; the first token set is a subset of the second token set; and the token sequence of the response content corresponding to the input instruction is obtained based on the verification results of the at least one token prediction model.

[0136] The token set can include candidate tokens that can be selected when performing token prediction. The first token set is a subset of the second token set, that is, the number of candidate tokens included in the first token set is not more than the number of candidate tokens included in the second token set.

[0137] Exemplarily, in the iterative token prediction by the at least one token prediction model in the large language model, the server can perform the iterative token prediction by each token prediction model based on the initial prediction result, the token sequence of the input instruction, and the first token set to obtain the respective token prediction result of the at least one token prediction model. In some embodiments, the server can perform the iterative token prediction by each token prediction model based on the initial prediction result, the token sequence of the input instruction, and each candidate token included in the first token set, that is, each time the iterative token prediction is performed, the token prediction is performed from the candidate token in the first token set. When the token sequence of the corresponding response content of the input instruction is obtained by the main model based on the respective token prediction result of the at least one token prediction model, the server can perform the verification by the main model based on the second token set for the respective token prediction result of the at least one token prediction model. In some embodiments, the server can perform the verification by the main model for the respective token prediction result of the at least one token prediction model, and the verification can be based on the candidate token in the second token set to obtain the respective verification result of each token prediction model. The server can obtain the token sequence of the corresponding response content of the input instruction based on the respective verification result of each token prediction model.

[0138] In the embodiment, each token prediction model can perform token prediction based on the subset in the second token set, and the main model can perform verification based on the second token set, which is beneficial to improve the token prediction accuracy of the token prediction model and ensure the reliability of the main model, thereby ensuring the accuracy of the inference processing of the large language model.

[0139] The present application also provides an application scenario applying the above-mentioned large language model processing method and instruction response method based on the large language model. Specifically, in the application scenario, the token prediction model is an MTP module, the MTP head in each MTP module shares the weight parameter, and when the MTP module is updated, the MTP head can be updated; the i-th iterative token prediction is performed by the token prediction model, specifically, the i-th token prediction (or simply prediction) is performed by the MTP module; and the application of the large language model processing method and the instruction response method based on the large language model in the application scenario is as follows:

[0140] Traditional large language models usually use a self-recursive step-by-step generation method for inference. In this process, the model generates one token at a time, and then uses it as the input for the generation of the next token. This step-by-step generation method can ensure the correctness of the generated token sequence, but it can slow down the inference speed. The existing self-recursive step-by-step generation method has some obvious shortcomings. First, because only one token is generated at a time, the inference process is very slow. Each time a token is generated, the model needs to repeat the calculation process, resulting in large consumption of computing resources and increasing memory overhead. In addition, although the multi-token prediction (MTP) method can accelerate inference to some extent, the existing MTP method still uses multiple independent modules for prediction, and does not fully utilize the recursive multi-step prediction ability learned during training in the inference stage.

[0141] MTP technology (Multi-Token Prediction) is a core technology for accelerating large language model inference and training by generating multiple tokens in parallel. Specifically, at least one token prediction model in the large language model is used as a lightweight draft model to generate multiple candidate tokens in advance, which are then verified by the main model (Main Model) in batches, achieving the output of multiple tokens in a single forward propagation. For example, after the main model (multi-layer Transformer) generates the first token (initial prediction result), multiple independent MTP modules (i.e., token prediction models) generate subsequent token drafts (token prediction results) in series, and the main model decides whether to accept after verification. For another example, K token prediction models can generate K token sequences in parallel, and the main model verifies all candidate token sequences in a single forward pass. If all are accepted, K times acceleration is achieved. In the MTP technology architecture design, after the main model extracts the context features, multiple independent token prediction models predict the subsequent tokens in parallel. Each token prediction model can reuse the parameters of the embedding layer (Embedding) and output head (Output Head) in the main model to reduce the amount of calculation. In each token prediction model, except for the first token prediction model, the input of each token prediction model depends on the output representation of the main model or the previous token prediction model. For example, the input of the kth token prediction model needs to combine the hidden state of the main model and the embedding vector of the future token.

[0142] In one specific application, taking the generation of "good, suitable for going out" from the user input "today's weather is very" as an example, the user input instruction "today's weather is very" is token standardized to obtain a token ID sequence, such as the initial token sequence [102, 301, 890]. The main model in the large language model can perform inference based on the initial token sequence [102, 301, 890] to obtain an initial prediction result, such as the token corresponding to "good" as the first token. The MTP module (token prediction model) in the large language model can perform iterative token prediction based on the token corresponding to "good" and the hidden state in the main model, respectively, to obtain the token 5 corresponding to "," the token 210 corresponding to "suitable", and the token 211 corresponding to "fit", thereby obtaining the draft sequence [5, 210, 211]. The main model in the large language model can batch verify and accept the draft sequence [5, 210, 211], such as splicing the token 120 corresponding to "good" and the draft sequence [5, 210, 211] to obtain a to-be-verified sequence. The main model can perform a forward propagation calculation of the probability distribution of all positions based on the to-be-verified sequence, and determine whether to accept the draft in the to-be-verified sequence according to the probability distribution of each position. If accepted, each draft in the to-be-verified sequence can be determined as a token of the completed inference, and the next token prediction processing is performed. After the large language model completes the inference processing, the token sequence corresponding to the response content of the input instruction is obtained, and the server can decode and restore the token sequence to obtain the response content in the form of natural language. The server can feed back the response content to the terminal to show the user.

[0143] Therefore, the large language model processing method provided by the present application can realize vocabulary compression and shared weight MTP acceleration, aiming to improve the LLM inference efficiency. Specifically, the present application provides a shared weight MTP head, so that all prediction steps use the same parameters, thereby reducing redundant computing resource consumption and improving prediction efficiency. Among them, the shared weight MTP head can allow the model to reuse the same parameters in multiple prediction steps, greatly reducing the consumption of calculation and memory. In some embodiments, such as Figure 9As shown, this is a comparison of the speedup ratios of different large language model processing methods for performing different types of subtasks. Among them, the basic MTP (Vanilla MTP) is an inference method that directly uses multiple independent MTP modules to execute sub-tasks; the self-data fine-tuned MTP (Self-data FT MTP) is an inference method that performs sub-tasks after fine-tuning the MTP modules based on self-data, where self-data can be training data obtained based on self-distillation; the fixed-data fine-tuned MTP (Fixed-data FT MTP) is an inference method that performs sub-tasks after fine-tuning the MTP modules based on fixed data; and the self-data fine-tuned MTP + frequency ranking (FR) is an inference method that performs sub-tasks after fine-tuning the MTP modules based on self-data and combining it with a frequency ranking speculative sampling algorithm. The sub-tasks to be compared can include various types such as retrieval-augmented generation, mathematical reasoning, question answering, summarization, coding, Chinese knowledge, and multi-turn conversation. It can be seen that the inference method of self-data fine-tuning MTP + frequency sorting speculative sampling has the highest speedup ratio in all subtasks.

[0144] like Figure 10 The figure shows the decoding speed (in tokens / s) and acceptance rate for different numbers of draft tokens. The basic MTP is the inference method that directly uses multiple independent MTP modules to execute subtasks, while the self-data fine-tuned MTP is the inference method in this application that fine-tunes the MTP module based on self-data before executing subtasks. K represents the amount of data in the MTP module, i.e., the number of draft tokens output by the MTP module. It can be seen that, for K values ​​from 0 to 7, the decoding speed and acceptance rate of the inference method in this application that fine-tunes the MTP module based on self-data before executing subtasks are both higher than the inference method that directly uses multiple independent MTP modules to execute subtasks. Therefore, the large language model processing method provided in this application significantly accelerates the inference process through recursive multi-step prediction and the design of a shared weighted MTP head. Experimental results show that the inference speed is improved by 2.03 times, while maintaining high-quality output consistent with standard generation quality.

[0145] Specifically, the existing autoregressive step-by-step generation method is slow in the inference process, and in the multi-label prediction (MTP) method, independent MTP modules are usually used, each of which has different weights and caches; and the application proposes an acceleration method using a shared weight MTP head, aiming to improve the inference processing efficiency of large language models by using a shared weight MTP head. The difficulty of using a shared weight MTP head is how to ensure that when the same weight is used in multiple prediction steps, a high token adoption rate can still be maintained. Specifically, it can easily lead to a lower accuracy of tokens generated in parallel inference, because the shared weight MTP module may not be fully trained to cope with different prediction steps, resulting in a low acceptance rate when verified by the main model.

[0146] Based on this, the large language model processing method provided by the application involves a progressive alignment training method and a recursive multi-step prediction training mechanism. As shown in Figure 11 , in each iteration of the label prediction processing, the input label sequence and the previous hidden state can be obtained, and the next label prediction can be recursively performed based on the input label sequence and the previous hidden state. The server can adaptively adjust the prediction weight of each iteration of the label prediction, and perform iterative label prediction based on a feedback recursive mechanism and a context enhancement mechanism to obtain a predicted label. The server can calculate a loss function based on the obtained predicted label, and recursively perform the next label prediction according to the updated model according to the loss function until the training ends when the training end condition is met.

[0147] The core idea of the progressive alignment is to train the MTP head in the MTP module (the MTP heads of each MTP module share weight parameters) by using the generated token as the target label, so that the prediction distribution of the MTP head aligns with the label distribution generated by the main model in the inference process. In the training process, the token sequence generated by the main model is used as label data to train the MTP head, so as to ensure that the MTP head can learn the same generation mode and generation quality as the main model. This training method effectively helps the MTP head to capture the context information required for multi-step recursive prediction, so that the generated token can have a higher acceptance rate in the inference process.

[0148] The goal of the training process is to minimize the difference between the token distribution generated by the main model and the token distribution generated by the MTP head, and the corresponding loss function is as follows:

[0149]

[0150] wherein, is the token distribution generated by the main model in the i-th prediction, is the token distribution of the i-th MTP head generation, KL denotes the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions; is the number of MTP modules (i.e., the number of MTP heads). By optimizing this loss function , the generation distribution of the MTP head can be as consistent as possible with the distribution of the main model.

[0151] Further, the present application also relates to a dynamic weight adjustment mechanism. In the traditional training method, the prediction step weight is fixed. In order to improve the alignment effect, the present application uses a dynamic weight adjustment mechanism. That is, the weight of the prediction step is dynamically adjusted according to the predicted distance, and the closer to the current step, the greater the weight. The weight of the k-th step prediction is ,

[0152]

[0153] wherein, is the decay factor, K is the total number of steps, i.e., the number of MTP modules, indicates that K-step label prediction is performed through K MTP modules, or K times of label prediction. In this way, the model can focus on short-term prediction accuracy while retaining attention to long-term prediction.

[0154] Further, in order to improve the accuracy of multi-step prediction, the present application uses a cross-step context enhancement mechanism to utilize context information to enhance the prediction of the current step. Assuming that in the k-th step prediction, the model needs to introduce the prediction results of the previous m steps in addition to the current step input and the output of the previous step to form a new input:

[0155]

[0156] wherein, is the final input of the k-th step prediction based on the cross-step context enhancement mechanism, is the input label sequence of the k-th step prediction, is the label prediction result of the k-1-th step prediction, which can specifically include the label of the k-1-th step prediction; is the label prediction result of the k-m-th step prediction. By combining the outputs of multiple prediction steps as context information, the model can better understand the long-term dependencies in the generated sequence and improve the consistency and accuracy of generation.

[0157] In the traditional backpropagation, the gradient is only passed from the current prediction step to the previous step. The present application adopts a recursive gradient propagation mechanism, which enables the gradient to recursively adjust each historical prediction step. This method can fully consider the effect of all prediction steps, rather than just local errors. Assuming that the gradient of the k-th step is The gradient is recursively passed from the kth step to all historical steps:

[0158]

[0159] wherein, is the loss of the i-th iteration of the label prediction; is the label prediction result of the i-th iteration of the label prediction, which can specifically include the label predicted in the i-th step; is the gradient of the loss of the i-th iteration of the label prediction with respect to the label prediction result of the i-th step. The recursive gradient propagation mechanism makes the optimization of each step consider historical errors, which can effectively improve the processing capability of long-term dependencies.

[0160] For deep recursive optimization prediction, the inference process is performed through a recursive multi-step prediction optimization model. The prediction of each step not only depends on the current input, but also depends on the previous prediction result, which gradually improves the generation effect. For the input and state transmission processing in the training process, the model receives the input of the current step and the prediction result of the previous step at each prediction step. The generated label of each step becomes the feedback of the next prediction. The present application provides a feedback recursive mechanism, which enables the model to self-correct during the generation process. Specifically, the output of the previous step is used as the context input of the current step, which helps the model to more accurately generate subsequent labels. The hidden state is propagated recursively to optimize the prediction of each step:

[0161]

[0162] wherein, is the hidden state of the kth step, is the hidden state of the k-1th step, is the current input, is the label prediction result of the previous step; represents the processing of the kth MTP head, that is, the MTP head processes the output , and according to .

[0163] Further, the weight of each prediction step is adjusted by a recursive decay mechanism, so that the influence of the long-term step gradually weakens, ensuring that the near prediction step learns first:

[0164]

[0165] wherein, W is a weight matrix, b is a bias term, is the label prediction result of the kth step. The adaptive decay mechanism in the recursive prediction step can adjust the focus of learning according to the distance of each step, which enables the model to better handle long-term dependencies.

[0166] At each step of prediction, not only the current input is used, but also the previous multiple generated tokens are used as context information to enhance the current prediction. This is achieved by the cross-step context enhancement mechanism, which combines the previous generated tokens to extract effective information from the context of multiple steps, reducing the error of relying on single-step local information:

[0167]

[0168]

[0169] wherein, is the pooling processing; is the token prediction result of the first step; is the token prediction result of the second step; is the token prediction result of the k-1th step. Cross-step context enhancement allows each step of prediction to integrate historical information, avoiding the limitations of single-step prediction, thereby improving the overall consistency of generation.

[0170] During training, the present application uses a recursive loss function to balance the contribution of each step. The recursive loss not only considers the prediction of the current step, but also adjusts the errors of historical steps to ensure that the model can maintain high accuracy in all steps:

[0171]

[0172] wherein, is the prediction weight of each iteration token prediction step, using a recursive decay factor to reduce the influence of long-term prediction, is the cross-entropy loss, is the token prediction result of the kth step, is the target label. The recursive loss function adopts a position-dependent weight, allowing the model to automatically adjust the difficulty of each prediction step and optimize the learning process.

[0173] In specific applications, such as Figure 12As shown, the large language model includes a main model and a plurality of MTP modules, the main model is used to predict the next token, the main model can generate an initial prediction by using the knowledge learned in the training stage; the MTP head in the MTP module recursively receives the next token predicted by the main model and generates draft tokens, which are verified in parallel to ensure accuracy. Among them, each MTP module can include an embedding layer, an MTP head and an output head, and the embedding layer and the output head in each MTP module can reuse the embedding layer and the output head in the main model, that is, the embedding layer in each MTP module can share the parameters of the embedding layer in the main model, and the output head in each MTP module can share the parameters of the output head in the main model. For each MTP module, the weight parameters of the MTP head can be shared, that is, the parameters in the MTP head in each MTP module are shared weight parameters included by each MTP module. After inputting an initial token sequence including 4 token samples, the main model can infer based on the initial token sequence to obtain an initial prediction result; the input token sequence for the first iteration token prediction can be obtained after the initial token sequence is offset by 1 token sample, and the initial prediction result and the previous hidden state output by the Transformer layer are combined to perform the first iteration token prediction through the first MTP module connected with the main model to obtain the token prediction result of the first iteration token prediction; the input token sequence for the second iteration token prediction can be obtained after the input token sequence for the first iteration token prediction is offset by 1 token sample, and the token prediction result of the first iteration token prediction and the previous hidden state of the first iteration token prediction are combined to perform the second iteration token prediction through the second MTP module to obtain the token prediction result of the second iteration token prediction. After traversing each MTP module, the token prediction result of each MTP module can be obtained.

[0174] Among them, the self-distillation method can be used to generate training data in the present application to ensure that the distribution generated by the MTP head and the main model is consistent, so that a higher acceptance rate can be obtained in the inference process. For example, diversified prompt words can be obtained from instruction fine-tuning data sets of multiple fields and languages, and the response content to the prompt words can be generated by the main model in the large language model itself, so that the training of the MTP head in the large language model can naturally match the generation distribution of the main model, capture specific generation patterns, style preferences, and features that are difficult to learn through external data.

[0175] In the inference application process, after the input includes the token sequence of the input instruction of 4 tokens, the main model can perform inference based on the token sequence of the input instruction to obtain an initial prediction result, including a predicted fifth token; the server can splice the fifth token predicted by the main model with the input second to fourth tokens to obtain the input token sequence of the first MTP module, so as to perform iterative token prediction by the first MTP module in combination with the previous hidden state output by the Transformer layer, and obtain the token prediction result of the first iteration token prediction. When the MTP module outputs the token prediction result, the prediction can be combined with the frequency-ranked (FR) vocabulary compression technology to reduce the search space and computational complexity by preferentially processing high-frequency vocabularies, thereby accelerating the draft generation process. After traversing each MTP module to obtain the respective token prediction result, the main model can perform parallel verification on each token prediction result to determine whether to accept the token prediction result predicted by the MTP module, thereby realizing inference acceleration.

[0176] The recursive multi-step prediction mechanism and the shared weight MTP head proposed in the application have a significant effect on improving inference efficiency and generation quality. Compared with the traditional step-by-step generation method, the application can reuse the same parameters in multiple prediction steps by using the shared weight MTP head, thereby greatly reducing memory consumption and computational overhead. Using the scheme of the application, the inference speed is improved by 2.03 times compared with the traditional method. Taking the MiMo-RL-7B model as an example, the standard next-step prediction method can only process about 31.28 tokens per second, while the method of the application can process more than 63.42 tokens per second.

[0177] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0178] Based on the same inventive concept, the embodiments of the present application also provide a large language model processing device for implementing the large language model processing method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more large language model processing device embodiments provided below can refer to the limitations of the large language model processing method described above, which will not be repeated here.

[0179] In one exemplary embodiment, as shown in Figure 13 A large language model processing device 1300 is provided, comprising: a token sequence sample acquisition module 1302, an initial processing data acquisition module 1304, an iterative token prediction module 1306, and a shared weight parameter update module 1308, wherein:

[0180] The token sequence sample acquisition module 1302 is configured to acquire a token sequence sample, the token sequence sample being obtained by reasoning based on a token sequence of an instruction sample;

[0181] The initial processing data acquisition module 1304 is configured to determine an initial token sequence from the token sequence sample, and obtain an initial prediction result obtained by reasoning by a main model in the large language model for the initial token sequence;

[0182] The iterative token prediction module 1306 is configured to sequentially perform iterative token prediction based on the token sequence sample and the initial prediction result by at least one token prediction model in the large language model, to obtain token prediction results of the at least one token prediction model respectively; wherein in each iterative token prediction, a target token prediction model of the present iterative token prediction performs token prediction for an input token sequence of the present iterative token prediction and a previous prediction result to obtain a token prediction result of the present iterative token prediction, the input token sequence being obtained by offsetting the initial token sequence in the token sequence sample according to a number sequence of the present iterative token prediction;

[0183] The shared weight parameter update module 1308 is configured to update shared weight parameters included in each token prediction model according to the token prediction result of each token prediction model and a target reasoning result, to obtain a trained large language model; the target reasoning result being obtained by the main model based on the token sequence sample and the initial prediction result through iterative token prediction; the trained large language model being configured to reason for a token sequence of an input instruction to output a token sequence of a corresponding response content of the input instruction.

[0184] In some embodiments, the shared weight parameter updating module 1308 is further configured to obtain, according to the respective label prediction result of each of the at least one label prediction model and the target inference result, a respective prediction result difference of each of the at least one label prediction model; obtain, according to the respective prediction result difference of each of the at least one label prediction model, a label prediction loss; and update the shared weight parameter included in each of the label prediction models based on the label prediction loss to obtain the large language model after training.

[0185] In some embodiments, the label prediction result comprises a label prediction distribution, and the target inference result comprises a target inference distribution; and the shared weight parameter updating module 1308 is further configured to obtain, according to the respective label prediction distribution of each of the at least one label prediction model and the target inference distribution, a respective distribution difference of each of the at least one label prediction model; and obtain, according to the respective distribution difference of each of the at least one label prediction model, the label prediction loss.

[0186] In some embodiments, the label prediction result comprises a predicted label, and the target inference result comprises a target label; and the shared weight parameter updating module 1308 is further configured to obtain, according to the respective predicted label of each of the at least one label prediction model and the target label, a respective sub-loss of each of the at least one label prediction model; and obtain, according to the respective sub-loss of each of the at least one label prediction model, the label prediction loss.

[0187] In some embodiments, the shared weight parameter updating module 1308 is further configured to determine a respective prediction weight of each of the at least one label prediction model, the value of the prediction weight being negatively correlated with the serial number of the number of times of the iterative label prediction performed by the label prediction model; and obtain, according to the respective prediction result difference of each of the at least one label prediction model and the prediction weight, the label prediction loss.

[0188] In some embodiments, the label prediction loss comprises a respective sub-loss of each of the iterative label predictions; the shared weight parameter updating module 1308 is further configured to determine, for each sub-loss in the label prediction loss, a loss gradient corresponding to the sub-loss; determine a respective historical gradient of the sub-loss of the previous label prediction, the previous label prediction being an iterative label prediction performed before the iterative label prediction corresponding to the sub-loss; obtain, according to the loss gradient and the historical gradient, a cumulative gradient of the sub-loss; obtain, based on the respective cumulative gradient of each sub-loss in the label prediction loss, a target gradient of the label prediction loss; and update the shared weight parameter included in each of the label prediction models through the target gradient to obtain the large language model after training.

[0189] In some embodiments, the iterative token prediction module 1306 is further configured to sequentially perform the iterative token prediction based on the token sequence sample, the initial prediction result, and the initial hidden state obtained by the main model for the initial token sequence by using at least one token prediction model in the large language model to obtain a token prediction result of each token prediction model; and perform token prediction on the input token sequence of the current iteration, the previous prediction result, and the previous hidden state by using a target token prediction model of the current iteration to obtain a token prediction result of the current iteration.

[0190] In some embodiments, the iterative token prediction module 1306 is further configured to determine the input token sequence of the current iteration, the previous prediction result, and the previous hidden state; perform token prediction on the input token sequence, the previous prediction result, and the previous hidden state by using a target token prediction model of the current iteration to obtain a target hidden state; and perform mapping processing on the target hidden state to obtain a token prediction result of the current iteration.

[0191] In some embodiments, the iterative token prediction module 1306 is further configured to determine the input token sequence of the current iteration, the previous hidden state, and at least one previous prediction result, the previous prediction result being a token prediction result obtained by performing the iterative token prediction before the current iteration; perform token prediction on the input token sequence, the previous hidden state, and the at least one previous prediction result by using a target token prediction model of the current iteration to obtain a token prediction result of the current iteration.

[0192] In some embodiments, the token prediction model includes at least two; the iterative token prediction module 1306 is further configured to perform a first offset on the initial token sequence in the token sequence sample to obtain an input token sequence of a first iteration, and determine the initial prediction result as a previous prediction result of the first iteration; perform token prediction on the input token sequence of the first iteration and the previous prediction result by using a first token prediction model to obtain a token prediction result of the first iteration; and for each iteration starting from the second iteration, perform an offset on the initial token sequence in the token sequence sample based on a number sequence of the current iteration to obtain an input token sequence of the current iteration, and determine a token prediction result of a previous iteration as a previous prediction result of the current iteration; perform token prediction on the input token sequence of the current iteration and the previous prediction result by using a target token prediction model of the current iteration to obtain a token prediction result of the current iteration.

[0193] Based on the same inventive concept, the embodiments of the present application also provide a large language model-based instruction response device for implementing the above-mentioned large language model-based instruction response method. The problem-solving implementation scheme provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more large language model-based instruction response device embodiments provided below can be referred to the limitations of the large language model-based instruction response method in the above, which will not be described here again.

[0194] In one exemplary embodiment, as shown in Figure 14 a large language model-based instruction response device 1400 is provided, comprising an instruction token sequence acquisition module 1402, an initial prediction module 1404, an iterative token prediction module 1406, and a response token sequence acquisition module 1408, wherein:

[0195] The instruction token sequence acquisition module 1402 is configured to acquire a token sequence of an input instruction, determine a large language model, and the large language model is obtained based on the large language model processing method as described above;

[0196] The initial prediction module 1404 is configured to perform reasoning on the token sequence of the input instruction by a main model in the large language model to obtain an initial prediction result;

[0197] The iterative token prediction module 1406 is configured to perform iterative token prediction based on the initial prediction result and the token sequence of the input instruction by at least one token prediction model in the large language model to obtain a respective token prediction result of each of the at least one token prediction model;

[0198] The response token sequence acquisition module 1408 is configured to obtain a token sequence of a corresponding response content of the input instruction based on the respective token prediction result of each of the at least one token prediction model by the main model.

[0199] In some embodiments, the iterative token prediction module 1406 is further configured to perform iterative token prediction based on the initial prediction result, the token sequence of the input instruction, and a first token set by at least one token prediction model in the large language model to obtain a respective token prediction result of each of the at least one token prediction model; the response token sequence acquisition module 1408 is further configured to verify the respective token prediction result of each of the at least one token prediction model based on a second token set by the main model to obtain a respective verification result of each of the at least one token prediction model; the first token set is a subset of the second token set; and the token sequence of the corresponding response content of the input instruction is obtained based on the respective verification result of each of the at least one token prediction model.

[0200] Each of the modules in the above various apparatuses can be implemented wholly or partially by software, hardware, and a combination thereof. The above modules can be embedded in or independent of a processor in a hardware form in a computer device, or stored in a memory in a software form in the computer device, so as to be called and executed by a processor to perform operations corresponding to the above modules.

[0201] In an exemplary embodiment, a computer device, which can be a server, has an internal structure diagram as shown in Figure 15 The computer device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with terminals outside through a network connection. The computer program is executed by the processor to implement a large language model processing method and / or a large language model-based instruction response method.

[0202] Those skilled in the art can understand that Figure 15 The structure shown in the above

[0203] In an embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above method embodiments.

[0204] In an embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0205] In an embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0206] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0207] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to memory, database or other medium used in the embodiments provided by the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, etc., without being limited thereto.

[0208] The technical features of the above embodiments can be combined in any way. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0209] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A large language model processing method, characterized in that, The method comprises: obtaining a mark sequence sample, the mark sequence sample being obtained by inference based on a mark sequence of an instruction sample; determining an initial mark sequence from the mark sequence sample, and obtaining an initial prediction result obtained by inference of a main model in a large language model on the initial mark sequence; iteratively performing mark prediction based on the mark sequence sample and the initial prediction result through at least one mark prediction model in the large language model to obtain mark prediction results of the at least one mark prediction model respectively; wherein in each iteration of mark prediction, a target mark prediction model of the current iteration of mark prediction performs mark prediction on an input mark sequence of the current iteration of mark prediction and a previous prediction result to obtain a mark prediction result of the current iteration of mark prediction, the input mark sequence being obtained by offsetting the initial mark sequence in the mark sequence sample according to a sequence number of the current iteration of mark prediction; updating shared weight parameters included in each of the mark prediction models according to the mark prediction results of the at least one mark prediction model respectively and a target inference result to obtain a trained large language model; the target inference result being obtained by iteratively performing mark prediction on the mark sequence sample and the initial prediction result through the main model; and the trained large language model being used to perform inference on a mark sequence of an input instruction to output a mark sequence of a corresponding response content of the input instruction.

2. The method of claim 1, wherein, The updating of the shared weight parameters included in each of the mark prediction models according to the mark prediction results of the at least one mark prediction model respectively and the target inference result to obtain the trained large language model comprises: obtaining prediction result differences of the at least one mark prediction model respectively according to the mark prediction results of the at least one mark prediction model respectively and the target inference result; obtaining mark prediction losses according to the prediction result differences of the at least one mark prediction model respectively; updating the shared weight parameters included in each of the mark prediction models based on the mark prediction losses to obtain the trained large language model.

3. The method of claim 2, wherein, The mark prediction results comprise mark prediction distributions, and the target inference result comprises a target inference distribution; the obtaining of the prediction result differences of the at least one mark prediction model respectively according to the mark prediction results of the at least one mark prediction model respectively and the target inference result comprises: obtaining distribution differences of the at least one mark prediction model respectively according to the mark prediction distributions of the at least one mark prediction model respectively and the target inference distribution; The obtaining of the mark prediction losses according to the prediction result differences of the at least one mark prediction model respectively comprises: obtaining the mark prediction losses according to the distribution differences of the at least one mark prediction model respectively.

4. The method of claim 2, wherein, The mark prediction results comprise predicted marks, and the target inference result comprises target marks; the obtaining of the prediction result differences of the at least one mark prediction model respectively according to the mark prediction results of the at least one mark prediction model respectively and the target inference result comprises: According to the prediction label of each of the at least one label prediction model and the target label, a sub-loss of each of the at least one label prediction model is obtained; The label prediction loss is obtained according to the prediction result difference of each of the at least one label prediction model, including: The label prediction loss is obtained according to the sub-loss of each of the at least one label prediction model.

5. The method of claim 2, wherein, The label prediction loss is obtained according to the prediction result difference of each of the at least one label prediction model, including: The prediction weight of each of the at least one label prediction model is determined, and the value of the prediction weight is negatively correlated with the number of times of the iterative label prediction performed by the label prediction model; The label prediction loss is obtained according to the prediction result difference and the prediction weight of each of the at least one label prediction model.

6. The method of claim 2, wherein, The label prediction loss includes the sub-loss of each of the iterative label prediction, and the shared weight parameter included in each of the label prediction model is updated based on the label prediction loss to obtain the large language model after training, including: For each sub-loss in the label prediction loss, a loss gradient corresponding to the sub-loss is determined; The historical gradient corresponding to the sub-loss of the previous label prediction is determined, and the previous label prediction is the iterative label prediction performed before the corresponding iterative label prediction of the sub-loss; The cumulative gradient of the sub-loss is obtained according to the loss gradient and the historical gradient; The target gradient of the label prediction loss is obtained based on the cumulative gradient of each sub-loss in the label prediction loss; The shared weight parameter included in each of the label prediction model is updated based on the target gradient to obtain the large language model after training.

7. The method of claim 1, wherein, The label prediction result of each of the at least one label prediction model is obtained by sequentially performing the iterative label prediction based on the label sequence sample and the initial prediction result through at least one label prediction model in the large language model, including: The label prediction result of each of the at least one label prediction model is obtained by sequentially performing the iterative label prediction through at least one label prediction model in the large language model based on the label sequence sample, the initial prediction result and the initial hidden state obtained by the main model performing reasoning on the initial label sequence; The target label prediction model of the current iterative label prediction performs label prediction on the input label sequence and the previous prediction result of the current iterative label prediction to obtain the label prediction result of the current iterative label prediction, including: The target label prediction model of the current iterative label prediction performs label prediction on the input label sequence, the previous prediction result and the previous hidden state of the current iterative label prediction to obtain the label prediction result of the current iterative label prediction.

8. The method of claim 7, wherein, The target label prediction model of the current iterative label prediction performs label prediction on the input label sequence, the previous prediction result and the previous hidden state of the current iterative label prediction to obtain the label prediction result of the current iterative label prediction, including: The input label sequence, the previous prediction result and the previous hidden state of the current iterative label prediction are determined. The target mark prediction model of the current iteration mark prediction is used for mark prediction on the input mark sequence, the previous prediction result and the previous hidden state, to obtain a target hidden state; The target hidden state is mapped to obtain a mark prediction result of the current iteration mark prediction.

9. The method of claim 7, wherein, The target mark prediction model of the current iteration mark prediction is used for mark prediction on the input mark sequence, the previous prediction result and the previous hidden state, to obtain a target hidden state; The target hidden state is mapped to obtain a mark prediction result of the current iteration mark prediction. The target mark prediction model of the current iteration mark prediction is used for mark prediction on the input mark sequence, the previous prediction result and the previous hidden state, to obtain a target hidden state; 10. The method according to any one of claims 1 to 9, characterized in that, The target hidden state is mapped to obtain a mark prediction result of the current iteration mark prediction. The mark prediction model includes at least two; the at least one mark prediction model in the large language model is used for iterative mark prediction based on the mark sequence sample and the initial prediction result, to obtain the mark prediction result of each of the at least one mark prediction model, including: The initial mark sequence is first offset in the mark sequence sample based on the initial mark sequence, to obtain an input mark sequence of first iteration mark prediction, and the initial prediction result is determined as a previous prediction result of first iteration mark prediction; The first mark prediction model is used for mark prediction on the input mark sequence of first iteration mark prediction and the previous prediction result, to obtain a mark prediction result of first iteration mark prediction; From the second iteration mark prediction, the input mark sequence of the current iteration mark prediction is obtained by offsetting the initial mark sequence in the mark sequence sample based on the number sequence of the current iteration mark prediction, and the mark prediction result of the last iteration mark prediction is determined as the previous prediction result of the current iteration mark prediction; 11.A method for responding to an instruction based on a large language model, the method comprising: The target mark prediction model of the current iteration mark prediction is used for mark prediction on the input mark sequence, the previous prediction result and the previous hidden state, to obtain a target hidden state; The method comprises: Obtain the mark sequence of the input instruction, determine the large language model, the large language model is obtained based on the large language model processing method in any one of claims 1 to 10; The main model in the large language model is used for reasoning on the mark sequence of the input instruction, to obtain an initial prediction result; The at least one mark prediction model in the large language model is used for iterative mark prediction based on the initial prediction result and the mark sequence of the input instruction, to obtain the mark prediction result of each of the at least one mark prediction model; The main model is used for obtaining the mark sequence of the corresponding response content of the input instruction based on the mark prediction result of each of the at least one mark prediction model.

12. The method of claim 11, wherein, The at least one token prediction model in the large language model is used for iteratively performing token prediction based on the initial prediction result and the token sequence of the input instruction, and obtaining a token prediction result of each of the at least one token prediction model. The at least one token prediction model in the large language model is used for iteratively performing token prediction based on the initial prediction result, the token sequence of the input instruction, and the first token set, and obtaining a token prediction result of each of the at least one token prediction model. The main model is used for obtaining a token sequence of the corresponding response content of the input instruction based on the token prediction result of each of the at least one token prediction model. The main model is used for verifying the token prediction result of each of the at least one token prediction model based on the second token set, and obtaining a verification result of each of the at least one token prediction model; and the first token set is a subset of the second token set. The token sequence of the corresponding response content of the input instruction is obtained based on the verification result of each of the at least one token prediction model.

13. A large language model processing apparatus, comprising: The device comprises: a token sequence sample acquisition module configured to acquire a token sequence sample, the token sequence sample being obtained by inference based on a token sequence of an instruction sample; an initial processing data acquisition module configured to determine an initial token sequence from the token sequence sample, and obtain an initial prediction result obtained by inference of a main model in a large language model on the initial token sequence; an iterative token prediction module configured to use at least one token prediction model in the large language model to iteratively perform token prediction based on the token sequence sample and the initial prediction result, and obtain a token prediction result of each of the at least one token prediction model; in each iteration of token prediction, a target token prediction model of the current iteration of token prediction is used to perform token prediction on an input token sequence of the current iteration of token prediction and a previous prediction result to obtain a token prediction result of the current iteration of token prediction, the input token sequence being obtained by offsetting the initial token sequence in the token sequence sample according to a sequence number of the current iteration of token prediction; a shared weight parameter updating module configured to update shared weight parameters included in each of the token prediction models according to the token prediction result of each of the at least one token prediction model and a target inference result, and obtain a trained large language model; the target inference result is obtained by iteratively performing token prediction on the token sequence sample and the initial prediction result by using the main model; and the trained large language model is used to perform inference on a token sequence of an input instruction to output a token sequence of a corresponding response content of the input instruction.

14. A large language model-based instruction response apparatus, characterized by, The device comprises: an instruction token sequence acquisition module configured to acquire a token sequence of an input instruction, and determine a large language model, the large language model being obtained based on the large language model processing method in any one of claims 1 to 10; An initial prediction module is configured to perform reasoning on a token sequence of the input instruction by a main model in the large language model to obtain an initial prediction result. An iterative token prediction module is configured to perform iterative token prediction based on the initial prediction result and the token sequence of the input instruction by at least one token prediction model in the large language model to obtain a token prediction result of each token prediction model. A response token sequence acquisition module is configured to obtain a token sequence of the input instruction corresponding to the response content by the main model based on the token prediction result of each token prediction model.

15. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1-10 or 11-12.

16. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1-10 or 11-12.

17. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1-10 or 11-12.

Images