Diffusion model-based large language model reasoning method and device, terminal and medium

Abstract

The application provides a large language model reasoning method and device based on a diffusion model, a terminal and a medium. The application generates a draft sequence by using a draft model based on a diffusion model framework, and verifies the draft sequence by using a large language model. The application takes advantage of the characteristic that the diffusion model naturally supports parallel processing, so that the length of the draft sequence is significantly increased, thereby reducing the number of large model verifications, improving the reasoning efficiency of the large language model, and improving the utilization efficiency of computing resources.

Description

Technical Field

[0001] This application relates to the field of large language model technology, and in particular to a large language model reasoning method, device, terminal and medium based on diffusion model. Background Technology

[0002] Large Language Models (LLMs), such as those based on the Transformer architecture, have demonstrated outstanding performance in various fields, including natural language processing. However, their autoregressive inference mechanism, which generates text token by token, results in high inference latency, limiting their deployment in applications with high real-time requirements.

[0003] To alleviate this problem, speculative decoding techniques have emerged. This technique typically employs a small, fast draft model to pre-generate several candidate token sequences, which are then validated in parallel by a target large language model (Target LLM). If the validation passes, multiple tokens can be accepted at once, thereby accelerating inference. In existing technologies, the draft model is usually a miniaturized version of the target large language model (such as a small model obtained through knowledge distillation, a model with fewer parameters and the same architecture) or a simplified autoregressive model.

[0004] However, existing speculative decoding techniques have the following problems:

[0005] (1) Limited draft generation length: Existing autoregressive draft models require k serial decoding steps to generate k tokens. In order to control the delay of draft generation so as not to offset the speedup effect, the value of k is usually very limited, such as only 3 to 5 tokens. This fundamentally limits the potential maximum speedup benefit that word inference decoding operations can bring.

[0006] (2) Limited number of tokens actually accepted: Since k itself is small, and there may be deviations between the generation quality and distribution of the draft model and the target large language model, the number of tokens n that are actually verified and accepted by the large language model is usually also small, with an average of only 2-3 tokens. This makes the effective acceleration of single-inference decoding very limited.

[0007] (3) The overall acceleration effect is not obvious: Since the number of tokens that can be received in a single speculation decoding is small, more decoding iterations are needed to generate a sequence of the target length. Therefore, the overall acceleration effect of speculation decoding is poor and it is difficult to meet the ever-increasing requirements for the reasoning speed of large language models.

[0008] (4) The challenge of balancing efficiency and quality in draft model generation: Existing draft models require a difficult trade-off between generation speed and quality. While overly simple draft models are fast, they produce poor quality, resulting in low acceptance rates. On the other hand, draft models with slightly higher quality may not be fast enough due to their autoregressive nature, which weakens the significance of accelerating speculative decoding, and the number of tokens generated at one time is still limited. Summary of the Invention

[0009] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, device, terminal and medium for large language model inference based on diffusion model, so as to solve the problem that the existing draft model has limited candidate token sequence length and small number of tokens actually received in a single time due to its autoregressive characteristics, which leads to poor overall acceleration effect.

[0010] To achieve the above and other related objectives, a first aspect of this application provides a large language model inference method based on a diffusion model, comprising: obtaining a pre-trained draft model based on a diffusion model architecture and determining the large language model to be used; configuring the draft model according to pre-obtained draft model configuration parameters; obtaining context information to be inferred and converting it into a context sequence to be inferred; using the context sequence to be inferred as the current context sequence; performing a decoding operation on the current context sequence to obtain an intermediate output sequence; performing a termination judgment operation based on the intermediate output sequence according to a preset termination condition to obtain a corresponding judgment result; if the obtained judgment result satisfies the termination condition, then using the intermediate output sequence as the final output sequence; if the obtained judgment result does not satisfy the termination condition, then using the intermediate output sequence as the current context sequence and performing a decoding operation on it until a judgment result that satisfies the termination condition is obtained; wherein, the decoding operation includes: sequentially executing a draft sequence generation operation based on the draft model after parameter configuration, a verification operation based on the large language model, a receiving prefix determination operation, and a condition judgment and correction operation; performing text conversion on the final output sequence, and using the conversion result as the inference result text information corresponding to the context information to be inferred.

[0011] In some embodiments of the first aspect of this application, the draft model configuration parameters include: the length of the draft sequence and the number of denoising steps of the draft model.

[0012] In some embodiments of the first aspect of this application, the long sequence draft generation operation based on the parameter-configured draft model includes: performing reverse diffusion processing on the current context sequence using the parameter-configured draft model to generate multiple token representations, and converting the multiple token representations into a draft sequence according to a vocabulary.

[0013] In some embodiments of the first aspect of this application, the verification operation based on a large language model includes: inputting the current context sequence and the draft sequence into the large language model for parallel evaluation to obtain the optimal token sequence.

[0014] In some embodiments of the first aspect of this application, the operation of determining the receiving prefix includes: outputting the portion of the draft sequence that is consistent with the optimal token sequence starting from the first token as a prefix sequence, and outputting the number of tokens in the prefix sequence as the number of tokens accepted by the large language model; concatenating the prefix sequence with the current context sequence to obtain a preliminary intermediate output sequence.

[0015] In some embodiments of the first aspect of this application, the condition judgment operation includes: if the number of accepted tokens is less than the length of the draft sequence, then a correction operation is performed to obtain a corrected intermediate output sequence, and the corrected intermediate output sequence is used as the final intermediate output sequence; if the number of accepted tokens is equal to the length of the draft sequence, then the preliminary intermediate output sequence is used as the final intermediate output sequence.

[0016] In some embodiments of the first aspect of this application, the correction operation includes: inputting the prefix sequence and the current context sequence into a large language model to obtain a correction token; concatenating the correction token with the preliminary intermediate output sequence to obtain a corrected intermediate output sequence.

[0017] To achieve the above and other related objectives, a second aspect of this application provides a large language model inference apparatus based on a diffusion model, comprising: a model acquisition module for acquiring a pre-trained draft model based on a diffusion model architecture and determining the large language model to be used; a model configuration module for configuring the draft model according to pre-acquired draft model configuration parameters; and an inference module for acquiring context information to be inferred and converting it into a context sequence to be inferred; using the context sequence to be inferred as the current context sequence; performing a decoding operation on the current context sequence to obtain an intermediate output sequence; and performing a termination judgment operation based on the intermediate output sequence according to a preset termination condition. The system obtains the corresponding judgment result; if the obtained judgment result satisfies the termination condition, the intermediate output sequence is used as the final output sequence; if the obtained judgment result does not satisfy the termination condition, the intermediate output sequence is used as the current context sequence and decoded until a judgment result satisfying the termination condition is obtained; wherein, the decoding operation includes: sequentially executing a draft sequence generation operation based on the draft model after parameter configuration, a verification operation based on the large language model, a receiving prefix determination operation, and a condition judgment and correction operation; the result generation module is used to perform text conversion on the final output sequence and use the conversion result as the inference result text information corresponding to the context information to be inferred.

[0018] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the diffusion-based large language model reasoning method.

[0019] To achieve the above and other related objectives, a fourth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the diffusion-based large language model reasoning method.

[0020] As described above, the large language model reasoning method, apparatus, terminal, and medium based on the diffusion model of this application have the following beneficial effects:

[0021] This application leverages the inherent parallel processing capability of diffusion models, which significantly increases the length of the draft sequence, thereby reducing the number of large model validations, thus improving the inference efficiency of large language models and enhancing the utilization efficiency of computing resources. Attached Figure Description

[0022] Figure 1 The diagram shown is a flowchart illustrating a large language model reasoning method based on a diffusion model in one embodiment of this application.

[0023] Figure 2The diagram shown is a schematic representation of the decoding operation in one embodiment of this application.

[0024] Figure 3 The diagram shown is a schematic block diagram of a large language model inference device based on a diffusion model according to an embodiment of this application.

[0025] Figure 4 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0026] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0027] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0028] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0029] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0030] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0031] <1> In language models, a token is the smallest unit of text processing. A token is the result of the model segmenting the input text.

[0032] <2> Denoising steps: Denoising steps are an inherent parameter of the diffusion model. They refer to the number of iterations required for the model to progressively denoise from pure noise to the final image during the generation process.

[0033] <3> Contextual information: Contextual information refers to the input text (i.e., user-provided dialogue history, document content, etc.) that the model references when generating responses. It is the key basis for the model to understand the current task, maintain dialogue coherence, and provide accurate answers.

[0034] <4> Logits: In machine learning and deep learning, logits are an important concept. They typically refer to the raw predicted values ​​output by the model in classification problems, values ​​that have not yet been normalized, such as by the softmax function. Logits are the outputs of the model at the last layer; they are unscaled log probabilities.

[0035] <5> Vocabulary: The vocabulary is the collection of all recognizable and usable words, subwords, characters, or symbols in a language model. It defines the range of language units that the model can process.

[0036] <6> Token ID: A Token ID is a unique identifier (usually an integer) for each word, subword, or character in the vocabulary. It is a mapping that converts linguistic units (such as words, subwords, or characters) in text into numerical values ​​that the model can understand.

[0037] <7> Sequence terminator: In language models, text is typically processed as a series of tokens. A sequence terminator (often called an EOS, or "End of Sequence") is used to explicitly tell the model that the text sequence has ended. This helps the model know when to stop generating text.

[0038] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1 This document illustrates a flowchart of a large language model inference method based on a diffusion model, as described in an embodiment of the present invention. The large language model inference method based on a diffusion model in this embodiment mainly includes the following steps:

[0039] Step S11: Obtain a pre-trained draft model based on the diffusion model architecture and determine the large language model to be used.

[0040] It should be understood that a pre-trained, high-performance large language model can be selected for use based on actual needs, and this invention does not limit this choice. Types of large language models include, but are not limited to: Mistral AI models, the Llama 3 model from the MetaLlama series, and Llama 2 models.

[0041] In one embodiment, the method of training the draft model specifically includes: constructing a training set for the draft model using the large language model determined in the above process; and training the diffusion model framework using the training set to obtain a trained draft model.

[0042] In one embodiment, constructing a training set for a draft model using a defined large language model includes: acquiring multiple context sequences (token sequences) and inputting the multiple context sequences into the defined large language model to obtain multiple corresponding large language model output token sequences; wherein, the multiple large language model output token sequences cover large language model output token sequences of different lengths; the multiple context sequences and the multiple large language model output token sequences corresponding to the multiple context sequences constitute the training set of the draft model. It should be understood that the context sequences can be obtained by transforming context information.

[0043] In one embodiment, the diffusion model framework may include, but is not limited to, the U-Net framework, the non-autoregressive Transformer structure, etc.

[0044] In one embodiment, the draft model is optimized by minimizing the denoising error. Specifically, the multiple contexts are input into the diffusion model framework to obtain corresponding multiple model framework output token sequences. Minimizing the denoising error reflects the error between the multiple model framework output token sequences and the corresponding multiple large language model output token sequences.

[0045] In one embodiment, reinforcement learning can also be used to directly optimize the final acceleration effect of speculative decoding (e.g., maximizing the expected number of accepted tokens).

[0046] Step S12: Configure the draft model according to the pre-obtained draft model configuration parameters.

[0047] In one embodiment, the draft model configuration parameters include: the length of the draft sequence and the number of denoising steps of the draft model.

[0048] It should be understood that the configuration parameters of the draft model can be set according to actual needs. However, it is important to note that the length of the draft sequence should be set much larger than the draft length in existing speculative decoding (the existing draft length is generally 2 to 3), for example, the length of the draft sequence can be set to 8, 12, 16 or longer. The number of denoising steps in the draft model should be set much smaller than the number of steps in existing diffusion models used for tasks such as image generation. It should be noted that the existing denoising steps of diffusion models are usually between 50 and 1000. It should also be noted that the length of the draft sequence and the number of denoising steps in the draft model are two key hyperparameters. The choice of the draft sequence length k_D needs to be weighed against the increased benefit of the increased number of accepted tokens and the increased overhead of validating k_D tokens in the target LLM M_L. The choice of the number of denoising steps N_S needs to be balanced between draft generation speed and generation quality. These two parameters can be jointly tuned experimentally to maximize the overall inference speedup.

[0049] Step S13: Obtain the context information to be reasoned and convert it into a context sequence to be reasoned; use the context sequence to be reasoned as the current context sequence; perform a decoding operation on the current context sequence to obtain an intermediate output sequence; according to a preset termination condition, perform a termination judgment operation based on the intermediate output sequence to obtain the corresponding judgment result; if the obtained judgment result satisfies the termination condition, then use the intermediate output sequence as the final output sequence; if the obtained judgment result does not satisfy the termination condition, then use the intermediate output sequence as the current context sequence and perform a decoding operation on it until a judgment result that satisfies the termination condition is obtained.

[0050] In one embodiment, the context information to be inferred is linguistic information. The context information to be inferred can come from various fields, including but not limited to the medical field, the biological field, the gaming field, and the chip field.

[0051] In one embodiment, the context sequence to be reasoned is a token sequence. It should be noted that those skilled in the art can convert the context information into the corresponding token using existing conversion methods, which will not be elaborated upon here.

[0052] In one embodiment, the decoding operation includes: a draft sequence generation operation based on the draft model with configured parameters, a verification operation based on a large language model, a receiving prefix determination operation, and a condition judgment and correction operation, which are executed sequentially.

[0053] In one embodiment, the long sequence draft generation operation based on the parameter-configured draft model includes: performing reverse diffusion processing on the current context sequence using the parameter-configured draft model to generate multiple token representations, and converting the multiple token representations into a draft sequence according to the vocabulary.

[0054] It should be noted that the draft model generation process leverages the inherent characteristics of diffusion models (e.g., some variants can non-autoregressively predict the noise of the entire sequence at once, and then denoise in parallel), avoiding the strict stepwise sequential dependency of traditional autoregressive models, thus allowing for more efficient parallel computation. In summary, the diffusion model framework used in the draft model naturally supports parallel processing of the elements of the entire sequence, enabling the generation of token sequences of various lengths either at once or in parallel, effectively controlling the additional time overhead of generating long sequences. Furthermore, it is worth noting that the number of tokens generated by the draft model is equal to the pre-configured length of the draft sequence. The length of the draft sequence is the pre-configured length of the draft sequence.

[0055] In one embodiment, the draft model runs on a specific type of hardware accelerator (e.g., GPU, TPU). It should be noted that the diffusion model, especially the highly parallelized variant, can exhibit better computational efficiency and throughput than existing autoregressive models when running on the aforementioned hardware accelerators, thereby helping to accelerate the inference efficiency of large language models.

[0056] In one embodiment, the verification operation based on a large language model includes: inputting the current context sequence and the draft sequence into the large language model for parallel evaluation to obtain the optimal token sequence.

[0057] In one embodiment, the operation of determining the receiving prefix includes: outputting the portion of the draft sequence that matches the optimal token sequence starting from the first token as a prefix sequence, and outputting the number of tokens in the prefix sequence as the number of tokens accepted by the large language model; concatenating the prefix sequence with the current context sequence to obtain a preliminary intermediate output sequence.

[0058] In one embodiment, the condition judgment operation includes: if the number of accepted tokens is less than the length of the draft sequence, then a correction operation is performed to obtain a corrected intermediate output sequence, and the corrected intermediate output sequence is used as the final intermediate output sequence; if the number of accepted tokens is equal to the length of the draft sequence, then the preliminary intermediate output sequence is used as the final intermediate output sequence.

[0059] In one embodiment, the correction operation includes: inputting the prefix sequence and the current context sequence into a large language model to obtain a correction token; concatenating the correction token with the initial intermediate output sequence to obtain a corrected intermediate output sequence.

[0060] The following will be combined with the appendix Figure 2 Explain the reasoning process:

[0061] Before inference begins, obtain the context information to be inferred and convert it into a context sequence C_0. Set the target generation length L_target.

[0062] The context sequence C_0 to be inferred is used as the current context sequence C. The current context sequence C is input into the draft model. The draft model performs a back-diffusion process, generating K_D token representations. The process by which the diffusion model processes data is the back-diffusion process, which is an inherent characteristic of the diffusion model. The back-diffusion process includes: starting from a randomly sampled noise sequence (or a predefined noise pattern), combining it with the current context sequence C, and performing N_S iterative denoising steps to generate K_D token representations. It should be understood that the number of token representations is equal to the length of the draft sequence in the draft model configuration parameters, and the number of iterative denoising steps is the number of denoising steps in the draft model configuration parameters.

[0063] Further, calculate the logits distribution of each of the generated K_D token representations on the vocabulary; perform an argmax operation or sampling on the logits distribution of each token representation on the vocabulary to obtain the token ID of each token representation; and output the token IDs of each token representation as a draft sequence.

[0064] Furthermore, the draft sequence T_D and the current context sequence C are input into the large language model for parallel evaluation, outputting what the large language model considers the optimal token sequence, token t_i^L. It should be understood that parallel evaluation is an inherent feature of the large language model's processing; as long as the above sequences are input into the large language model, it can automatically complete the parallel evaluation. The length of the optimal token sequence is equal to the length of the draft sequence in the draft model's configuration parameters.

[0065] Further, initialize the prefix sequence T_A. The draft sequence is T_D = (t_1^D,...,t_{k_D}^D), and the optimal token sequence is (token t_1^L,……,token t_{k_D}^L). Following the order from 1 to k_D, compare the i-th token in the draft sequence with the i-th token in the optimal token sequence. If they match, store t_1^D in the prefix sequence as the i-th element of prefix sequence T_A. Stop the comparison when a token in the draft sequence is inconsistent with the corresponding token in the optimal token sequence for the first time. The prefix sequence T_A at the point of stopping the comparison is the final prefix sequence, and the number of tokens in prefix sequence T_A at the point of stopping the comparison is the number of tokens accepted by the large language model. For example, if the first two tokens in the draft sequence match the corresponding tokens in the optimal token sequence, the matching process stops immediately when the third token does not match the third token in the optimal token sequence. The final prefix sequence consists only of the first two tokens from the draft sequence. The large language model accepts a maximum of 2 tokens.

[0066] The consistency of two tokens can be determined by comparing their token IDs. Alternatively, a greedy matching algorithm can be used (where t_i^D is the value of a large language model). The consistency of two tokens can be determined by using the highest probability token generated under certain conditions. Alternatively, token consistency can be determined through random sampling and matching.

[0067] Furthermore, the prefix sequence T_A is concatenated with the current context sequence C to obtain the preliminary intermediate output sequence S_out: The prefix sequence T_A is appended to the current context sequence C.

[0068] Furthermore, if the number of accepted tokens is less than the length of the draft sequence, the prefix sequence and the current context sequence are input into the large language model to obtain a correction token (the number of correction tokens is generally one); the correction token is then concatenated with the initial intermediate output sequence to obtain the corrected intermediate output sequence. (The corrected token is appended to the initial intermediate output sequence); the corrected intermediate output sequence is used as the final intermediate output sequence; if the number of accepted tokens is equal to the length of the draft sequence, the initial intermediate output sequence is used as the final intermediate output sequence.

[0069] Furthermore, if the length of the final intermediate output sequence reaches the target generation length L_target, or if the final intermediate output sequence contains a sequence end symbol, then the final intermediate output sequence is used as the final output sequence; if the length of the final intermediate output sequence does not reach the target generation length L_target, or if no sequence end symbol appears, then the final intermediate output sequence is used as the current context sequence, and the above decoding operation is repeated until the length of the final intermediate output sequence reaches the target generation length L_target or the final intermediate output sequence contains a sequence end symbol, at which point the loop ends.

[0070] It should be noted that the draft model of this invention can generate longer token sequences than existing autoregressive draft models. The longer draft k_D provides a basis for obtaining more accepted tokens n. Even if the acceptance rate n / k_D remains constant, n will increase with the increase of k_D. If the diffusion model improves the draft quality, the increase in n will be even more significant, which is key to acceleration. Due to the substantial increase in n, the number of "large model validations" required to generate text of the same length will be significantly reduced, thus achieving a non-linear, step-like improvement in inference efficiency. Because a single decoding operation can process and accept more information (i.e., more tokens), the "output" (i.e., the number of accepted tokens) of each interaction with the relatively computationally expensive large language model (validation phase) is higher, improving the utilization efficiency of computing resources.

[0071] For applications that require real-time interaction with large language models (such as chatbots, real-time translation, code assistants, etc.), the significant improvement in inference speed will directly translate into faster system response speed, thereby greatly improving the user experience.

[0072] In one specific embodiment, a control module may be provided to coordinate the workflow of the draft model and the large language model, manage context information, perform the operation of determining the receive prefix, and control the entire iterative process of speculative decoding.

[0073] Step S14: Perform text conversion on the final output sequence, and use the conversion result as the inference result text information corresponding to the context information to be inferred.

[0074] It should be understood that the final output sequence is a token sequence, which can be transformed using existing text conversion methods, which will not be elaborated here.

[0075] Figure 3 This is a schematic block diagram of a large language model inference device based on a diffusion model provided in an embodiment of this application. Figure 3 As shown, the large language model inference device based on the diffusion model includes:

[0076] Model acquisition module 31 is used to obtain a pre-trained draft model based on a diffusion model architecture and determine the large language model to be used.

[0077] The model configuration module 32 is used to configure the draft model according to the pre-obtained draft model configuration parameters;

[0078] The reasoning module 33 is used to acquire the context information to be reasoned and convert it into a context sequence to be reasoned; use the context sequence to be reasoned as the current context sequence; perform a decoding operation on the current context sequence to obtain an intermediate output sequence; perform a termination judgment operation based on the intermediate output sequence according to a preset termination condition to obtain a corresponding judgment result; if the obtained judgment result satisfies the termination condition, then the intermediate output sequence is used as the final output sequence; if the obtained judgment result does not satisfy the termination condition, then the intermediate output sequence is used as the current context sequence and a decoding operation is performed on it until a judgment result that satisfies the termination condition is obtained.

[0079] The decoding operation includes: a draft sequence generation operation based on the draft model after parameter configuration, a verification operation based on the large language model, a receiving prefix determination operation, and a condition judgment and correction operation, which are executed sequentially.

[0080] The result generation module 34 is used to perform text conversion on the final output sequence and use the conversion result as the inference result text information corresponding to the context information to be inferred.

[0081] It should be understood that the specific process of each module performing the above-mentioned steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0082] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0083] In one embodiment, the draft model configuration parameters include: the length of the draft sequence and the number of denoising steps of the draft model.

[0084] In one embodiment, the long sequence draft generation operation based on the parameter-configured draft model includes: performing reverse diffusion processing on the current context sequence using the parameter-configured draft model to generate multiple token representations, and converting the multiple token representations into a draft sequence according to the vocabulary.

[0085] In one embodiment, the verification operation based on a large language model includes: inputting the current context sequence and the draft sequence into the large language model for parallel evaluation to obtain the optimal token sequence.

[0086] In one embodiment, the operation of determining the receiving prefix includes: outputting the portion of the draft sequence that matches the optimal token sequence starting from the first token as a prefix sequence, and outputting the number of tokens in the prefix sequence as the number of tokens accepted by the large language model; concatenating the prefix sequence with the current context sequence to obtain a preliminary intermediate output sequence.

[0087] In one embodiment, the condition judgment operation includes: if the number of accepted tokens is less than the length of the draft sequence, then a correction operation is performed to obtain a corrected intermediate output sequence, and the corrected intermediate output sequence is used as the final intermediate output sequence; if the number of accepted tokens is equal to the length of the draft sequence, then the preliminary intermediate output sequence is used as the final intermediate output sequence.

[0088] In one embodiment, the correction operation includes: inputting the prefix sequence and the current context sequence into a large language model to obtain a correction token; concatenating the correction token with the initial intermediate sequence to obtain a corrected intermediate output sequence.

[0089] Figure 4 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 4 As shown, the electronic terminal includes at least one processor 401, a memory 402, at least one network interface 403, and a user interface 405. The various components in the device are coupled together via a bus system 404. It is understood that the bus system 404 is used to implement communication between these components. In addition to a data bus, the bus system 404 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 4 The general will label all buses as bus systems.

[0090] The user interface 405 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0091] It is understood that memory 402 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0092] In this embodiment of the invention, the memory 402 is used to store various types of data to support the operation of the electronic terminal 400. Examples of this data include: any executable program for operation on the electronic terminal 400, such as the operating system 4021 and application programs 4022; the operating system 4021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 4022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The large language model inference method based on the diffusion model provided in this embodiment of the invention can be included in the application program 4022.

[0093] The methods disclosed in the above embodiments of the present invention can be applied to processor 401, or implemented by processor 401. Processor 401 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 401 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 401 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 401 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0094] In an exemplary embodiment, the electronic terminal 400 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0095] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figure 1 The large language model reasoning method based on the diffusion model in the illustrated embodiment.

[0096] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when executed on a computer, causes the computer to perform... Figure 1 The large language model reasoning method based on the diffusion model in the illustrated embodiment.

[0097] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0098] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0099] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0100] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0103] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0104] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0106] In summary, this application provides a method, apparatus, terminal, and medium for large language model inference based on a diffusion model. This application utilizes a draft model based on a diffusion model framework to generate a draft sequence, and then verifies the draft sequence using a large language model. This application leverages the inherent parallel processing capability of the diffusion model, significantly increasing the length of the draft sequence and thus reducing the number of large model verifications, thereby improving the inference efficiency of the large language model and the utilization efficiency of computing resources. Therefore, this application effectively overcomes the various shortcomings of existing technologies and has high industrial applicability.

[0107] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A large language model reasoning method based on a diffusion model, characterized in that, include: Obtain a pre-trained draft model based on the diffusion model architecture and determine the large language model to be used; wherein, the method of training the draft model specifically includes: constructing a training set for the draft model using the large language model; and training the diffusion model framework using the training set to obtain the trained draft model. Configure the draft model according to the pre-obtained draft model configuration parameters; The process involves: acquiring the context information to be reasoned and converting it into a context sequence; using the context sequence as the current context sequence; decoding the current context sequence to obtain an intermediate output sequence; performing a termination judgment operation based on the intermediate output sequence according to a preset termination condition to obtain a corresponding judgment result; if the obtained judgment result satisfies the termination condition, using the intermediate output sequence as the final output sequence; if the obtained judgment result does not satisfy the termination condition, using the intermediate output sequence as the current context sequence and decoding it until a judgment result that satisfies the termination condition is obtained. The decoding operation includes: a draft sequence generation operation based on the draft model after parameter configuration, a verification operation based on the large language model, a receiving prefix determination operation, and a condition judgment and correction operation, which are executed sequentially. Furthermore, the long-sequence draft generation operation based on the parameter-configured draft model includes: performing reverse diffusion processing on the current context sequence using the parameter-configured draft model to generate multiple token representations, and converting the multiple token representations into a draft sequence according to the vocabulary; The verification operation based on the large language model includes: inputting the current context sequence and the draft sequence into the large language model for parallel evaluation to obtain the optimal token sequence; The operation of determining the receiving prefix includes: outputting the portion of the draft sequence that matches the optimal token sequence starting from the first token as a prefix sequence, and outputting the number of tokens in the prefix sequence as the number of tokens accepted by the large language model; concatenating the prefix sequence with the current context sequence to obtain a preliminary intermediate output sequence; The conditional judgment operation includes: if the number of accepted tokens is less than the length of the draft sequence, then a correction operation is performed to obtain a corrected intermediate output sequence, and the corrected intermediate output sequence is used as the final intermediate output sequence; if the number of accepted tokens is equal to the length of the draft sequence, then the preliminary intermediate output sequence is used as the final intermediate output sequence. The final output sequence is converted into text, and the conversion result is used as the inference result text information corresponding to the context information to be inferred.

2. The large language model reasoning method based on diffusion model according to claim 1, characterized in that, The draft model configuration parameters include: the length of the draft sequence and the number of denoising steps in the draft model.

3. The large language model reasoning method based on diffusion model according to claim 1, characterized in that, The correction operation includes: inputting the prefix sequence and the current context sequence into the large language model to obtain the correction token; concatenating the correction token with the initial intermediate output sequence to obtain the corrected intermediate output sequence.

4. A large language model reasoning device based on a diffusion model, characterized in that, include: The model acquisition module is used to obtain a pre-trained draft model based on the diffusion model architecture and determine the large language model to be used; wherein, the method of training the draft model specifically includes: constructing a training set for the draft model using the large language model; and training the diffusion model framework using the training set to obtain the trained draft model. The model configuration module is used to configure the draft model according to the pre-obtained draft model configuration parameters; The reasoning module is used to acquire the context information to be reasoned and convert it into a context sequence to be reasoned; use the context sequence to be reasoned as the current context sequence; perform a decoding operation on the current context sequence to obtain an intermediate output sequence; perform a termination judgment operation based on the intermediate output sequence according to a preset termination condition to obtain a corresponding judgment result; if the obtained judgment result satisfies the termination condition, then the intermediate output sequence is used as the final output sequence; if the obtained judgment result does not satisfy the termination condition, then the intermediate output sequence is used as the current context sequence and a decoding operation is performed on it until a judgment result that satisfies the termination condition is obtained. The decoding operation includes: a draft sequence generation operation based on the draft model after parameter configuration, a verification operation based on the large language model, a receiving prefix determination operation, and a condition judgment and correction operation, which are executed sequentially. Furthermore, the long-sequence draft generation operation based on the parameter-configured draft model includes: performing reverse diffusion processing on the current context sequence using the parameter-configured draft model to generate multiple token representations, and converting the multiple token representations into a draft sequence according to the vocabulary; The verification operation based on the large language model includes: inputting the current context sequence and the draft sequence into the large language model for parallel evaluation to obtain the optimal token sequence; The operation of determining the receiving prefix includes: outputting the portion of the draft sequence that matches the optimal token sequence starting from the first token as a prefix sequence, and outputting the number of tokens in the prefix sequence as the number of tokens accepted by the large language model; concatenating the prefix sequence with the current context sequence to obtain a preliminary intermediate output sequence; The conditional judgment operation includes: if the number of accepted tokens is less than the length of the draft sequence, then a correction operation is performed to obtain a corrected intermediate output sequence, and the corrected intermediate output sequence is used as the final intermediate output sequence; if the number of accepted tokens is equal to the length of the draft sequence, then the preliminary intermediate output sequence is used as the final intermediate output sequence. The result generation module is used to perform text conversion on the final output sequence and use the conversion result as the inference result text information corresponding to the context information to be inferred.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 3.

6. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1 to 3.

Images