A large language model reasoning acceleration method and system based on a progressive drafting tree

Abstract

The application discloses a large language model reasoning acceleration method and system based on a progressive draft tree, and the method comprises the following steps: constructing an initial draft tree, the root node of the initial draft tree is associated with a decoded context sequence, and the remaining nodes are generated by random initialization to generate perturbed words; setting attention masks and position encodings based on the dependency relationship between nodes, so that each node can only focus on the previous nodes on its path; using a large language model to reason the initial draft tree, generating draft words, and expanding the draft tree and updating the candidate pool according to the draft words; pruning the expanded draft tree through preset width constraints and depth constraints; extracting candidate draft trees from the candidate pool. The application converts the traditional autoregressive single-step prediction task into a parallel drafting and verification process, proposes a draft generation mechanism based on a progressive draft tree, and realizes efficient reasoning acceleration while maintaining the original model architecture and parameters.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, specifically a method and system for accelerating reasoning in large language models based on progressive drafting trees. Background Technology

[0002] Current large-scale language models have significantly improved training efficiency and text generation capabilities through parallel training on massive amounts of data. For example... Figure 2 As shown, (a) is the traditional autoregressive method and (b) is the general speculative decoding method. In the autoregressive decoding process, since it is necessary to generate labels sequentially and load the complete model parameters into the on-chip cache for each decoding step, the inference process is continuously limited by GPU bandwidth, and the system is in a memory-constrained state for a long time.

[0003] To address the aforementioned issues, existing technologies primarily focus on two directions: model parameter compression and computational resource optimization. Model parameter compression refers to reducing the model size through methods such as pruning, distillation, and quantization to decrease data transmission volume, but this carries the risk of performance loss. Computational resource optimization, on the other hand, employs techniques such as data / model parallelization, operator fusion, caching mechanisms, and speculative decoding to improve GPU utilization. Among these, speculative decoding has attracted widespread attention due to its lossless nature and ease of deployment.

[0004] Existing speculative decoding techniques primarily construct candidate sequences for validation using three methods. First, the simplest approach is to pre-build a candidate draft sequence based on a corpus. However, its effectiveness is limited by the matching degree between the corpus and the current context, exhibiting strong dependence on external corpora and making it difficult to balance generality with contextual relevance. Second, some methods quickly generate candidate drafts using small draft models. However, acquiring draft models typically requires additional training overhead, and serial communication between large and small models introduces additional hardware resource requirements and communication latency overhead, which often significantly limits the performance of this method. Furthermore, some self-drafting decoding works aim to enable large language models to reason in parallel by modifying the basic architecture of the model. However, this usually requires additional training and may affect the model's inference quality.

[0005] Experiments on the robustness of large language models to noise perturbations revealed that even after applying random perturbations to the input sequence, the model can still generate effective content highly correlated with the original output. Furthermore, in autoregressive decoding methods, GPUs are typically memory-constrained, meaning there are still sufficient computational resources available for parallelization. Summary of the Invention

[0006] This embodiment provides a method, system, electronic device, and storage medium for accelerating inference of a large language model based on progressive drafting trees. It can utilize the robustness of the large language model itself to transform the autoregressive decoding process into an efficient drafting and verification process under tree perturbation, achieving efficient inference acceleration while maintaining the original model architecture and parameters.

[0007] In a first aspect, embodiments of the present invention provide a method for accelerating large language model inference based on progressive drafting trees, the method comprising:

[0008] An initial drafting tree is constructed, wherein the root node of the initial drafting tree is associated with the decoded context sequence, and the remaining nodes are generated by randomly initializing perturbation tokens;

[0009] Based on the dependencies between nodes, attention masks and positional encodings are set so that each node can only pay attention to the preceding nodes on its path.

[0010] The initial draft tree is inferred using a large language model to generate draft lexical units, and the draft tree is expanded and the candidate pool is updated based on the draft lexical units.

[0011] The expanded drafting tree is pruned by pre-setting width and depth constraints in order to control the size of the tree structure and semantic coherence.

[0012] Candidate draft trees are extracted from the candidate pool, and the validity of each node is verified in parallel. Continuous paths that are consistent with the model predictions are retained as the final decoding sequence.

[0013] In an optional embodiment, the structure of the initial drafting tree is as follows: ;in,

[0014] It is a set of nodes containing the root node and randomly initialized perturbation lexical units;

[0015] Let be the set of edges representing the partial order relationship between nodes.

[0016] In an optional embodiment, the attention mask is implemented through dependencies between nodes in the drafting tree to identify hierarchical relationships and path dependencies between tree nodes.

[0017] In an optional embodiment, the specific process of expanding the drafting tree includes:

[0018] Add the draft lexical units generated by each node as child nodes under that node;

[0019] Update node Sum of edges ,in, For nodes The generated draft words.

[0020] In an optional embodiment, the pruning operation includes:

[0021] Limit the number of child nodes of each node to no more than a preset width threshold;

[0022] Limit the depth of each subtree to no more than a preset depth threshold;

[0023] When the depth of a subtree reaches the depth threshold, its root node is deleted and part of the subtree is retained to maintain semantic coherence.

[0024] In an optional embodiment, the parallel verification includes:

[0025] The same approach is used to set attention masks and positional codes for both the draft tree and the candidate draft tree;

[0026] The output of the large language model is used to verify whether the child nodes of each node match the predicted words, and the continuous valid paths starting from the root node are preserved.

[0027] In an optional embodiment, a candidate pool maintenance step is also included:

[0028] Merge the subtrees of each node in the drafting tree with the corresponding candidate trees in the candidate pool, and add any missing nodes to update the candidate pool.

[0029] Compared with existing technologies, the beneficial effects of the large language model reasoning acceleration method based on progressive drafting trees in this invention are as follows:

[0030] During the drafting phase, this invention leverages the robustness of large language models under input perturbations to overcome the limitations of traditional linear decoding by constructing a tree-like drafting structure (referred to as the drafting tree in this invention) for efficient drafting. First, a randomly initialized drafting tree is introduced at the end of the input sequence as an input perturbation. An attention mask is designed using a depth-first traversal rule to ensure that each node only focuses on the preceding nodes of its path, enabling the Transformer architecture to support this non-sequential data structure. Simultaneously, a dual constraint is designed on the drafting tree to ensure that the tree structure can efficiently generate high-quality drafts: limiting the number of child nodes of each node to limit the overall tree width and filter low-quality drafts, and designing a pruning algorithm to ensure the relevance of the drafting structure to the contextual semantics and the coherence of the draft content.

[0031] The candidate validation phase employs a parallel processing mechanism. After extracting candidate draft trees from the draft pool, the same attention masking strategy is used for validation. The output of the large language model is used to validate the validity of each node based on the child node matching principle. Only the continuous path starting from the root node of the draft tree that is consistent with the model prediction is retained as the final decoding sequence.

[0032] Compared to other speculative decoding methods, the method proposed in this invention requires no additional training or auxiliary models or pre-built caches to obtain context-appropriate drafts for validation. Furthermore, compared to other self-drafting decoding methods, the method proposed in this invention does not require modification or fine-tuning of model parameters, and the solution is fully compatible with mainstream Transformer architectures, supporting plug-and-play deployment.

[0033] Secondly, embodiments of the present invention provide a large language model reasoning acceleration system based on progressive drafting trees, comprising:

[0034] The drafting module is used to build an initial draft tree and generate draft lexical units;

[0035] The pruning module is used to prune the draft tree according to preset constraints;

[0036] The validation module is used to validate the validity of candidate draft trees in parallel.

[0037] The maintenance module is used to update the candidate tree in the candidate pool.

[0038] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a bus, wherein the processor, the communication interface, and the memory communicate with each other through the bus, and the processor can call logical instructions in the memory to execute the steps of the method provided in the first aspect.

[0039] Fourthly, embodiments of the present invention provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the large language model inference acceleration method based on progressive drafting trees as described in the first aspect.

[0040] Compared with the prior art, the beneficial effects of the large language model reasoning acceleration system, electronic device and storage medium based on progressive drafting tree of the present invention are the same as those of the large language model reasoning acceleration method based on progressive drafting tree described in the first aspect, so they will not be repeated here. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of the method for accelerating large language model inference based on progressive drafting trees in an embodiment of the present invention;

[0043] Figure 2 This is a comparison diagram of the large language model inference acceleration method proposed in this embodiment of the invention with traditional autoregressive methods and general speculative decoding methods;

[0044] Figure 3 This is an example of the expansion and pruning of the draft tree and the attention mask matrix generated by depth-first search, as used in embodiments of the present invention.

[0045] Figure 4 This is a structural block diagram of the large language model inference acceleration system based on progressive drafting trees in an embodiment of the present invention;

[0046] Figure 5 This is a structural block diagram of the electronic device in an embodiment of the present invention. Detailed Implementation

[0047] To better understand the purpose, technical solution, and advantages of this application, the application is described and explained below in conjunction with the accompanying drawings and embodiments.

[0048] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0049] This invention provides a method for accelerating reasoning in large language models based on progressive drafting trees. Figure 1 This is a flowchart of the large language model reasoning acceleration method based on progressive drafting trees of the present invention. Figure 2 Image (c) is a schematic diagram of the large language model reasoning acceleration method proposed in this invention, as shown below. Figure 1 and Figure 2 As shown in (c), the process includes the following steps:

[0050] S100. Construct an initial draft tree. The root node of the initial draft tree is associated with the decoded context sequence, and the remaining nodes generate perturbation tokens through random initialization.

[0051] It should be noted that the initial draft tree structure was as follows: ;in,

[0052] It is a set of nodes containing the root node and randomly initialized perturbation lexical units;

[0053] Let be the set of edges representing the partial order relationship between nodes.

[0054] Specifically, the present invention first creates an initial drafting tree structure. ,in, This represents a randomly initialized perturbation word. In other words, the edges between nodes represent the partial order relationships between them. The receptive field of each node's context is determined by the path from the root node to that node, and all nodes on the path constitute the node's dependency sequence. The root node is associated with the decoded context sequence, while other nodes are generated through random initialization to introduce diverse perturbations to ensure the diversity of subsequently generated content.

[0055] S200: Based on the dependencies between nodes, attention masks and position codes are set so that each node can only pay attention to the preceding nodes on its path.

[0056] In this embodiment, attention masking is implemented through the dependency relationships between nodes in the drafting tree to identify the hierarchical relationships and path dependencies between tree nodes.

[0057] Specifically, a depth-first traversal approach is used to generate attention masks that support tree-structured inputs. Each lexical can only access its preceding nodes on its path when calculating attention. For example, in... Figure 3 In the tree structure shown, the attention scope of a third-level node only includes the root node, parent node, and itself along its path. Relative position encoding is also introduced to ensure correct partial order relationships and the model's ability to recognize hierarchical relationships between tree nodes.

[0058] S300. Use a large language model to reason about the initial draft tree, generate draft lexical units, and expand the draft tree based on the draft lexical units.

[0059] In this embodiment, the specific process of expanding the drafting tree includes:

[0060] Add the draft lexical units generated by each node as child nodes under that node;

[0061] Update node Sum of edges ,in, For nodes The generated draft words.

[0062] In traditional autoregressive decoding methods, each large language model is based on the already generated sequences. The probability distribution of the next word is obtained, and then the next word is obtained according to a certain sampling method. This process can be described as follows:

[0063] ,

[0064] in, Representation Model In the input The probability distribution of the output under the given condition, This indicates the sampling strategy, with common strategies including greedy strategies, top-p, and top-k.

[0065] In the method proposed in this invention, during each model inference, all nodes in the drafting tree obtain corresponding draft terms based on the probability distribution of the model's inference results. The drafting tree is then expanded based on these draft terms. Because the strategy in this invention limits the receptive field of each node, it ensures the relevance and coherence of the generated draft content with the context. This process can be formally described as follows:

[0066] ,

[0067] in Indicates the first The drafting status of the steps, This represents all the draft lexical units generated by the large language model based on this draft tree.

[0068] S400. Prune the expanded drafting tree by setting width and depth constraints to control the size of the tree structure and semantic coherence.

[0069] In this embodiment, the pruning operation includes:

[0070] Limit the number of child nodes of each node to no more than a preset width threshold;

[0071] Limit the depth of each subtree to no more than a preset depth threshold;

[0072] When the depth of a subtree reaches a depth threshold, its root node is deleted and part of the subtree is retained to maintain semantic coherence.

[0073] Specifically, in order to control the number of nodes in the drafting tree, this invention controls the size of the drafting tree and the coherence of the drafting content from two aspects: width and depth. Regarding width, this invention limits the maximum number of child nodes for each node in the drafting tree to 4 (this limitation is the parameter used in this embodiment and can be adjusted). Once the maximum number of child nodes is reached, no new child nodes are added.

[0074] Regarding depth, this invention limits the depth of each subtree in the drafting tree to 6 (this limitation is the parameter used in this example and can be adjusted). When the depth of a subtree reaches the maximum value, this invention will perform a step-by-step operation on the drafting tree, that is, delete the root node of the subtree, and then select a child node and all its successor nodes from its child nodes according to the first-in-first-out principle as a new subtree, and delete the remaining subtrees (as shown in Figure 2). This step-by-step strategy preserves the semantic information in the original drafting tree as much as possible, which can ensure the semantic coherence of the drafted content.

[0075] S500: Extract candidate draft trees from the candidate pool, verify the effectiveness of each node in parallel, and retain the continuous path consistent with the model prediction as the final decoding sequence.

[0076] In this embodiment, parallel verification includes:

[0077] The same approach is used to set attention masks and positional codes for both the draft tree and the candidate draft tree;

[0078] The output of the large language model is used to verify whether the child nodes of each node match the predicted words, and the continuous valid paths starting from the root node are preserved.

[0079] It also includes the candidate pool maintenance steps:

[0080] Merge the subtrees of each node in the drafting tree with the corresponding candidate trees in the candidate pool, and add any missing nodes to update the candidate pool.

[0081] Specifically, during each forward propagation of the model, the draft tree generates new draft lexical units from the large language model within the current context, thus bringing new semantic information. This invention also updates the draft pool after each draft tree update. Specifically, each node in the draft tree, as the root node, corresponds to a subtree structure that organizes a series of semantically related lexical units. This invention fuses all subtree structures in the draft tree with the corresponding candidate semantic trees located in the candidate pool. The specific fusion process is as follows:

[0082] Each node in the drafting tree Extract its corresponding subtree structure as the root. .

[0083] Query whether the candidate pool contains the following: Candidate trees with roots If not, just put Add to candidate pool

[0084] If the candidate pool contains [a certain type of candidate] If a candidate tree is found to be the root, then a tree merging operation will be performed, which means traversing the tree starting from the root node. and recursively Nodes that do not exist in the pool are added to the candidate pool.

[0085] In addition to normal decoding and drafting, this invention also performs parallel verification of candidate draft trees in each model forward propagation. This is the reason why this invention can decode multiple tokens in a single forward propagation. Specifically, this invention first selects from the candidate pool based on the currently decoded sequence. Extract the corresponding candidate draft tree To enable the large language model to support candidate draft trees, similar to the draft tree, this invention sets the same attention mask matrix for the candidate draft trees in the same way, so that each node in the draft tree can only pay attention to its predecessor node. Then, after each forward propagation of the model, each node in the draft tree generates a candidate lexical based on its prefix. This invention describes this process together with the aforementioned drafting stage as follows:

[0086] ,

[0087] in, This represents the validation tokens obtained from all nodes in the candidate draft tree based on their prefixes. If a node's validation token already exists in its child nodes, then this token is considered valid. The path of valid tokens starting from the root node is the validation result, which is the decoding result of this invention in one model forward propagation process.

[0088] In summary, this invention transforms the autoregressive decoding process into two processes: drafting and verification.

[0089] During the drafting phase, this invention leverages the robustness of large language models under input perturbations to overcome the limitations of traditional linear decoding by constructing a tree-like drafting structure (referred to as the drafting tree in this invention) for efficient drafting. First, a randomly initialized drafting tree is introduced at the end of the input sequence as an input perturbation. An attention mask is designed using a depth-first traversal rule to ensure that each node only focuses on the preceding nodes of its path, enabling the Transformer architecture to support this non-sequential data structure. Simultaneously, a dual constraint is designed on the drafting tree to ensure that the tree structure can efficiently generate high-quality drafts: limiting the number of child nodes of each node to limit the overall tree width and filter low-quality drafts, and designing a pruning algorithm to ensure the relevance of the drafting structure to the contextual semantics and the coherence of the draft content.

[0090] The candidate validation phase employs a parallel processing mechanism. After extracting candidate draft trees from the draft pool, the same attention masking strategy is used for validation. The output of the large language model is used to validate the validity of each node based on the child node matching principle. Only the continuous path starting from the root node of the draft tree that is consistent with the model prediction is retained as the final decoding sequence.

[0091] As can be seen from the technical solution provided by the present invention, compared with other speculative decoding methods, the method proposed in this invention does not require any additional training and auxiliary models or pre-built caches to obtain a context-compliant draft for verification. Furthermore, compared with other self-drafting decoding methods, the method proposed in this invention does not require modification or fine-tuning of model parameters, and the solution is fully compatible with mainstream Transformer architectures, supporting plug-and-play deployment.

[0092] This invention also provides a large language model inference acceleration system based on progressive drafting trees, which is used to implement the above-described method embodiments; details already described will not be repeated. The terms "module," "unit," and "subunit," etc., used below refer to combinations of software and / or hardware that perform predetermined functions. Although the system described in the following embodiments is preferably implemented in software, hardware implementation or a combination of software and hardware is also possible and contemplated.

[0093] like Figure 4 As shown, Figure 4 This is a block diagram of the large language model inference acceleration system based on progressive drafting trees in this invention. The system includes:

[0094] Drafting module 101 is used to construct the initial drafting tree and generate draft lexical units;

[0095] The pruning module 102 is used to prune the draft tree according to preset constraints;

[0096] The verification module 103 is used to verify the validity of the candidate draft tree in parallel.

[0097] Maintenance module 104 is used to update the candidate tree in the candidate pool.

[0098] The beneficial effects of the large language model reasoning acceleration system based on progressive drafting tree of the present invention are the same as those of the large language model reasoning acceleration method based on progressive drafting tree described above, so they will not be repeated here.

[0099] Figure 5 A structural block diagram of the electronic device provided in the embodiments of the present invention, such as... Figure 5 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the following methods:

[0100] An initial drafting tree is constructed, wherein the root node of the initial drafting tree is associated with the decoded context sequence, and the remaining nodes are generated by randomly initializing perturbation tokens;

[0101] Based on the dependencies between nodes, attention masks and positional encodings are set so that each node can only pay attention to the preceding nodes on its path.

[0102] The initial draft tree is inferred using a large language model to generate draft lexical units, and the draft tree is expanded and the candidate pool is updated based on the draft lexical units.

[0103] The expanded drafting tree is pruned by pre-setting width and depth constraints in order to control the size of the tree structure and semantic coherence.

[0104] Candidate draft trees are extracted from the candidate pool, and the validity of each node is verified in parallel. Continuous paths that are consistent with the model predictions are retained as the final decoding sequence.

[0105] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0107] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for accelerating reasoning in large language models based on progressive drafting trees, characterized in that, The method for accelerating inference in large language models based on progressive drafting trees includes: An initial drafting tree is constructed, wherein the root node of the initial drafting tree is associated with the decoded context sequence, and the remaining nodes are generated by randomly initializing perturbation tokens; Based on the dependencies between nodes, attention masks and positional encodings are set so that each node can only pay attention to the preceding nodes on its path. The initial draft tree is inferred using a large language model to generate draft lexical units, and the draft tree is expanded and the candidate pool is updated based on the draft lexical units. The expanded drafting tree is pruned by pre-setting width and depth constraints in order to control the size of the tree structure and semantic coherence. Candidate draft trees are extracted from the candidate pool, and the validity of each node is verified in parallel. Continuous paths that are consistent with the model predictions are retained as the final decoding sequence.

2. The method for accelerating large language model inference based on progressive drafting trees according to claim 1, characterized in that, The structure of the initial drafting tree is as follows: ;in, It is a set of nodes containing the root node and randomly initialized perturbation lexical units; Let be the set of edges representing the partial order relationship between nodes.

3. The method for accelerating reasoning in large language models based on progressive drafting trees according to claim 1, characterized in that, The attention mask is implemented through the dependencies between nodes in the drafting tree to identify hierarchical relationships and path dependencies between tree nodes.

4. The method for accelerating large language model inference based on progressive drafting trees according to claim 1, characterized in that, The specific process of expanding the drafting tree includes: Add the draft lexical units generated by each node as child nodes under that node; Update node Sum of edges ,in, For nodes The generated draft words.

5. The method for accelerating reasoning in large language models based on progressive drafting trees according to claim 1, characterized in that, The pruning operation includes: Limit the number of child nodes of each node to no more than a preset width threshold; Limit the depth of each subtree to no more than a preset depth threshold; When the depth of a subtree reaches the depth threshold, its root node is deleted and part of the subtree is retained to maintain semantic coherence.

6. The method for accelerating large language model inference based on progressive drafting trees according to claim 1, characterized in that, The parallel verification includes: The same approach is used to set attention masks and positional codes for both the draft tree and the candidate draft tree; The output of the large language model is used to verify whether the child nodes of each node match the predicted words, and the continuous valid paths starting from the root node are preserved.

7. The method for accelerating reasoning in large language models based on progressive drafting trees according to claim 1, characterized in that, It also includes the candidate pool maintenance steps: Merge the subtrees of each node in the drafting tree with the corresponding candidate trees in the candidate pool, and add any missing nodes to update the candidate pool.

8. A large language model reasoning acceleration system based on progressive drafting trees, used to implement the method described in any one of claims 1-7, characterized in that, include: The drafting module is used to build an initial draft tree and generate draft lexical units; The pruning module is used to prune the draft tree according to preset constraints; The validation module is used to validate the validity of candidate draft trees in parallel. The maintenance module is used to update the candidate tree in the candidate pool.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for accelerating large language model inference based on progressive drafting trees as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for accelerating large language model inference based on progressive drafting trees as described in any one of claims 1 to 7.

Images