Large language model reasoning acceleration method and system based on progressive grassy tree

Abstract

The invention discloses a large language model reasoning acceleration method and system based on a progressive grass-up tree, and the method comprises the steps: constructing an initial grass-up tree, associating a root node of the initial grass-up tree with a decoded context sequence, and generating disturbance lexical elements through random initialization of other nodes; an attention mask and a position code are set based on the dependency relationship between the nodes, so that each node can only pay attention to a preorder node on a path where the node is located; reasoning the initial draft tree by using a large language model, generating draft lemma, and expanding the draft tree and updating the candidate pool according to the draft lemma; pruning the expanded grass raising tree through preset width constraint and depth constraint; a candidate draft tree is extracted from the candidate pool. According to the method, a traditional autoregression single-step prediction task is converted into a parallel draft and verification process, a draft generation mechanism based on a progressive draft tree is provided, and efficient reasoning acceleration is achieved under the condition that an original model architecture and parameters are kept.

Description

Technical Field

[0001] The present application relates to the field of natural language processing technology, and more specifically to a large language model reasoning acceleration method and system based on a progressive drafting tree. Background Art

[0002] Current large-scale language models have significantly improved training efficiency and text generation capabilities through parallel training of massive data. Figure 2 As shown in the figure, (a) is the traditional autoregressive method and (b) is the general speculative decoding method. During the autoregressive decoding process, since labels need to be generated sequence by sequence and the complete model parameters need to be loaded into the on-chip cache for each decoding step, the inference process is continuously limited by the GPU bandwidth and the system is in a memory-limited state for a long time.

[0003] To address these issues, existing research focuses on two main areas: model parameter compression and computing resource optimization. Model parameter compression involves reducing the size of the model through methods such as pruning, distillation, and quantization to reduce data transmission, but this carries the risk of performance loss. Computing resource optimization, on the other hand, utilizes technologies such as data / model parallelism, operator fusion, caching mechanisms, and speculative decoding to improve GPU utilization. Speculative decoding, in particular, has attracted widespread attention due to its lossless nature and ease of deployment.

[0004] Existing speculative decoding technology mainly uses the following three methods to construct candidate sequences for verification. First, the simplest method is to pre-construct a candidate draft sequence based on a corpus, but its effectiveness is limited by the matching degree between the corpus and the current context. It is highly dependent on the external corpus and it is difficult to balance universality and context relevance. Secondly, some methods quickly generate candidate drafts through some small draft models, but the acquisition of draft models usually requires additional training overhead, and the serial communication between large and small models will introduce additional hardware resource requirements and communication delay overhead, which often greatly limits the performance of this method. In addition, other self-drafting decoding works are committed to modifying the basic architecture of the model so that large language models can be inferred in parallel, but this usually requires additional training and may affect the inference quality of the model.

[0005] Experiments examining the robustness of large language models to noise perturbations have shown that, even after random perturbations are applied to the input sequence, the model can still generate valid content that is highly correlated with the original output. Furthermore, with autoregressive decoding, GPUs are typically memory-constrained, meaning ample computing resources are still available for parallelization. Summary of the Invention

[0006] In this embodiment, a large language model reasoning acceleration method, system, electronic device and storage medium based on a progressive drafting tree are provided. The method can utilize the robustness of the large language model itself to convert the autoregressive decoding process into an efficient drafting and verification process under tree perturbations, thereby achieving efficient reasoning acceleration while maintaining the original model architecture and parameters.

[0007] In a first aspect, an embodiment of the present invention provides a large language model reasoning acceleration method based on a progressive drafting tree, the large language model reasoning acceleration method based on a progressive drafting tree comprising: Constructing an initial drafting tree, wherein the root node of the initial drafting tree is associated with the decoded context sequence, and the remaining nodes are randomly initialized to generate perturbation tokens; Attention masks and position encodings are set based on the dependencies between nodes, so that each node can only focus on the previous nodes on its path; Using a large language model to reason on the initial draft tree to generate draft tokens, and expanding the draft tree and updating the candidate pool based on the draft tokens; Prune the expanded drafting tree by presetting width and depth constraints to control the scale and semantic coherence of the tree structure; Candidate draft trees are extracted from the candidate pool, the validity of each node is verified in parallel, and the continuous path consistent with the model prediction is retained as the final decoding sequence.

[0008] In an optional embodiment, the structure of the initial drafting tree is ;in, is a node set containing the root node and randomly initialized perturbation tokens; is a set of edges that represent the partial order relationship between nodes.

[0009] In an optional embodiment, the attention mask is implemented by drafting the dependencies between nodes in the tree to identify the hierarchical relationships and path dependencies between tree nodes.

[0010] In an optional embodiment, the specific process of extending the drafting tree includes: Add the draft token generated by each node as a child node under the node; Update Node and edge sets ,in, For nodes Generated draft tokens.

[0011] In an optional embodiment, the pruning operation includes: Limit the number of child nodes of each node to no more than the preset width threshold; Limit the depth of each subtree to no more than a preset depth threshold; When the depth of a subtree reaches the depth threshold, its root node is deleted and part of the subtree is retained to maintain semantic coherence.

[0012] In an optional embodiment, the parallel verification includes: The same method is used to set the attention mask and position encoding for the draft tree and the candidate draft tree respectively; Verify whether the child nodes of each node match the predicted word based on the output of the large language model, and retain the continuous valid path starting from the root node.

[0013] In an optional embodiment, the method further includes the following steps: Merge the subtree of each node in the drafting tree with the corresponding candidate tree in the candidate pool, and supplement the missing nodes to update the candidate pool.

[0014] Compared with the prior art, the large language model reasoning acceleration method based on progressive drafting tree of the present invention has the following beneficial effects: During the drafting phase, the present invention leverages the robustness of large language models under input perturbations. By constructing a tree-like drafting structure (referred to as a drafting tree in this invention), this approach overcomes the limitations of traditional linear decoding and enables efficient drafting. First, a randomly initialized drafting tree is introduced at the end of the input sequence as an input perturbation. An attention mask is designed using a depth-first traversal rule to ensure that each node focuses only on the preceding node in its path, enabling the Transformer architecture to support this non-sequential data structure. Simultaneously, dual constraints are placed on the drafting tree to ensure that the tree structure can efficiently generate high-quality drafts. This involves limiting the number of children per node, thereby limiting the overall tree width and filtering out low-quality drafts. Furthermore, a pruning algorithm is designed to ensure the relevance of the drafting structure to the contextual semantics and the coherence of the draft content.

[0015] The candidate verification phase uses a parallel processing mechanism. After extracting candidate draft trees from the draft pool, they are verified using the same attention mask setting strategy. The output of the large language model is used to verify the validity of each node based on the child node matching principle. Only the continuous path starting from the root node of the draft tree that is consistent with the model prediction is retained as the final decoding sequence.

[0016] Compared to other speculative decoding methods, the proposed method does not require any additional training, auxiliary models, or pre-built caches to obtain context-appropriate drafts for verification. Furthermore, compared to other self-drafting decoding methods, the proposed method does not require model parameter modifications or fine-tuning. The solution is fully compatible with the mainstream Transformer architecture and supports plug-and-play deployment.

[0017] In a second aspect, an embodiment of the present invention provides a large language model reasoning acceleration system based on a progressive drafting tree, comprising: Drafting module, used to build the initial draft tree and generate draft tokens; Pruning module, used to prune the drafting tree according to preset constraints; Verification module, used to verify the validity of candidate draft trees in parallel; The maintenance module is used to update the candidate trees in the candidate pool.

[0018] In a third aspect, an embodiment of the present invention provides an electronic device comprising a processor, a communication interface, a memory and a bus, wherein the processor, the communication interface and the memory communicate with each other through the bus, and the processor can call logic instructions in the memory to execute the steps of the method provided in the first aspect.

[0019] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the large language model reasoning acceleration method based on the progressive drafting tree as described in the first aspect.

[0020] Compared with the prior art, the beneficial effects of the large language model reasoning acceleration system, electronic device and storage medium based on the progressive drafting tree of the present invention are the same as those of the large language model reasoning acceleration method based on the progressive drafting tree described in the first aspect, so they will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 Flowchart of a large language model reasoning acceleration method based on progressive drafting tree in an embodiment of the present invention; Figure 2 A comparison chart of the large language model inference acceleration method proposed in an embodiment of the present invention, the traditional autoregressive method, and the general speculative decoding method; Figure 3 An example of the expansion and pruning of the scratch tree and the depth-first generation of the attention mask matrix used in an embodiment of the present invention; Figure 4 This is a structural block diagram of a large language model reasoning acceleration system based on a progressive drafting tree in an embodiment of the present invention; Figure 52 is a structural block diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to more clearly understand the purpose, technical solutions and advantages of this application, this application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0024] Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meanings as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," "these," and similar expressions in this application do not denote limitations on quantity and may be singular or plural. The terms "comprise," "include," "have," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include unlisted steps or modules (units) or other steps or modules (units) inherent to the process, method, product, or device. The terms "connected," "connected," "coupled," and similar expressions used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used in this application, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone; A and B exist simultaneously; or B exists alone. Generally, the character " / " indicates that the objects in the preceding and following relationship are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0025] In an embodiment of the present invention, a large language model reasoning acceleration method based on progressive drafting tree is provided. Figure 1 is a flowchart of the large language model reasoning acceleration method based on progressive drafting tree of the present invention, Figure 2 (c) is a schematic diagram of the large language model reasoning acceleration method proposed by the present invention, as shown in FIG. Figure 1 and Figure 2 As shown in (c), the process includes the following steps: S100, constructing an initial drafting tree, where the root node of the initial drafting tree is associated with the decoded context sequence, and the remaining nodes are randomly initialized to generate perturbation tokens; It should be noted that the structure of the initial drafting tree is ;in, is a node set containing the root node and randomly initialized perturbation tokens; is a set of edges that represent the partial order relationship between nodes.

[0026] Specifically, the present invention first creates an initial drafting tree structure ,in, represents a randomly initialized perturbation token, In other words, edges between nodes represent a partial order relationship between them. The receptive field of each node's context is determined by the path from the root node to that node. All nodes on the path constitute the node's dependency sequence. The root node is associated with the decoded context sequence, and other nodes are generated through random initialization to introduce diverse perturbations to ensure the diversity of the subsequent generated content.

[0027] S200, setting an attention mask and position encoding based on the dependency relationship between nodes so that each node can only focus on the previous node on its path; In this embodiment, the attention mask is implemented by drafting the dependency relationships between nodes in the tree to identify the hierarchical relationships and path dependencies between tree nodes.

[0028] Specifically, a depth-first traversal method is used to generate an attention mask that supports tree-structured input. When calculating attention, each word can only access the previous node on its path. For example, in Figure 3 In the tree structure shown, the attention scope of the third-layer node only includes the root node, parent node, and itself in its path. Relative position encoding is also introduced to ensure the correct partial order relationship and enable the model to recognize the hierarchical relationship between tree nodes.

[0029] S300: Using the large language model to infer the initial draft tree, generate draft tokens, and expand the draft tree based on the draft tokens; In this embodiment, the specific process of extending the drafting tree includes: Add the draft token generated by each node as a child node under the node; Update Node and edge sets ,in, For nodes Generated draft tokens.

[0030] In the traditional autoregressive decoding method, each large language model will be generated based on the sequence Get the probability distribution of the next word, and then get the next word according to a certain sampling method. This process can be described as follows: , in, Representation Model On input The probability distribution of the output under the condition of Represents the sampling strategy, common ones include greedy strategy, top-p, top-k, etc.

[0031] In the method proposed in this invention, during each model inference, all nodes in the drafting tree obtain corresponding draft tokens based on the probability distribution of the model's inference results. The drafting tree is then expanded based on the draft tokens. Because the strategy in this invention limits the receptive field of each node, the invention can ensure that the generated draft content is relevant and coherent with the context. This process can be formally described as follows: , in Indicates the The drafting tree status of the step, Represents all draft tokens generated by the large language model based on the draft tree.

[0032] S400, pruning the expanded drafting tree by presetting width constraints and depth constraints to control the scale and semantic coherence of the tree structure; In this embodiment, the pruning operation includes: Limit the number of child nodes of each node to no more than the preset width threshold; Limit the depth of each subtree to no more than a preset depth threshold; When the depth of a subtree reaches a depth threshold, its root node is deleted and part of the subtree is retained to maintain semantic coherence.

[0033] Specifically, to control the number of nodes in the drafting tree, the present invention manages the tree's size and the coherence of the drafting content from two perspectives: width and depth. Regarding width, the present invention limits the maximum number of child nodes per node in the drafting tree to four (this limit is a parameter used in this embodiment and is adjustable). Once the maximum number of child nodes is reached, no new child nodes are added.

[0034] Regarding depth, the present invention limits the depth of each subtree in the drafting tree to 6 (this limitation is a parameter used in this example and is adjustable). When the depth of a subtree reaches the maximum value, the present invention will perform a step operation on the drafting tree, that is, delete the root node of the subtree, and then select a child node and all its successor nodes from its child nodes according to the first-in-first-out principle as a new subtree, and delete the remaining subtrees (as shown in Figure 2). This step strategy retains the semantic information in the original drafting tree as much as possible, and can ensure the semantic coherence of the drafting content.

[0035] S500: Extract candidate draft trees from the candidate pool, verify the validity of each node in parallel, and retain the continuous path consistent with the model prediction as the final decoding sequence.

[0036] In this embodiment, parallel verification includes: The same method is used to set the attention mask and position encoding for the draft tree and the candidate draft tree respectively; Verify whether the child nodes of each node match the predicted word based on the output of the large language model, and retain the continuous valid path starting from the root node.

[0037] Also includes candidate pool maintenance steps: Merge the subtree of each node in the drafting tree with the corresponding candidate tree in the candidate pool, and supplement the missing nodes to update the candidate pool.

[0038] Specifically, during each forward propagation of the model, the draft tree generates new draft words in the current context by the large language model, thereby bringing new semantic information. The present invention also updates the draft pool after each draft tree update. Specifically, each node in the draft tree is a root node, corresponding to a subtree structure, which organizes a series of words with semantic relevance. The present invention fuses all subtree structures in the draft tree with the corresponding candidate semantic trees in the candidate pool. The specific fusion process is as follows: To draft each node in the tree Extract the corresponding subtree structure as the root .

[0039] Check whether the candidate pool contains Candidate tree with root If not, just Add to candidate pool If there is a candidate pool with For the candidate tree with the root, the tree will be merged, that is, traversing from the root node respectively. and , recursively Nodes that do not exist in the pool are added to the candidate pool.

[0040] In addition to normal decoding and drafting, the present invention will also perform parallel verification on the candidate draft tree in each forward propagation of the model, which is also the ability of the present invention to decode multiple words in one forward propagation. Specifically, the present invention first selects the candidate word from the candidate pool according to the current decoded sequence. Extract the corresponding candidate draft tree In order to enable the large language model to support candidate draft trees, similar to the drafting tree, the present invention also sets the same attention mask matrix for the candidate draft tree in the same way, so that each node in the draft tree can only pay attention to the previous node. Then, after each forward propagation of the model, each node in the draft tree will generate a candidate word based on its prefix. The present invention describes this process together with the aforementioned drafting stage as follows: , in, Represents the verification tokens obtained from the prefixes of all nodes in the candidate draft tree. If the verification token of a node already exists in its child nodes, then the token is considered valid. The path of valid tokens starting from the root node is the verification result, which is also the decoding result of the present invention in the forward propagation process of the model.

[0041] In summary, the present invention converts the autoregressive decoding process into two processes: drafting and verification.

[0042] During the drafting phase, the present invention leverages the robustness of large language models under input perturbations. By constructing a tree-like drafting structure (referred to as a drafting tree in this invention), this approach overcomes the limitations of traditional linear decoding and enables efficient drafting. First, a randomly initialized drafting tree is introduced at the end of the input sequence as input perturbation. An attention mask is designed using a depth-first traversal rule to ensure that each node focuses only on the preceding node in its path, enabling the Transformer architecture to support this non-sequential data structure. Simultaneously, dual constraints are placed on the drafting tree to ensure that the tree structure can efficiently generate high-quality drafts. This involves limiting the number of children per node, thereby limiting the overall tree width and filtering out low-quality drafts. Furthermore, a pruning algorithm is designed to ensure the relevance of the drafting structure to the contextual semantics and the coherence of the draft content.

[0043] The candidate verification phase uses a parallel processing mechanism. After extracting candidate draft trees from the draft pool, they are verified using the same attention mask setting strategy. The output of the large language model is used to verify the validity of each node based on the child node matching principle. Only the continuous path starting from the root node of the draft tree that is consistent with the model prediction is retained as the final decoding sequence.

[0044] As can be seen from the technical solution provided by the present invention, compared to other speculative decoding methods, the method proposed by the present invention does not require any additional training, auxiliary models, or pre-built caches to obtain context-appropriate drafts for verification. Furthermore, compared to other self-drafting decoding methods, the method proposed by the present invention does not require any changes to model parameters or fine-tuning. The solution is fully compatible with the mainstream Transformer architecture and supports plug-and-play deployment.

[0045] The embodiment of the present invention also provides a large language model reasoning acceleration system based on a progressive drafting tree, which is used to implement the above-mentioned method embodiment, and the details that have been explained will not be repeated here. The terms "module", "unit", "sub-unit", etc. used below can implement a combination of software and / or hardware for predetermined functions. Although the system described in the following embodiments is preferably implemented in software, implementation by hardware or a combination of software and hardware is also possible and conceivable.

[0046] like Figure 4 As shown, Figure 4 : This is a structural block diagram of the large language model reasoning acceleration system based on progressive drafting tree in the present invention, which includes: A drafting module 101 is used to construct an initial drafting tree and generate draft tokens; A pruning module 102, configured to prune the drafting tree according to preset constraints; Verification module 103, used for verifying the validity of candidate draft trees in parallel; The maintenance module 104 is configured to update the candidate trees in the candidate pool.

[0047] The beneficial effects of the large language model reasoning acceleration system based on progressive drafting tree of the present invention are the same as those of the large language model reasoning acceleration method based on progressive drafting tree described above, so they will not be described in detail here.

[0048] Figure 5 A structural block diagram of an electronic device provided by an embodiment of the present invention, such as Figure 5 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the following method: Constructing an initial drafting tree, wherein the root node of the initial drafting tree is associated with the decoded context sequence, and the remaining nodes are randomly initialized to generate perturbation tokens; Attention masks and position encodings are set based on the dependencies between nodes, so that each node can only focus on the previous nodes on its path; Using a large language model to reason on the initial draft tree to generate draft tokens, and expanding the draft tree and updating the candidate pool based on the draft tokens; Prune the expanded drafting tree by presetting width and depth constraints to control the scale and semantic coherence of the tree structure; Candidate draft trees are extracted from the candidate pool, the validity of each node is verified in parallel, and the continuous path consistent with the model prediction is retained as the final decoding sequence.

[0049] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0050] An embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method provided in the above embodiments is implemented.

[0051] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or certain parts of the embodiment.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A large language model reasoning acceleration method based on progressive drafting tree, characterized by: The large language model reasoning acceleration method based on progressive drafting tree includes: Constructing an initial drafting tree, wherein the root node of the initial drafting tree is associated with the decoded context sequence, and the remaining nodes are randomly initialized to generate perturbation tokens; Attention masks and position encodings are set based on the dependencies between nodes, so that each node can only focus on the previous nodes on its path; Using a large language model to reason on the initial draft tree to generate draft tokens, and expanding the draft tree and updating the candidate pool based on the draft tokens; Prune the expanded drafting tree by presetting width and depth constraints to control the scale and semantic coherence of the tree structure; Candidate draft trees are extracted from the candidate pool, the validity of each node is verified in parallel, and the continuous path consistent with the model prediction is retained as the final decoding sequence.

2. The large language model reasoning acceleration method based on progressive drafting tree according to claim 1 is characterized in that: The structure of the initial drafting tree is ;in, is a node set containing the root node and randomly initialized perturbation tokens; is a set of edges that represent the partial order relationship between nodes.

3. The large language model reasoning acceleration method based on progressive drafting tree according to claim 1 is characterized in that: The attention mask is implemented by drafting the dependencies between nodes in the tree to identify the hierarchical relationships and path dependencies between tree nodes.

4. The large language model reasoning acceleration method based on progressive drafting tree according to claim 1 is characterized in that: The specific process of extending the drafting tree includes: Add the draft token generated by each node as a child node under the node; Update Node and edge sets ,in, For nodes Generated draft tokens.

5. The large language model reasoning acceleration method based on progressive drafting tree according to claim 1 is characterized in that: The pruning operation includes: Limit the number of child nodes of each node to no more than the preset width threshold; Limit the depth of each subtree to no more than a preset depth threshold; When the depth of a subtree reaches the depth threshold, its root node is deleted and part of the subtree is retained to maintain semantic coherence.

6. The large language model reasoning acceleration method based on progressive drafting tree according to claim 1 is characterized in that: The parallel verification includes: The same method is used to set the attention mask and position encoding for the draft tree and the candidate draft tree respectively; Verify whether the child nodes of each node match the predicted word based on the output of the large language model, and retain the continuous valid path starting from the root node.

7. The large language model reasoning acceleration method based on progressive drafting tree according to claim 1 is characterized in that: Also includes candidate pool maintenance steps: Merge the subtree of each node in the drafting tree with the corresponding candidate tree in the candidate pool, and supplement the missing nodes to update the candidate pool.

8. A large language model reasoning acceleration system based on progressive drafting tree, used to implement the method described in any one of claims 1 to 7, characterized in that: include: Drafting module, used to build the initial draft tree and generate draft tokens; Pruning module, used to prune the drafting tree according to preset constraints; Verification module, used to verify the validity of candidate draft trees in parallel; The maintenance module is used to update the candidate trees in the candidate pool.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the large language model reasoning acceleration method based on progressive drafting tree is implemented according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large language model reasoning acceleration method based on progressive drafting tree are implemented as described in any one of claims 1 to 7.

Images