A large language model reasoning acceleration method based on context semantic guidance

Abstract

The application provides a large language model reasoning acceleration method based on context semantic guidance, which comprises the following steps: S1, obtaining current request input and context information thereof, extracting semantic information therefrom, and constructing a context semantic graph; S2, generating one or more candidate draft sequences under the guidance of the context semantic graph structure, the candidate draft sequences being associated with different semantic paths in the context semantic graph; and S3, inputting the candidate draft sequences and a current decoding prefix into a target large language model for single forward propagation, verifying the candidate draft sequences in parallel, and accepting the longest continuous prefix that passes the verification as the decoding result of the current round. The application guides and constrains the reasoning process of the large language model without modifying the internal structure of the large language model and without additional training, thereby improving the quality and efficiency of the generated draft, and achieving the improvement of the overall reasoning speed and resource utilization.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically a method for accelerating reasoning in large language models based on contextual semantic guidance. Background Technology

[0002] Large Language Models (LLMs) excel in natural language understanding and generation, but their inference process typically employs autoregressive decoding: each new token requires a complete forward propagation, loading all model parameters into the computation unit for calculation. In practical deployments, this process is primarily limited by memory bandwidth and computational overhead, resulting in high inference latency and limited throughput, making it difficult to meet the real-time requirements of online services.

[0003] To alleviate the aforementioned problems, the industry has proposed various inference acceleration techniques, such as: Model compression and quantization: reducing the number of parameters and computational load through pruning, low-bit quantization, etc., but often requiring additional training and potentially sacrificing generation quality. Cache reuse and pipelining: sharing a KV cache among batch requests or employing system optimization techniques such as tensor parallelism and pipelining, but placing high demands on task structure and model architecture. Speculative decoding: using a small model or auxiliary module to generate a draft, which is then verified in parallel by a large model, thereby accepting multiple tokens in a single forward computation, breaking the bottleneck of sequential decoding per token.

[0004] Existing speculative decoding methods often utilize small draft models or retrieved corpora to generate candidate draft sequences, which are then validated by a large model. However, these methods generally require additional training and maintenance of auxiliary models, increasing engineering complexity and deployment costs. Furthermore, distribution mismatches may exist between the auxiliary and large models, affecting draft quality and validation efficiency. Building on this, previous research has proposed multi-branch self-drafting and progressive tree-structured drafting methods. These methods leverage the robustness of the large model itself, adding perturbations to the input to achieve multi-branching or drafting tree structures for parallel drafting, thus eliminating dependence on auxiliary small models and significantly improving draft utilization and overall throughput. However, these methods still primarily rely on adding random perturbations to the input to leverage the robustness of the large language model to obtain high-quality drafts. They do not fully utilize key semantic clues, logical relationships, and entity structure information in the context, resulting in a large amount of homogeneous draft content, limited semantic diversity, and insufficient draft quality. Moreover, the validation stage typically relies only on surface token matching, lacking adaptive adjustments to task type and context structure, and failing to fully explore the potential of semantic guidance in accelerating reasoning.

[0005] In summary, current large language model inference optimization techniques urgently need a plug-and-play, universal acceleration solution that organically integrates contextual semantic structure with the decoding process without any additional training or fine-tuning. This solution would allow contextual semantic information to directly participate in candidate generation and selection, reducing invalid computations and increasing the effective content confirmed in each forward propagation without sacrificing generation quality. This would improve inference speed and resource utilization without changing hardware configuration. Summary of the Invention

[0006] The problem addressed by this invention is how to construct a contextual semantic analysis and guidance link without modifying the internal structure of a large language model and without additional training. This link allows contextual semantic structure information to be integrated into the decoding process, thereby guiding and constraining the reasoning process of the large language model, improving the quality and efficiency of the generated draft, and enhancing the overall reasoning speed and resource utilization.

[0007] To address the aforementioned problems, this invention provides a method, system, electronic device, and storage medium for accelerating large language model reasoning based on contextual semantic guidance.

[0008] In a first aspect, the present invention provides a method for accelerating reasoning in large language models based on contextual semantic guidance, comprising the following steps:

[0009] S1. Context semantic structure construction: Obtain the current request input and its context information, extract the semantic information, and construct a context semantic graph;

[0010] S2. Semantic-guided candidate draft generation: Under the guidance of the context semantic graph structure, one or more candidate draft sequences are generated, and the candidate draft sequences are associated with different semantic paths in the context semantic graph;

[0011] S3. Parallel Draft Verification and Acceptance: Input the candidate draft sequence and the current decoding prefix into the target large language model for a single forward propagation, verify each candidate draft sequence in parallel, and accept the longest consecutive prefix that passes the verification as the decoding result of this round.

[0012] Optionally, constructing the context semantic graph includes:

[0013] By semantic parsing and entity and relation recognition, key semantic elements and their relationships in the context information are extracted. Semantic elements are represented by nodes and the relationships between semantic elements are represented by edges, forming a graph structure.

[0014] Optionally, generating a candidate draft sequence under the guidance of the context semantic graph structure includes:

[0015] Using the structure of the context semantic graph as a skeleton, the direction and order of branch expansion are determined. By applying guiding perturbations corresponding to the semantic path to the input state or decoding prefix of the large language model, multiple candidate branches are generated in parallel to obtain a set of candidate draft sequences.

[0016] Optionally, prior to the single forward propagation, the method further includes:

[0017] Based on the current decoding prefix or the context semantic graph, one or more matching candidate draft sequences are retrieved from the candidate draft sequence set for verification.

[0018] Optionally, performing a single forward propagation includes:

[0019] Construct a joint input sequence comprising the current decoding prefix and / or at least one candidate draft sequence and a linearized representation of the context semantic graph;

[0020] Configure a joint attention mask for the joint input sequence so that the target large language model can simultaneously perform parallel verification of the candidate draft sequence and generate draft content under the guidance of the semantic graph in one forward computation.

[0021] Optionally, the joint attention mask is configured as multiple partitions, including at least:

[0022] Context region: Allows causal autoregressive attention computation to be performed on the portion of the joint input sequence that belongs to the historical context;

[0023] Prefix Visibility Region: Allows the portion of the joint input sequence that belongs to the candidate draft sequence and the semantic graph linearized representation to focus on the entire historical context;

[0024] Parallel verification region: Allows causal autoregressive attention calculation within the same candidate draft sequence in the joint input sequence, and masks attention between different candidate draft sequences;

[0025] Semantic graph topology region: Allows the portion of the joint input sequence that belongs to the linearized representation of the semantic graph to perform attention calculation based on the dependencies between nodes in the semantic graph.

[0026] Optionally, the method further includes the steps of: dynamically updating and pruning the context semantic graph based on the confidence level of the target large language model when generating lexical units corresponding to the semantic graph-guided content; the updating includes expanding semantic graph nodes or extending node content based on the generated lexical units; the pruning includes removing nodes whose generation confidence level is lower than a preset threshold.

[0027] Optionally, it also includes a draft caching step: storing the generated sequences corresponding to the high-confidence branches retained after pruning in the context semantic graph in the draft cache library, with a prefix of a preset length as the key and the remaining suffix as the value, for retrieval and reuse in subsequent inference steps.

[0028] Optionally, the execution frequency of the semantically guided candidate draft generation step in step S2 is dynamically adjustable, and whether it is executed in this decoding step is determined based on the computational load, cache hit rate, or preset strategy.

[0029] Secondly, embodiments of the present invention provide a large language model reasoning acceleration system based on contextual semantic guidance, comprising:

[0030] The semantic structure construction module is used to obtain the current request input and its context information, extract the semantic information, and construct a context semantic graph.

[0031] A candidate draft generation module is used to generate one or more candidate draft sequences under the guidance of the context semantic graph structure, wherein the candidate draft sequences are associated with different semantic paths in the context semantic graph;

[0032] The parallel verification and acceptance module is used to input the candidate draft sequence and the current decoding prefix into the target large language model for a single forward propagation, verify each candidate draft sequence in parallel, and accept the longest consecutive prefix that passes the verification as the decoding result of this round.

[0033] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a bus, wherein the processor, the communication interface, and the memory communicate with each other through the bus, and the processor can call logical instructions in the memory to execute the steps of the method provided in the first aspect.

[0034] Fourthly, embodiments of the present invention provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the context-semantic guided large language model reasoning acceleration method as described in the first aspect.

[0035] The beneficial effects of the context-semantic guided method for accelerating large language model inference in this invention are as follows:

[0036] This invention guides the generation and selection of candidate drafts by constructing and utilizing a contextual semantic graph, directly injecting deep semantic structure information into the inference acceleration link. This effectively reduces redundancy and homogenization branches caused by random perturbations, significantly improving the quality and validation pass rate of candidate drafts. By designing a joint attention mask mechanism, draft validation and semantic-guided drafting are completed in parallel during a single forward propagation. Without increasing model parameters or changing the architecture, this significantly increases the number of valid lexical units confirmed in a single computation, thereby reducing the overall inference latency. At the same time, this method is implemented entirely on the inference side, is compatible with existing models and inference frameworks, and has the advantages of simple deployment and low invasiveness. Attached Figure Description

[0037] Figure 1 This is a flowchart of the method for accelerating large language model inference based on contextual semantic guidance in an embodiment of the present invention;

[0038] Figure 2 This is a schematic diagram of the principle of the large language model reasoning acceleration method based on contextual semantic guidance in an embodiment of the present invention;

[0039] Figure 3 This is an example diagram of the attention mask matrix in an embodiment of the present invention;

[0040] Figure 4 This is a structural block diagram of the large language model reasoning acceleration system based on contextual semantic guidance in an embodiment of the present invention;

[0041] Figure 5 This is a structural block diagram of the electronic device in an embodiment of the present invention. Detailed Implementation

[0042] To better understand the purpose, technical solution, and advantages of this application, the application is described and explained below in conjunction with the accompanying drawings and embodiments.

[0043] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0044] like Figure 1 and Figure 2 As shown in the embodiment of the present invention, a method for accelerating large language model inference based on contextual semantic guidance includes the following steps:

[0045] S1. Context semantic structure construction: Obtain the current request input and its context information, extract the semantic information, and construct a context semantic graph;

[0046] In this embodiment, constructing a context semantic graph includes:

[0047] By semantic parsing and entity and relation recognition, key semantic elements and their relationships in the context information are extracted. Semantic elements are represented by nodes and the relationships between semantic elements are represented by edges, forming a graph structure.

[0048] The system acquires the current request input and related contextual information (such as historical dialogue turns, document content, and external knowledge). Through techniques like semantic parsing, entity recognition and relation extraction, event extraction, or knowledge graph alignment, it extracts key entities and relationships from the context and constructs a contextual semantic graph or an equivalent graph structure. Based on this, a guiding drafting structure for inference decoding is generated using this semantic graph as a skeleton. This structure introduces various biased perturbations into the nodes, edges, or local subgraphs of the graph during subsequent drafting. The topological relationships of the semantic graph determine the branching order and direction, thus driving the construction of a tree-like candidate structure with the contextual semantic graph, replacing the existing method of generating branches based on random perturbations.

[0049] For example, first, user input and a sequence of historical dialogue contexts are received, denoted as... The semantic parsing module extracts entities and relationships from the context to construct an initial context semantic graph. .in, Represents a set of semantic nodes, each node It corresponds to a semantic concept or entity (such as "strategy guide" or "Erhai Lake"). Denotes the set of edges. Represents a node To the node There is a semantic dependency.

[0050] S2. Semantic-guided candidate draft generation: Based on the structure of the context semantic graph, one or more candidate draft sequences are generated, and the candidate draft sequences are associated with different semantic paths in the context semantic graph;

[0051] The execution frequency of the semantically guided candidate draft generation step in step S2 is dynamically adjustable, and it is determined whether to execute it in this decoding step based on the computational load, cache hit rate, or preset strategy.

[0052] In this embodiment, the generation of candidate draft sequences based on the structure of the context semantic graph includes:

[0053] Using the structure of the context semantic graph as a skeleton, the direction and order of branch expansion are determined. By applying guiding perturbations corresponding to the semantic path to the input state or decoding prefix of the large language model, multiple candidate branches are generated in parallel to obtain a set of candidate draft sequences.

[0054] Candidate draft sequences are generated based on the structure of the context semantic graph, including:

[0055] Using the structure of the context semantic graph as a skeleton, the direction and order of branch expansion are determined. By applying guiding perturbations corresponding to the semantic path to the input state or decoding prefix of the large language model, multiple candidate branches are generated in parallel to obtain a set of candidate draft sequences.

[0056] During the inference and decoding phase, the aforementioned contextual semantic graph structure is used to apply controlled perturbations to the input state or decoding prefix of the large language model. Multiple candidate branches are then expanded in parallel along different semantic directions during a single forward propagation, ensuring that candidate drafts cover semantic paths closely related to the context and reducing invalid or homogenized branches caused by random perturbations. In this way, the inference and expansion results on the contextual semantic graph are systematically mapped into a set of candidate draft sequences that correspond one-to-one with different semantic paths, providing input for subsequent verification and acceptance.

[0057] In summary, to obtain a high-accuracy validation draft with minimal computational overhead, the system optimizes the current context sequence. A string suffix matching strategy is used. (Definition) For context end A sequence of tokens ( (This is the preset window size).

[0058] The system in the draft cache library Searching for key-value pairs Records that are exact matches:

[0059] ;

[0060] in, For the historical context suffix key stored in the cache, This is a draft of the corresponding subsequent text.

[0061] The mechanism ensures that the retrieved drafts are generated in the exact same local context in history, thus achieving a very high verification pass rate. Moreover, the retrieval process only involves hash lookup or trie matching, with no vector computation overhead.

[0062] The present invention then transforms the two parts of prior information into a token sequence to construct the input for this reasoning:

[0063] Semantic graph sequence Semantic graph The nodes in the graph are mapped to the input word sequence. Furthermore, by reconstructing the semantic dependencies between nodes through topological attention masks, the model can perform inference according to the non-linear graph topology when processing linear input sequences.

[0064] Candidate text draft sequence : The draft text fragment obtained from the above retrieval steps.

[0065] System constructs joint input sequence :

[0066] ;

[0067] in, This indicates sequence concatenation. After the input is constructed, the model will simultaneously process the draft during a single forward propagation using a subsequently defined joint attention mask (explained in detail below). Verification and The semantic guidance was used in the drafting process.

[0068] S3. Parallel Draft Verification and Acceptance: Input the candidate draft sequence and the current decoding prefix into the target large language model for a single forward propagation, verify each candidate draft sequence in parallel, and accept the longest consecutive prefix that passes the verification as the decoding result of this round.

[0069] Before a single forward propagation, it also includes:

[0070] Based on the current decoding prefix or context semantic graph, retrieve one or more matching candidate draft sequences from the candidate draft sequence set for verification.

[0071] Perform a single forward propagation, including:

[0072] Construct a joint input sequence containing the current decoding prefix and / or at least one candidate draft sequence and a linearized representation of the context semantic graph; it should be noted that there may be no candidate draft sequence, and if it cannot be retrieved, there is no candidate sequence.

[0073] Configure a joint attention mask for the joint input sequence so that the target large language model can simultaneously perform parallel verification of the candidate draft sequence and generate draft content under the guidance of the semantic graph in a single forward computation.

[0074] The method also includes the steps of: dynamically updating and pruning the context semantic graph based on the confidence level of the target large language model when generating lexical units corresponding to the semantic graph-guided content; the updating includes expanding semantic graph nodes or extending node content based on the generated lexical units; the pruning includes removing nodes whose generation confidence level is lower than a preset threshold.

[0075] It also includes a draft caching step: the generated sequences corresponding to the high-confidence branches retained after pruning in the context semantic graph are stored in the draft cache library with a prefix of a preset length as the key and the remaining suffix as the value, for retrieval and reuse in subsequent inference steps.

[0076] In each round of decoding, the system first selects one or more candidate branches that best match the current context from the candidate draft set obtained in the second step, based on the current context semantic graph and decoding prefix. If there is no corresponding candidate for the current prefix, normal decoding is used directly. Subsequently, the selected candidate drafts and the current decoding prefix are input into the target large language model, and multiple candidates are verified simultaneously through a forward computation: for each candidate draft, the token output by the model is compared with the token in the draft from the starting position, the longest continuous prefix that matches the first one is found, and this part of the token is taken as the directly acceptable decoding result for this round, while the inconsistent parts are discarded.

[0077] Specifically, this stage is the core of the invention, achieved by designing a specific attention mask matrix. This allows the large language model to perform both "verification" and "drafting" simultaneously during a single forward propagation.

[0078] In this embodiment, the joint attention mask is configured as multiple partitions, including at least:

[0079] Context region: Allows causal autoregressive attention computation on the portion of the joint input sequence that belongs to the historical context;

[0080] Prefix Visibility Region: Allows the portion of the joint input sequence that belongs to the candidate draft sequence and the semantic graph linearization representation to focus on the entire historical context;

[0081] Parallel Validation Region: Allows causal autoregressive attention computation within the same candidate draft sequence in the joint input sequence, and masks attention between different candidate draft sequences;

[0082] Semantic graph topology region: Allows the portion of the joint input sequence that belongs to the linearized representation of the semantic graph, and performs attention calculation based on the dependencies between nodes in the semantic graph.

[0083] Specifically, in order to simultaneously support contextual autoregression, parallel verification of multiple drafts, and semantic graph drafting in a single forward propagation, such as Figure 3 As shown, this invention designs a partitioned joint attention mask matrix. Assume a joint input sequence The total length is For the first in the sequence The first token (as a query) and the second Each token (as the key), mask value The calculation formula is defined and illustrated with examples as follows:

[0084] ;

[0085] Region A (Context Region): When and All belong to context sequences At that time, the rule is: follow the standard causal masking rule ( This ensures that the encoding of historical information conforms to the chronological order.

[0086] Region B (prefix visibility): when It belongs to any draft sequence or semantic graph sequence, and Belongs to the context sequence hour. This means that all candidate branches (whether drafts or semantic graphs) can fully "see" the historical context, ensuring that the generated content is coherent with the preceding context.

[0087] Region C (Multi-Draft Parallel Validation Region): When and All belong to the text draft collection When. Only when and They belong to the same draft fragment and satisfy the causal order ( When ), the mask is 1.

[0088] Region D (semantic graph topological region): when and All belong to semantic graph sequences At that time, a mapping function was introduced. Retrieve the graph node corresponding to the Token and define it. For nodes The set of direct or indirect predecessor nodes in the semantic graph. Only if the node... It is a node When the node is a predecessor or itself, the mask is 1. If the node It is a fusion node (such as "Summary") that may have multiple parent nodes (such as "Branch A" and "Branch B"), and the formula allows... By focusing on the content of both branches, logical integration and reasoning can be achieved.

[0089] Parallel Inference: Based on Joint Input Sequence and joint attention mask Using large language models Perform one forward propagation and calculate the output probability distribution of the entire sequence. :

[0090] ;

[0091] in, , Given the length of the input sequence, This refers to the size of the vocabulary. Indicates the location The predicted probability distribution for the next word. This is achieved thanks to the partition mask. The design of the model decouples and parallelizes the verification and drafting tasks in a single forward propagation. The system uses position indexes... The region to which it belongs, and the output probability distribution Execute different processing logic.

[0092] For the input sequence that belongs to the text draft Location index The model performs a parallel validation task of the text draft. Assume the position in the input sequence... The actual word units are (i.e., the next real word in the draft), the system checks whether the model prediction matches the draft:

[0093] ;

[0094] in This is an indicator function. If the predicted result matches the draft content, the validation at that location passes. This process utilizes the causal constraints of the mask to verify the validity of the draft content in the current context in parallel.

[0095] For the input sequence that belongs to the semantic graph drafting sequence Location index The model performs the task of drafting the semantic graph. Let this position correspond to a node in the semantic graph. Due to the mask By shielding this location from connections to other unrelated branches, the effective context of the model is restricted to... This refers to the global context and the predecessor path of the node. The system operates based on a probability distribution. Sampling generates new drafting morphemes :

[0096] ;

[0097] in, For sampling strategies (such as Greedy Search or Nucleus Sampling).

[0098] Conditional updates to the semantic graph state to obtain draft lexical units. Then, the system according to Semantic attribute execution graph state transition function Transform semantic graphs from states Updated to The process is defined as follows:

[0099] ;

[0100] in, A predefined set of topology trigger words (such as colons ":", item separators "1.", key entity names, etc.) are used to indicate new semantic levels. Topology expansion refers to when... When it belongs to a trigger word, the system in the node Create new child nodes and establish directed edges At this point, the graph's topology changes. Content extension refers to when... When it is a regular word, the system appends it to the node. text content In this process, the graph's topology remains unchanged, with only node attributes updated. Through the aforementioned state transition mechanism, the semantic graph can adaptively grow along with the reasoning process, transforming from simple text sequence generation into structured semantic space exploration.

[0101] To optimize computational overhead and block invalid semantic exploration paths, the system updates the semantic graph. Perform confidence-based pruning to remove a set of nodes from the graph that have insufficient confidence. Thus, the semantic graph after pruning is obtained. :

[0102] ;

[0103] Among them, the set of nodes to be pruned Defined as all generated confidence levels below a safety threshold Nodes:

[0104] ;

[0105] Thus ensure We always focus on the semantic space with high probability.

[0106] The system extracts and caches drafts, traversing each active branch node in the semantic graph. If the confidence level of the drafting lemmas generated by this node meets the preset requirements, the system backtracks from the root node of the semantic graph (or the starting point of the current drafting fragment) to the node. The complete generation path is obtained, all tokens on that path are extracted, and a cumulative generation sequence is constructed. The system then adjusts the index window size according to the preset parameters. ( ), for sequence Perform the segmentation operation: extract the sequence The former Each token serves as an index key for matching: and extract the sequence The remaining suffixes are used as draft content to be verified: The extracted key-value pairs are stored in the draft cache library. :

[0107] ;

[0108] Through the above mechanism, high-confidence paths in the semantic graph reasoning process are transformed into reusable serialized experience. When subsequent reasoning encounters similar contextual prefixes, the system can directly recall the corresponding sequence drafts for parallel verification, thereby significantly improving the reasoning efficiency of text generation.

[0109] The decoupling of drafting and verification, along with adaptive frequency adjustment, requires special attention. While the verification and drafting tasks in this invention execute in parallel, they are logically relatively independent. To further optimize computational resources, the system supports adaptive drafting frequency adjustment: the model does not need to activate the semantic graph drafting module at every inference step, but can dynamically adjust the drafting frequency based on the current computational load or cache hit rate (e.g., only every...). The semantic graph sequence is only included in the input when a step or cache miss occurs. This decoupling mechanism enables the system to "draft on demand," significantly reducing the average computational cost of the model while ensuring accelerated inference.

[0110] To better understand and apply this method, this invention takes its application in a dialogue system as an example, and uses it to accelerate the reasoning process of large language models in online dialogue interaction scenarios.

[0111] In this embodiment, the dialogue system includes a user terminal, a dialogue server, and a large language model inference module deployed on the dialogue server. The user sends a dialogue request in natural language form to the dialogue server through the user terminal. After receiving the request, the dialogue server obtains the current request input and the corresponding historical dialogue context information, which serves as the context information in the method of this invention.

[0112] The dialogue server invokes the context-semantic guided large language model reasoning acceleration method provided by this invention to perform semantic parsing on the current request input and historical dialogue context, extracting key semantic elements and their relationships, and constructing a context semantic graph. This context semantic graph is used to characterize the entities, topics, and their semantic relationships involved in the historical dialogue.

[0113] Based on this, guided by the context semantic graph structure, the dialogue server generates one or more candidate draft sequences. These candidate draft sequences correspond to different semantic paths in the context semantic graph and are stored or maintained in a candidate draft set. During subsequent decoding, the dialogue server selects a candidate draft sequence from the candidate draft set based on the current decoding prefix, the context semantic state, or a preset strategy. The selected candidate draft sequence, along with the current decoding prefix, is input into the target large language model for a single forward propagation. The candidate draft sequences are then validated in parallel, and the longest consecutive prefix that passes the validation is accepted as the decoding result of this round of dialogue response.

[0114] By employing the above methods, the dialogue system can identify multiple valid lexical units in a single forward propagation while maintaining the semantic consistency and coherence of the generated dialogue content. This reduces the redundant computational overhead caused by sequential decoding of each lexical unit, thereby significantly reducing the response latency of a single-turn dialogue and improving the processing capacity of the dialogue system. This application example is particularly suitable for application scenarios such as online dialogue services, intelligent customer service systems, and real-time interactive question-and-answer systems with high latency requirements.

[0115] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0116] 1. Semantic graph-driven candidate draft generation directly participates in the inference acceleration process.

[0117] This invention constructs an explicit semantic graph based on the current input and historical context, and generates candidate sequences using the contextual semantic graph as a skeleton. This allows nodes and edges in the semantic graph to correspond to different branches and expansion directions during the candidate generation process. In this way, the construction of candidate drafts is directly driven by the contextual semantic structure, replacing the method of generating branches by random perturbation in the prior art. This allows contextual semantic information to be truly embedded into the complete chain of "candidate generation → draft verification → inference acceleration".

[0118] 2. Reduce candidate redundancy and improve the effective draft hit rate.

[0119] In this invention, candidate branches are expanded along entity and relation paths on the context semantic graph, prioritizing coverage of semantic directions closely related to the current task, thus avoiding a large number of similar or irrelevant branches introduced by random perturbation methods. This reduces redundant candidates that significantly deviate from the context semantics, lowering the ineffective overhead of tree-based validation and backoff; and increases the proportion of high-quality drafts validated by the main model, thereby improving the hit rate of effective drafts in each round of speculative decoding.

[0120] 3. An efficient draft utilization mechanism based on candidate retrieval and parallel validation

[0121] In each round of decoding, this invention rapidly retrieves one or more draft sequences that best match the current semantic path from the candidate set generated by the semantic graph, based on the current context semantic graph or decoding prefix. These draft sequences, along with the decoding prefix, are then input into the target large language model. Multiple candidates are validated in parallel through a single forward computation, and the longest prefix consistent with the model output in each branch is extracted as the acceptable draft fragment. Compared to sequential decoding token by token, this mechanism significantly increases the number of tokens that can be confirmed in a single forward propagation without increasing model parameters or structural complexity, thereby reducing overall inference latency.

[0122] 4. Inference acceleration can be achieved without modifying the large model architecture, and it is compatible with existing inference frameworks and methods.

[0123] This invention only modifies the candidate generation and draft selection strategies on the inference side, without requiring modification to the network structure and parameters of the large language model. It uses a context semantic graph to replace random perturbations to generate and organize candidate branches, and features simple implementation and minimal intrusion into existing systems, making it easy to integrate directly into existing large language model inference services.

[0124] This invention also provides a context-semantic guided large language model reasoning acceleration system, which is used to implement the above-described method embodiments; details already described will not be repeated. The terms "module," "unit," and "subunit," etc., used below refer to combinations of software and / or hardware that perform predetermined functions. Although the system described in the following embodiments is preferably implemented in software, hardware implementation or a combination of software and hardware is also possible and contemplated.

[0125] like Figure 4 As shown, Figure 4 This is a structural block diagram of the context-semantic guided large language model reasoning acceleration system of the present invention, which includes:

[0126] The semantic structure construction module 101 is used to obtain the current request input and its context information, extract the semantic information therein, and construct a context semantic graph;

[0127] The candidate draft generation module 102 is used to generate one or more candidate draft sequences under the guidance of the context semantic graph structure, wherein the candidate draft sequences are associated with different semantic paths in the context semantic graph;

[0128] The parallel verification and acceptance module 103 is used to input the candidate draft sequence and the current decoding prefix into the target large language model for a single forward propagation, verify each candidate draft sequence in parallel, and accept the longest consecutive prefix that passes the verification as the decoding result of this round.

[0129] like Figure 5 As shown in the figure, an electronic device provided by an embodiment of the present invention includes: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the following method:

[0130] S1. Context semantic structure construction: Obtain the current request input and its context information, extract the semantic information, and construct a context semantic graph;

[0131] S2. Semantic-guided candidate draft generation: Under the guidance of the context semantic graph structure, one or more candidate draft sequences are generated, and the candidate draft sequences are associated with different semantic paths in the context semantic graph;

[0132] S3. Parallel Draft Verification and Acceptance: Input the candidate draft sequence and the current decoding prefix into the target large language model for a single forward propagation, verify each candidate draft sequence in parallel, and accept the longest consecutive prefix that passes the verification as the decoding result of this round.

[0133] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0135] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for accelerating reasoning in large language models based on contextual semantic guidance, characterized in that, Includes the following steps: S1. Context semantic structure construction: Obtain the current request input and its context information, extract the semantic information, and construct a context semantic graph; S2. Semantic-guided candidate draft generation: Under the guidance of the context semantic graph structure, one or more candidate draft sequences are generated, and the candidate draft sequences are associated with different semantic paths in the context semantic graph; S3. Parallel Draft Verification and Acceptance: Input the candidate draft sequence and the current decoding prefix into the target large language model for a single forward propagation, verify each candidate draft sequence in parallel, and accept the longest consecutive prefix that passes the verification as the decoding result of this round.

2. The method according to claim 1, characterized in that, The construction of the context semantic graph includes: By semantic parsing and entity and relation recognition, key semantic elements and their relationships in the context information are extracted. Semantic elements are represented by nodes and the relationships between semantic elements are represented by edges, forming a graph structure.

3. The method according to claim 1, characterized in that, Guided by the context semantic graph structure, a candidate draft sequence is generated, including: Using the structure of the context semantic graph as a skeleton, the direction and order of branch expansion are determined. By applying guiding perturbations corresponding to the semantic path to the input state or decoding prefix of the large language model, multiple candidate branches are generated in parallel to obtain a set of candidate draft sequences.

4. The method according to claim 1, characterized in that, Prior to the single forward propagation, it also includes: Based on the current decoding prefix or the context semantic graph, one or more matching candidate draft sequences are retrieved from the candidate draft sequence set for verification.

5. The method according to claim 1, characterized in that, The single forward propagation includes: Construct a joint input sequence comprising the current decoding prefix and / or at least one candidate draft sequence and a linearized representation of the context semantic graph; Configure a joint attention mask for the joint input sequence so that the target large language model can simultaneously perform parallel verification of the candidate draft sequence and generate draft content under the guidance of the semantic graph in one forward computation.

6. The method according to claim 5, characterized in that, The joint attention mask is configured as multiple partitions, including at least: Context region: Allows causal autoregressive attention computation to be performed on the portion of the joint input sequence that belongs to the historical context; Prefix Visibility Region: Allows the portion of the joint input sequence that belongs to the candidate draft sequence and the semantic graph linearized representation to focus on the entire historical context; Parallel verification region: Allows causal autoregressive attention calculation within the same candidate draft sequence in the joint input sequence, and masks attention between different candidate draft sequences; Semantic graph topology region: Allows the portion of the joint input sequence that belongs to the linearized representation of the semantic graph to perform attention calculation based on the dependencies between nodes in the semantic graph.

7. The method according to claim 1, characterized in that, The method also includes the steps of: dynamically updating and pruning the context semantic graph based on the confidence level of the target large language model when generating lexical units corresponding to the semantic graph-guided content; the updating includes expanding semantic graph nodes or extending node content based on the generated lexical units; the pruning includes removing nodes whose generation confidence level is lower than a preset threshold.

8. The method according to claim 7, characterized in that, It also includes a draft caching step: the generated sequences corresponding to the high-confidence branches retained after pruning in the context semantic graph are stored in the draft cache library with a prefix of a preset length as the key and the remaining suffix as the value, for retrieval and reuse in subsequent inference steps.

9. The method according to claim 1, characterized in that, The execution frequency of the semantically guided candidate draft generation step in step S2 is dynamically adjustable, and it is determined whether to execute it in this decoding step based on the computational load, cache hit rate, or preset strategy.

10. An electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the context-semantic guided large language model reasoning acceleration method as described in any one of claims 1 to 9.

Images