A method and apparatus for optimizing question answering in large language models
By constructing a thought chain of local knowledge graph compression and pruning large language models, the problem of balancing reasoning efficiency and accuracy in existing technologies is solved, achieving efficient and accurate question-answering processing.
Patent Information
- Application Number
- CN202611146687.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-30
- Publication Date
- 2026-11-17
AI Technical Summary
Existing large language model question answering methods struggle to balance reasoning efficiency and accuracy, resulting in redundant text and repetitive reasoning steps, which increases computational overhead. Furthermore, early reasoning errors can easily accumulate, leading to incorrect answers.
By constructing a local knowledge graph, compressing and pruning thought chains, merging equivalent reasoning steps, eliminating redundant nodes, and using the local knowledge graph to constrain the large language model to generate reasoning steps, and performing factual and logical verification, the reliability of the reasoning basis is ensured.
It reduces the computational load and latency of the question-and-answer process, improves the response speed and answer quality of the question-and-answer system, and ensures the accuracy and reliability of the answers.
Smart Images

Figure CN123115308A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing technology, and in particular to a method and apparatus for optimizing question answering in a large language model. Background Technology
[0002] Thought chain hinting technology, by guiding large language models to output intermediate reasoning steps for problem-solving, has become an important means to improve the accuracy of large language model question answering. However, existing large language model question answering methods face a dilemma in practical deployment: on the one hand, the thought chains generated by large language models often contain a large amount of redundant text and repetitive reasoning steps unrelated to the core reasoning logic, leading to increased token consumption and computational overhead during the question answering process, making it difficult to meet the real-time question answering requirements in low-computing-power scenarios; on the other hand, in long-chain reasoning processes, factual biases in early steps accumulate and propagate along the reasoning chain, ultimately leading to incorrect answers. Therefore, an optimization method for large language model question answering is urgently needed to solve the aforementioned technical problems. Summary of the Invention
[0003] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solutions, nor is it intended to determine the scope of protection of the claimed technical solutions.
[0004] Firstly, this application proposes a method for optimizing question answering in large language models, the method comprising: Obtain the original question text; Based on the pre-defined knowledge graph and the original question text, a corresponding local knowledge graph is constructed; Based on a local knowledge graph, a large language model is invoked to generate an initial thought chain corresponding to the original question text. Based on a local knowledge graph, the initial thought chain is compressed to obtain a compressed thought chain. The compressed thinking chain is verified, and the compressed thinking chain is pruned based on the verification results to obtain the target thinking chain. Based on the target thinking chain, the large language model is invoked to determine the answer information corresponding to the original question text.
[0005] In some implementations, a corresponding local knowledge graph is constructed based on a pre-defined knowledge graph and the original question text, including: Based on a pre-defined knowledge graph, entity recognition and linking are performed on the original question text to identify at least one target entity node. Based on each target entity node and a pre-defined knowledge graph, a local knowledge graph corresponding to the original question text is constructed.
[0006] In some implementations, based on a pre-defined knowledge graph, entity recognition and linking are performed on the original question text to determine at least one target entity node, including: The original question text is preprocessed to generate standardized question text; Based on the few-sample prompt template, a large language model is called to extract standardized question text and obtain a candidate entity set; Based on the sequence labeling model, boundary correction and type calibration are performed on each candidate entity in the candidate entity set to generate a calibration entity set. For each calibration entity in the calibration entity set, the target entity node corresponding to the calibration entity is determined from the preset knowledge graph.
[0007] In some implementations, for each calibration entity in the calibration entity set, a target entity node corresponding to the calibration entity is determined from a preset knowledge graph, including: For each calibration entity in the calibration entity set, a search is performed in the preset knowledge graph based on the entity name of the calibration entity to obtain the initial node set; Candidate nodes that match the type information of the calibration entity are selected from the initial node set to generate a candidate node set; Based on the semantic coding model, the question semantic vector of the standardized question text and the node semantic vector corresponding to the attribute description text of each candidate node in the candidate node set are extracted. Calculate the first cosine similarity between the semantic vector of the question and the semantic vectors of each node; The candidate node with the highest first cosine similarity is determined as the target entity node corresponding to the calibration entity.
[0008] In some implementations, a local knowledge graph corresponding to the original question text is constructed based on each target entity node and a pre-defined knowledge graph, including: Classify the complexity of standardized problem texts and determine the level of reasoning complexity; Based on the reasoning complexity level, determine the number of retrieval jumps for each target entity node in the preset knowledge graph; Starting from each target entity node, retrieve knowledge triples from the preset knowledge graph whose number of relationship edges with each target entity node is no greater than the number of retrieval hops to obtain an initial set of triples; wherein each knowledge triple includes a target entity node, another entity node associated with the target entity node, and a relationship edge connecting the target entity node and the other entity node; Perform correlation filtering on the initial set of triples to generate the target set of triples; Based on the target triple set, a local knowledge graph corresponding to the original question text is constructed.
[0009] In some implementations, relevance filtering is performed on the initial set of triples to generate a target set of triples, including: Based on the preset knowledge graph, the relation confidence of each knowledge triple in the initial triple set is determined. Knowledge triples with relation confidence less than the preset confidence threshold are removed from the initial triple set to generate a filtered triple set. The semantic encoding model is used to extract the question semantic vector of the standardized question text and the group semantic vector of each knowledge triple in the filtered triple set. Calculate the second cosine similarity between the semantic vector of the question and each group of semantic vectors; Remove knowledge triplets with a second cosine similarity less than a preset similarity threshold from the filtered triplet set to generate the target triplet set.
[0010] In some implementations, based on a local knowledge graph, a large language model is invoked to generate an initial thought chain corresponding to the original question text, including: Convert each knowledge triple in the local knowledge graph into constraint prompt words; Based on constraint prompts and the original question text, the large language model is invoked to output an initial thought chain consisting of multiple reasoning steps according to a preset structured format.
[0011] In some implementations, based on a local knowledge graph, the initial thought chain is compressed to obtain a compressed thought chain, including: When the starting entity node and ending entity node corresponding to at least two consecutive reasoning steps in the initial thought chain are the same and the relational semantics are consistent, the at least two consecutive reasoning steps are merged, and the first compressed thought chain is generated based on the merged reasoning steps. Redundant entity nodes corresponding to each reasoning step in the first compressed thinking chain are removed, and a second compressed thinking chain is generated based on the removed first compressed thinking chain. When there are at least two reasoning paths between any two entity nodes in a local knowledge graph, the path with the fewest relation edges is selected as the target reasoning path. The reasoning steps in the second compressed thinking chain are then replaced with the reasoning steps corresponding to the target reasoning path to obtain the compressed thinking chain.
[0012] In some implementations, the compressed thought chain is validated, and based on the validation results, the compressed thought chain is pruned to obtain the target thought chain, including: Determine whether the knowledge triples corresponding to each reasoning step in the compressed thought chain exist in the local knowledge graph. If not, remove the reasoning steps corresponding to knowledge triples that do not exist in the local knowledge graph from the compressed thinking chain and generate the first verification thinking chain; Determine whether there is a logical contradiction or a break in the reasoning chain between adjacent reasoning steps in the first verification thought chain; If so, remove the reasoning steps that contain logical contradictions or have broken reasoning chains from the first verification thinking chain, and generate the second verification thinking chain. Determine whether there are any gaps in the connection between adjacent reasoning steps in the second verification thought chain; If so, remove the reasoning steps located at the connection gaps from the second verification thinking chain, generate the third verification thinking chain, and complete the reasoning steps at the connection gaps based on the local knowledge graph. Based on the completed reasoning steps and the third verification thinking chain, generate the target thinking chain.
[0013] Secondly, this application proposes a large language model question-answering optimization device, the device comprising: The question text acquisition module is used to obtain the original question text; The local knowledge graph construction module is used to construct the corresponding local knowledge graph based on the preset knowledge graph and the original question text; The initial thought chain generation module is used to generate an initial thought chain corresponding to the original question text by calling a large language model based on a local knowledge graph. The path compression processing module is used to compress the initial thought chain based on the local knowledge graph to obtain a compressed thought chain. The pruning and verification processing module is used to verify the compressed thinking chain and prune the compressed thinking chain according to the verification results to obtain the target thinking chain. The text answer generation module is used to determine the answer information corresponding to the original question text by calling a large language model based on the target thought chain.
[0014] Thirdly, this application proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program stored in the memory to implement the steps of the large language model question-answering optimization method of any of the first aspects.
[0015] Fourthly, this application also proposes a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the large language model question-answering optimization method of any one of the first aspects.
[0016] In summary, this application provides a large language model question-answering optimization method. Based on a pre-defined knowledge graph and the original question text, a corresponding local knowledge graph is constructed. A set of factual evidence relevant to the current question is selected from the global knowledge graph, narrowing the knowledge retrieval scope for reasoning and reducing computational overhead during the question-answering process. An initial thought chain is generated by calling the large language model based on the local knowledge graph. This constrains the large language model's generation of reasoning steps by the factual information in the local knowledge graph, suppressing the generation of redundant content unrelated to the core reasoning logic and reducing the length of the thought chain. The initial thought chain is compressed based on the local knowledge graph. By merging equivalent reasoning steps, eliminating redundant entity nodes, and replacing multi-step reasoning paths with direct relation edges, the computational load of question-answering processing is reduced, the latency of question-answering reasoning is lowered, and the response speed of the question-answering system is improved. The compressed thought chain is then verified, and pruning is performed based on the verification results. Unreliable reasoning steps are eliminated through factual verification and logical consistency verification, ensuring the reliability of the reasoning evidence used in the question-answering process and avoiding incorrect answers due to erroneous reasoning. The system uses a target-oriented thinking chain to call a large language model to determine the answer information. The large language model quickly generates the answer based on concise and correct reasoning, which improves the efficiency and quality of question answering. Attached Figure Description
[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A schematic diagram of a large language model question-answering optimization method provided in an embodiment of this application; Figure 2 A schematic diagram of a large language model question-answering optimization device provided in an embodiment of this application; Figure 3 This is a schematic diagram of an electronic device for optimizing large language model question answering, provided as an embodiment of this application. Detailed Implementation
[0018] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.
[0019] This application relates to the field of natural language processing technology, and in particular to a method and apparatus for optimizing question answering using a large language model. The method provided in this application can be widely applied in various practical scenarios such as intelligent question answering, logical reasoning, code generation, compliance auditing, and in-vehicle human-computer interaction, and is especially suitable for intelligent question answering systems that have high requirements for both reasoning efficiency and accuracy.
[0020] CoT (Chain of Thought) is a technique that guides large language models to generate intermediate reasoning steps before outputting the final answer. By breaking down complex problems into a series of simpler subproblems and solving them step by step, CoT improves the performance of large language models in complex tasks such as arithmetic reasoning, common sense reasoning, and symbolic reasoning. However, the CoTs directly generated by large language models often contain a large amount of redundant text and repetitive reasoning steps unrelated to the core reasoning logic, leading to increased token consumption and computational overhead in the question-and-answer process, making it difficult to meet the real-time question-and-answer requirements in low-computing-power scenarios. Furthermore, in long-chain reasoning processes, factual biases in early reasoning steps accumulate and propagate along the reasoning chain, ultimately leading to incorrect answers.
[0021] To facilitate understanding of the technical solution of this application, the terms involved in this application are explained below.
[0022] The large language model described in this application refers to a natural language processing model with a large number of parameters built based on deep learning technology. It receives natural language text input and generates corresponding text output through internal computation and reasoning processes, such as generating question answers, reasoning steps, or structured content. Large language models typically contain billions to hundreds of billions of parameters and possess strong language understanding and generation capabilities.
[0023] The "thought chain" in this application refers to the sequence of intermediate reasoning steps generated by a large language model in answering complex questions. These steps record the model's step-by-step reasoning process from question to answer in natural language or structured form. Each reasoning step in the thought chain typically corresponds to a logical leap or factual connection, and multiple steps are linked together to form a complete reasoning path. The thought chain is an intermediate product connecting the question and the answer, and its quality directly affects the accuracy of the final answer.
[0024] The question-answering optimization in this application refers to the process of compressing the length and enhancing the reliability of the above-mentioned thought chain. Its purpose is to shorten the length of the thought chain and eliminate redundant, repetitive or erroneous reasoning steps while ensuring the accuracy of reasoning, thereby improving the overall efficiency and answer quality of the question-answering system.
[0025] The knowledge graph of this application refers to a semantic network database that stores structured factual information in the form of "entity node-relation edge-entity node" triples. Entity nodes are used to represent specific objects or abstract concepts in the real world, and relation edges are used to represent semantic relationships between entities. For example, the knowledge graph can store the triple "Yao Ming-wife-Ye Li", where "Yao Ming" and "Ye Li" are entity nodes, and "wife" is the relation edge connecting the two entities.
[0026] The local knowledge graph in this application refers to a subset of fact triples that are retrieved and filtered from the global knowledge graph based on the original question text, containing only those relevant to the current question. Compared to the global knowledge graph, the local knowledge graph is significantly smaller in size, retaining only the entity nodes and relation edges necessary for reasoning, thereby reducing the scope and complexity of knowledge retrieval for subsequent reasoning and computation.
[0027] The path compression of the thought chain in this application refers to a set of operations that simplify the reasoning steps in the initial thought chain. This process eliminates duplicate content by merging consecutive reasoning steps with the same starting and ending points and consistent relational semantics, removes entity nodes that do not contribute logically to the reasoning path to shorten the length of the reasoning chain, and uses direct relational edges existing in the local knowledge graph to replace multi-step indirect reasoning paths to reduce redundant reasoning steps. Thus, the overall length of the thought chain is significantly shortened while preserving the integrity of the core reasoning logic.
[0028] The verification and pruning of the thought chain in this application refers to the process of verifying the factual accuracy and logical consistency of the compressed thought chain, and removing unreliable reasoning steps based on the verification results. Specifically, the factual accuracy verification verifies whether the factual triples upon which each reasoning step in the thought chain is based exist in the local knowledge graph; the logical consistency verification verifies whether there are logical contradictions or breaks in the reasoning chain between adjacent reasoning steps. Reasoning steps that fail the verification are removed from the thought chain. If a break in the reasoning chain occurs after removal, it is completed based on the local knowledge graph to ensure the integrity and reliability of the final thought chain. Through the above verification and pruning process, incorrect answers caused by factual or logical errors propagating along the reasoning chain are avoided.
[0029] Please see Figure 1 This is a flowchart illustrating a large language model question-answering optimization method provided in an embodiment of this application, which may specifically include: S110. Obtain the original question text; For example, in a large language model question-answering scenario, the user inputs a question to be answered in natural language, and the original question text serves as the input to the question-answering process. Specifically, the natural language question input by the user is received through a human-computer interaction interface or API interface, and can be a factual question, a logical reasoning question, or a multi-hop complex question; this application does not impose any restrictions on this.
[0030] S120. Based on the preset knowledge graph and the original question text, construct the corresponding local knowledge graph; For example, a pre-defined knowledge graph is a semantic network database that stores structured facts in the form of entity nodes and relation edges, and the scale of a global knowledge graph is usually very large. Directly performing reasoning and retrieval on the entire global knowledge graph would increase unnecessary computational overhead and be difficult to meet the response requirements of real-time question-answering scenarios. Therefore, entity recognition and linking processing is performed on the original question text to extract target entity nodes related to the question, and these target entity nodes are then mapped to their corresponding positions in the pre-defined knowledge graph. Using these target entity nodes as the starting point for retrieval, a local subgraph containing only knowledge triples related to the current question is retrieved from the pre-defined knowledge graph and constructed. This reduces the retrieval space of the large-scale global knowledge graph to a small-scale subgraph highly relevant to the current question, reducing the knowledge search scope and computational complexity of subsequent processing.
[0031] S130. Based on the local knowledge graph, call the large language model to generate the initial thought chain corresponding to the original question text; For example, after constructing a local knowledge graph, this local knowledge graph contains a set of factual evidence necessary to answer the current question. The knowledge triples in this local knowledge graph are converted into constraint prompts. These constraint prompts limit the large language model to using only the factual information already provided in the local knowledge graph when generating reasoning steps. Simultaneously, the reasoning steps are output according to a preset structured format, ensuring that each output reasoning step corresponds to a knowledge triple in the local knowledge graph. This suppresses redundant embellishments, irrelevant descriptions, and repetitive expressions in the thought chain, reducing the text length and redundancy of the thought chain.
[0032] S140. Based on the local knowledge graph, the initial thought chain is compressed to obtain the compressed thought chain. For example, the initial thought chain may still contain issues such as repetitive reasoning, redundant nodes, or detours. For instance, the starting and ending entities of multiple consecutive reasoning steps may be identical and semantically consistent; or there may be entity nodes in the reasoning path that contribute no logically to the final reasoning result; or there may be indirect reasoning paths from the starting entity to the ending entity that pass through multiple intermediate entities, but shorter direct relation edges actually exist in the local knowledge graph. Using the local knowledge graph as a benchmark, the initial thought chain is processed by merging equivalent reasoning steps, eliminating redundant entity nodes, and replacing multi-step indirect reasoning paths with direct relation edges. This shortens the number of reasoning steps and the path length of the thought chain while preserving the integrity of the core reasoning logic.
[0033] S150. Verify the compressed thinking chain, and prune the compressed thinking chain according to the verification results to obtain the target thinking chain. For example, while path compression shortens the length of the thought chain, it may introduce logical breaks or retain reasoning steps that are inconsistent with the facts. Therefore, the compressed thought chain needs to be verified and pruned. The verification process involves verifying whether the knowledge triples corresponding to each reasoning step in the compressed thought chain exist in the local knowledge graph. If not, it indicates that the reasoning step lacks factual basis and is removed from the thought chain. The verification process also involves checking for logical contradictions or breaks in the reasoning chain between adjacent reasoning steps. If these exist, it indicates that the reasoning step cannot form a coherent reasoning chain with the preceding and following steps and is removed from the thought chain. Finally, the pruned thought chain undergoes a connection check. If a connection gap is detected between adjacent reasoning steps due to the pruning operation, the missing reasoning steps are filled in at the gap based on the local knowledge graph, ensuring that the final target thought chain possesses both a streamlined structure and complete reasoning logic.
[0034] S160. Based on the target thinking chain, call the large language model to determine the answer information corresponding to the original question text.
[0035] For example, the target thinking chain simplifies the reasoning steps while retaining the complete reasoning logic. It also ensures the factual accuracy and logical consistency of each reasoning step through factual verification and logical consistency verification. The target thinking chain is used as the reasoning basis and input into the large language model. The large language model is then called to generate the natural language answer corresponding to the original question text based on the reasoning basis.
[0036] In summary, this application provides a large language model question-answering optimization method. Based on a pre-defined knowledge graph and the original question text, a corresponding local knowledge graph is constructed. A set of factual evidence related to the current question is selected from the global knowledge graph, narrowing the knowledge retrieval scope for reasoning and reducing computational overhead during the question-answering process. An initial thought chain is generated by calling the large language model based on the local knowledge graph. This constrains the large language model's generation of reasoning steps by the factual information in the local knowledge graph, suppressing the generation of redundant content unrelated to the core reasoning logic and reducing the length of the thought chain. The initial thought chain is compressed based on the local knowledge graph. By merging equivalent reasoning steps, eliminating redundant entity nodes, and replacing multi-step reasoning paths with direct relation edges, the computational load of question-answering processing is reduced, the latency of question-answering reasoning is lowered, and the response speed of the question-answering system is improved. The compressed thought chain is then verified, and pruning is performed based on the verification results. Unreliable reasoning steps are eliminated through factual verification and logical consistency verification, ensuring the reliability of the reasoning evidence used in the question-answering process and avoiding incorrect answers due to erroneous reasoning. The system uses a target-oriented thinking chain to call a large language model to determine the answer information. The large language model generates the answer based on concise and correct reasoning, which improves the efficiency and quality of question answering.
[0037] In some embodiments, a corresponding local knowledge graph is constructed based on a preset knowledge graph and the original question text, including: Based on a pre-defined knowledge graph, entity recognition and linking are performed on the original question text to identify at least one target entity node. Based on each target entity node and a pre-defined knowledge graph, a local knowledge graph corresponding to the original question text is constructed.
[0038] For example, a local knowledge graph corresponding to the question is constructed based on a pre-defined knowledge graph to filter out factual information irrelevant to the current question from the global knowledge graph, thus reducing the knowledge retrieval scope for subsequent reasoning. The original question text is preprocessed to obtain a standardized question text with a unified format. Then, target entity nodes matching the question's semantics are located from the pre-defined knowledge graph through entity recognition and linking. Starting from each target entity node, the retrieval hop count is adaptively determined based on the question's complexity. Knowledge triples within the retrieval hop count range are searched from the pre-defined knowledge graph. After relevance filtering, knowledge triples that are highly relevant and reliable to the question's semantics are retained. These knowledge triples constitute the local knowledge graph corresponding to the original question text. This local knowledge graph serves as the factual constraint benchmark for the generation, compression, and verification of the thought chain. All reasoning steps are performed based on the factual information in this local knowledge graph. The large language model can only reason within the factual scope defined by this local knowledge graph, and is prohibited from introducing factual information irrelevant to this local knowledge graph. Thus, the factual scope upon which the large language model's reasoning is based is constrained from the global knowledge graph to a local scope highly relevant to the current question.
[0039] In some embodiments, based on a preset knowledge graph, entity recognition and linking are performed on the original question text to determine at least one target entity node, including: The original question text is preprocessed to generate standardized question text; Based on the few-sample prompt template, a large language model is called to extract standardized question text and obtain a candidate entity set; Based on the sequence labeling model, boundary correction and type calibration are performed on each candidate entity in the candidate entity set to generate a calibration entity set. For each calibration entity in the calibration entity set, the target entity node corresponding to the calibration entity is determined from the preset knowledge graph.
[0040] For example, the original question text undergoes preprocessing to eliminate noise interference. The preprocessing operations include full-width / half-width character conversion, special character filtering, removal of meaningless interjections, Chinese word segmentation, stop word filtering, and sentence normalization. Full-width / half-width character conversion uniformly converts full-width characters to half-width characters to avoid inconsistencies in subsequent processing due to encoding differences of the same character; special character filtering removes punctuation marks and special characters that contribute no semantic value; meaningless interjection removal removes common colloquial interjections from the question text; the Chinese word segmentation tool divides continuous sequences of Chinese characters into independent word sequences; stop word filtering removes common words with extremely high frequency but very low semantic contribution, such as auxiliary words and conjunctions; and sentence normalization standardizes the sentence structure of the question expression. After these preprocessing operations, the original question text is converted into a standardized question text with no noise interference and a uniform format.
[0041] A hybrid architecture combining large-scale model-based coarse extraction with sequence labeling-based fine calibration is employed for entity extraction. First, a coarse extraction stage is performed, where a large language model is invoked based on a few-shot prompt template to extract standardized question text. This template contains several question samples with labeled entities. The large language model identifies and outputs candidate entities from the standardized question text by referring to the correspondence between questions and entities in these samples. The large language model possesses strong language understanding and context awareness capabilities, enabling it to handle coarse extraction scenarios involving complex sentence structures, rare entities, and nested entities, outputting a broad set of candidate entities. However, the candidate entities output by the large language model in the coarse extraction stage may have imprecise boundaries or inaccurate type classifications. For example, entity names may contain extra or missing characters, or entities may be classified into incorrect semantic types. Therefore, fine calibration of the candidate entities is necessary.
[0042] For the candidate entity set obtained in the coarse extraction stage, a sequence labeling model is used to perform boundary correction and type calibration on each candidate entity, generating a calibrated entity set. The sequence labeling model is a pre-trained language model fine-tuned with domain data. This model labels each word in the input standardized question text with its position label and entity type label within the entity. The position label identifies the start and end positions of the entity, allowing the system to determine the accuracy of the candidate entity's boundary. If boundary offsets exist, they are corrected to ensure the entity boundary corresponds to the actual word position in the standardized question text. The type label identifies the semantic category of the entity, allowing the system to determine the accuracy of the candidate entity's type classification. If type misclassification exists, calibration is performed to ensure the entity's type information aligns with its actual semantic category in the question context. After boundary correction and type calibration by the sequence labeling model, the position boundaries and semantic types of each candidate entity are adjusted, generating the calibrated entity set.
[0043] For each calibration entity in the calibration entity set, the target entity node corresponding to that calibration entity is determined from a pre-defined knowledge graph. The pre-defined knowledge graph stores factual information in the form of entity nodes and relational edges. Each entity node has a unique node identifier and attribute description text, and nodes are connected by relational edges. Since different entity nodes may share the same entity name, the target entity node cannot be uniquely determined solely by the entity name. Therefore, it is necessary to filter candidate nodes with matching names based on the calibration entity's type information, and then calculate and match the semantic similarity between the standardized question text and the candidate node's attribute description text. The candidate node with the highest semantic similarity is selected as the target entity node corresponding to that calibration entity, thus completing entity identification and linking.
[0044] In summary, through the entity recognition and linking processes described above, the original question text in natural language form is converted into a set of target entity nodes in a pre-defined knowledge graph. Specifically, the preprocessing stage ensures the standardization of the input text through format unification and noise reduction; the few-sample coarse extraction stage utilizes the generalization ability of the large language model to recall candidate entities, reducing the risk of entity omission; the sequence labeling fine calibration stage corrects the inaccuracy of boundaries and types in the coarse extraction results of the large language model, improving the accuracy of entity location and classification; and the entity linking stage uses type filtering and semantic similarity calculation to disambiguate entities with the same name.
[0045] In some embodiments, for each calibration entity in the calibration entity set, determining the target entity node corresponding to the calibration entity from a preset knowledge graph includes: For each calibration entity in the calibration entity set, a search is performed in the preset knowledge graph based on the entity name of the calibration entity to obtain the initial node set; Candidate nodes that match the type information of the calibration entity are selected from the initial node set to generate a candidate node set; Based on the semantic coding model, the question semantic vector of the standardized question text and the node semantic vector corresponding to the attribute description text of each candidate node in the candidate node set are extracted. Calculate the first cosine similarity between the semantic vector of the question and the semantic vectors of each node; The candidate node with the highest first cosine similarity is determined as the target entity node corresponding to the calibration entity.
[0046] For example, the entity name of each calibration entity may have multiple different nodes with the same name in the preset knowledge graph. For instance, the entity name "apple" may correspond to different entity nodes such as "fruit apple," "Apple Inc.," and "Apple Movie." Therefore, disambiguation processing is needed for each calibration entity to determine its unique target entity node in the preset knowledge graph. Using the entity name of the calibration entity as the search condition, a string matching query is performed in the preset knowledge graph to obtain all entity nodes whose names are the same as or match the entity name of the calibration entity. These entity nodes are then grouped into an initial node set. This initial node set contains all possible candidate entity nodes for the calibration entity in the preset knowledge graph, covering all potentially ambiguous nodes.
[0047] The initial node set is filtered using the type information of the calibration entities. In the entity recognition stage, candidate entities are calibrated using a sequence labeling model, with each calibration entity labeled with type information, such as person type, organization type, geographical location type, or work type. This type information serves as the first-level filtering criterion for entity disambiguation, quickly eliminating candidate nodes in the initial node set whose types do not match those of the calibration entities. Specifically, each candidate node in the initial node set is traversed, and its predefined type label in a predefined knowledge graph is obtained. This type label is compared with the type information of the calibration entity, retaining only candidate nodes with matching types and eliminating those with mismatched types, thus generating a candidate node set. This candidate node set retains only candidate nodes with types consistent with the calibration entities, narrowing the candidate range for semantic matching and reducing the complexity of semantic computation.
[0048] Different types of nodes have different attribute description text formats. For example, the attribute description text for a person-type node includes attributes such as date of birth, occupation, and nationality, while the attribute description text for an organization-type node includes attributes such as establishment date, headquarters location, and business scope. The attribute description text records detailed factual information about the entity node in natural language form. A semantic encoding model is used to map natural language text into fixed-length semantic vectors. This model is built on a deep neural network structure and learns through pre-training on a large corpus of text, mapping semantically similar texts to similar positions in the vector space. Specifically, standardized question text is input into the semantic encoding model, which performs word segmentation and context encoding on the input text, outputting a question semantic vector representing the semantics of the question text.
[0049] Simultaneously, the attribute description text of each candidate node in the candidate node set is input into the semantic encoding model. For each candidate node, the semantic encoding model extracts the text content of each attribute field from the node's attribute description text, encodes the entire attribute description text, and generates a node semantic vector corresponding to that candidate node. This node semantic vector represents all the attribute information of the candidate node and reflects the semantic content of the entity node in the knowledge graph. The standardized question text corresponds to a question semantic vector, and each candidate node in the candidate node set corresponds to a node semantic vector. Each pair of vectors resides in the same semantic vector space.
[0050] After obtaining the question semantic vector and the corresponding node semantic vectors for each candidate node, the first cosine similarity between the question semantic vector and each node semantic vector is calculated. Cosine similarity measures the directional consistency between two vectors in a high-dimensional space. The theoretical range of cosine similarity is [-1, 1], but in this scenario, the non-negative interval [0, 1] is typically used as the basis for measuring relevance. When two vectors are in the same direction, the cosine similarity approaches 1; when two vectors are orthogonal, the cosine similarity approaches 0. The more semantically relevant a candidate node's attribute description text is to the standardized question text, the smaller the angle between its corresponding node semantic vector and the question semantic vector in the vector space, and the higher the cosine similarity value. Conversely, if a candidate node is unrelated to the question semantics, the angle between its node semantic vector and the question semantic vector is larger, and the cosine similarity value is lower. By calculating the first cosine similarity between the node semantic vector and the question semantic vector for each candidate node in the candidate node set, the similarity calculation result for each candidate node is obtained. This calculation result quantitatively reflects the semantic association strength between each candidate node and the original question text.
[0051] After obtaining the first cosine similarity of each candidate node in the candidate node set, the first cosine similarity of all candidate nodes is compared, and the candidate node with the highest cosine similarity value is determined. This candidate node is then used as the target entity node corresponding to the calibration entity. Through a triple screening process of entity name retrieval, type filtering, and semantic similarity matching, the final determined target entity node matches in name, is consistent in type, and is semantically closest to the context of the original question text, thus achieving the alignment of the calibration entity from textual description to knowledge graph entity node. Each calibration entity in the calibration entity set is mapped to a target entity node in the preset knowledge graph.
[0052] In summary, this embodiment of the application, through the aforementioned entity recognition and linking processing, sequentially performs name retrieval on each calibration entity in the calibration entity set to ensure the completeness of candidate coverage. Irrelevant candidate nodes are eliminated through type filtering to reduce the complexity of semantic computation. Then, a semantic encoding model is used to extract the question semantic vector of the standardized question text and the node semantic vector of each candidate node, and cosine similarity is compared. Disambiguation is achieved using semantic association strength as the criterion. By progressively narrowing the candidate range from name, type, and semantic dimensions, the final determined target entity node remains consistent with the original question text in terms of string name, entity type, and contextual semantics. This avoids entity linking errors caused by confusion of entities with the same name or semantic misunderstanding, thus facilitating the construction of a local knowledge graph.
[0053] In some embodiments, a local knowledge graph corresponding to the original question text is constructed based on each target entity node and a preset knowledge graph, including: Classify the complexity of standardized problem texts and determine the level of reasoning complexity; Based on the reasoning complexity level, determine the number of retrieval jumps for each target entity node in the preset knowledge graph; Starting from each target entity node, retrieve knowledge triples from the preset knowledge graph whose number of relationship edges with each target entity node is no greater than the number of retrieval hops to obtain an initial set of triples; wherein each knowledge triple includes a target entity node, another entity node associated with the target entity node, and a relationship edge connecting the target entity node and the other entity node; Perform correlation filtering on the initial set of triples to generate the target set of triples; Based on the target triple set, a local knowledge graph corresponding to the original question text is constructed.
[0054] For example, since different questions require significantly different levels of knowledge, simple questions only require a small amount of factual information to complete the reasoning, while complex questions require a broader range of knowledge coverage. Therefore, standardized question texts are classified by complexity to determine the scope of knowledge retrieval. Specifically, a pre-trained text classification model is used to evaluate the reasoning complexity of standardized question texts. This model is trained on a large corpus of questions with labeled complexity and categorizes the input text into three levels: simple common sense, medium reasoning, and complex multi-hop. Simple common sense questions typically involve only a direct inquiry into a single fact and do not require multiple steps of reasoning; medium reasoning questions require two or three steps of reasoning to reach a conclusion; and complex multi-hop questions involve chain-like relationships between multiple entities, requiring traversal of multiple hops to complete the reasoning. Through this classification process, the semantic complexity of the original question text is transformed into level labels.
[0055] The number of retrieval hops in the preset knowledge graph is configured for each target entity node based on its reasoning complexity level. The number of retrieval hops is a parameter controlling the depth of knowledge retrieval, representing the maximum number of steps required to expand outwards from the target entity node along relational edges. For simple common-sense questions, which can be answered with only a single fact, a smaller number of retrieval hops is set, such as a one-hop retrieval, retrieving only knowledge triples directly connected to the target entity node. For medium-level reasoning questions, whose reasoning process involves the transitive association of multiple entities, a moderate number of retrieval hops is set, such as a two-hop retrieval, retrieving all knowledge triples reachable from the target entity node via two relational edges. For complex multi-hop questions, which require traversing long reasoning chains to reach the answer entity from the question entity, a larger number of retrieval hops is set, such as a three-hop retrieval, retrieving all knowledge triples reachable from the target entity node via three relational edges. This mapping between question complexity and the number of retrieval hops ensures that the scope of knowledge retrieval matches the actual needs of the question.
[0056] Starting from each target entity node, a neighborhood search is performed within a pre-defined knowledge graph to obtain all knowledge triples for each target entity node within the hop count range. The knowledge graph represents factual information as a directed graph of entity nodes and relational edges. Entity nodes represent concrete objects or abstract concepts in the real world, and relational edges represent semantic connections between entities. Each knowledge triple consists of a target entity node, another entity node associated with the target entity node, and a relational edge connecting the two entity nodes. During the search, starting from the target entity node, the search expands outward hop by hop along the direction of the relational edges, recording each entity node and each relational edge traversed. All reachable entity nodes and their connections within the hop count range are extracted as knowledge triples. The search results for all target entity nodes are merged to obtain an initial set of triples, which covers all the factual information needed to answer the current question.
[0057] The initial set of triples may contain far more knowledge triples than needed for actual reasoning. Some of these triples may have low relevance to the current problem or insufficient factual reliability, requiring relevance filtering to remove low-value factual information. Relevance filtering involves two stages. The first stage is based on predefined relation confidence levels for each relation edge in a predefined knowledge graph. This confidence level reflects the reliability of the fact within the knowledge graph. Each knowledge triple in the initial set is traversed, and the relation confidence level corresponding to its relation edge is obtained. Triples with relation confidence levels below a predefined confidence threshold are removed to filter out unreliable factual evidence. The second stage is based on semantic relevance measurement. A semantic encoding model is used to extract the question semantic vector from the standardized question text and the group semantic vectors from the remaining knowledge triples. The cosine similarity between the question semantic vector and each group semantic vector is calculated. Triples with cosine similarity levels below a predefined similarity threshold are removed to filter out factual evidence irrelevant to the question semantics. The remaining knowledge triples after filtering constitute the target triple set.
[0058] A directed weighted local knowledge graph is constructed based on the target triple set. The node set of this local knowledge graph consists of the target entity nodes and related entity nodes involved in all knowledge triples in the target triple set. The edge set consists of the relation edges in each knowledge triple, and each relation edge retains its corresponding relation name and relation confidence as edge weight. The size of this local knowledge graph is much smaller than the global size of the preset knowledge graph, and it only contains entity nodes and relation edges that are highly related to the original question text, serving as the factual constraint benchmark for the generation and compression of thought chains.
[0059] In summary, this application's embodiments, through the aforementioned adaptive retrieval and relevance filtering processes, achieve the filtering and construction of a local knowledge graph from a global knowledge graph. Determining the retrieval hop count based on problem complexity avoids unnecessary computational overhead due to excessive retrieval for simple problems, while ensuring sufficient knowledge coverage for complex problems to guarantee reasoning integrity, thus achieving an adaptation between the knowledge retrieval scope and problem complexity. Reliability-based elimination of unreliable factual evidence ensures the accuracy of knowledge triples in the local knowledge graph; semantic similarity-based elimination of factual evidence irrelevant to the problem ensures the relevance of knowledge triples in the local knowledge graph to the problem. These filtering processes screen the initial triple set from both reliability and relevance dimensions, enabling the final constructed local knowledge graph to retain high-quality factual evidence while reducing its size, providing a factual benchmark for the generation, compression, and verification of thought chains.
[0060] In some embodiments, relevance filtering is performed on the initial set of triples to generate a target set of triples, including: Based on the preset knowledge graph, the relation confidence of each knowledge triple in the initial triple set is determined. Knowledge triples with relation confidence less than the preset confidence threshold are removed from the initial triple set to generate a filtered triple set. The semantic encoding model is used to extract the question semantic vector of the standardized question text and the group semantic vector of each knowledge triple in the filtered triple set. Calculate the second cosine similarity between the semantic vector of the question and each group of semantic vectors; Remove knowledge triplets with a second cosine similarity less than a preset similarity threshold from the filtered triplet set to generate the target triplet set.
[0061] For example, after retrieving an initial set of triples from a pre-defined knowledge graph, although all triples in this set have a path to the target entity node within the retrieval hop count range, the reliability of facts varies among different triples. During the construction of the pre-defined knowledge graph, triples from different sources have different levels of credibility. For instance, high-quality knowledge triples constructed manually have higher relational confidence, while knowledge triples extracted from unstructured text using automated information extraction techniques have relatively lower relational confidence. Relational confidence is a predefined numerical attribute of the pre-defined knowledge graph for each relational edge, used to represent the reliability of the fact. Based on the pre-defined knowledge graph, the relational confidence corresponding to each triple in the initial set of triples is obtained. This relational confidence is compared with a pre-set confidence threshold. Triples with relational confidence below the threshold are removed, while those with relational confidence at or above the threshold are retained, generating a filtered set of triples. This filters the initial set of triples from the perspective of factual reliability.
[0062] While all knowledge triples in the filtered triple set possess high factual reliability, some may still contain triples with low semantic relevance to the original question text. Because the knowledge triples in the pre-defined knowledge graph cover a wide range of factual information across multiple domains, during multi-hop retrieval starting from the target entity node, knowledge triples within the hop count range may contain factual content that differs from the current question domain or has weak semantic relevance. For example, when asking about a person's occupation, the search results might include the person's birthplace. While these knowledge triples are factually reliable, they offer no substantial help in answering the current question. The purpose of semantic relevance filtering is to eliminate knowledge triples irrelevant to the question's semantics, retaining only factual evidence that contributes to the reasoning process.
[0063] A semantic encoding model is used to encode both the standardized question text and each knowledge triple in the filter triple set. The semantic encoding model is built on a deep neural network structure and pre-trained on a large-scale text corpus, mapping natural language texts of different lengths and expressions into fixed-length semantic vectors. Texts with similar semantics have a small spatial distance in the vector space. Specifically, the standardized question text is input into the semantic encoding model, which performs contextual encoding and semantic aggregation on the entire question text, outputting a question semantic vector representing the overall semantics of the question. Simultaneously, each knowledge triple in the filter triple set is traversed, and the target entity node name, relation edge name, and another entity node name in the knowledge triple are concatenated into a text sequence according to a preset order. This concatenated text sequence is then input into the semantic encoding model, outputting a group semantic vector representing the overall semantics of the knowledge triple.
[0064] The second cosine similarity is calculated between the question semantic vector and each group semantic vector. Both the second and first cosine similarities measure the degree of directional consistency between the two vectors in high-dimensional space. When the standardized question text is semantically highly correlated with a knowledge triple, the question semantic vector and the group semantic vector of that knowledge triple are aligned in high-dimensional space, and the value of the second cosine similarity approaches 1. When the standardized question text is semantically unrelated to a knowledge triple, the question semantic vector and the group semantic vector of that knowledge triple are not aligned in high-dimensional space, and the value of the second cosine similarity approaches 0. By calculating the second cosine similarity between the group semantic vector corresponding to each knowledge triple in the filtered triple set and the question semantic vector, the semantic association strength value between each knowledge triple and the original question text is obtained. This value is used for semantic relevance filtering.
[0065] Each second cosine similarity is compared with a preset similarity threshold. The preset similarity threshold is a pre-defined numerical parameter used to determine whether there is sufficient semantic association between the knowledge triple and the original question text. When the second cosine similarity of a knowledge triple is less than the preset similarity threshold, it indicates that the knowledge triple has a low degree of semantic association with the original question text and has limited contribution to the subsequent reasoning process; therefore, it is removed from the filter triple set. When the second cosine similarity of a knowledge triple is not lower than the preset similarity threshold, it indicates that the knowledge triple is highly semantically related to the original question text; therefore, it is retained in the set. After traversing all knowledge triples in the filter triple set, all retained knowledge triples constitute the target triple set, where each knowledge triple simultaneously satisfies the requirements of factual reliability and semantic relevance.
[0066] In summary, this embodiment of the application filters the initial set of triples from two dimensions: factual reliability and semantic relevance, through the aforementioned relevance filtering process. Unreliable knowledge triples are eliminated based on a relation confidence threshold, ensuring the accuracy of the factual basis in the target triple set and preventing low-quality facts from misleading subsequent reasoning. Knowledge triples irrelevant to the question are eliminated based on a cosine similarity threshold between the question semantic vector and the group semantic vector, ensuring a high degree of relevance between the factual basis in the target triple set and the question, and preventing irrelevant facts from interfering with subsequent reasoning. This filtering process enables the final generated target triple set to achieve a high degree of semantic matching with the question while ensuring factual accuracy.
[0067] In some embodiments, based on a local knowledge graph, a large language model is invoked to generate an initial thought chain corresponding to the original question text, including: Convert each knowledge triple in the local knowledge graph into constraint prompt words; Based on constraint prompts and the original question text, the large language model is invoked to output an initial thought chain consisting of multiple reasoning steps according to a preset structured format.
[0068] For example, the structured factual information stored in the local knowledge graph is transformed into textual constraints that the large language model can understand and follow. Each knowledge triple in the local knowledge graph consists of a target entity node, another entity node associated with the target entity node, and a relational edge connecting the two nodes. This triple is represented using a graph structure, while the input format of the large language model is a sequence of natural language text. Therefore, the factual information in the graph structure is converted into prompts in the form of textual descriptions.
[0069] This constraint tells the large language model that it can only use knowledge triples provided in the local knowledge graph as factual basis when making inferences, and prohibits inference based on factual information unrelated to the local knowledge graph stored in the model's own parametric memory. Through this transformation process, the structured facts stored in the local knowledge graph are transferred to the input of the large language model, providing factual boundary constraints for inference generation.
[0070] The large language model receives input based on constraint prompts and the original question text. It then outputs an initial thought chain consisting of multiple reasoning steps according to a pre-defined structured format. The input to the large language model contains two parts: the factual boundary constraints provided by the constraint prompts and the user's query intent represented by the original question text. These two parts determine the scope of the model's output. The pre-defined structured format requires each reasoning step output by the large language model to be expressed according to a specified structured form, preventing the model from outputting the reasoning process in free text form and forcing it to follow the pre-defined format for each step. Under the constraints of the pre-defined structured format, each reasoning step in the initial thought chain output by the large language model corresponds to a knowledge triple in a local knowledge graph, and multiple reasoning steps constitute a complete reasoning path according to the reasoning order.
[0071] In summary, this embodiment of the application, through the aforementioned constraint cue word conversion and preset structured format output, embeds the structured factual information of the local knowledge graph into the reasoning generation process of the large language model. The constraint cue words limit the source of facts upon which the model's reasoning is based, ensuring that the model can only reason within the scope of facts provided by the local knowledge graph, preventing the introduction of irrelevant external facts and thus reducing the probability of factual errors in the thought process. The preset structured format limits the expression of the model's output reasoning steps, ensuring that each output reasoning step corresponds to a knowledge triple, facilitating path compression and verification pruning processes for identifying and analyzing the reasoning steps.
[0072] In some embodiments, based on a local knowledge graph, the initial thought chain is compressed to obtain a compressed thought chain, including: When the starting entity node and ending entity node corresponding to at least two consecutive reasoning steps in the initial thought chain are the same and the relational semantics are consistent, the at least two consecutive reasoning steps are merged, and the first compressed thought chain is generated based on the merged reasoning steps. Redundant entity nodes corresponding to each reasoning step in the first compressed thinking chain are removed, and a second compressed thinking chain is generated based on the removed first compressed thinking chain. When there are at least two reasoning paths between any two entity nodes in a local knowledge graph, the path with the fewest relation edges is selected as the target reasoning path. The reasoning steps in the second compressed thinking chain are then replaced with the reasoning steps corresponding to the target reasoning path to obtain the compressed thinking chain.
[0073] For example, after obtaining the initial thought chain, although the thought chain already has a structured format, it may still contain problems such as repetitive statements, redundant nodes, or detours in reasoning. Therefore, path compression processing is needed to shorten the number of reasoning steps. Each reasoning step in the initial thought chain records the logical relationship from the starting point to the ending point of reasoning in the form of entity nodes and relation edges. When the starting and ending entity nodes corresponding to at least two consecutive reasoning steps are the same and the relation semantics are consistent, it indicates that these two reasoning steps logically express the same reasoning content. For example, if the first consecutive reasoning step is entity A reaching entity B through relation A, and the second reasoning step is entity A reaching entity B through relation B, and relations A and B are semantically equivalent, then these two reasoning steps constitute repetitive reasoning.
[0074] The process iterates through all reasoning steps in the initial thought chain, checking in sequence whether the starting entity nodes, ending entity nodes, and semantic relationships of any two adjacent reasoning steps are identical. Adjacent reasoning steps that simultaneously meet these conditions are identified as equivalent steps to be merged. Multiple reasoning steps to be merged are then combined into one, retaining the same starting entity nodes, ending entity nodes, and semantic relationships. After the iteration is complete, all unmerged reasoning steps and the merged reasoning step are rearranged according to their original reasoning order to generate the first compressed thought chain.
[0075] After obtaining the first compressed thought chain, there may be entity nodes in this chain that do not substantially contribute to the logical progression of the reasoning path from the starting point to the ending point. Although these entity nodes appear in the reasoning chain, their presence or absence does not affect the logical deduction relationship from the starting entity to the ending entity. After removing them, the logical chain of the thought chain remains complete. To identify such redundant entity nodes, we traverse each reasoning step in the first compressed thought chain, obtain the starting entity node, relation edge, and ending entity node corresponding to each reasoning step, and check whether there exists an entity node that is both the ending entity node of one reasoning step and the starting entity node of the next adjacent reasoning step, but whose existence does not provide necessary logical deduction from the starting entity node of the previous reasoning step to the ending entity node of the next reasoning step.
[0076] For example, in the reasoning path where entity A reaches entity C via relation A and then reaches entity D via relation B, if the existence of entity C does not substantially contribute to the derivation from entity A to entity D, then entity C is a redundant entity node. After identifying the redundant entity node, it is removed from the reasoning chain, and the preceding and following reasoning steps of the redundant entity node are directly connected, so that the endpoint entity node of the preceding reasoning step is directly updated to the endpoint entity node of the following reasoning step, while maintaining the reasoning order of each reasoning step. After traversing and completing the removal of all redundant entity nodes, the second compressed thought chain is generated based on the removed reasoning steps.
[0077] After obtaining the second compressed thought chain, there may still be cases of detour reasoning in the thought chain. That is, the reasoning path from the starting entity node to the ending entity node is an indirect reasoning path that passes through multiple intermediate entity nodes. However, there are actually direct relationship edges in the local knowledge graph from the starting entity node to the ending entity node.
[0078] For each reasoning step or the reasoning path formed by multiple consecutive reasoning steps in the second compressed thought chain, the starting entity node and the ending entity node of the reasoning path are extracted, and these two entity nodes are used as query conditions for retrieval in the local knowledge graph. The local knowledge graph may store multiple reasoning paths from the starting entity node to the ending entity node, including indirect reasoning paths that pass through multiple intermediate entity nodes and direct reasoning paths that directly connect two entity nodes through only one relation edge.
[0079] When an indirect reasoning path and a direct relation edge are detected simultaneously from the starting entity node to the ending entity node in the local knowledge graph, it indicates that the corresponding reasoning step in the second compressed thinking chain has adopted a non-optimal path. All reasoning paths from the starting entity node to the ending entity node are obtained from the local knowledge graph, and the number of relation edges contained in each reasoning path is counted. The reasoning path with the fewest relation edges is determined as the target reasoning path.
[0080] After determining the target reasoning path, the corresponding reasoning steps in the second compressed thinking chain are replaced with the reasoning steps corresponding to the target reasoning path. When the target reasoning path is a direct relation edge, the original multi-step reasoning steps in the second compressed thinking chain are replaced with a single reasoning step consisting of a starting entity node, a direct relation edge, and an ending entity node, significantly shortening the length of the reasoning chain. When the target reasoning path is an indirect reasoning path containing fewer relation edges, the longer sequence of reasoning steps in the second compressed thinking chain is replaced with the sequence of reasoning steps corresponding to the shorter sequence of relation edges in the target reasoning path.
[0081] During the replacement process, the content and order of other reasoning steps in the reasoning chain that are not involved in the replacement remain unchanged; only the reasoning paths that meet the replacement conditions are updated. After traversing all reasoning paths in the second compressed thought chain and completing the replacement of all replaceable paths, a compressed thought chain is generated based on all the replaced reasoning steps.
[0082] In summary, the embodiments of this application simplify the initial thought chain through the aforementioned path compression processing. Merging equivalent steps eliminates repetitive reasoning content caused by semantic repetition in the initial thought chain, merging multiple reasoning steps with the same starting and ending points into a single reasoning step, thus reducing the number of redundant reasoning steps in the thought chain. Eliminating redundant nodes removes intermediate entity nodes in the thought chain that do not substantially contribute to the reasoning logic, shortening the overall length of the reasoning chain and making its structure more compact. Direct relations replace long reasoning by using direct relation edges stored in the local knowledge graph to replace detour-like multi-step reasoning paths with the shortest reasoning paths, optimizing the reasoning chain at the path level. The three compression operations are executed sequentially, processing three different types of redundancy in the thought chain: repeated expressions, redundant nodes, and detour paths. This reduces the length of the thought chain while preserving the integrity of the original reasoning logic, lowering the computational load of verification pruning and the token consumption of the reasoning process.
[0083] In some embodiments, the compressed thought chain is validated, and the compressed thought chain is pruned based on the validation results to obtain the target thought chain, including: Determine whether the knowledge triples corresponding to each reasoning step in the compressed thought chain exist in the local knowledge graph. If not, remove the reasoning steps corresponding to knowledge triples that do not exist in the local knowledge graph from the compressed thinking chain and generate the first verification thinking chain; Determine whether there is a logical contradiction or a break in the reasoning chain between adjacent reasoning steps in the first verification thought chain; If so, remove the reasoning steps that contain logical contradictions or have broken reasoning chains from the first verification thinking chain, and generate the second verification thinking chain. Determine whether there are any gaps in the connection between adjacent reasoning steps in the second verification thought chain; If so, remove the reasoning steps located at the connection gaps from the second verification thinking chain, generate the third verification thinking chain, and complete the reasoning steps at the connection gaps based on the local knowledge graph. Based on the completed reasoning steps and the third verification thinking chain, generate the target thinking chain.
[0084] For example, although the reasoning steps in the compressed thinking chain undergo optimization based on local knowledge graphs during path compression, the compression operation itself may introduce new problems. For instance, when merging equivalent steps, two fundamentally different steps may be mistakenly merged into one, resulting in the merged reasoning step lacking corresponding factual basis in the local knowledge graph; or when removing redundant entity nodes, intermediate entity nodes with necessary logical support may be mistakenly removed, causing the preceding and following reasoning steps to lose factual connection. Therefore, it is necessary to perform factual verification on each reasoning step in the compressed thinking chain.
[0085] Specifically, each reasoning step in the compressed thought chain is traversed to obtain the corresponding knowledge triple. This knowledge triple contains the starting entity node, relation edge, and ending entity node of the reasoning step. Using this knowledge triple as a query condition, a matching search is performed in the local knowledge graph to determine whether the knowledge triple exists in the local knowledge graph. If it exists in the local knowledge graph, it indicates that the reasoning step has objective factual basis; if it does not exist in the local knowledge graph, it indicates that the reasoning step lacks factual basis.
[0086] After factually verifying each reasoning step in the compressed thought chain, the chain is pruned based on the verification results. For reasoning steps that fail factual verification, since their corresponding knowledge triples do not exist in the local knowledge graph, it indicates that the facts upon which these reasoning steps are based cannot be found to support them in the local knowledge graph, making the content of these reasoning steps unreliable. These reasoning steps that fail factual verification are removed from the compressed thought chain, while all reasoning steps that pass factual verification are retained. The retained reasoning steps are then rearranged according to their original reasoning order to generate the first verification thought chain. Through this process, all reasoning steps lacking objective factual basis in the compressed thought chain are removed, eliminating factual errors that may have been introduced during the compression process or were already present, thus preventing unfounded reasoning steps from misleading logical verification and answer generation.
[0087] After obtaining the first verification thought chain, although each reasoning step has been factually verified—meaning each step, when viewed individually, has objective factual basis—the logical relationships between the steps still need to be examined. The reasoning steps in the first verification thought chain are arranged according to the reasoning order. The output entity node of each reasoning step should serve as the input entity node for subsequent reasoning steps or have a logical connection with them. A logical contradiction exists when there is a factual conflict between two adjacent reasoning steps. For example, if the endpoint entity node of the preceding reasoning step and the starting entity node of the following reasoning step point to the same entity but provide different attribute descriptions, or if the reasoning conclusion of the preceding and following reasoning steps contradict each other semantically.
[0088] When there is no logical connection between the entity nodes of two adjacent reasoning steps, for example, the end entity node of the previous reasoning step and the start entity node of the next reasoning step are semantically unrelated, then the reasoning chain is broken. Traverse each adjacent reasoning step in the first verification thought chain to determine whether there is the aforementioned logical contradiction or reasoning chain break between each pair of adjacent reasoning steps.
[0089] After verifying the logical consistency of adjacent reasoning steps in the first verification thought chain, the first verification thought chain is pruned based on the verification results. Adjacent reasoning steps identified as having logical contradictions or broken reasoning chains are removed from the first verification thought chain because they cannot form a coherent logical derivation relationship with the preceding and following steps. When a reasoning step has a logical contradiction or broken reasoning chain with the preceding reasoning step and also has a logical problem with the following reasoning step, that reasoning step is also removed. After traversing all adjacent reasoning steps and removing all reasoning steps with logical problems, the remaining reasoning steps are rearranged according to their original reasoning order to generate the second verification thought chain. Through this process, all reasoning steps in the first verification thought chain that cannot form a coherent logical relationship with adjacent steps are removed, eliminating logical contradictions that may have been introduced during the compression process or that already existed, and preventing logical errors from propagating along the reasoning chain.
[0090] After obtaining the second verification thought chain, due to the removal operations of factual pruning and logical pruning, there may be gaps in the connection between inference steps due to the removal of inference steps. That is, the originally coherent inference chain becomes logically missing after the removal of some inference steps. To address this, a connection check is performed on the second verification thought chain. Specifically, each adjacent inference step in the second verification thought chain is traversed sequentially to determine whether there is a gap between each pair of adjacent inference steps. That is, whether there is a direct logical relationship or entity consistency between the endpoint entity node of the previous inference step and the starting entity node of the next inference step.
[0091] When a connection gap is detected, the endpoint entity node of the preceding reasoning step and the starting entity node of the following reasoning step at that point are obtained. Using these two entity nodes as search criteria, a reasoning path connecting the two entity nodes is found in the local knowledge graph. One or more reasoning steps corresponding to this reasoning path are then added to the connection gap, creating a coherent logical connection between the preceding and added reasoning steps, as well as between the added and following reasoning steps. After completing the completion of all connection gaps, the completed reasoning steps are rearranged and combined with the original reasoning steps retained in the second verification thought chain according to the reasoning order to generate the target thought chain.
[0092] In summary, this application's embodiments validate the compressed thought chain from both factual and logical dimensions through the aforementioned verification and pruning processes. Factual verification determines the existence of factual basis for each reasoning step based on the local knowledge graph; pruning removes all reasoning steps lacking factual basis, ensuring the reliability of the thought chain from the perspective of factual accuracy. Logical consistency verification detects logical contradictions or breaks in the reasoning chain between adjacent reasoning steps; pruning removes all reasoning steps with logical problems, ensuring the continuity of the thought chain from the perspective of logical self-consistency. The gap detection and completion mechanism repairs any logical deficiencies that may arise after pruning, supplementing missing reasoning steps at gaps based on the local knowledge graph, ensuring that the final generated target thought chain maintains a complete logical structure even after removing erroneous steps. The above verification and processing simplify the thought chain from three levels: factual basis, logical relationship, and structural integrity, ensuring that the target thought chain, while streamlining reasoning steps, also maintains factual accuracy and logical completeness.
[0093] In some embodiments, based on the target thought chain, a large language model is invoked to determine the answer information corresponding to the original question text, including: For example, a structured sequence of reasoning steps is transformed into a user-readable natural language answer. Each reasoning step in the target thought chain records the derivation process from the original question text to the reasoning conclusion in the form of knowledge triples. Each reasoning step includes the name of the starting entity node, the name of the relation edge, and the name of the ending entity node. Adjacent reasoning steps are linked together to form a reasoning chain. This target thought chain has undergone path compression to simplify the reasoning steps and has been verified for factual accuracy and logical consistency to ensure the factual accuracy and logical self-consistency of each reasoning step. The reasoning steps in the target thought chain are combined according to the reasoning order to form a structured reasoning basis text, and an answer generation instruction is attached to form an answer generation prompt. This answer generation prompt instructs the large language model to generate the corresponding natural language answer based on the provided sequence of reasoning steps.
[0094] After generating the answer generation prompt, the prompt is input into the large language model, which then performs the answer generation operation. Upon receiving the answer generation prompt containing the target thought chain, the large language model's internal decoder generates natural language text word by word based on the provided sequence of reasoning steps. Since the target thought chain already records the complete reasoning logic from question to answer, the large language model does not need to perform complex multi-step reasoning calculations itself; it only needs to convert the structured sequence of reasoning steps into fluent natural language expressions. Specifically, the large language model maps the entity node names and relation edge names in each reasoning step to natural language phrases according to the order of the reasoning steps. For example, it combines the starting entity node name, relation edge name, and ending entity node name into a semantically coherent statement. Then, based on the logical relationship between the preceding and following reasoning steps, it adds connectors and adjusts the word order of each generated statement to form a continuous and fluent answer text. The semantic content of this answer text is consistent with the reasoning logic of the target thought chain, and each factual statement in the answer text can be traced back to the corresponding reasoning step in the target thought chain.
[0095] In summary, the target thought chain in this embodiment undergoes path compression, reducing the number of reasoning steps and shortening the input sequence length of the large language model during answer generation, thereby reducing computational overhead and token consumption during model reasoning. Furthermore, since the target thought chain undergoes factual verification and logical consistency verification, each reasoning step has factual basis in the local knowledge graph, and adjacent reasoning steps are logically coherent. Therefore, the answer content generated by the large language model based on this reasoning basis does not exhibit any deviation in factual accuracy or logical correctness due to reasoning errors.
[0096] The technical solution of this application will be further described in detail below through specific embodiments.
[0097] The user entered the following original question text through the interactive interface of the intelligent question-and-answer system: What communication method does the lunar rover mainly rely on to transmit data with the ground control center when operating on the lunar surface?
[0098] After obtaining the original question text, the question-answering system performs entity recognition and link processing. Preprocessing operations such as full-width / half-width character conversion, special symbol filtering, and stop word filtering are performed on the original question text to obtain the standardized question text: Which communication method does the lunar rover primarily rely on for data transmission from the ground control center?
[0099] A large language model is invoked to perform coarse extraction on the standardized question text based on a few-shot prompt template. This template contains a preset number of examples comparing questions with the extracted entity results. The large language model extracts candidate entities from the standardized question text according to the format of the examples, resulting in a candidate entity set that includes the lunar rover and the ground control center. A sequence labeling model is then invoked to perform boundary correction and type calibration on each candidate entity. Boundary correction confirms that the lunar rover is a complete entity representation and that the ground control center is a single entity rather than two independent entities. Type calibration determines that the lunar rover's type information is a probe type and that the ground control center's type information is a ground facility type, generating a calibration entity set that includes both the lunar rover and the ground control center.
[0100] For each calibration entity in the calibration entity set, the lunar rover searches a pre-defined knowledge graph based on the entity name, obtaining an initial node set containing multiple candidate nodes with similar names such as lunar rover, lunar probe, and lunar surface rover. A type information detector then filters candidate nodes with matching types from this initial node set, generating a candidate node set. A semantic encoding model extracts the question semantic vector from the standardized question text, and simultaneously extracts the node semantic vector corresponding to the attribute description text of each candidate node. The first cosine similarity between the question semantic vector and the semantic vectors of each node is calculated, and the candidate node with the highest first cosine similarity is determined as the target entity node, completing the entity linking of the lunar rover to the target entity node. The same processing method is used to determine the target entity node corresponding to the ground control center.
[0101] After identifying each target entity node, a local knowledge graph construction process is performed. The standardized question text is classified by complexity, determining it to be of medium reasoning level. This is because answering the question requires first confirming that the rover's data transmission target is the ground control center, and then confirming the communication method used between them. Based on this reasoning complexity level, the retrieval hop count for each target entity node in the pre-defined knowledge graph is determined to be two hops. Starting with the rover's target entity node, knowledge triples with no more than two hops of relational edges between the rover and the rover are retrieved from the pre-defined knowledge graph, resulting in an initial set of triples. This initial set of triples includes the following knowledge triples: the signal forwarding relationship between the rover and the relay satellite; the data transmission communication relationship between the rover and the ground control center; the equipment connection relationship between the ground control center and the communication base station; the equipment composition relationship between the communication base station and the data transmission radio; and the membership relationship between the rover and the lunar surface operation system. In these knowledge triples, the relay satellite, communication base station, data transmission radio, and lunar surface operation system are all associated entity nodes retrieved from the pre-defined knowledge graph and do not require pre-identification during the entity identification stage.
[0102] The initial set of triples undergoes relevance filtering. Based on a pre-defined knowledge graph, the relation confidence of each knowledge triple is determined. Triples with relation confidence below a pre-set confidence threshold are removed. All triples in the initial set whose relation confidence meets the pre-defined requirements are retained, generating a filtered triple set. Then, a semantic encoding model is used to extract the question semantic vector from the standardized question text and the group semantic vector from each knowledge triple in the filtered triple set. The second cosine similarity between the question semantic vector and each group semantic vector is calculated. The highest second cosine similarity is found for the data transmission communication relationship between the lunar rover and the ground control center; the medium second cosine similarity is found for the signal forwarding relationship between the lunar rover and the relay satellite; and the low second cosine similarity is found for the equipment connection relationship between the ground control center and the communication base station, and the equipment composition relationship between the communication base station and the data transmission radio, both below the pre-set similarity threshold.
[0103] Knowledge triplets with a second cosine similarity below a preset similarity threshold are removed from the filtered triplet set. The data transmission and communication relationships between the lunar rover and the ground control center, as well as the signal forwarding relationships between the lunar rover and the relay satellite, are retained to generate a target triplet set. A local knowledge graph is constructed based on this target triplet set. This local knowledge graph contains three entity nodes: the lunar rover, the relay satellite, and the ground control center, as well as edges representing the signal forwarding relationships between the lunar rover and the relay satellite, and the data transmission and communication relationships between the lunar rover and the ground control center.
[0104] After constructing the local knowledge graph, an initial thought chain generation process is performed. Each knowledge triple in the local knowledge graph is converted into constraint prompts. These constraint prompts restrict the large language model to use only the factual information already provided in the local knowledge graph when generating reasoning steps. Based on the constraint prompts and the original question text, the large language model is invoked to output an initial thought chain consisting of multiple reasoning steps according to a preset structured format. Under the constraint of the constraint prompts, the large language model can only use the signal forwarding relationship between the lunar rover and the relay satellite, and the data transmission communication relationship between the lunar rover and the ground control center, stored in the local knowledge graph, to perform reasoning. The reasoning steps are output according to the preset structured format, generating the initial thought chain as follows: The lunar rover and the relay satellite establish a communication link through signal forwarding; the relay satellite forwards the signals emitted by the lunar rover to the ground control center; and the ground control center receives data from the lunar rover through data transmission communication.
[0105] After generating the initial thought chain, path compression is performed. Equivalence is assessed for at least two consecutive reasoning steps in the initial thought chain. It is checked whether there are consecutive reasoning steps with identical starting and ending entity nodes and consistent relational semantics. If the starting and ending entity nodes of each reasoning step in the initial thought chain are different, there are no equivalent steps, and merging is unnecessary. A first compressed thought chain is then generated, which is identical to the initial thought chain.
[0106] Then, redundant entity node detection is performed on each reasoning step in the first compressed thought chain. The initial reasoning path in the thought chain is that the lunar rover establishes a communication link with the ground control center through a relay satellite, the relay satellite forwards the signal to the ground control center, and the ground control center receives the lunar rover's data through data transmission communication. It is detected that the relay satellite acts as an intermediate entity node connecting the lunar rover and the ground control center in the reasoning step. However, a data transmission communication relationship edge between the lunar rover and the ground control center also exists in the local knowledge graph. The lunar rover can directly establish a communication connection with the ground control center without going through the relay satellite, so the relay satellite's existence does not substantially contribute to the reasoning path and is therefore identified as a redundant entity node. This entity node is removed from the inference step preceding and following the one containing the relay satellite, allowing the preceding and following inference steps to connect directly, generating the second compressed thought chain. After removing the relay satellite, the reasoning steps in the second compressed thought chain are adjusted to: the lunar rover establishes a communication link with the ground control center through data transmission communication, and the ground control center receives the lunar rover's data through data transmission communication.
[0107] Then, direct relationship substitution processing is performed on the second compressed thought chain. The reasoning path from the starting entity node (lunar rover) to the ending entity node (ground control center) in the second compressed thought chain is obtained. This reasoning path currently involves the lunar rover reaching the ground control center via data transmission communication. The starting entity node of this reasoning path is the lunar rover, and the ending entity node is the ground control center. A search is performed in the local knowledge graph using the lunar rover and ground control center as query conditions. The local knowledge graph stores data transmission communication relationship edges between the lunar rover and the ground control center. These relationship edges directly connect the two entity nodes, and there is one such edge. The corresponding reasoning steps in the second compressed thought chain include two reasoning steps: the lunar rover and the ground control center establishing a communication link via data transmission communication, and the ground control center receiving data from the lunar rover via data transmission communication. There are two such relationship edges.
[0108] The two reasoning steps in the second compressed thought chain are replaced with single-step reasoning steps corresponding to direct relation edges in the local knowledge graph. The replaced reasoning steps involve the lunar rover directly transmitting data with the ground control center via data transmission communication. After these three types of compression processing, a compressed thought chain is generated, which represents the lunar rover directly transmitting data with the ground control center via data transmission communication.
[0109] The compressed thought chain is validated and pruned. Each reasoning step in the compressed thought chain is factually verified, and the corresponding knowledge triples are obtained. The starting entity node of this knowledge triple is the lunar rover, the relation edge is data transmission and communication, and the ending entity node is the ground control center. Using this knowledge triple as a query condition, a matching search is performed in the local knowledge graph. If the knowledge triple exists in the local knowledge graph, the reasoning step passes the factual verification, generating the first validated thought chain.
[0110] Logical consistency verification is performed on adjacent reasoning steps in the first verification thought chain. Since the first verification thought chain contains only one reasoning step and there are no adjacent reasoning steps, there is no need to detect logical contradictions or breaks in the reasoning chain; therefore, the first verification thought chain is directly identified as the second verification thought chain. Connection discontinuity detection is performed on the second verification thought chain. Since the second verification thought chain contains only one reasoning step and there are no connections between adjacent reasoning steps, there is no need to perform connection discontinuity completion processing; therefore, the second verification thought chain is directly identified as the target thought chain. This target thought chain involves the lunar rover directly transmitting data with the ground control center via data transmission communication.
[0111] After obtaining the target thought chain, the answer information generation process is executed. The reasoning steps in the target thought chain are combined in the order of reasoning into a structured reasoning basis text. This reasoning basis text represents the data transmission between the lunar rover and the ground control center via data transmission communication, along with an answer generation instruction. This instruction instructs the large language model to generate the corresponding natural language answer based on the provided reasoning steps. The reasoning basis text and the answer generation instruction together constitute the answer generation prompt, which is then input into the large language model. Upon receiving the answer generation prompt containing the target thought chain, the large language model maps the reasoning steps in the target thought chain into natural language expressions, generating the natural language answer corresponding to the original question text: The lunar rover primarily relies on data transmission communication with the ground control center during its lunar surface operations. This answer content is consistent with the reasoning logic of the target thought chain, and the factual statements in the answer can be traced back to the corresponding reasoning steps in the target thought chain, i.e., the data transmission communication relationship between the lunar rover and the ground control center. Simultaneously, the structured reasoning path and the source information of the knowledge graph triples corresponding to each reasoning step are output. The source information records that the starting entity node of the knowledge triples on which the reasoning steps are based is the lunar rover, the relation edge is data transmission and communication, and the ending entity node is the ground control center. Moreover, the knowledge triples originate from the direct relation edges between the lunar rover and the ground control center stored in the local knowledge graph.
[0112] Please see Figure 2 This is a schematic diagram of a large language model question-answering optimization device 20 provided in an embodiment of this application. The device includes: Problem text acquisition module 21 is used to acquire the original problem text; The local knowledge graph construction module 22 is used to construct the corresponding local knowledge graph based on the preset knowledge graph and the original question text; The initial thought chain generation module 23 is used to generate an initial thought chain corresponding to the original question text by calling a large language model based on a local knowledge graph. The path compression processing module 24 is used to perform path compression processing on the initial thought chain based on the local knowledge graph to obtain the compressed thought chain. The pruning verification processing module 25 is used to verify the compressed thinking chain and prune the compressed thinking chain according to the verification results to obtain the target thinking chain. The text answer generation module 26 is used to determine the answer information corresponding to the original question text by calling a large language model based on the target thinking chain.
[0113] Please see Figure 3This application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and running on the processor. When the processor 320 executes the computer program 311, it implements the steps of the large language model question answering optimization method.
[0114] Since the electronic device described in this embodiment is the device used to implement the large language model question answering optimization device in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this application is within the scope of protection of this application.
[0115] In practice, when the computer program 311 is executed by the processor, it can implement any of the embodiments corresponding to the first aspect.
[0116] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing computer program instructions. When the computer program instructions are loaded and executed by a processor, they implement the steps of the large language model question-answering optimization method described above.
[0117] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0118] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
[0119] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications beyond the scope of this specification.
[0120] Obviously, those skilled in the art can make various modifications to this specification without departing from its spirit and scope. Therefore, this specification also intends to include any modifications that fall within the scope of the claims and their equivalents.
Claims
1. A question-answering optimization method for large language models, characterized in that, The method includes: Obtain the original question text; Based on the preset knowledge graph and the original question text, a corresponding local knowledge graph is constructed; Based on the local knowledge graph, a large language model is invoked to generate an initial thought chain corresponding to the original question text; Based on the local knowledge graph, the initial thought chain is compressed to obtain a compressed thought chain. The compressed thinking chain is verified, and the compressed thinking chain is pruned according to the verification results to obtain the target thinking chain. Based on the target thought chain, the large language model is invoked to determine the answer information corresponding to the original question text.
2. The method according to claim 1, characterized in that, The construction of a corresponding local knowledge graph based on a preset knowledge graph and the original question text includes: Based on a pre-defined knowledge graph, entity recognition and linking are performed on the original question text to determine at least one target entity node; Based on each of the target entity nodes and the preset knowledge graph, a local knowledge graph corresponding to the original question text is constructed.
3. The method according to claim 2, characterized in that, The step of performing entity recognition and linking on the original question text based on a preset knowledge graph to determine at least one target entity node includes: The original question text is preprocessed to generate standardized question text; Based on the few-sample prompt template, a large language model is invoked to extract and process the standardized question text to obtain a candidate entity set; Based on the sequence labeling model, boundary correction and type calibration are performed on each candidate entity in the candidate entity set to generate a calibrated entity set. For each calibration entity in the calibration entity set, a target entity node corresponding to the calibration entity is determined from a preset knowledge graph.
4. The method according to claim 3, characterized in that, The step of determining the target entity node corresponding to each calibration entity in the calibration entity set from a preset knowledge graph includes: For each calibration entity in the calibration entity set, an initial node set is obtained by searching the preset knowledge graph based on the entity name of the calibration entity. Candidate nodes that match the type information of the calibration entity are selected from the initial node set to generate a candidate node set; Based on the semantic coding model, the question semantic vector of the standardized question text and the node semantic vector corresponding to the attribute description text of each candidate node in the candidate node set are extracted. Calculate the first cosine similarity between the semantic vector of the question and the semantic vector of each node; The candidate node with the highest first cosine similarity is determined as the target entity node corresponding to the calibration entity.
5. The method according to claim 3, characterized in that, The step of constructing a local knowledge graph corresponding to the original question text based on each of the target entity nodes and a preset knowledge graph includes: The standardized problem texts are classified by complexity to determine the reasoning complexity level; Based on the reasoning complexity level, determine the number of retrieval jumps for each target entity node in the preset knowledge graph; Starting from each of the target entity nodes, knowledge triples with a relationship edge number no greater than the retrieval hop count are retrieved from the preset knowledge graph to obtain an initial set of triples; wherein each of the knowledge triples includes the target entity node, another entity node associated with the target entity node, and a relationship edge connecting the target entity node and the other entity node; The initial set of triples is subjected to relevance filtering to generate the target set of triples; Based on the target triple set, a local knowledge graph corresponding to the original question text is constructed.
6. The method according to claim 5, characterized in that, The step of performing relevance filtering on the initial set of triples to generate the target set of triples includes: Based on the preset knowledge graph, the relation confidence level corresponding to each knowledge triple in the initial triple set is determined. Knowledge triples with relation confidence levels less than a preset confidence threshold are removed from the initial triple set to generate a filtered triple set. The semantic vectors of the standardized question text and the group semantic vectors of each knowledge triple in the filter triple set are extracted using a semantic encoding model. Calculate the second cosine similarity between the semantic vector of the question and each of the groups of semantic vectors; Knowledge triplets with a second cosine similarity less than a preset similarity threshold are removed from the filtered triplet set to generate the target triplet set.
7. The method according to claim 1, characterized in that, The step of generating an initial thought chain corresponding to the original question text by calling a large language model based on the local knowledge graph includes: Each knowledge triple in the local knowledge graph is converted into a constraint prompt word; Based on the constraint prompts and the original question text, the large language model is invoked to output an initial thought chain consisting of multiple reasoning steps according to a preset structured format.
8. The method according to claim 1, characterized in that, The step of performing path compression processing on the initial thought chain based on the local knowledge graph to obtain a compressed thought chain includes: When the starting entity node and ending entity node corresponding to at least two consecutive reasoning steps in the initial thought chain are the same and the relational semantics are consistent, the at least two consecutive reasoning steps are merged, and a first compressed thought chain is generated based on the merged reasoning steps. Redundant entity nodes corresponding to each reasoning step in the first compressed thinking chain are removed, and a second compressed thinking chain is generated based on the removed first compressed thinking chain. When there are at least two reasoning paths between any two entity nodes in the local knowledge graph, the path with the fewest relation edges is determined as the target reasoning path, and the reasoning steps in the second compressed thinking chain are replaced with the reasoning steps corresponding to the target reasoning path to obtain the compressed thinking chain.
9. The method according to claim 1, characterized in that, The process of verifying the compressed thought chain and pruning it based on the verification results to obtain the target thought chain includes: Determine whether the knowledge triples corresponding to each reasoning step in the compressed thinking chain exist in the local knowledge graph. If not, remove the reasoning steps corresponding to knowledge triples that do not exist in the local knowledge graph from the compressed thinking chain, and generate the first verification thinking chain; Determine whether there is a logical contradiction or a break in the reasoning chain between adjacent reasoning steps in the first verification thought chain; If so, remove the reasoning steps that contain logical contradictions or have broken reasoning chains from the first verification thinking chain, and generate the second verification thinking chain. Determine whether there are any gaps in the connection between adjacent reasoning steps in the second verification thought chain; If so, remove the reasoning step located at the connection gap from the second verification thinking chain, generate a third verification thinking chain, and complete the reasoning step at the connection gap based on the local knowledge graph. Based on the completed reasoning step and the third verification thinking chain, generate the target thinking chain.
10. A large language model question-answering optimization device, characterized in that, The device includes: The question text acquisition module is used to obtain the original question text; The local knowledge graph construction module is used to construct a corresponding local knowledge graph based on the preset knowledge graph and the original question text; The initial thought chain generation module is used to generate an initial thought chain corresponding to the original question text by calling the large language model based on the local knowledge graph. The path compression processing module is used to perform path compression processing on the initial thought chain based on the local knowledge graph to obtain a compressed thought chain. The pruning verification processing module is used to verify the compressed thinking chain and prune the compressed thinking chain according to the verification result to obtain the target thinking chain. The text answer generation module is used to determine the answer information corresponding to the original question text by calling the large language model based on the target thought chain.
