Intelligent agent multi-round reasoning consistency maintenance method and device based on large language model

Abstract

The invention relates to an agent multi-round reasoning consistency maintenance method based on a large language model, and the method comprises the steps: receiving the initial input of a user, analyzing the information, such as intentions and entities, and constructing a structured knowledge graph in combination with domain knowledge; during multi-round reasoning of the intelligent agent, current subtask key information in a graph is extracted in each round and fused into large model input to generate an intermediate result; semantic analysis is carried out on the intermediate result, the intermediate result is converted into a structured entry, consistency verification is carried out according to a preset rule set and a graph, and correction is carried out when conflicts occur; the items passing verification are fused into the atlas to achieve dynamic extension; after reasoning is finished, candidate answers are generated based on the final atlas, and final answers are output after consistency verification. According to the method, the consistency and accuracy of the multi-round reasoning process of the intelligent agent can be effectively guaranteed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and apparatus for maintaining consistency in multi-round reasoning of intelligent agents based on a large language model. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent agent models based on large language models have become important tools for handling complex tasks. In practical applications such as intelligent customer service, personalized assistants, and complex question-and-answer systems, intelligent agents can autonomously complete user requests through multi-turn dialogue and reasoning, such as tasks like travel planning, technical support, and data queries. This model typically employs mechanisms such as chain thinking or planned execution, progressively breaking down the problem and invoking external tools to generate the final answer. Its efficiency and flexibility have led to its widespread application across numerous industries, making it one of the key technologies for improving automation levels.

[0003] However, in multi-round reasoning, agents often face the severe challenge of maintaining context consistency. Due to the iterative progression of reasoning steps, agents are prone to compressing or rewriting previous information each time they generate intermediate results, leading to a deviation between the final answer and the user's initial intent. For example, during task execution, key constraints such as budget limitations or user preferences may be gradually forgotten or misinterpreted, causing the output to deviate from the original requirements. Furthermore, most existing systems rely on plain text-based context recordings, lacking structured representations. This makes it difficult for models to accurately remember and reference early information in long dialogues, and the lack of automatic verification mechanisms for intermediate results means that errors, once generated, can accumulate and affect overall reliability. These problems are particularly prominent in high-risk domains requiring strict adherence to rules, such as medical or financial scenarios.

[0004] Faced with the aforementioned problems, ensuring the consistency of contextual semantics throughout multi-turn reasoning has become a core challenge that urgently needs to be addressed by engineers in the field. While existing technologies have made progress in functional implementation, they have not yet provided a universal and efficient consistency maintenance scheme, limiting the practicality of agents in demanding environments and user trust. Therefore, developing a method capable of dynamically tracking, verifying, and updating contextual information to improve the accuracy and controllability of agent output is of significant practical importance. Summary of the Invention

[0005] To address the existing technical problems, this application provides a method and apparatus for agent-based multi-turn reasoning consistency based on a large-scale language model.

[0006] In a first aspect, embodiments of this application provide an agent-based multi-turn reasoning consistency method based on a large-scale language model, including:

[0007] Receive initial user input, parse the user intent, entity information, background data and constraints in the initial user input, and construct a structured knowledge graph by combining the domain knowledge corresponding to the initial user input;

[0008] When the agent performs multi-round reasoning, it extracts key information related to the current subtask from the structured knowledge graph before each round of reasoning, and integrates the key information into the input keywords of the large language model to guide the large language model to generate intermediate results.

[0009] Semantic parsing is performed on the intermediate results to extract one or more of the newly added facts, decisions, and attribute information, and then they are converted into structured entries.

[0010] Based on a pre-defined set of semantic consistency rules, the consistency between structured entries and structured knowledge graphs is checked, and if there is a conflict, a correction strategy is executed.

[0011] Structured entries that pass consistency verification are integrated into the structured knowledge graph, dynamically expanding the nodes, edges, and relationships of the structured knowledge graph;

[0012] After the agent completes multiple rounds of reasoning, it generates candidate answers based on the final structured knowledge graph. After the graph consistency verification is passed, the final answer is output.

[0013] Optionally, the domain knowledge includes a pre-defined domain ontology, which defines the node types, edge types, and semantic association rules of the structured knowledge graph. The node types include intent nodes, entity nodes, constraint nodes, and background information nodes.

[0014] Optionally, the pre-defined semantic consistency rule set includes one or more of the following: entity attribute consistency rules, constraint scope compliance rules, target association rules, and fact conflict exclusion rules.

[0015] Optionally, the correction strategy includes two implementation methods: the first is to feed back conflict information to a large language model and indicate corrections; the second is to adjust the weighted structured entries through a rule engine to meet the requirements of a pre-defined semantic consistency rule set.

[0016] Optionally, the method is applicable to multi-agent collaborative scenarios. The structured knowledge graph serves as the global shared context. Each sub-agent reads relevant information in the global shared context based on its own responsibilities and submits the structured entries it generates to the consistency verification module. After the verification is passed, the entries are updated to the global shared context.

[0017] Optionally, each sub-agent can only write to a pre-defined exclusive field in the global shared context, and only has read access to other fields in the global shared context.

[0018] Optionally, the graph consistency verification includes one or more of the following: constraint satisfaction verification, intent coverage verification, factual accuracy verification, and integrity verification.

[0019] Optionally, the methods for extracting key information include graph summary extraction and retrieval-based prompt extraction. Graph summary extraction involves semantically compressing nodes and edges related to the current subtask to generate structured summary text. Retrieval-based prompt extraction is based on the semantic features of the current subtask, retrieving the node and edge information with the highest matching degree from the structured knowledge graph, and incorporating prompt words in the form of key-value pairs.

[0020] Optionally, the dynamic expansion of the structured knowledge graph includes node addition, edge addition, and node attribute update. Node addition is used to add new entities or decisions in intermediate results, edge addition is used to establish the association between new nodes and existing nodes, and node attribute update is used to modify the attribute values ​​of existing nodes.

[0021] Optionally, the method also includes an external tool call result integration step: after the agent calls an external tool to obtain data, it performs semantic parsing on the results returned by the tool, extracts valid information and converts it into structured entries, and after passing the consistency check, integrates it into the structured knowledge graph.

[0022] Optionally, the method can be applied to intelligent planning systems, intelligent customer service systems, data query systems, or legal consulting systems. Intelligent planning systems include tourism planning systems and project planning systems.

[0023] Optionally, the construction of a structured knowledge graph also includes an association step of introducing external knowledge graphs. By using entity alignment technology, relevant nodes and edges in the external knowledge graph are associated with the structured knowledge graph. The external knowledge graph includes domain-specific knowledge graphs and general knowledge graphs.

[0024] Optionally, constraint satisfaction verification is used to check whether the candidate answer complies with the conditions defined by the constraint node; intent coverage verification is used to check whether the candidate answer covers all the objectives of the intent node; factual accuracy verification is used to check whether the factual information in the candidate answer is consistent with the entity nodes and background information nodes of the structured knowledge graph; and completeness verification is used to check whether the candidate answer has omitted the key node information of the structured knowledge graph.

[0025] Secondly, embodiments of this application provide an apparatus for agent-based multi-turn reasoning consistency based on a large language model, comprising:

[0026] The context initialization module is configured to receive initial user input, parse the user intent, entity information, background data and constraints in the initial user input, and construct a structured knowledge graph by combining the domain knowledge corresponding to the initial user input.

[0027] The reasoning guidance module is configured to extract key information related to the current subtask from the structured knowledge graph before each round of reasoning when the agent performs multi-round reasoning, and integrate the key information into the input keywords of the large language model to guide the large language model to generate intermediate results.

[0028] The results parsing module is configured to perform semantic parsing on intermediate results, extract one or more of the new facts, decisions, and attribute information, and convert them into structured entries.

[0029] The consistency verification and correction module is configured to perform consistency verification on structured entries and structured knowledge graphs based on a preset semantic consistency rule set, and execute correction strategies if conflicts exist.

[0030] The graph update module is configured to integrate structured entries that have passed consistency checks into the structured knowledge graph, and dynamically expand the nodes, edges and relationships of the structured knowledge graph.

[0031] The answer generation and verification module is configured to generate candidate answers based on the final structured knowledge graph after the agent has performed multiple rounds of reasoning, and output the final answer after the graph consistency verification is passed.

[0032] This method models the user's initial needs by constructing a structured knowledge graph and continuously utilizes this graph for key information prompts, intermediate result parsing, and consistency verification during the agent's multi-turn reasoning process. It dynamically updates the graph content at each reasoning step, ensuring that new information is semantically consistent with existing knowledge, thus constraining the entire reasoning process within the scope of the user's initial intent and established rules. This method significantly improves the accuracy and reliability of the agent's output, effectively avoiding common problems in multi-turn dialogues such as deviating from the topic, forgetting constraints, or self-contradiction, and enhancing the controllability of complex task processing. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application or the background art will be described below.

[0034] Figure 1 The flowchart illustrates a method for achieving consensus in multi-turn reasoning of an intelligent agent based on a large language model, as provided in an embodiment of this application. Detailed Implementation

[0035] The embodiments of the present invention describe the provided methods, apparatus, and electronic devices through flowcharts and / or block diagrams.

[0036] It should be understood that each block of a flowchart and / or block diagram, as well as combinations of blocks in a flowchart and / or block diagram, can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine that, when executed by a computer or other programmable data processing apparatus, creates means for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0037] These computer-readable program instructions may also be stored in a computer-readable storage medium that enables a computer or other programmable data processing device to function in a particular manner. In this way, the instructions stored in the computer-readable storage medium produce an instruction apparatus product that includes the functions / operations specified in the blocks of a flowchart and / or block diagram.

[0038] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer-implemented process, such that the instructions that execute on the computer or other programmable data processing apparatus provide a process for implementing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0039] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0040] Figure 1 This document presents a flowchart of a multi-turn inference consistency method for intelligent agents based on a large language model, as provided in an embodiment of this application. As shown, it includes the following:

[0041] This method begins with deep analysis of the user's initial input and the construction of a knowledge graph. Upon receiving a user query, the system first identifies and extracts the core user intent, involved entities, relevant background information, and all explicit constraints. For example, in a travel planning scenario, this means accurately capturing key elements such as the destination, budget limit, time schedule, and personal preferences. Subsequently, the system introduces a predefined domain knowledge ontology, transforming this unstructured information into a structured knowledge graph composed of nodes and edges. Nodes represent specific concepts or entities, while edges define the semantic relationships between them. The core principle of this step is to transform the fuzzy natural language requirements into a "task blueprint" with a clear semantic structure that the machine can precisely understand and process. Its beneficial effect is that it establishes a unique source of facts and framework for the entire reasoning process, avoiding ambiguity from the outset and providing a structured foundation for subsequent consistency checks.

[0042] Before each step of a multi-round reasoning process, the system proactively retrieves the most relevant information for the current subtask from the constructed knowledge graph. This information typically includes constraints to be followed, established facts, and relevant entity attributes. The retrieval process can be based on the graph's topological structure or semantic similarity. This key information is then carefully organized and embedded into prompts submitted to the large language model, serving as contextual guidance for its reasoning. The principle behind this step is to leverage the structured retrieval capabilities of the knowledge graph to provide the large language model with precise and highly relevant "memory cues" at each step of its thinking, effectively preventing the model from forgetting early key information or deviating from the topic during long-chain reasoning. Its beneficial effect is a significant improvement in the accuracy and relevance of intermediate results in each round, reducing bias caused by contextual forgetting or distraction.

[0043] After a large language model generates intermediate results, the system performs semantic parsing to extract new knowledge. The parsing process aims to extract newly generated factual assertions, decision conclusions, or attribute descriptions from the natural language text output by the model. This information is then transformed into structured entries conforming to a predetermined pattern, such as a new "activity-cost" entity or an "entity-attribute-value" triple. The principle behind this step is to transform the freely generated, unstructured text output of the model into standardized data units that can be precisely understood and processed by the computational system. This transformation enables the machine to perform precise operations and logical judgments on the inference results. Its beneficial effect lies in making automated knowledge updates and consistency checks possible, transforming ambiguous semantic judgments into explicit data operations.

[0044] The system pre-defines a set of semantic consistency rules to compare and verify the new structured entries obtained in the previous step with the current state of the knowledge graph. These rules can cover various types, including logical consistency and domain constraints. If a new entry is detected to have a logical conflict with existing knowledge in the graph or violate constraints, the system will trigger a correction strategy. The principle behind this step is to introduce an automated "semantic firewall" into the agent's reasoning process, using predefined business logic and domain rules to audit the output of each step in real time. Its core benefit lies in its ability to instantly intercept and correct errors in the reasoning chain, preventing them from being amplified and propagated in subsequent steps, thereby greatly enhancing the robustness and reliability of complex reasoning tasks.

[0045] For new structured entries that pass the consistency check, the system formally integrates them into the knowledge graph. This process involves creating new nodes or edges in the graph, updating the attributes of existing nodes, and establishing their associations with existing elements. In this way, the knowledge graph is no longer a static initial state, but dynamically evolves and enriches as reasoning progresses. The principle behind this step is to make the knowledge graph a dynamically growing and continuously improving shared context carrier, faithfully recording every confirmed and valid step and discovery during the reasoning process. Its beneficial effect is that it ensures that the agent's understanding of the global state of the task is continuously synchronized and accumulated, providing an increasingly rich and consistent factual basis for subsequent reasoning steps and the final decision.

[0046] After all reasoning steps are completed, the system generates candidate answers based on the final knowledge graph. Since all elements in the answer originate directly from nodes and relationships in the graph, its internal consistency is already highly guaranteed. However, the system still performs a final consistency verification to ensure that the answer fully covers all the key requirements of the user's initial intent and that there are no internal contradictions. This step utilizes the knowledge graph as an objective standard to perform a comprehensive "quality check" on the final output, ensuring that it is not only correct in content but also semantically coherent with the entire reasoning history. Its most significant benefit is that the final answer delivered to the user has extremely high reliability and accuracy, completely avoiding irrelevant or contradictory answers, and greatly enhancing the user's trust in the agent in critical tasks.

[0047] In some embodiments, the domain knowledge optionally includes a pre-defined domain ontology. The pre-defined domain ontology defines the node types, edge types, and semantic association rules of the structured knowledge graph. Node types include intent nodes, entity nodes, constraint nodes, and background information nodes. The system constructs a high-quality structured knowledge graph by introducing a pre-defined domain ontology. This process first relies on deep abstraction and modeling of specific industry or task scenarios, thereby forming a standardized framework for defining node types, edge types, and semantic association rules. The pre-defined domain ontology is essentially a formalized domain knowledge model that pre-defines the basic vocabulary and relationships available in the knowledge graph. For example, in the field of tourism planning, the ontology might explicitly define "destination," "activity," and "budget" as core entity node types, and specify "located in," "belongs to," and "restricted by" as valid edge types, while also including semantic rules such as "activity costs must be subject to budget constraints." In practice, when the system receives user input, the parsing module annotates and classifies the natural language based on this ontology. It instantiates the user's intent as intent nodes, the mentioned specific objects (such as city names) as entity nodes, and various constraints as constraint nodes. Then, according to the association rules defined in the ontology, it establishes semantically clear edges between the nodes, thus systematically constructing the initial knowledge graph. The core advantage of this approach is that it provides strict semantic specifications and structural constraints for the construction of the knowledge graph, ensuring good logical consistency and machine readability from the outset. Its beneficial effects are multifaceted: it not only greatly improves the accuracy and efficiency of graph construction, avoiding semantic confusion caused by arbitrary structures, but more importantly, this standardized structure lays a solid foundation for subsequent automated reasoning and consistency verification. This allows the system to discover and correct contradictions based on explicit rules rather than fuzzy text matching, thereby significantly enhancing the reliability and interpretability of the agent throughout the multi-round reasoning task.

[0048] In some embodiments, the pre-defined semantic consistency rule set may optionally include one or more of entity attribute consistency rules, constraint scope compliance rules, target association rules, and fact conflict exclusion rules. The pre-defined semantic consistency rule set upon which the system relies plays the role of an automated logical arbiter. Its implementation involves transforming domain common sense, business logic, and user-specific constraints into executable computational rules. Specifically, entity attribute consistency rules ensure that the attribute values ​​of the same entity mentioned in different reasoning steps remain consistent, avoiding inconsistencies in descriptions. Constraint scope compliance rules verify whether newly generated data or decisions fall within preset limits, such as verifying whether an expenditure exceeds the budget limit. Target association rules continuously examine the logical connection between intermediate decisions and the end-user's intent, preventing the reasoning process from deviating from the core objective. Fact conflict exclusion rules focus on identifying and resolving direct contradictions between newly acquired information and confirmed facts in the knowledge graph. These rules collectively constitute a multi-layered verification network, automatically triggered after each round of reasoning, scanning for new content to be integrated into the knowledge graph. Once the rule engine detects a violation, it triggers a preset correction strategy, such as requiring the large language model to reconsider or automatically applying logical corrections. The beneficial effect of this mechanism is that it provides real-time, logically-based safeguards for multi-step reasoning of intelligent agents, nipping potential errors in the bud, thereby significantly improving the rigor of the entire reasoning chain and the reliability of the final output. This automated semantic monitoring is particularly important in application scenarios that require stringent accuracy and consistency.

[0049] In some embodiments, the correction strategy optionally includes two implementations: first, feeding back conflict information to a large language model and indicating corrections; second, adjusting the weighted structured entries through a rule engine to meet the requirements of a pre-defined semantic consistency rule set. The system is designed with a flexible correction strategy to address conflicts discovered during consistency checks. Its core implementation is divided into two paths based on the degree of automation of the intervention. The first path focuses on utilizing the reasoning and correction capabilities of the large language model itself. When the rule set detects a contradiction between a newly generated structured entry and the existing knowledge graph, the system constructs a feedback prompt containing a specific description of the conflict, such as explicitly pointing out that the proposed hotel price does not conform to the budget constraint, and feeds this prompt back to the model as new contextual input, requiring it to reconsider or adjust its output. This approach essentially guides the model to self-examine and correct itself, maintaining the semantic coherence of the generation process. The second approach is more deterministic. When the conflict resolution logic is clear and directly programmable, the system activates a rule engine to automatically adjust problematic structured entries. For example, if a date range exceeds a reasonable interval, the engine can directly trim it to the legal range; or when a synonym conflict is detected in an entity name, it automatically normalizes it to the standard terms defined in the graph. This rule-based direct intervention ensures processing efficiency and the determinism of the results. These two strategies can be selected or combined depending on the type and complexity of the conflict and the level of creativity required. The fundamental purpose is to build a resilient reasoning loop, enabling errors to be identified and effectively corrected in a timely manner, rather than accumulating and propagating. Its beneficial effects are twofold: firstly, the human-machine collaborative feedback mechanism preserves the model's flexibility in dealing with complex and ambiguous situations; secondly, the automatic execution of rules ensures efficient handling of simple and clear conflicts, jointly ensuring the continuous reliability of the agent's output results and the smoothness of the overall task progress in multi-round interactions.

[0050] In some embodiments, the method is optionally applicable to multi-agent collaborative scenarios. The structured knowledge graph serves as a global shared context. Each sub-agent reads relevant information from the global shared context based on its own responsibilities and submits its generated structured entries to a consistency verification module. After verification, the entries are updated to the global shared context. When applied to multi-agent collaborative scenarios, the core implementation mechanism is to establish the structured knowledge graph as the global shared context for the entire system. This shared context acts as the central coordinator and the sole source of facts for all sub-agent interactions. Each sub-agent, for example, in a data analysis task, might be responsible for defining the time range, setting filtering conditions, or selecting grouping dimensions, reads information fragments related to its own task from the global shared context based on pre-assigned responsibilities and permissions. Subsequently, during its independent reasoning process, each sub-agent transforms its intermediate results, such as a proposed time interval or a set of filtering conditions, into structured entries and submits them to a central consistency verification module. This module rigorously examines the compatibility of these new entries with the current state of the knowledge graph according to pre-defined semantic rules. Only entries that pass the verification are allowed to be updated to the global context. This design makes the knowledge graph a dynamic and authoritative knowledge base, with a controlled and orderly update process. The beneficial effect of this mechanism is that, at the architectural level, it ensures that in a distributed, multi-expert collaborative intelligent agent system, all decisions are based on a synchronous and consistent cognitive foundation. This effectively avoids decision conflicts and logical contradictions caused by information asymmetry or misunderstandings among agents, thereby significantly improving the coordination of complex task execution, the reliability of overall results, and the predictability of system operation.

[0051] In some embodiments, optionally, each sub-agent can only write to pre-defined exclusive fields in the global shared context, and only has read permissions for other fields in the global shared context. To ensure the integrity and consistency of the global shared context, the system sets granular data operation permissions for each sub-agent. Specifically, during system initialization, based on the pre-defined responsibilities of each sub-agent, the system clearly defines its exclusive fields or data ranges with write permissions in the knowledge graph. For example, a sub-agent specifically responsible for processing time information may be uniquely authorized to modify node attributes related to "time range"; while a sub-agent responsible for budget constraints is uniquely authorized to update node values ​​related to "cost". During operation, when any sub-agent submits its reasoning results, the system first verifies whether the field it attempts to modify is within its authorized write range. For its exclusive fields, it can add, modify, or delete; while for most other fields in the knowledge graph, the sub-agent is only granted read permissions, meaning it can obtain this information to assist its own reasoning, but cannot directly modify it. This responsibility-based access control mechanism aims to isolate potential conflicts and the risk of accidental modification at the source, ensuring that data updates in each professional domain are handled by the corresponding expert agent. This avoids logical confusion or data corruption that can occur when multiple agents modify the same data simultaneously. The beneficial effect is a significant enhancement of the orderliness and stability of multi-agent systems, enabling complex reasoning tasks requiring multi-expert collaboration to operate reliably like precision instruments, ultimately producing highly consistent and reliable results.

[0052] In some embodiments, the graph consistency verification optionally includes one or more of constraint satisfaction verification, intent coverage verification, factual accuracy verification, and integrity verification. The graph consistency verification performed during the final answer generation stage is a multi-dimensional and systematic quality assurance process. Its implementation first involves constraint satisfaction verification, which checks whether all nodes marked as constraints (such as budget limits and time ranges) in the knowledge graph are strictly adhered to in the final solution, ensuring that the output does not exceed any predetermined boundaries. Next, intent coverage verification is performed. By comparing the final graph state with the initial user intent nodes, the system assesses whether the core objective has been fully achieved and whether there are any unmet key requirements. Based on this, the system also initiates factual accuracy verification. This step may involve cross-referencing key fact nodes in the graph with external trusted knowledge sources to identify and eliminate potentially fictitious or erroneous information. Finally, integrity verification scans the entire graph structure to determine whether all key decision-making steps necessary to achieve the objective are complete and whether there are any breaks or omissions in the logical chain. This comprehensive verification mechanism acts as a rigorous quality control checkpoint for the final output of the intelligent agent. Its beneficial effect is that it can comprehensively guarantee the quality of the final answer from multiple key dimensions, ensuring not only its logical consistency and compliance with constraints, but also its ability to truly and completely respond to the user's core needs, thereby significantly improving the credibility of the results and user satisfaction.

[0053] In some embodiments, the extraction of key information may optionally include graph summarization extraction and retrieval-based prompt extraction. Graph summarization extraction generates structured summary text by semantically compressing nodes and edges related to the current subtask. Retrieval-based prompt extraction retrieves the nodes and edges with the highest matching degree from the structured knowledge graph based on the semantic features of the current subtask and incorporates them into prompt words in the form of key-value pairs. The system employs two complementary paths to extract key information from the structured knowledge graph to guide the reasoning of large-scale language models. The implementation of graph summarization extraction is similar to automatically generating a concise task briefing. The system first identifies nodes and edges that are highly semantically related to the current subtask, and then uses natural language generation technology to compress this discrete structured information into a coherent and easy-to-understand text summary that summarizes the core facts and constraints that need to be addressed in the current step. On the other hand, retrieval-based prompt extraction is more like a precise information retrieval process. Its process is based on the semantic features (such as keywords and intent classification) contained in the current subtask, searching in real time in the graph and returning the nodes with the highest matching degree and their attributes, and directly embedding them into the instructions given to the model in the form of clear key-value pairs. These two approaches each have their own emphasis: summary extraction provides a coherent macro-level perspective, while retrieval-based prompts ensure the accurate presentation of key data points. The beneficial effect of this flexible information extraction mechanism is that it can provide the most suitable background information input format for large language models based on the specific nature and complexity of the subtasks. This avoids both information overload leading to model attention distraction and inference bias caused by missing information, thus significantly improving the relevance and accuracy of each inference step and laying a reliable foundation for the smooth progress of the entire multi-turn dialogue.

[0054] In some embodiments, the dynamic expansion of the structured knowledge graph optionally includes node addition, edge addition, and node attribute updating. Node addition is used to add new entities or decisions in the intermediate results; edge addition is used to establish associations between new nodes and existing nodes; and node attribute updating is used to modify the attribute values ​​of existing nodes. During the multi-round reasoning process of the agent, the structured knowledge graph is not static but continuously evolves through a dynamic expansion mechanism, which is specifically implemented through three core operations. When the intermediate results generated by a large language model contain new entities or decisions, the system performs a node addition operation, adding these new elements as independent nodes to the graph, making them part of the shared context. Subsequently, the edge addition operation establishes explicit semantic connections between these new nodes and existing related nodes in the graph, thereby outlining a complete relationship network between information. Furthermore, for existing nodes, if new information indicates that their attributes need correction or enrichment, the system triggers a node attribute updating operation to ensure that the information state carried by the node remains consistent with the latest reasoning conclusion. This dynamic expansion process enables the knowledge graph to act like a continuously growing organism, faithfully recording and integrating every effective discovery and decision along the reasoning path. The benefits are twofold: on the one hand, it enables the cognitive state of the agent to deepen and become more precise as the task progresses, effectively avoiding information stagnation; on the other hand, this structured accumulation provides richer and more accurate background knowledge for each subsequent step of reasoning, significantly enhancing the coherence and reliability of the entire system in handling complex, multi-step tasks.

[0055] In some embodiments, the method optionally includes an external tool call result integration step: after the agent calls an external tool to obtain data, it performs semantic parsing on the tool's returned results, extracts valid information, converts it into structured entries, and integrates them into a structured knowledge graph after passing a consistency check. The method further includes an external tool call result integration step, which plays a crucial bridging role in the interaction between the agent and the external environment. When the agent calls an external tool (such as a database query interface, real-time data API, or professional computing service) according to reasoning needs, the system receives the raw data returned by the tool. This data is often unstructured or has a specific format. Subsequently, the system initiates a semantic parsing process to identify and extract valuable information for the current task from these raw results, such as a specific numerical value, the state of an object, or a key description. This extracted information is then converted into structured entries that conform to the knowledge graph pattern definition, such as a new node representing a tool output entity or a set of attribute key-value pairs. After this, these structured entries are not directly adopted but need to pass a consistency check with the internal knowledge graph to ensure that they do not logically conflict with existing user intentions, facts, and constraints. Only entries that pass the verification process are formally integrated into the knowledge graph, becoming part of the globally shared context. The beneficial effect of this integration step lies in its creative combination of the objective data acquisition capabilities of external tools with the logical reasoning framework of the internal knowledge graph. This allows agents to dynamically absorb and utilize the latest information from the external world to correct or enrich their cognitive state, greatly expanding the system's knowledge boundaries and application scope. Simultaneously, rigorous consistency verification ensures the accuracy and reliability of internal and external information fusion, thereby significantly improving the agent's adaptability and decision-making quality when solving complex real-world problems.

[0056] In some embodiments, the method can optionally be applied to intelligent planning systems, intelligent customer service systems, data query systems, or legal consulting systems. Intelligent planning systems include tourism planning systems and project planning systems. This method has broad applicability and can be deployed in various intelligent systems requiring complex reasoning and strict adherence to context. For example, in the field of intelligent planning systems, whether it's a tourism planning system for personalized travel itineraries or a project planning system for setting task nodes and resource allocations for enterprises, this method ensures that the final solution always aligns with the user's initial goals, budget constraints, and preferences by maintaining a dynamic knowledge graph. In intelligent customer service systems, it helps customer service assistants accurately remember the user's product model, historical problem descriptions, and attempted solutions during multi-turn conversations, avoiding contradictions in responses and thus improving efficiency and user experience. For data query systems, this method ensures that the filtering conditions, time ranges, and grouping dimensions of database query statements converted from natural language remain logically consistent throughout the generation process, ultimately outputting accurate and reliable statistical data. In high-risk areas such as legal consulting, the system can use this method to rigorously verify the consistency between legal advice and case facts and legal citations, significantly improving the accuracy and reliability of professional services. These application scenarios share the common requirement of high accuracy, reliability, and contextual consistency in the output results. Our method, by introducing a structured knowledge graph and its consistency maintenance mechanism, provides a stable and reliable reasoning foundation for these systems, enabling them to complete complex tasks with high quality.

[0057] In some embodiments, the construction of a structured knowledge graph may optionally include an association step involving external knowledge graphs. This involves establishing associations between relevant nodes and edges in the external knowledge graph and the structured knowledge graph using entity alignment technology. External knowledge graphs include domain-specific knowledge graphs and general knowledge graphs. To enhance the breadth and depth of the structured knowledge graph, the system also introduces an association step with external knowledge graphs. This process begins with the system identifying core entities or concepts in its initial knowledge graph that require further enrichment. Then, using entity alignment technology, it searches for corresponding nodes representing the same real-world object in selected external knowledge graphs (e.g., domain-specific knowledge graphs containing detailed geographical information or product specifications, or general knowledge graphs like encyclopedic data sources). Entity alignment technology acts like a translator fluent in multiple languages, identifying and associating different representations of the same thing from different data sources. Once successfully aligned, the system can, within permissible permissions, establish referential connections between the numerous valuable edges (i.e., relationships) and attribute information of the node in the external knowledge graph and the internal knowledge graph, rather than simply copying them. This significantly expands the information boundaries of the internal knowledge graph. The beneficial effects of this approach are obvious: it enables agents to overcome the limitations of their initial knowledge or private data, dynamically extracting structured information from a vast external knowledge base. This provides richer background knowledge and factual evidence for their reasoning, significantly improving the professionalism, accuracy, and breadth of their answers, especially in question-and-answer or consultation scenarios where broad knowledge coverage is required.

[0058] In some embodiments, optionally, constraint satisfaction verification is used to check whether the candidate answer complies with the conditions defined by the constraint nodes; intent coverage verification is used to check whether the candidate answer covers all the goals of the intent nodes; factual accuracy verification is used to check whether the factual information in the candidate answer is consistent with the entity nodes and background information nodes of the structured knowledge graph; and completeness verification is used to check whether the candidate answer has omitted any key node information of the structured knowledge graph. During the graph consistency verification stage before the final answer is generated, the system executes a series of rigorous verification processes. The first step is constraint satisfaction verification, which is like a comprehensive compliance review. Its core task is to verify that each item in the candidate answer strictly follows the conditions set by all constraint nodes in the knowledge graph, such as budget limits or time constraints, ensuring that the final solution does not exceed any explicit boundaries. Based on this, the system performs intent coverage verification, which aims to assess whether the candidate answer has fully responded to all the core demands raised by the user at the beginning of the task, that is, whether the goals represented by the intent nodes in the graph have been substantially reflected and resolved in the final answer. Simultaneously, factual accuracy verification is performed, focusing on comparing specific factual statements mentioned in the answer, such as events, data, or attributes, to ensure they fully match the information carried by entity nodes and background information nodes recorded in the knowledge graph, thus preventing factual errors. Finally, completeness verification acts as a quality inspector, systematically scanning the entire knowledge graph structure to ensure that all key node information essential for a complete and credible answer has not been unintentionally omitted from candidate responses. The beneficial effect of this combined verification mechanism is that it provides systematic assurance for the quality of the final output from multiple key dimensions, ensuring that the generated answers are not only logically consistent and meet requirements, but also truthful, reliable, detailed, and complete, thereby significantly improving the credibility of the results and user satisfaction.

[0059] This application provides a device for agent-based multi-turn reasoning consistency based on a large language model, comprising:

[0060] The context initialization module is configured to receive initial user input, parse the user intent, entity information, background data and constraints in the initial user input, and construct a structured knowledge graph by combining the domain knowledge corresponding to the initial user input.

[0061] The reasoning guidance module is configured to extract key information related to the current subtask from the structured knowledge graph before each round of reasoning when the agent performs multi-round reasoning, and integrate the key information into the input keywords of the large language model to guide the large language model to generate intermediate results.

[0062] The results parsing module is configured to perform semantic parsing on intermediate results, extract one or more of the new facts, decisions, and attribute information, and convert them into structured entries.

[0063] The consistency verification and correction module is configured to perform consistency verification on structured entries and structured knowledge graphs based on a preset semantic consistency rule set, and execute correction strategies if conflicts exist.

[0064] The graph update module is configured to integrate structured entries that have passed consistency checks into the structured knowledge graph, and dynamically expand the nodes, edges and relationships of the structured knowledge graph.

[0065] The answer generation and verification module is configured to generate candidate answers based on the final structured knowledge graph after the agent has performed multiple rounds of reasoning, and output the final answer after the graph consistency verification is passed.

[0066] The device's data execution process begins with the context initialization module, which receives and deeply analyzes the user's initial request, identifying the core intent, key entities, relevant background, and all constraints. Based on this, it invokes a pre-built domain knowledge model to construct a structured knowledge graph representing the initial task blueprint using nodes and edges. Subsequently, during multiple rounds of reasoning, the reasoning guidance module proactively intervenes. Before each round of reasoning, it precisely retrieves the most relevant information fragments from the knowledge graph to the current subtask and integrates them as key hints into the instructions sent to the large language model, thereby guiding the model to generate more targeted intermediate results. The result parsing module performs semantic analysis on the model's output text, extracting newly generated facts, decisions, or attributes, and transforming them into standardized structured data entries. These new entries are not immediately adopted but are sent to the consistency verification and correction module. This module rigorously examines whether the new entries conflict logically with the existing knowledge graph based on a predefined set of semantic rules. If a contradiction is found, a corresponding correction strategy is triggered. Only entries that pass the verification are received by the graph update module, which is responsible for systematically integrating this new knowledge into the graph, dynamically expanding and enriching the global context by adding nodes, edges, or updating attributes. After all reasoning steps are completed, the answer generation and verification module constructs candidate answers based on the finally converged knowledge graph and performs a comprehensive graph consistency verification to ensure that the answers meet requirements in terms of constraint satisfaction, intent coverage, factual accuracy, and completeness before outputting the final answer to the user. This entire process, through close collaboration among the modules, forms a closed-loop quality management system from requirement analysis to result delivery, ensuring high reliability and consistency of the agent's output.

[0067] Example

[0068] This application provides a method for consistent multi-turn reasoning of intelligent agents based on a large language model. This method can be applied to intelligent planning systems, such as tourism planning systems or project planning systems, to solve the problems of context shift and inconsistency in multi-turn dialogues. In this embodiment, the method ensures that the agent's reasoning process remains consistent with the user's initial intent by constructing and maintaining a structured knowledge graph. Specific implementation details are as follows.

[0069] The system receives initial user input, parses the user's intent, entity information, background data, and constraints, and constructs a structured knowledge graph based on domain knowledge corresponding to the initial input. For example, in a travel planning scenario, if a user inputs "I want to go to Paris for 3 days with a budget of no more than $1,000, and I like museums," the system parses the user's intent as "planning a trip to Paris," the entity information includes "Paris," "3 days," and "$1,000," the background data includes the user's historical preferences, and the constraints include the budget limit and museum preferences. The system also introduces a travel domain ontology to define node and edge types, constructing an initial knowledge graph. Nodes include intent nodes (e.g., "travel planning"), entity nodes (e.g., "Paris"), constraint nodes (e.g., "budget ≤ $1,000"), and background information nodes (e.g., "likes museums"). Edges represent relationships between nodes, such as "destination - Paris" linking "budget - constraint." This step transforms fuzzy requirements into a machine-processable blueprint through structured representation, providing an accurate foundation for subsequent reasoning.

[0070] When the agent performs multi-round reasoning, before each round, it extracts key information relevant to the current subtask from the structured knowledge graph. This key information is then integrated into the input keywords of the large language model to guide the model in generating intermediate results. For example, in the itinerary planning subtask, the system generates compressed text such as "Current task: Planning activities for day one; Constraints: Budget remaining $800, museum preference" through graph summarization, or extracts matching key-value pairs such as "{Destination: Paris, Daily budget: $300}" through retrieval-based prompts. This information is then integrated into prompt words to guide the model in generating reasonable suggestions such as "Visit the Louvre, cost $30". This step utilizes the semantic structure of the graph to provide accurate contextual cues, reducing model forgetting or deviation and improving the reliability of intermediate results.

[0071] The intermediate results undergo semantic parsing to extract one or more of the following: new facts, decisions, and attribute information, which are then converted into structured entries. For example, after the model outputs "Recommend the Louvre, tickets are $30," the parsing module extracts new facts such as "Activity: Louvre," decisions such as "Select this activity," and attributes such as "Cost: $30," and converts them into structured entries such as the triple "{Activity: Louvre, Type: Museum, Cost: $30}." This step transforms unstructured text into standard data units, facilitating machine processing and validation.

[0072] Based on a pre-defined set of semantic consistency rules, the system performs consistency checks on structured entries and the structured knowledge graph. If conflicts are found, corrective strategies are executed. For example, the pre-defined rules include entity attribute consistency rules (ensuring consistency of attributes for the same entity), constraint compliance rules (e.g., costs not exceeding the budget), goal association rules (e.g., activities must conform to user intent), and fact conflict exclusion rules (e.g., avoiding duplicate activities). If a new entry's cost exceeds the budget, the system triggers corrective strategies, such as providing conflict information to the model for correction, or automatically adjusting the cost to a reasonable range through the rule engine. This step uses automated rule verification to promptly intercept errors and prevent their accumulation and propagation.

[0073] Structured entries that pass consistency checks are integrated into the structured knowledge graph, dynamically expanding the graph's nodes, edges, and relationships. For example, a conflict-free entry like "Louvre event" is added as a new node, and edges are established between it and existing nodes such as "budget constraint" and "travel intention," while related attributes like cumulative cost are updated. This step allows the graph to evolve dynamically with the reasoning process, maintaining contextual integrity and timeliness.

[0074] After the agent performs multiple rounds of reasoning, it generates candidate answers based on the final structured knowledge graph. Once the graph passes consistency verification, the final answer is output. For example, if the final graph contains all itinerary details, the system might generate a candidate answer such as "Total cost of 3-day trip: $950, including museum activities." It then performs verifications including constraint satisfaction (checking budget compliance), intent coverage (confirming museum preferences are met), factual accuracy (verifying attraction information), and completeness (ensuring no key nodes are omitted). Once these verifications pass, the final answer is output. This multi-dimensional verification ensures answer quality and enhances user trust.

[0075] Furthermore, the method includes an external tool call result integration step: after the agent calls an external tool to obtain data, it performs semantic parsing on the tool's returned results, extracts valid information, converts it into structured entries, and integrates them into a structured knowledge graph after passing consistency verification; for example, calling a flight ticket API returns data, parsing out "flight fee: $400", verifying it, and then updating the knowledge graph. This method also supports multi-agent collaborative scenarios, where the knowledge graph serves as a globally shared context, and each sub-agent reads and writes its own fields based on its responsibilities, ensuring collaborative consistency; and it can introduce external knowledge graphs to expand information sources through entity alignment.

[0076] The beneficial effects of this method are that, through the dynamic maintenance and consistency verification of the structured knowledge graph, it significantly improves the accuracy and reliability of multi-turn reasoning of intelligent agents, reduces context offset and error propagation, is applicable to high-risk fields such as medical or financial fields, and enhances system usability and user satisfaction.

[0077] The above description is merely a specific implementation of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of the present invention should be included within the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.

Claims

1. A method for maintaining consistency in multi-turn reasoning of intelligent agents based on a large language model, characterized in that, include: Receive initial user input, parse the user intent, entity information, background data and constraints of the initial user input, and construct a structured knowledge graph by combining the domain knowledge corresponding to the initial user input; When the agent performs multi-round reasoning, before each round of reasoning, it extracts key information related to the current subtask from the structured knowledge graph and integrates the key information into the input keywords of the large language model to guide the large language model to generate intermediate results. The intermediate results are semantically parsed to extract one or more of the newly added facts, decisions, and attribute information, and then converted into structured entries. Based on a pre-set semantic consistency rule set, the consistency between the structured entries and the structured knowledge graph is checked, and if there is a conflict, a correction strategy is executed. The structured entries that pass the consistency check are integrated into the structured knowledge graph, and the nodes, edges, and relationships of the structured knowledge graph are dynamically expanded. After the agent completes multiple rounds of reasoning, it generates candidate answers based on the final structured knowledge graph. After the graph consistency verification is passed, the final answer is output.

2. The method according to claim 1, characterized in that, The domain knowledge includes a pre-defined domain ontology, which defines the node types, edge types, and semantic association rules of the structured knowledge graph. The node types include intent nodes, entity nodes, constraint nodes, and background information nodes.

3. The method according to claim 1, characterized in that, The pre-defined semantic consistency rule set includes one or more of the following: entity attribute consistency rules, constraint scope compliance rules, target association rules, and fact conflict exclusion rules.

4. The method according to claim 1, characterized in that, The correction strategy includes two implementation methods: first, feeding back conflict information to the large language model and indicating correction; second, adjusting the structured entries through a rule engine to meet the requirements of the preset semantic consistency rule set.

5. The method according to claim 1, characterized in that, The method is applicable to multi-agent collaborative scenarios. The structured knowledge graph serves as a global shared context. Each sub-agent reads relevant information from the global shared context based on its own responsibilities and submits its generated structured entries to the consistency verification module. After successful verification, the entries are updated to the global shared context.

6. The method according to claim 5, characterized in that, Each sub-agent can only write to a pre-defined exclusive field in the global shared context, and only has read permission for other fields in the global shared context.

7. The method according to claim 1, characterized in that, The graph consistency verification includes one or more of the following: constraint satisfaction verification, intent coverage verification, factual accuracy verification, and integrity verification.

8. The method according to claim 1, characterized in that, The methods for extracting the key information include graph summary extraction and retrieval-based prompt extraction. Graph summary extraction involves semantically compressing nodes and edges related to the current subtask to generate structured summary text. Retrieval-based prompt extraction is based on the semantic features of the current subtask, retrieving the node and edge information with the highest matching degree from the structured knowledge graph, and incorporating prompt words in the form of key-value pairs.

9. The method according to claim 1, characterized in that, The dynamic expansion of the structured knowledge graph includes node addition, edge addition, and node attribute update. Node addition is used to add new entities or decisions in intermediate results, edge addition is used to establish the association between new nodes and existing nodes, and node attribute update is used to modify the attribute values ​​of existing nodes.

10. The method according to claim 1, characterized in that, The method further includes an external tool call result integration step: after the agent calls an external tool to obtain data, it performs semantic parsing on the tool's returned results, extracts valid information and converts it into structured entries, and after passing the consistency check, integrates it into the structured knowledge graph.

11. The method according to claim 1, characterized in that, The method is applied to intelligent planning systems, intelligent customer service systems, data query systems, or legal consulting systems, wherein the intelligent planning systems include tourism planning systems and project planning systems.

12. The method according to claim 1, characterized in that, The construction of the structured knowledge graph also includes an association step of introducing external knowledge graphs. By using entity alignment technology, relevant nodes and edges in the external knowledge graph are associated with the structured knowledge graph. The external knowledge graph includes domain-specific knowledge graphs and general knowledge graphs.

13. The method according to claim 7, characterized in that, The constraint satisfaction verification is used to check whether the candidate answer complies with the conditions defined by the constraint node; the intent coverage verification is used to check whether the candidate answer covers all the objectives of the intent node; the factual accuracy verification is used to check whether the factual information in the candidate answer is consistent with the entity nodes and background information nodes of the structured knowledge graph; the completeness verification is used to check whether the candidate answer has omitted the key node information of the structured knowledge graph.

14. A device for maintaining consistency in multi-turn reasoning of intelligent agents based on a large language model, characterized in that, include: The context initialization module is configured to receive initial user input, parse the user intent, entity information, background data and constraints of the initial user input, and construct a structured knowledge graph by combining the domain knowledge corresponding to the initial user input. The reasoning guidance module is configured to extract key information related to the current subtask from the structured knowledge graph before each round of reasoning when the agent performs multi-round reasoning, and integrate the key information into the input keywords of the large language model to guide the large language model to generate intermediate results. The result parsing module is configured to perform semantic parsing on the intermediate results, extract one or more of the newly added facts, decisions, and attribute information, and convert them into structured entries. The consistency verification and correction module is configured to perform consistency verification between the structured entries and the structured knowledge graph based on a preset semantic consistency rule set, and to execute a correction strategy if a conflict exists. The graph update module is configured to integrate the structured entries that have passed the consistency check into the structured knowledge graph, and dynamically expand the nodes, edges and associations of the structured knowledge graph. The answer generation and verification module is configured to generate candidate answers based on the final structured knowledge graph after the agent has performed multiple rounds of reasoning, and output the final answer after the graph consistency verification is passed.

Images