Multi-agent training method and system fusing knowledge graph

Abstract

The invention discloses a multi-agent training method and system fusing a knowledge graph. The system comprises a knowledge graph management module, an agent training module and an interaction and feedback module. And the knowledge graph management module is responsible for organizing, updating and the like of the knowledge graph and covers data acquisition, cleaning, storage, construction, updating, query, reasoning and feedback collection. The agent training module comprises an environment simulator, a plurality of learning modules, a knowledge graph interaction module, a strategy generation module and a feedback mechanism, and helps an agent to learn and optimize a decision strategy in a simulation environment. The interaction and feedback module is provided with an interaction interface, a feedback calculation module, a knowledge graph association module and a performance monitoring and visualization module, and is used for realizing communication and performance evaluation between the intelligent agent and the environment; and the interaction and feedback module realizes environment interaction, feedback calculation, knowledge graph association and performance monitoring evaluation and adjustment.

Description

Technical Field

[0001] The present invention belongs to the technical field related to the field of artificial intelligence, and specifically relates to a multi-agent training method and system integrating knowledge graphs. Background Art

[0002] Traditional multi-agent training systems have deficiencies in knowledge integration and utilization, and it is difficult to efficiently process multi-source and complex data, resulting in limited understanding of the environment by the agents and insufficient basis for decision-making. At the same time, the interaction and collaboration between agents are not optimized, and there is a lack of effective feedback mechanisms to continuously improve performance. During the training process, the ability to adapt to environmental changes is weak, and strategies cannot be adjusted in time according to real-time conditions. Moreover, the lack of intuitive performance monitoring and evaluation methods makes it difficult to accurately analyze training effects and optimization directions. These limitations make traditional multi-agent training systems perform poorly when dealing with complex and changing tasks and environments, and more advanced technologies are needed to improve and enhance them. Summary of the invention

[0003] The purpose of the present invention is to provide a multi-agent training method and system that integrates knowledge graphs, so as to solve the problems that the traditional multi-agent training system proposed in the above background technology has deficiencies in knowledge integration and utilization, and it is difficult to efficiently process multi-source and complex data, resulting in limited understanding of the environment by the intelligent agent and insufficient decision-making basis.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A multi-agent training system integrating knowledge graphs,

[0006] The system includes a knowledge graph management module: responsible for organizing and updating the knowledge graph, including data collection, cleaning and storage;

[0007] Agent training module: Based on the information provided by the knowledge graph, the agent can better understand the environment, make decisions and interact;

[0008] Interaction and feedback module: realizes information exchange and collaboration between intelligent agents, and improves overall performance through continuous feedback loops.

[0009] Preferably, the knowledge graph management module mainly includes:

[0010] Data acquisition module: responsible for the interface with external data sources to obtain raw data, including API calls, web crawling, and sensor input;

[0011] Knowledge processing module: realizes data cleaning, extraction and conversion, converting raw data into the standard format of knowledge graph.

[0012] Graph construction module: Use graph database technology to store processed data in a graph structure and establish effective indexes to cover various query requirements;

[0013] Update management module: monitors changes in external data sources and automatically triggers updates to the knowledge graph;

[0014] Query and reasoning module: Provides a RESTful API or SPARQL interface, allowing the agent to perform complex query and reasoning tasks.

[0015] Preferably, the agent training module mainly includes:

[0016] Environment simulator: simulates the real environment, allowing the agent to be trained and tested, providing feedback. This can be a simple rule-based environment or a complex physics engine.

[0017] Learning Engine:

[0018] Reinforcement learning module: implements Q learning, deep Q network (DQN), policy gradient method, etc., to help the intelligent agent learn the optimal strategy through interaction with the environment;

[0019] Supervised learning module: used to train the agent’s decision model when labeled data is available, especially when classification or regression tasks are required;

[0020] Knowledge graph interaction module: allows the agent to access the knowledge graph to obtain background knowledge and supports the agent to use contextual information when making decisions;

[0021] Strategy generation module: Generates new decision-making strategies and optimizes existing strategies based on the experience and knowledge acquired by the agent;

[0022] Feedback mechanism: A mechanism that provides feedback by analyzing the performance of the intelligent agent in the environment, including evaluation of different indicators such as success rate and error rate.

[0023] Preferably, the interaction and feedback module mainly includes:

[0024] Interaction interface: responsible for communication with the environment, supporting different types of input (such as sensor data, environmental status) and output (such as instructions to perform actions);

[0025] Feedback calculation module:

[0026] Reward mechanism: Calculate rewards and penalties based on the results of the agent's actions to guide the agent's learning process;

[0027] Subsequent state evaluation: Evaluate the new state of the agent after affecting the environment and confirm the effectiveness of its decision;

[0028] Knowledge graph association module: associates the agent’s behavior and experience with the knowledge in the knowledge graph to provide enhanced decision-making basis;

[0029] Performance monitoring and visualization module: monitors the learning and decision-making process of the intelligent agent in real time, records key performance indicators (such as success rate, learning progress, etc.) and visualizes the data for analysis.

[0030] Preferably, a training method for a multi-agent training system integrating a knowledge graph, the specific steps of the knowledge graph management module are as follows:

[0031] Data collection:

[0032] Collect raw data from different data sources, pre-process and clean it;

[0033] Knowledge Modeling:

[0034] Analyze the collected data to identify key entities and their attributes and relationships;

[0035] Build a preliminary knowledge graph model and verify it;

[0036] Graph construction:

[0037] Store the confirmed entities and their relationships in a graph database to form a knowledge graph;

[0038] Create indexes for nodes and edges in the graph to optimize query efficiency;

[0039] Knowledge update:

[0040] Monitor changes in data sources in real time and use incremental update strategies to update knowledge graphs to ensure the timeliness of information;

[0041] Record each update through version control to facilitate management of change history;

[0042] Query and inference runs:

[0043] Accept query requests from the agent and call the query and reasoning module for processing;

[0044] Return query results for agent decision-making and learning;

[0045] Feedback Mechanism:

[0046] Collect feedback from intelligent agents when using knowledge graphs to improve data collection and knowledge updating strategies.

[0047] Preferably, the specific steps of the agent training module are as follows:

[0048] initialization:

[0049] Configure the agent, including setting learning parameters (learning rate, exploration rate, etc.) and environment parameters (state space, action space, etc.);

[0050] Initialize the agent’s strategy and value function;

[0051] Environment Interaction:

[0052] The agent selects actions in the environment and executes them according to the current policy;

[0053] The environment returns a new state and corresponding reward based on the agent's behavior;

[0054] Knowledge Acquisition:

[0055] After each interaction, the agent calls the knowledge graph interaction module to query relevant information to assist in decision making. This may involve using knowledge graph reasoning to increase understanding of the current state;

[0056] Learning Update:

[0057] Based on the environmental feedback (rewards and new states), the agent updates its learning model, which can be adjusted by reward signals in reinforcement learning or error feedback in supervised learning;

[0058] Strategy Optimization:

[0059] Continuously iterate to adjust strategies and improve decisions based on new knowledge and experience;

[0060] Use the strategy generation module to generate new strategies and compare them with previous strategies;

[0061] Evaluation and Adjustment:

[0062] Regularly evaluate the learning effect of the agent and test its performance by setting up verification environments and evaluation indicators;

[0063] Adjust parameters such as learning rate and exploration strategy to improve training results.

[0064] The specific steps of the interaction and feedback module are as follows:

[0065] Environment Interaction:

[0066] The agent sends action instructions to the environment through the interactive interface, and the environment changes its state according to the instruction and returns new state information;

[0067] In some settings, the environment can also actively provide feedback data (e.g., state changes, event notifications);

[0068] Feedback calculation:

[0069] Calculate rewards or penalties based on the actions taken by the agent and evaluate its impact on the environment;

[0070] At each time step, the feedback calculation module evaluates the difference between the new state of the agent and the previous state, and gives a corresponding score based on previous knowledge;

[0071] Knowledge graph association:

[0072] Every time the agent performs an action, the interaction and feedback module will automatically consult the knowledge graph to identify knowledge information related to the current state;

[0073] In this way, the agent can obtain new knowledge or important hints after interacting with the environment, enhancing the quality of its subsequent decisions;

[0074] Performance monitoring and evaluation:

[0075] Continuously track and record the agent's performance and analyze the success or failure of each decision;

[0076] Generate performance reports and visualization charts to help developers and researchers evaluate the training progress and agent optimization effects;

[0077] Adjustment mechanism: Based on the results of performance monitoring, the agent’s behavior strategy or learning parameters can be adjusted to improve its performance in the environment. For example, if the agent’s success rate is lower than expected, the exploration behavior can be enhanced or the reward strategy can be adjusted.

[0078] Compared with the prior art, the present invention provides a multi-agent training method and system integrating knowledge graphs, which has the following beneficial effects:

[0079] Knowledge integration and optimization: The knowledge graph management module can effectively integrate multi-source data, provide accurate and timely updated knowledge, and optimize the decision-making basis of intelligent agents;

[0080] Enhanced agent capabilities: Through multiple learning methods and knowledge graph interactions in the training module, the agent's decision-making ability, environmental adaptability, and strategy optimization capabilities are significantly enhanced;

[0081] Efficient interaction and collaboration: The interaction and feedback module promotes efficient information exchange and collaboration between agents and between agents and the environment, improving overall performance;

[0082] Accurate decision support: Intelligent agents can use the rich information in the knowledge graph to make more accurate inferences and decisions, improving the quality of decisions;

[0083] Dynamic adaptability: The system can adjust in real time according to environmental changes and agent performance, and has good dynamic adaptability;

[0084] Visualization and evaluability: The performance monitoring and visualization module facilitates intuitive evaluation and analysis of the training process and results, helping to quickly identify problems and optimize strategies;

[0085] Innovation and scalability: It provides new ideas and methods for the research and application of multi-agent systems, and has broad expansion space and application prospects;

[0086] Improved efficiency and effectiveness: The overall efficiency and effectiveness of multi-agent training have been improved, and it performs better in complex tasks and environments. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0088] The present invention provides a technical solution:

[0089] A multi-agent training system integrating knowledge graphs,

[0090] The system includes a knowledge graph management module: responsible for organizing and updating the knowledge graph, including data collection, cleaning and storage;

[0091] Agent training module: Based on the information provided by the knowledge graph, the agent can better understand the environment, make decisions and interact;

[0092] Interaction and feedback module: realizes information exchange and collaboration between intelligent agents, and improves overall performance through continuous feedback loops.

[0093] The knowledge graph management module mainly includes:

[0094] Data acquisition module: responsible for the interface with external data sources to obtain raw data, including API calls, web crawling, and sensor input;

[0095] Knowledge processing module: realizes data cleaning, extraction and conversion, converting raw data into the standard format of knowledge graph.

[0096] Graph construction module: Use graph database technology to store processed data in a graph structure and establish effective indexes to cover various query requirements;

[0097] Update management module: monitors changes in external data sources and automatically triggers updates to the knowledge graph;

[0098] Query and reasoning module: Provides a RESTful API or SPARQL interface, allowing the agent to perform complex query and reasoning tasks.

[0099] The main functions of the knowledge graph management module include:

[0100] Data integration: Collect and integrate data from different sources (such as databases, websites, sensors, etc.) to build basic information for knowledge graphs;

[0101] Knowledge modeling: Establish connections between entities, attributes, and relationships, and transform the collected information into structures that can be expressed in the knowledge graph;

[0102] Knowledge update: Real-time update of knowledge graph to ensure the timeliness and accuracy of its content. Support dynamic modification, addition or deletion of entities and their relationships in the graph;

[0103] Query and reasoning: Provides a query interface to support intelligent agents to query and reason about knowledge graphs and obtain relevant information.

[0104] Version control: Manage the versions of the knowledge graph to track change history and support rollback.

[0105] The knowledge graph management module plays a fundamental role in providing rich knowledge and information in the multi-agent training system. Through efficient data processing, accurate knowledge modeling and dynamic knowledge updating, the agents can better understand their environment and enhance their learning and collaboration capabilities. This modular design makes the entire system highly flexible and scalable, and can adapt to the needs of different fields.

[0106] The agent training module mainly includes:

[0107] Environment simulator: simulates the real environment, allowing the agent to be trained and tested, providing feedback. This can be a simple rule-based environment or a complex physics engine.

[0108] Learning Engine:

[0109] Reinforcement learning module: implements Q learning, deep Q network (DQN), policy gradient method, etc., to help the intelligent agent learn the optimal strategy through interaction with the environment;

[0110] Supervised learning module: used to train the agent’s decision model when labeled data is available, especially when classification or regression tasks are required;

[0111] Knowledge graph interaction module: allows the agent to access the knowledge graph to obtain background knowledge and supports the agent to use contextual information when making decisions;

[0112] Strategy generation module: Generates new decision-making strategies and optimizes existing strategies based on the experience and knowledge acquired by the agent;

[0113] Feedback mechanism: A mechanism that provides feedback by analyzing the performance of the intelligent agent in the environment, including evaluation of different indicators such as success rate and error rate.

[0114] The main functions of the agent training module include:

[0115] Agent configuration and initialization: Setting and initializing the agent’s basic properties, environment configuration, and learning tasks.

[0116] Learning and adaptation mechanism: Implement different learning algorithms (such as reinforcement learning, deep learning, transfer learning, etc.) to enable the agent to adapt to environmental changes.

[0117] Knowledge acquisition and application: Extract relevant information from the knowledge graph to help the intelligent agent use useful knowledge in the decision-making process.

[0118] Behavior selection and execution: Select actions based on the learned policy and execute operations to interact with the environment.

[0119] Feedback and evaluation mechanism: collect the behavior results of the agent, calculate rewards or penalties, evaluate the effectiveness of decisions, and optimize strategies.

[0120] In the multi-agent training system, the agent training module allows agents to optimize their behavior through continuous learning and adaptation, while combining the rich background knowledge provided by the knowledge graph to improve decision-making capabilities. Through effective environmental interaction and feedback mechanisms, this module can ensure that agents can respond flexibly in dynamic environments and gradually evolve to more advanced intelligent performance.

[0121] The interaction and feedback module mainly includes:

[0122] Interaction interface: responsible for communication with the environment, supporting different types of input (such as sensor data, environmental status) and output (such as instructions to perform actions);

[0123] Feedback calculation module:

[0124] Reward mechanism: Calculate rewards and penalties based on the results of the agent's actions to guide the agent's learning process;

[0125] Subsequent state evaluation: Evaluate the new state of the agent after affecting the environment and confirm the effectiveness of its decision;

[0126] Knowledge graph association module: associates the agent’s behavior and experience with the knowledge in the knowledge graph to provide enhanced decision-making basis;

[0127] Performance monitoring and visualization module: monitors the learning and decision-making process of the intelligent agent in real time, records key performance indicators (such as success rate, learning progress, etc.) and visualizes the data for analysis.

[0128] The main functions of the interaction and feedback module include:

[0129] Interaction between the agent and the environment: Processing the signals and interactive responses between the agent and the environment to ensure that the agent can obtain real-time environmental information.

[0130] Response feedback mechanism: collects the behavior results of the intelligent agent, calculates and feeds back reward or penalty signals to support the optimization of the learning algorithm.

[0131] Knowledge graph feedback: Combine the information obtained by the agent in the environment with the data in the knowledge graph to provide context for the agent's decision-making process.

[0132] Performance monitoring and evaluation: Monitor the performance of the agent in real time, evaluate its decision-making effects, and make necessary adjustments based on the evaluation results.

[0133] The interaction and feedback module plays a vital role in the multi-agent training system. By managing the effective interaction between the agent and the environment and providing timely feedback information, the module ensures that the agent can quickly adapt to environmental changes, optimize decision-making strategies and improve overall performance. By combining with the knowledge graph, this module not only enhances the learning ability of the agent, but also improves its flexibility and intelligence in complex environments.

[0134] A training method for a multi-agent training system integrating a knowledge graph, the specific steps of the knowledge graph management module are as follows:

[0135] Data collection:

[0136] Collect raw data from different data sources, pre-process and clean it;

[0137] Knowledge Modeling:

[0138] Analyze the collected data to identify key entities and their attributes and relationships;

[0139] Build a preliminary knowledge graph model and verify it;

[0140] Graph construction:

[0141] Store the confirmed entities and their relationships in a graph database to form a knowledge graph;

[0142] Create indexes for nodes and edges in the graph to optimize query efficiency;

[0143] Knowledge update:

[0144] Monitor changes in data sources in real time and use incremental update strategies to update knowledge graphs to ensure the timeliness of information;

[0145] Record each update through version control to facilitate management of change history;

[0146] Query and inference runs:

[0147] Accept query requests from the agent and call the query and reasoning module for processing;

[0148] Return query results for agent decision-making and learning;

[0149] Feedback Mechanism:

[0150] Collect feedback from intelligent agents when using knowledge graphs to improve data collection and knowledge updating strategies.

[0151] The specific steps of the agent training module are as follows:

[0152] initialization:

[0153] Configure the agent, including setting learning parameters (learning rate, exploration rate, etc.) and environment parameters (state space, action space, etc.);

[0154] Initialize the agent’s strategy and value function;

[0155] Environment Interaction:

[0156] The agent selects actions in the environment and executes them according to the current policy;

[0157] The environment returns a new state and corresponding reward based on the agent's behavior;

[0158] Knowledge Acquisition:

[0159] After each interaction, the agent calls the knowledge graph interaction module to query relevant information to assist in decision making. This may involve using knowledge graph reasoning to increase understanding of the current state;

[0160] Learning Update:

[0161] Based on the environmental feedback (rewards and new states), the agent updates its learning model, which can be adjusted by reward signals in reinforcement learning or error feedback in supervised learning;

[0162] Strategy Optimization:

[0163] Continuously iterate to adjust strategies and improve decisions based on new knowledge and experience;

[0164] Use the strategy generation module to generate new strategies and compare them with previous strategies;

[0165] Evaluation and Adjustment:

[0166] Regularly evaluate the learning effect of the agent and test its performance by setting up verification environments and evaluation indicators;

[0167] Adjust parameters such as learning rate and exploration strategy to improve training results.

[0168] The specific steps of the interaction and feedback module are as follows:

[0169] Environment Interaction:

[0170] The agent sends action instructions to the environment through the interactive interface, and the environment changes its state according to the instruction and returns new state information;

[0171] In some settings, the environment can also actively provide feedback data (e.g., state changes, event notifications);

[0172] Feedback calculation:

[0173] Calculate rewards or penalties based on the actions taken by the agent and evaluate its impact on the environment;

[0174] At each time step, the feedback calculation module evaluates the difference between the new state of the agent and the previous state, and gives a corresponding score based on previous knowledge;

[0175] Knowledge graph association:

[0176] Every time the agent performs an action, the interaction and feedback module will automatically consult the knowledge graph to identify knowledge information related to the current state;

[0177] In this way, the agent can obtain new knowledge or important hints after interacting with the environment, enhancing the quality of its subsequent decisions;

[0178] Performance monitoring and evaluation:

[0179] Continuously track and record the agent's performance and analyze the success or failure of each decision;

[0180] Generate performance reports and visualization charts to help developers and researchers evaluate the training progress and agent optimization effects;

[0181] Adjustment mechanism: Based on the results of performance monitoring, the agent’s behavior strategy or learning parameters can be adjusted to improve its performance in the environment. For example, if the agent’s success rate is lower than expected, the exploration behavior can be enhanced or the reward strategy can be adjusted.

[0182] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-agent training system integrating knowledge graph, characterized by: The system includes a knowledge graph management module: responsible for organizing and updating the knowledge graph, including data collection, cleaning and storage; Agent training module: Based on the information provided by the knowledge graph, the agent can better understand the environment, make decisions and interact; Interaction and feedback module: realizes information exchange and collaboration between intelligent agents, and improves overall performance through continuous feedback loops.

2. A multi-agent training system integrating knowledge graph according to claim 1, characterized in that: The knowledge graph management module mainly includes: Data acquisition module: responsible for the interface with external data sources to obtain raw data, including API calls, web crawling, and sensor input; Knowledge processing module: realizes data cleaning, extraction and conversion, converting raw data into the standard format of knowledge graph. Graph building module: Use graph database technology to store processed data in a graph structure and build effective indexes to cover various query requirements; Update management module: monitors changes in external data sources and automatically triggers updates to the knowledge graph; Query and reasoning module: Provides a RESTful API or SPARQL interface, allowing the agent to perform complex query and reasoning tasks.

3. A multi-agent training system integrating knowledge graph according to claim 1, characterized in that: The agent training module mainly includes: Environment simulator: simulates the real environment, allowing the agent to be trained and tested, providing feedback. This can be a simple rule-based environment or a complex physics engine. Learning Engine: Reinforcement learning module: implements Q-learning, deep Q-network, and policy gradient methods to help agents learn optimal strategies through interaction with the environment; Supervised learning module: used to train the agent’s decision model when labeled data is available, especially when classification or regression tasks are required; Knowledge graph interaction module: allows the agent to access the knowledge graph to obtain background knowledge and supports the agent to use contextual information when making decisions; Strategy generation module: Generates new decision-making strategies and optimizes existing strategies based on the experience and knowledge acquired by the agent; Feedback mechanism: A mechanism that provides feedback by analyzing the performance of the intelligent agent in the environment, including evaluation of different indicators such as success rate and error rate.

4. The multi-agent training system integrating knowledge graph according to claim 1, characterized in that: The interaction and feedback module mainly includes: Interaction interface: responsible for communication with the environment and supports different types of input and output; Feedback calculation module: Reward mechanism: Calculate rewards and penalties based on the results of the agent's actions to guide the agent's learning process; Subsequent state evaluation: Evaluate the new state of the agent after affecting the environment and confirm the effectiveness of its decision; Knowledge graph association module: associates the agent’s behavior and experience with the knowledge in the knowledge graph to provide enhanced decision-making basis; Performance Monitoring and Visualization Module: monitors the learning and decision-making process of the intelligent agent in real time, records key performance indicators and visualizes the data for analysis.

5. A training method for a multi-agent training system integrating a knowledge graph according to any one of claims 1 to 4, characterized in that: The specific steps of the knowledge graph management module are as follows: Data collection: Collect raw data from different data sources, pre-process and clean it; Knowledge Modeling: Analyze the collected data to identify key entities and their attributes and relationships; Build a preliminary knowledge graph model and verify it; Graph construction: Store the confirmed entities and their relationships in a graph database to form a knowledge graph; Create indexes for nodes and edges in the graph to optimize query efficiency; Knowledge update: Monitor changes in data sources in real time and use incremental update strategies to update knowledge graphs to ensure the timeliness of information; Record each update through version control to facilitate management of change history; Query and inference runs: Accept query requests from the agent and call the query and reasoning module for processing; Return query results for agent decision-making and learning; Feedback Mechanism: Collect feedback from intelligent agents when using knowledge graphs to improve data collection and knowledge updating strategies.

6. A training method for a multi-agent training system integrating a knowledge graph according to any one of claims 1 to 4, characterized in that: The specific steps of the agent training module are as follows: initialization: Configure the agent, including setting learning parameters and environment parameters; Initialize the agent’s strategy and value function; Environment Interaction: The agent selects actions in the environment and executes them according to the current policy; The environment returns a new state and corresponding reward based on the agent's behavior; Knowledge Acquisition: After each interaction, the agent calls the knowledge graph interaction module to query relevant information to assist decision-making; Learning Update: Based on the feedback from the environment, the agent updates its learning model, using reward signals in reinforcement learning or making adjustments through error feedback in supervised learning; Strategy Optimization: Continuously iterate to adjust strategies and improve decisions based on new knowledge and experience; Use the strategy generation module to generate new strategies and compare them with previous strategies; Evaluation and Adjustment: Regularly evaluate the learning effect of the agent and test its performance by setting up verification environments and evaluation indicators; Adjust parameters such as learning rate and exploration strategy to improve training results.

7. A training method for a multi-agent training system integrating a knowledge graph according to any one of claims 1 to 4, characterized in that: The specific steps of the interaction and feedback module are as follows: Environment Interaction: The agent sends action instructions to the environment through the interactive interface, and the environment changes its state according to the instruction and returns new state information; In some settings, the environment can also actively provide feedback data; Feedback calculation: Calculate rewards or penalties based on the actions taken by the agent and evaluate its impact on the environment; At each time step, the feedback calculation module evaluates the difference between the new state of the agent and the previous state, and gives a corresponding score based on previous knowledge; Knowledge graph association: Every time the agent performs an action, the interaction and feedback module will automatically consult the knowledge graph to identify knowledge information related to the current state; In this way, the agent can obtain new knowledge or important hints after interacting with the environment, enhancing the quality of its subsequent decisions; Performance monitoring and evaluation: Continuously track and record the agent's performance and analyze the success or failure of each decision; Generate performance reports and visualization charts to help developers and researchers evaluate the training progress and agent optimization effects; Adjustment mechanism: Based on the results of performance monitoring, the agent’s behavior strategy or learning parameters can be adjusted to improve its performance in the environment.