Intelligent agent autonomous decision optimization method and system

Abstract

The embodiment of the invention provides an agent autonomous decision optimization method and system. Firstly, multi-source heterogeneous data is acquired to construct a domain knowledge graph, a structured domain knowledge basis is provided for subsequent decision making, and decision making basis missing caused by knowledge fragmentation is avoided; secondly, collecting sensor signals in real time, associating the sensor signals with a domain knowledge graph, generating environment cognition information containing entity states, environment constraints and task targets, and ensuring deep fusion of real-time environment data and domain knowledge required by decision making; on the basis of environment cognitive information, path reasoning is carried out on the domain knowledge graph by utilizing a decision reasoning model to determine a target decision strategy, and the reasonability of the decision strategy is guaranteed by virtue of the structured reasoning capability of the knowledge graph and the logic analysis capability of the model; and finally, executing the target strategy to obtain multi-dimensional feedback information, and updating the multi-dimensional feedback information to the domain knowledge graph and the decision inference model, thereby avoiding the decision solidification problem caused by lack of feedback adjustment in a traditional system, and improving the overall reliability of the system.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and system for optimizing autonomous decision-making by an intelligent agent. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent agent technology, with its autonomous perception, decision-making and execution capabilities, has been widely penetrated into various autonomous decision-making systems such as intelligent customer service, industrial automation and smart homes, directly affecting the efficiency and depth of digital transformation in various industries.

[0003] Currently, traditional intelligent agents mainly include rule-based reasoning expert systems and deep learning-based end-to-end data-driven systems. Expert systems rely on manually encoding a large number of domain rules and expert knowledge, while deep learning-based end-to-end data-driven systems rely on a large amount of labeled data to train models.

[0004] However, traditional intelligent agents lack effective feedback and adjustment mechanisms during actual execution, making it difficult to balance the reliability and flexibility of decision-making in complex scenarios, which leads to insufficient overall system reliability. Summary of the Invention

[0005] This invention provides an intelligent agent autonomous decision-making optimization method and system, which can improve the overall reliability of the system.

[0006] In a first aspect, embodiments of the present invention provide an intelligent agent autonomous decision-making optimization method, the method comprising:

[0007] Acquire multi-source heterogeneous data to construct a domain knowledge graph, wherein the domain knowledge graph includes: domain entities, entity attributes, and relationships between entities;

[0008] Real-time acquisition of at least one sensor signal and association with the domain knowledge graph to generate environmental cognitive information including entity state, environmental constraints and task objectives;

[0009] Based on the environmental cognitive information, a decision reasoning model is used to perform path reasoning on the domain knowledge graph to determine the target decision strategy;

[0010] The target decision-making strategy is executed to obtain multi-dimensional feedback information, and the domain knowledge graph and the decision reasoning model are updated based on the multi-dimensional feedback information.

[0011] Preferably,

[0012] The acquisition of multi-source heterogeneous data to construct a domain knowledge graph includes:

[0013] The multi-source heterogeneous data is acquired and adapted data preprocessing operations are performed to generate standardized multi-source heterogeneous data, wherein the multi-source heterogeneous data includes: structured database data, semi-structured document data, unstructured text data and real-time sensor data;

[0014] Perform knowledge extraction operations on the standardized multi-source heterogeneous data to identify and extract the domain entities, entity attributes, and relationships between entities;

[0015] Based on a preset knowledge fusion strategy, knowledge fusion is performed on the domain entities, the entity attributes, and the relationships between the entities;

[0016] The merged domain entities, entity attributes, and relationships between entities are used to construct the domain knowledge graph and stored in a graph database.

[0017] Preferably,

[0018] The real-time acquisition of at least one sensor signal and association with the domain knowledge graph to generate environmental cognitive information including entity state, environmental constraints, and task objectives includes:

[0019] The at least one sensor signal is acquired in real time, wherein the at least one sensor signal includes: visual signal, voice signal and position signal;

[0020] The at least one sensor signal is analyzed to generate at least one structured sensor signal;

[0021] The at least one structured sensor signal is mapped to the domain knowledge graph to generate environmental cognitive information that includes entity state, environmental constraints, and task objectives.

[0022] Preferably,

[0023] The step of using a decision reasoning model to perform path reasoning on the domain knowledge graph based on the environmental cognitive information to determine the target decision strategy includes:

[0024] Based on the preset rule base that associates the entity state, the environmental constraints, the task objective, and the domain knowledge graph contained in the environmental cognitive information, rule reasoning operations are performed and rule reasoning results are generated.

[0025] An experience replay pool is built based on historical execution data. A hierarchical strategy network architecture is used to perform reinforcement learning inference and generate reinforcement learning inference results.

[0026] Based on the rule confidence and model confidence of the decision-making scenario, set corresponding rule inference weights and reinforcement learning inference weights;

[0027] Based on the rule reasoning results, the reinforcement learning reasoning results, the rule reasoning weights, and the reinforcement learning reasoning weights, the decision reasoning model is used to perform path reasoning on the domain knowledge graph to determine the target decision strategy.

[0028] Based on the target decision-making strategy and preset template, a corresponding visual decision tree and natural language explanation are generated.

[0029] Preferably,

[0030] The step of executing the target strategy to obtain multi-dimensional feedback information, and updating the domain knowledge graph and the decision reasoning model based on the multi-dimensional feedback information, includes:

[0031] Construct a multi-dimensional monitoring system, which includes: task dimension, resource dimension, and quality dimension;

[0032] The target decision-making strategy is executed, and the execution results are monitored in real time using the multi-dimensional monitoring system to obtain the multi-dimensional feedback information.

[0033] Determine the task completion rate, resource utilization rate, user satisfaction, task completion rate weight, resource utilization rate weight, and user satisfaction weight, and calculate the performance index using the first formula;

[0034] The first formula is: P = α × A + β × (1 - B) + γ × C;

[0035] Wherein, P is the performance index, α is the task completion rate weight, A is the task completion rate, β is the resource occupancy weight, B is the resource occupancy, γ is the user satisfaction weight, and C is the user satisfaction.

[0036] The multidimensional feedback information is transformed into structured triples, and the domain knowledge graph is updated based on the performance index, wherein the structured triples include: scenario, decision, and effect;

[0037] Simultaneously extract failure experiences from multidimensional feedback information and historical data of similar scenarios from the experience replay pool to retrain the decision reasoning model.

[0038] Secondly, embodiments of the present invention provide an intelligent agent autonomous decision-making optimization system, the system comprising:

[0039] Knowledge graph construction module: used to acquire multi-source heterogeneous data to construct a domain knowledge graph, wherein the domain knowledge graph includes: domain entities, entity attributes and relationships between entities;

[0040] Environmental perception module: used to collect signals from at least one sensor in real time and associate them with the domain knowledge graph to generate environmental cognitive information including entity state, environmental constraints and task objectives;

[0041] Decision reasoning module: Based on the environmental cognitive information, it uses a decision reasoning model to perform path reasoning on the domain knowledge graph to determine the target decision strategy;

[0042] The execution optimization module is used to execute the target decision-making strategy to obtain multi-dimensional feedback information, and update the domain knowledge graph and the decision reasoning model based on the multi-dimensional feedback information.

[0043] Preferably,

[0044] The knowledge graph construction module is used to perform:

[0045] The multi-source heterogeneous data is acquired and adapted data preprocessing operations are performed to generate standardized multi-source heterogeneous data, wherein the multi-source heterogeneous data includes: structured database data, semi-structured document data, unstructured text data and real-time sensor data;

[0046] Perform knowledge extraction operations on the standardized multi-source heterogeneous data to identify and extract the domain entities, entity attributes, and relationships between entities;

[0047] Based on a preset knowledge fusion strategy, knowledge fusion is performed on the domain entities, the entity attributes, and the relationships between the entities;

[0048] The merged domain entities, entity attributes, and relationships between entities are used to construct the domain knowledge graph and stored in a graph database.

[0049] Preferably,

[0050] The environment perception module is used to perform:

[0051] The at least one sensor signal is acquired in real time, wherein the at least one sensor signal includes: visual signal, voice signal and position signal;

[0052] The at least one sensor signal is analyzed to generate at least one structured sensor signal;

[0053] The at least one structured sensor signal is mapped to the domain knowledge graph to generate environmental cognitive information that includes entity state, environmental constraints, and task objectives.

[0054] Preferably,

[0055] The decision reasoning module is used to execute:

[0056] Based on the preset rule base that associates the entity state, the environmental constraints, the task objective, and the domain knowledge graph contained in the environmental cognitive information, rule reasoning operations are performed and rule reasoning results are generated.

[0057] An experience replay pool is built based on historical execution data. A hierarchical strategy network architecture is used to perform reinforcement learning inference and generate reinforcement learning inference results.

[0058] Based on the rule confidence and model confidence of the decision-making scenario, set corresponding rule inference weights and reinforcement learning inference weights;

[0059] Based on the rule reasoning results, the reinforcement learning reasoning results, the rule reasoning weights, and the reinforcement learning reasoning weights, the decision reasoning model is used to perform path reasoning on the domain knowledge graph to determine the target decision strategy.

[0060] Based on the target decision-making strategy and preset template, a corresponding visual decision tree and natural language explanation are generated.

[0061] Preferably,

[0062] The execution optimization module is used to perform:

[0063] Construct a multi-dimensional monitoring system, which includes: task dimension, resource dimension, and quality dimension;

[0064] The target decision-making strategy is executed, and the execution results are monitored in real time using the multi-dimensional monitoring system to obtain the multi-dimensional feedback information.

[0065] Determine the task completion rate, resource utilization rate, user satisfaction, task completion rate weight, resource utilization rate weight, and user satisfaction weight, and calculate the performance index using the first formula;

[0066] The first formula is: P = α × A + β × (1 - B) + γ × C;

[0067] Wherein, P is the performance index, α is the task completion rate weight, A is the task completion rate, β is the resource occupancy weight, B is the resource occupancy, γ is the user satisfaction weight, and C is the user satisfaction.

[0068] The multidimensional feedback information is transformed into structured triples, and the domain knowledge graph is updated based on the performance index, wherein the structured triples include: scenario, decision, and effect;

[0069] Simultaneously extract failure experiences from multidimensional feedback information and historical data of similar scenarios from the experience replay pool to retrain the decision reasoning model.

[0070] This invention provides an intelligent agent autonomous decision-making optimization method and system. The method first constructs a domain knowledge graph containing domain entities, entity attributes, and relationships between entities by acquiring multi-source heterogeneous data, providing a structured domain knowledge foundation for subsequent decision-making and avoiding the lack of decision-making basis due to knowledge fragmentation. Next, it collects sensor signals in real time and associates them with the domain knowledge graph to generate environmental cognitive information containing entity states, environmental constraints, and task objectives, ensuring deep integration of real-time environmental data and domain knowledge required for decision-making. Then, based on the environmental cognitive information, it uses a decision reasoning model to perform path reasoning on the domain knowledge graph to determine the target decision strategy. Leveraging the structured reasoning capabilities of the knowledge graph and the logical analysis capabilities of the model, the rationality of the decision strategy is ensured. Finally, it executes the target decision strategy to obtain multi-dimensional feedback information and updates it to the domain knowledge graph and the decision reasoning model, avoiding the decision-fixation problem caused by the lack of feedback adjustment in traditional systems, thereby improving the overall reliability of the system. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 This is a flowchart of an intelligent agent autonomous decision-making optimization method provided in an embodiment of the present invention;

[0073] Figure 2 This is a flowchart of another intelligent agent autonomous decision-making optimization method provided in an embodiment of the present invention;

[0074] Figure 3 This is a schematic diagram of an intelligent agent autonomous decision-making optimization system provided in an embodiment of the present invention. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0076] like Figure 1 As shown, this embodiment of the invention provides an intelligent agent autonomous decision-making optimization method, which may include the following steps:

[0077] Step 101: Acquire multi-source heterogeneous data to construct a domain knowledge graph, wherein the domain knowledge graph includes: domain entities, entity attributes, and relationships between entities;

[0078] Step 102: Collect at least one sensor signal in real time and associate it with the domain knowledge graph to generate environmental cognitive information containing entity state, environmental constraints and task objectives;

[0079] Step 103: Based on environmental cognitive information, use a decision reasoning model to perform path reasoning on the domain knowledge graph to determine the target decision strategy;

[0080] Step 104: Execute the target decision-making strategy to obtain multi-dimensional feedback information, and update the domain knowledge graph and decision reasoning model based on the multi-dimensional feedback information.

[0081] This invention provides an intelligent agent autonomous decision-making optimization method. First, it constructs a domain knowledge graph containing domain entities, entity attributes, and relationships between entities by acquiring multi-source heterogeneous data. This provides a structured domain knowledge foundation for subsequent decision-making, avoiding the lack of decision-making basis due to knowledge fragmentation. Next, it collects sensor signals in real time and associates them with the domain knowledge graph to generate environmental cognitive information containing entity states, environmental constraints, and task objectives, ensuring deep integration of real-time environmental data and domain knowledge required for decision-making. Then, based on the environmental cognitive information, it uses a decision reasoning model to perform path reasoning on the domain knowledge graph to determine the target decision strategy. Leveraging the structured reasoning capabilities of the knowledge graph and the logical analysis capabilities of the model, the rationality of the decision strategy is guaranteed. Finally, it executes the target decision strategy to obtain multi-dimensional feedback information and updates it to the domain knowledge graph and the decision reasoning model. This avoids the decision-fixation problem caused by the lack of feedback adjustment mechanisms in traditional systems, thereby improving the overall reliability of the system.

[0082] To provide high-quality structured knowledge, in one embodiment of the present invention, step 101 in the above embodiment includes:

[0083] The multi-source heterogeneous data is acquired and adapted data preprocessing operations are performed to generate standardized multi-source heterogeneous data, which includes: structured database data, semi-structured document data, unstructured text data and real-time sensor data.

[0084] Perform knowledge extraction operations on the standardized multi-source heterogeneous data to identify and extract the domain entities, entity attributes, and relationships between entities;

[0085] Based on a preset knowledge fusion strategy, knowledge fusion is performed on the domain entities, the entity attributes, and the relationships between the entities;

[0086] The merged domain entities, entity attributes, and relationships between entities are used to construct the domain knowledge graph and stored in a graph database.

[0087] In this embodiment of the invention, to provide high-quality structured knowledge: First, adaptive preprocessing operations are performed on multi-source heterogeneous data to generate standardized data, solving the problem of inconsistent formats among multi-source data and avoiding knowledge bias caused by messy raw data (for example, structured data can be directly extracted through JDBC / ODBC interfaces; unstructured text can be obtained using web crawling technology and noise filtered based on the TF-IDF algorithm; sensor data can be accessed in real time through the MQTT protocol and noise reduced using Kalman filtering); Second, knowledge extraction operations are performed on the standardized data (for example, BiLSTM+CRF can be used) to identify and extract domain entities (e.g., devices, fault types, operation instructions). The system first extracts entity attributes and relationships between entities, providing a reliable data foundation for the construction of the knowledge graph. Then, it fuses the extracted domain entities, entity attributes, and relationships between entities according to a preset knowledge fusion strategy to achieve consistent knowledge fusion (for example, it can calculate the similarity of entity structure and attributes based on the SimRank algorithm and set a similarity threshold to achieve entity alignment; at the same time, it can use a weighted average method to fuse attribute values ​​based on the credibility of the data source to achieve conflict resolution). Finally, it constructs the fused domain entities, entity attributes, and relationships between entities into a domain knowledge graph and stores it in a graph database (such as Neo4j) to achieve the traceability of domain knowledge and facilitate efficient retrieval and path analysis during subsequent decision-making and reasoning.

[0088] To ensure the real-time nature and accuracy of environmental awareness information, in one embodiment of the present invention, step 102 in the above embodiment includes:

[0089] The real-time acquisition of at least one sensor signal and association with the domain knowledge graph to generate environmental cognitive information including entity state, environmental constraints, and task objectives includes:

[0090] The at least one sensor signal is acquired in real time, wherein the at least one sensor signal includes: visual signal, voice signal and position signal;

[0091] The at least one sensor signal is analyzed to generate at least one structured sensor signal;

[0092] The at least one structured sensor signal is mapped to the domain knowledge graph to generate environmental cognitive information that includes entity state, environmental constraints, and task objectives.

[0093] In this embodiment of the invention, to ensure the real-time nature and accuracy of environmental cognition information: First, multi-dimensional sensor signals, including visual signals, speech signals, and location signals, are collected in real time to avoid incomplete environmental cognition due to single signal acquisition (for example, visual signals can be used for target detection using the YOLOv5 model, speech signals can be converted into text instructions using the WFST decoder, and location signals can be noise-removed using Kalman filtering); then, the collected sensor signals are parsed to generate structured sensor signals (for example, the pre-trained language model BERT can be used to parse user intent and environmental semantics), solving the problem of messy original sensor signal formats and difficulty in directly associating knowledge, laying a data foundation for subsequent knowledge mapping; finally, the structured sensor signals are accurately mapped to domain entities, entity attributes, and relationships between entities in the domain knowledge graph, thereby generating environmental cognition information including entity states (for example, server A-CPU utilization rate of 95%), environmental constraints (for example, data center temperature threshold of 28℃), and task objectives (for example, ensuring service availability), ensuring that the environmental cognition information can directly match the structured logic of the knowledge graph, providing accurate input for subsequent decision-making and reasoning based on the knowledge graph.

[0094] To ensure the reliability of the decision-making strategy, in one embodiment of the present invention, step 103 in the above embodiment may specifically include the following steps:

[0095] Based on the preset rule base that associates the entity state, the environmental constraints, the task objective, and the domain knowledge graph contained in the environmental cognitive information, rule reasoning operations are performed and rule reasoning results are generated.

[0096] An experience replay pool is built based on historical execution data. A hierarchical strategy network architecture is used to perform reinforcement learning inference and generate reinforcement learning inference results.

[0097] Based on the rule confidence and model confidence of the decision-making scenario, set corresponding rule inference weights and reinforcement learning inference weights;

[0098] Based on the rule reasoning results, the reinforcement learning reasoning results, the rule reasoning weights, and the reinforcement learning reasoning weights, the decision reasoning model is used to perform path reasoning on the domain knowledge graph to determine the target decision strategy.

[0099] Based on the target decision-making strategy and preset template, a corresponding visual decision tree and natural language explanation are generated.

[0100] In this embodiment of the invention, to ensure the reliability of the decision-making strategy: First, based on the entity state, environmental constraints, and task objectives in the environmental cognitive information, and combined with a preset rule base associated with the domain knowledge graph (e.g., the SWRL rule base), rule reasoning is performed. Rule reasoning results are generated through structured logic matching to ensure that the decision is supported by clear domain knowledge and business rules (e.g., triggering the "suggest expansion" rule when "CPU utilization > 90%" and "no expansion record" are detected). Second, an experience replay pool is built based on historical execution data, and a hierarchical strategy network architecture is used to conduct reinforcement learning reasoning. Historical experience is used to optimize the reasoning model, generating reinforcement learning reasoning results adapted to dynamic scenarios, thus compensating for the lack of flexibility of traditional rule reasoning in complex and ever-changing scenarios. Finally, rule reasoning weights and reinforcement learning reasoning weights are set based on the rule confidence and model confidence of the decision-making scenario to avoid single-faceted reasoning. The limitations of the first reasoning method are then addressed. The decision reasoning model, based on a domain knowledge graph, integrates weighted rule-based reasoning results with reinforcement learning reasoning results. Path reasoning is performed within the domain knowledge graph to determine the target decision strategy (e.g., searching for feasible paths on the knowledge graph using Dijkstra's algorithm, evaluating the expected utility of different paths using the Graph Attention Network (GAT), and generating the optimal strategy, i.e., the target decision strategy, using Q-learning reinforcement learning). This ensures that the decision conforms to domain knowledge logic and is adaptable to real-time scenarios. Finally, a visual decision tree (e.g., using Graphviz) and natural language explanation are generated based on the target decision strategy and a preset template (e.g., because server A's CPU utilization reaches 95% > the threshold of 90%, and the historical expansion operation success rate is 92%, it is recommended to prioritize the expansion operation). This enables traceability and transparency of the decision-making process.

[0101] To improve the reliability of system decision-making, in one embodiment of the present invention, step 104 in the above embodiment includes:

[0102] Construct a multi-dimensional monitoring system, which includes: task dimension, resource dimension, and quality dimension;

[0103] The target decision-making strategy is executed, and the execution results are monitored in real time using the multi-dimensional monitoring system to obtain the multi-dimensional feedback information.

[0104] Determine the task completion rate, resource utilization rate, user satisfaction, task completion rate weight, resource utilization rate weight, and user satisfaction weight, and calculate the performance index using the first formula;

[0105] The first formula is: P = α × A + β × (1 - B) + γ × C;

[0106] Wherein, P is the performance index, α is the task completion rate weight, A is the task completion rate, β is the resource occupancy weight, B is the resource occupancy, γ is the user satisfaction weight, and C is the user satisfaction.

[0107] The multidimensional feedback information is transformed into structured triples, and the domain knowledge graph is updated based on the performance index, wherein the structured triples include: scenario, decision, and effect;

[0108] Simultaneously extract failure experiences from multidimensional feedback information and historical data of similar scenarios from the experience replay pool to retrain the decision reasoning model.

[0109] In this embodiment of the invention, to improve the reliability of system decision-making: First, a multi-dimensional monitoring system is constructed, covering task dimensions (e.g., execution time / expected time of expansion tasks), resource dimensions (e.g., hardware indicators such as CPU / memory utilization and network latency), and quality dimensions (e.g., user satisfaction ratings and error rate statistics), laying a data foundation for subsequent feedback information; then, when executing the target decision-making strategy, the multi-dimensional monitoring system is used to monitor the execution results in real time, accurately collecting multi-dimensional feedback information including task execution status, resource consumption, and user evaluation feedback, avoiding deviations in optimization direction due to incomplete feedback data; then, the multi-dimensional feedback information is quantified into a performance index using the first formula; then... Multidimensional feedback information is transformed into structured triples (e.g., <Server A, perform operation, scale up>, <effect, CPU utilization decreases by 35%>), and combined with performance indices to update the domain knowledge graph (e.g., using backpropagation to update the weights of relevant entities and relationships in the knowledge graph) to maintain the timeliness and accuracy of domain knowledge. Finally, failure experiences from multidimensional feedback information and historical data of similar scenarios from the experience replay pool are extracted simultaneously (e.g., using an experience replay mechanism to store execution cases for subsequent learning) to retrain the decision-making reasoning model, making up for the model's decision-making bias in specific scenarios, thereby improving the system's decision reliability and adaptability in complex dynamic scenarios.

[0110] like Figure 2 As shown, to more clearly illustrate the technical solution and advantages of the present invention, the following is a detailed description of an intelligent agent autonomous decision-making optimization method provided by an embodiment of the present invention, which may specifically include the following steps:

[0111] Step 201: Acquire multi-source heterogeneous data and perform appropriate data preprocessing operations to generate standardized multi-source heterogeneous data, which includes: structured database data, semi-structured document data, unstructured text data and real-time sensor data;

[0112] Step 202: Perform knowledge extraction operations on standardized multi-source heterogeneous data to identify and extract domain entities, entity attributes, and relationships between entities;

[0113] Step 203: Perform knowledge fusion on domain entities, entity attributes, and relationships between entities based on a preset knowledge fusion strategy;

[0114] Step 204: Construct the merged domain entities, entity attributes, and relationships between entities into a domain knowledge graph and store it in a graph database;

[0115] Step 205: Real-time acquisition of at least one sensor signal, wherein the at least one sensor signal includes: visual signal, voice signal and position signal;

[0116] Step 206: Analyze at least one sensor signal to generate at least one structured sensor signal;

[0117] Step 207: Map at least one structured sensor signal to a domain knowledge graph to generate environmental cognitive information containing entity states, environmental constraints, and task objectives;

[0118] Step 208: Based on the preset rule base that associates entity states, environmental constraints, task objectives, and domain knowledge graph contained in the environmental cognitive information, perform rule reasoning operations and generate rule reasoning results;

[0119] Step 209: Construct an experience replay pool based on historical execution data, execute reinforcement learning inference using a hierarchical strategy network architecture, and generate reinforcement learning inference results;

[0120] Step 210: Set the corresponding rule inference weights and reinforcement learning inference weights based on the rule confidence and model confidence of the decision-making scenario;

[0121] Step 211: Based on the rule reasoning results, reinforcement learning reasoning results, rule reasoning weights, and reinforcement learning reasoning weights, use the decision reasoning model to perform path reasoning on the domain knowledge graph to determine the target decision strategy;

[0122] Step 212: Generate a corresponding visual decision tree and natural language explanation based on the target decision strategy and preset template;

[0123] Step 213: Construct a multi-dimensional monitoring system, which includes: task dimension, resource dimension, and quality dimension;

[0124] Step 214: Execute the target decision-making strategy and use a multi-dimensional monitoring system to monitor the execution results in real time to obtain multi-dimensional feedback information;

[0125] Step 215: Determine the task completion rate, resource utilization rate, user satisfaction, task completion rate weight, resource utilization rate weight, and user satisfaction weight, and calculate the performance index using the first formula;

[0126] Specifically, the first formula is: P = α × A + β × (1 - B) + γ × C;

[0127] Where P is the performance index, α is the task completion rate weight, A is the task completion rate, β is the resource utilization weight, B is the resource utilization, γ is the user satisfaction weight, and C is the user satisfaction.

[0128] Step 216: Transform the multidimensional feedback information into structured triples and update the domain knowledge graph based on the performance index. The structured triples include: scenario, decision, and effect.

[0129] Step 217: Simultaneously extract failure experiences from the multidimensional feedback information and historical data of similar scenarios from the experience replay pool to retrain the decision reasoning model.

[0130] like Figure 3 As shown, this embodiment of the invention provides an intelligent agent autonomous decision-making optimization system, which includes:

[0131] Knowledge graph construction module 301: used to acquire multi-source heterogeneous data to construct a domain knowledge graph, wherein the domain knowledge graph includes: domain entities, entity attributes and relationships between entities;

[0132] Environmental perception module 302: used to collect at least one sensor signal in real time and associate it with the domain knowledge graph to generate environmental cognitive information including entity state, environmental constraints and task objectives;

[0133] Decision reasoning module 303: Based on the environmental cognitive information, it uses a decision reasoning model to perform path reasoning on the domain knowledge graph to determine the target decision strategy;

[0134] Execution optimization module 304: used to execute the target decision strategy to obtain multi-dimensional feedback information, and update the domain knowledge graph and the decision reasoning model based on the multi-dimensional feedback information.

[0135] like Figure 3 As shown, the knowledge graph construction module 301 is used to perform:

[0136] The multi-source heterogeneous data is acquired and adapted data preprocessing operations are performed to generate standardized multi-source heterogeneous data, wherein the multi-source heterogeneous data includes: structured database data, semi-structured document data, unstructured text data and real-time sensor data;

[0137] Perform knowledge extraction operations on the standardized multi-source heterogeneous data to identify and extract the domain entities, entity attributes, and relationships between entities;

[0138] Based on a preset knowledge fusion strategy, knowledge fusion is performed on the domain entities, the entity attributes, and the relationships between the entities;

[0139] The merged domain entities, entity attributes, and relationships between entities are used to construct the domain knowledge graph and stored in a graph database.

[0140] like Figure 3 As shown, the environment perception module 302 is used to perform:

[0141] The at least one sensor signal is acquired in real time, wherein the at least one sensor signal includes: visual signal, voice signal and position signal;

[0142] The at least one sensor signal is analyzed to generate at least one structured sensor signal;

[0143] The at least one structured sensor signal is mapped to the domain knowledge graph to generate environmental cognitive information that includes entity state, environmental constraints, and task objectives.

[0144] like Figure 3 As shown, the decision reasoning module 303 is used to execute:

[0145] Based on the preset rule base that associates the entity state, the environmental constraints, the task objective, and the domain knowledge graph contained in the environmental cognitive information, rule reasoning operations are performed and rule reasoning results are generated.

[0146] An experience replay pool is built based on historical execution data. A hierarchical strategy network architecture is used to perform reinforcement learning inference and generate reinforcement learning inference results.

[0147] Based on the rule confidence and model confidence of the decision-making scenario, set corresponding rule inference weights and reinforcement learning inference weights;

[0148] Based on the rule reasoning results, the reinforcement learning reasoning results, the rule reasoning weights, and the reinforcement learning reasoning weights, the decision reasoning model is used to perform path reasoning on the domain knowledge graph to determine the target decision strategy.

[0149] Based on the target decision-making strategy and preset template, a corresponding visual decision tree and natural language explanation are generated.

[0150] like Figure 3 As shown, the execution optimization module 304 is used to execute:

[0151] Construct a multi-dimensional monitoring system, which includes: task dimension, resource dimension, and quality dimension;

[0152] The target decision-making strategy is executed, and the execution results are monitored in real time using the multi-dimensional monitoring system to obtain the multi-dimensional feedback information.

[0153] Determine the task completion rate, resource utilization rate, user satisfaction, task completion rate weight, resource utilization rate weight, and user satisfaction weight, and calculate the performance index using the first formula;

[0154] The first formula is: P = α × A + β × (1 - B) + γ × C;

[0155] Wherein, P is the performance index, α is the task completion rate weight, A is the task completion rate, β is the resource occupancy weight, B is the resource occupancy, γ is the user satisfaction weight, and C is the user satisfaction.

[0156] The multidimensional feedback information is transformed into structured triples, and the domain knowledge graph is updated based on the performance index, wherein the structured triples include: scenario, decision, and effect;

[0157] Simultaneously extract failure experiences from multidimensional feedback information and historical data of similar scenarios from the experience replay pool to retrain the decision reasoning model.

[0158] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on an intelligent agent autonomous decision-making optimization system. In other embodiments of the present invention, an intelligent agent autonomous decision-making optimization system may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0159] The information interaction and execution process between the various units in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description of the method embodiment of the present invention, and will not be repeated here.

[0160] This invention also provides an intelligent agent autonomous decision-making optimization system, comprising: at least one memory and at least one processor;

[0161] At least one memory for storing machine-readable programs;

[0162] At least one processor is configured to invoke a machine-readable program to execute an agent autonomous decision-making optimization method according to any embodiment of the present invention.

[0163] This invention also provides a computer-readable medium storing computer instructions, which, when executed by a processor, cause the processor to perform an intelligent agent autonomous decision-making optimization method according to any embodiment of this invention.

[0164] Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0165] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0166] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0167] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0168] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0169] The various embodiments of the present invention have at least the following beneficial effects:

[0170] 1. In this embodiment of the invention, an intelligent agent autonomous decision-making optimization method is provided. This method first constructs a domain knowledge graph containing domain entities, entity attributes, and relationships between entities by acquiring multi-source heterogeneous data, providing a structured domain knowledge foundation for subsequent decision-making and avoiding the lack of decision-making basis due to knowledge fragmentation. Next, sensor signals are collected in real time and associated with the domain knowledge graph to generate environmental cognitive information containing entity states, environmental constraints, and task objectives, ensuring deep integration of real-time environmental data and domain knowledge required for decision-making. Then, based on the environmental cognitive information, a decision reasoning model is used to perform path reasoning on the domain knowledge graph to determine the target decision strategy. The structured reasoning capability of the knowledge graph and the logical analysis capability of the model ensure the rationality of the decision strategy. Finally, the target decision strategy is executed to obtain multi-dimensional feedback information, which is then updated to the domain knowledge graph and the decision reasoning model, avoiding the decision-fixation problem caused by the lack of feedback adjustment in traditional systems, thereby improving the overall reliability of the system.

[0171] 2. In this embodiment of the invention, to provide high-quality structured knowledge: First, adaptive preprocessing operations are performed on multi-source heterogeneous data to generate standardized data, solving the problem of inconsistent formats among multi-source data and avoiding knowledge bias caused by messy original data; Second, knowledge extraction operations are performed on the standardized data to identify and extract domain entities, entity attributes, and relationships between entities, providing a reliable data foundation for the construction of knowledge graphs; Then, the extracted domain entities, entity attributes, and relationships between entities are fused according to a preset knowledge fusion strategy to achieve consistent knowledge fusion; Finally, the fused domain entities, entity attributes, and relationships between entities are used to construct a domain knowledge graph and store it in a graph database to achieve traceability of domain knowledge, facilitating efficient retrieval and path analysis during subsequent decision-making and reasoning.

[0172] 3. In this embodiment of the invention, to ensure the real-time nature and accuracy of environmental cognition information: First, multi-dimensional sensor signals, including visual signals, voice signals, and location signals, are collected in real time to avoid incomplete environmental cognition due to single signal acquisition; then, the collected sensor signals are parsed to generate structured sensor signals, solving the problem of messy original sensor signal formats and difficulty in directly associating knowledge, thus laying a data foundation for subsequent knowledge mapping; finally, the structured sensor signals are accurately mapped to domain entities, entity attributes, and relationships between entities in the domain knowledge graph, thereby generating environmental cognition information containing entity states, environmental constraints, and task objectives, ensuring that the environmental cognition information can directly match the structured logic of the knowledge graph, providing accurate input for subsequent decision-making and reasoning based on the knowledge graph.

[0173] It should be noted that not all steps and modules in the above processes and system structure diagrams are mandatory; some steps or modules can be omitted as needed. The execution order of each step is not fixed and can be adjusted as required. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0174] In the above embodiments, the hardware units can be implemented mechanically or electrically. For example, a hardware unit may include permanent dedicated circuitry or logic (such as a dedicated processor, FPGA, or ASIC) to perform the corresponding operation. The hardware unit may also include programmable logic or circuitry (such as a general-purpose processor or other programmable processor), which can be temporarily configured by software to perform the corresponding operation. The specific implementation method (mechanical, dedicated permanent circuitry, or temporarily configured circuitry) can be determined based on cost and time considerations.

[0175] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for autonomous decision-making optimization of an intelligent agent, characterized in that, The method comprises: acquiring multi-source heterogeneous data to construct a domain knowledge graph, wherein the domain knowledge graph comprises domain entities, entity attributes, and inter-entity relationships; real-time collection of at least one sensor signal and association with the domain knowledge graph to generate environment cognition information comprising entity states, environmental constraints, and task targets; based on the environment cognition information, path reasoning on the domain knowledge graph using a decision reasoning model to determine a target decision strategy; execution of the target decision strategy to obtain multi-dimensional feedback information, and updating of the domain knowledge graph and the decision reasoning model based on the multi-dimensional feedback information.

2. The method of claim 1, wherein: the acquisition of multi-source heterogeneous data to construct a domain knowledge graph comprises: acquisition of the multi-source heterogeneous data and execution of adaptive data preprocessing operations to generate standardized multi-source heterogeneous data, wherein the multi-source heterogeneous data comprises structured database data, semi-structured document data, unstructured text data, and real-time sensor data; performance of knowledge extraction operations on the standardized multi-source heterogeneous data to identify and extract the domain entities, the entity attributes, and the inter-entity relationships; knowledge fusion of the domain entities, the entity attributes, and the inter-entity relationships based on a pre-set knowledge fusion strategy; construction of the fused domain entities, entity attributes, and inter-entity relationships into the domain knowledge graph and storage into a graph database.

3. The method of claim 1, wherein: the real-time collection of at least one sensor signal and association with the domain knowledge graph to generate environment cognition information comprising entity states, environmental constraints, and task targets comprises: real-time collection of the at least one sensor signal, wherein the at least one sensor signal comprises visual signals, voice signals, and location signals; analysis of the at least one sensor signal to generate at least one structured sensor signal; mapping of the at least one structured sensor signal to the domain knowledge graph to generate the environment cognition information comprising entity states, environmental constraints, and task targets.

4. The method of claim 1, wherein: the path reasoning on the domain knowledge graph using a decision reasoning model based on the environment cognition information to determine a target decision strategy comprises: based on the entity states, environmental constraints, and task targets contained in the environment cognition information and a pre-set rule base associated with the domain knowledge graph, execution of rule reasoning operations and generation of rule reasoning results; construction of an experience replay pool based on historical execution data, execution of reinforcement learning reasoning using a hierarchical policy network architecture, and generation of reinforcement learning reasoning results; based on the rule confidence and model confidence of the decision scenario, setting of corresponding rule reasoning weights and reinforcement learning reasoning weights; based on the rule reasoning results, the reinforcement learning reasoning results, the rule reasoning weights, and the reinforcement learning reasoning weights, path reasoning on the domain knowledge graph using the decision reasoning model to determine the target decision strategy. generate a corresponding visual decision tree and a natural language explanation based on the target decision strategy and a preset template.

5. The method of claim 4, wherein, the execution of the target decision strategy obtains multi-dimensional feedback information, and the domain knowledge graph and the decision reasoning model are updated based on the multi-dimensional feedback information, including: constructing a multi-dimensional monitoring system, wherein the multi-dimensional monitoring system includes a task dimension, a resource dimension, and a quality dimension; executing the target decision strategy and monitoring the execution result in real time using the multi-dimensional monitoring system to obtain the multi-dimensional feedback information; determining a task completion rate, a resource occupancy rate, a user satisfaction degree, a task completion rate weight, a resource occupancy rate weight, and a user satisfaction degree weight, and calculating a performance index using a first formula; the first formula is: P = a x A + b x (1-B) + g x C; wherein P is the performance index, a is the task completion rate weight, A is the task completion rate, b is the resource occupancy rate weight, B is the resource occupancy rate, g is the user satisfaction degree weight, and C is the user satisfaction degree; transforming the multi-dimensional feedback information into structured triples, and updating the domain knowledge graph based on the performance index, wherein the structured triples include a scenario, a decision, and an effect; synchronously extracting failure experience from the multi-dimensional feedback information and historical data of similar scenarios in the experience replay pool to retrain the decision reasoning model.

6. An autonomous decision optimization system for an agent, the system comprising: The system comprises: a knowledge graph construction module for obtaining multi-source heterogeneous data to construct a domain knowledge graph, wherein the domain knowledge graph includes domain entities, entity attributes, and entity relationships; an environment perception module for real-time collection of at least one sensor signal and association with the domain knowledge graph to generate environment cognition information including entity states, environmental constraints, and task targets; a decision reasoning module for path reasoning on the domain knowledge graph based on the environment cognition information using a decision reasoning model to determine a target decision strategy; an execution optimization module for executing the target decision strategy to obtain multi-dimensional feedback information, and updating the domain knowledge graph and the decision reasoning model based on the multi-dimensional feedback information.

7. The system of claim 6, wherein, the knowledge graph construction module is configured to: obtain the multi-source heterogeneous data and perform an adaptive data preprocessing operation to generate standardized multi-source heterogeneous data, wherein the multi-source heterogeneous data includes structured database data, semi-structured document data, unstructured text data, and real-time sensor data; perform a knowledge extraction operation on the standardized multi-source heterogeneous data to identify and extract the domain entities, the entity attributes, and the entity relationships; perform knowledge fusion on the domain entities, the entity attributes, and the entity relationships based on a preset knowledge fusion strategy; construct the fused domain entities, entity attributes, and entity relationships into the domain knowledge graph and store them in a graph database.

8. The system of claim 6, wherein, the environment perception module is configured to: collecting the at least one sensor signal in real time, wherein the at least one sensor signal comprises a visual signal, a voice signal, and a position signal; parsing the at least one sensor signal to generate at least one structured sensor signal; mapping the at least one structured sensor signal to the domain knowledge graph to generate the environment cognitive information containing the entity state, the environmental constraint, and the task target.

9. The system of claim 6, wherein the decision reasoning module is configured to perform: based on the entity state, the environmental constraint, and the task target contained in the environment cognitive information and a preset rule base associated with the domain knowledge graph, performing rule reasoning operation and generating a rule reasoning result; based on historical execution data, constructing an experience replay pool, and performing reinforcement learning reasoning using a hierarchical policy network architecture and generating a reinforcement learning reasoning result; based on the rule confidence and the model confidence of the decision scenario, setting a corresponding rule reasoning weight and reinforcement learning reasoning weight; based on the rule reasoning result, the reinforcement learning reasoning result, the rule reasoning weight, and the reinforcement learning reasoning weight, using the decision reasoning model to perform path reasoning on the domain knowledge graph to determine the target decision strategy; based on the target decision strategy and a preset template, generating a corresponding visual decision tree and natural language explanation.

10. The system of claim 9, wherein the execution optimization module is configured to perform: constructing a multi-dimensional monitoring system, wherein the multi-dimensional monitoring system comprises a task dimension, a resource dimension, and a quality dimension; executing the target decision strategy and using the multi-dimensional monitoring system to monitor the execution result in real time to obtain the multi-dimensional feedback information; determining a task completion rate, a resource occupancy rate, a user satisfaction degree, a task completion rate weight, a resource occupancy rate weight, and a user satisfaction degree weight, and using a first formula to calculate a performance index; the first formula is: P = α × A + β × (1-B) + γ × C; wherein P is the performance index, α is the task completion rate weight, A is the task completion rate, β is the resource occupancy rate weight, B is the resource occupancy rate, γ is the user satisfaction degree weight, and C is the user satisfaction degree; converting the multi-dimensional feedback information into a structured triple, and updating the domain knowledge graph based on the performance index, wherein the structured triple comprises a scenario, a decision, and an effect; synchronously extracting failure experience in the multi-dimensional feedback information and historical data of the same type of scenario in the experience replay pool to retrain the decision reasoning model. ​ ​

Images