Multi-agent autonomous decision-making method based on deep reinforcement learning

Abstract

The invention relates to the technical field of multi-agent cooperative control, and discloses a multi-agent autonomous decision-making method based on deep reinforcement learning. The method comprises the steps of synchronously detecting an initial collaborative state of a cluster, performing joint situation assessment, and judging a collaborative operation mode according to a quantitative situation. And analyzing the capability of each agent and the real-time task load, and constructing a distributed task knowledge graph. And utilizing the atlas to drive a deep reinforcement learning network, coupling computing resource allocation and a task path, and generating a preliminary behavior strategy of each agent. And performing cluster-level conflict detection and iterative negotiation adjustment, and finally issuing an executable action instruction sequence. According to the method, integrated optimization of resource allocation and action paths is realized, the situation understanding and negotiation mechanism is enhanced through the knowledge graph to guarantee the collaborative consistency, and the collaborative decision-making efficiency and task execution robustness of a multi-agent system in a dynamic environment are improved.

Description

Technical Field

[0001] This invention relates to the field of multi-agent cooperative control technology, specifically to a multi-agent autonomous decision-making method based on deep reinforcement learning. Background Technology

[0002] Existing multi-agent collaborative task decision-making primarily relies on centralized scheduling or distributed negotiation frameworks. Centralized methods typically allocate tasks based on global information at a central node, with each agent then independently planning its course according to the assignment. Distributed methods often use auctions or rule matching to achieve task acceptance, with path planning often serving as a subsequent step or cost estimation factor. These methods treat resource allocation and path generation as two independent, sequential or weakly correlated processes, relying mainly on the static attributes of agents and the discrete requirements of the task, failing to adequately consider the real-time changing states of individuals during dynamic task execution.

[0003] The key flaw in the current technical solution lies in the limitations of the information model and decision-making mechanism. The system lacks a refined and structured representation of the overall situation of the agent cluster. The real-time capacity load, remaining resources of each agent, and the dynamic constraints between them and tasks and other agents are not effectively integrated into a reasonable cognitive network, resulting in one-sided and lagging information on which decisions are based. The decoupling of task allocation and path planning makes the allocation scheme prone to resource competition, spatiotemporal conflicts, and efficiency losses in subsequent execution. This usually requires additional conflict detection and resolution modules for post-event remediation, affecting the overall efficiency and adaptability of collaboration.

[0004] The key to improving the autonomous collaborative performance of multi-agent systems lies in how to construct a cluster cognitive model that can deeply integrate the real-time state of individuals and task constraints in a dynamic environment, and on this basis, realize the integrated joint optimization of resource allocation and action path and the real-time conflict coordination. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-agent autonomous decision-making method based on deep reinforcement learning to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, this invention provides a multi-agent autonomous decision-making method based on deep reinforcement learning, the method comprising:

[0007] It senses the external environment and task instructions, triggering a synchronous detection of the initial collaborative state of the multi-agent cluster;

[0008] Based on the initial cooperative state obtained from the synchronous detection, a joint situation assessment is performed on the dynamic environment in which the multi-agent cluster is located, generating a quantitative description of the environmental situation.

[0009] Based on the quantitative description of the environmental situation, determine the cooperative operation mode that the multi-agent cluster should currently adopt;

[0010] Based on the cooperative operation mode, the individual capabilities and real-time task load of each agent in the multi-agent cluster are analyzed.

[0011] By integrating the individual capabilities and the results of real-time task load analysis, a distributed task knowledge graph for the multi-agent cluster is constructed.

[0012] Using the distributed task knowledge graph, a deep reinforcement learning network is driven to perform coupled computation of resource allocation and task paths within the cluster;

[0013] Based on the resource allocation and task path obtained from the coupled calculation, a preliminary behavioral strategy is generated for each agent in the multi-agent cluster.

[0014] Cluster-level conflict detection and compatibility verification are performed on the preliminary behavior strategy.

[0015] Based on the results of the conflict detection and compatibility verification, the initial behavioral strategies that have conflicts are iteratively negotiated and adjusted.

[0016] Based on the results of iterative negotiation and adjustment, an executable sequence of autonomous action instructions is issued to each agent.

[0017] Preferably, the step of sensing the external environment and task instructions to trigger the synchronous detection of the initial collaborative state of the multi-agent cluster includes:

[0018] Continuously collect multimodal data of the physical environment through a deployed sensor array;

[0019] Simultaneously, it receives top-level mission instruction packets from the command system;

[0020] Parse the top-level task instruction package to extract task objectives, constraints, and priority information;

[0021] Based on the multimodal data and the task objectives, constraints, and priority information, a synchronous detection trigger signal is generated;

[0022] In response to the synchronization detection trigger signal, a status query request is broadcast to all agents in the multi-agent cluster;

[0023] Collect local state information from each agent, including location, energy level, health status of functional modules, and current load.

[0024] Preferably, the step of performing a joint situation assessment of the dynamic environment in which the multi-agent cluster is located based on the initial cooperative state obtained by the synchronous detection, and generating a quantitative description of the environmental situation, includes:

[0025] The local state information of all the agents collected is spatiotemporally aligned and fused to form a global state view of the cluster;

[0026] Analyze the global state view of the cluster to identify the spatial relationships and functional dependencies between agents;

[0027] In parallel, feature extraction and threat source identification are performed on continuously collected multimodal data of the physical environment;

[0028] The comprehensive environmental complexity index is calculated by combining the spatial relative relationships and functional dependencies between the intelligent agents and the threat source identification results.

[0029] Based on the comprehensive environmental complexity index and task constraints, a quantitative description of the environmental situation is generated. The quantitative description of the environmental situation includes a map of passable areas, a dynamic obstacle distribution field, and a heat map of key task points.

[0030] Preferably, the step of determining the cooperative operation mode that the multi-agent cluster should currently adopt based on the quantitative description of the environmental situation includes:

[0031] Read the comprehensive environmental complexity index from the quantitative description of the environmental situation;

[0032] The overall environmental complexity index is compared with multiple preset complexity threshold ranges;

[0033] Based on the comparison results, the corresponding collaborative operation mode is selected from the predefined collaborative operation mode library, which includes tight formation mode, regional decentralized search mode, hierarchical command mode and dynamic alliance mode.

[0034] The selection of the collaborative operation mode determines the granularity and principles of subsequent resource allocation and task planning.

[0035] Preferably, the step of analyzing the individual capabilities and real-time task load of each agent in the multi-agent cluster based on the cooperative operation mode includes:

[0036] Based on the selected cooperative operation mode, determine the set of agent capability dimensions that need to be evaluated;

[0037] Query the capability model library for each agent to obtain its static capability parameters under the set of capability dimensions;

[0038] At the same time, monitor the list of subtasks that each agent is executing or queuing, and estimate its real-time computing and physical resource utilization.

[0039] The static capability parameters are combined with the real-time computing and physical resource utilization rates to calculate the instantaneous available capability vector for each agent.

[0040] The instantaneous availability vector represents the agent's potential and efficiency in undertaking new tasks at the current moment.

[0041] Preferably, the step of integrating the individual capabilities with the results of real-time task load analysis to construct a distributed task knowledge graph for the multi-agent cluster includes:

[0042] The task objective is used as the root node, and multiple levels of sub-task nodes are decomposed into it.

[0043] Associate each subtask node with the capability dimensions and resource requirements needed to complete the subtask;

[0044] Each agent is mapped to an execution entity node in the graph, and its readily available capability vector is used as an attribute of the execution entity node;

[0045] Weighted virtual connections are established between subtask nodes and execution entity nodes based on capability matching and resource sufficiency.

[0046] The distributed task knowledge graph dynamically reflects the matching relationship network between task structure, resource requirements, and agent capabilities.

[0047] Preferably, the step of using the distributed task knowledge graph to drive a deep reinforcement learning network to couple the computation of resource allocation and task paths within the cluster includes:

[0048] The current state of the distributed task knowledge graph is encoded as the input feature of a deep reinforcement learning network;

[0049] The deep reinforcement learning network includes a value network for evaluating the value of the global state and a policy network for generating specific allocation policies.

[0050] Based on the input features and considering the constraints of the cooperative operation mode, the policy network outputs two suggestions: first, the probability distribution of assigning execution entity nodes to sub-task nodes; and second, the preliminary task execution path for the assigned agents.

[0051] The value network then evaluates the long-term returns of the allocation and path combination of the strategy network output, and guides the strategy network to optimize its output through network parameter updates.

[0052] Preferably, the step of generating a preliminary behavioral strategy for each agent in the multi-agent cluster based on the resource allocation and task path obtained from the coupled calculation includes:

[0053] Analyze the allocation results output by the policy network to determine the set of subtasks assigned to each agent;

[0054] For each agent and its assigned set of subtasks, the policy network is extracted as the preliminary task execution path it plans;

[0055] The subtask set is integrated with the preliminary task execution path and encapsulated into a preliminary behavior strategy package specific to the intelligent agent;

[0056] The preliminary behavioral strategy includes the sequence of actions, target points, and expected resource consumption that the agent plans to execute over a period of time.

[0057] Preferably, the step of performing cluster-level conflict detection and compatibility verification on the preliminary behavior strategy includes:

[0058] Collect preliminary behavioral policy packages from all agents;

[0059] Establish a virtual cluster behavior simulation environment, load the preliminary behavior strategy package, and perform time-series simulation.

[0060] During the simulation, it is detected whether there are spatial trajectory conflicts, resource usage conflicts, or target logic conflicts between intelligent agents;

[0061] Record all detected conflict types, the identities of the agents involved, and the time window in which the conflict occurred;

[0062] A conflict detection report is generated, which is used to guide the subsequent negotiation and adjustment process.

[0063] Preferably, the step of iteratively negotiating and adjusting the initial behavioral strategy with conflicting results based on the conflict detection and compatibility verification includes:

[0064] Read the conflict detection report to locate the conflicting agent combinations and the content of the conflict;

[0065] Initiate a negotiation tuner based on rules or deep reinforcement learning networks;

[0066] The negotiation adjuster takes the initial behavioral strategies and conflict content of the conflicting agents as input, and fine-tunes the behavioral strategy parameters of the conflicting parties while ensuring the progress of the global task.

[0067] The fine-tuned behavior strategy was re-injected into the virtual cluster behavior simulation environment for verification.

[0068] If the conflict is not fully resolved, a new round of iterative adjustments will be initiated until all conflicts are resolved or reduced to an acceptable level. At this point, the final adjusted behavioral strategy will be issued as an executable sequence of autonomous action instructions.

[0069] Compared with the prior art, the beneficial effects of the present invention are:

[0070] By constructing a distributed task knowledge graph that integrates the real-time capability state and dynamic task load of each agent, this method transforms the discrete, static state parameters in traditional decision-making into a networked cognitive structure that dynamically reflects the multidimensional constraints and relationships between "individual-task-environment". This graph-based knowledge representation enables the system to gain an interpretable and reasonable deep understanding of the overall operational status of the cluster, thus providing a precise information foundation with inherent logical connections for subsequent decisions. It ensures that the input to decisions is no longer limited to instantaneous observations but includes potential capability changes and collaborative constraints derived from real-time loads.

[0071] This dynamic knowledge graph drives a deep reinforcement learning model to couple resource allocation decisions with individual task paths. This technique enables the model to simultaneously optimize the traditionally separate decision dimensions of "assignment" and "path" during policy exploration, directly searching the joint solution space for the globally optimal strategy combination, avoiding local optima and policy conflicts caused by phased decision-making. After the initial joint policy is generated, distributed fine-tuning is performed through an explicit cluster-level conflict detection and iterative negotiation process. This mechanism does not simply overwrite the model output, but provides a collaborative verification and optimization layer based on the current global policy and oriented towards practical executability. It compensates for potential policy inconsistencies in end-to-end models under complex dynamic scenarios, enhancing the coordination consistency of the final action instruction sequence and the overall system's adaptability and robustness. Attached Figure Description

[0072] Figure 1 This is a schematic diagram illustrating the working principle of the multi-agent autonomous decision-making method based on deep reinforcement learning described in this invention.

[0073] Figure 2 A flowchart for synchronously detecting the initial cooperative state;

[0074] Figure 3 A flowchart for the joint situation assessment;

[0075] Figure 4 A radar chart for assessing the capabilities of an intelligent agent;

[0076] Figure 5A dual-indicator trend graph for the multi-agent decision-making iterative process. Detailed Implementation

[0077] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0078] Please see Figure 1 This invention provides a multi-agent autonomous decision-making method based on deep reinforcement learning. The method includes: perceiving the external environment and task instructions to trigger synchronous detection of the initial cooperative state of a multi-agent cluster; performing joint situational assessment of the dynamic environment of the multi-agent cluster based on the initial cooperative state obtained from the synchronous detection, generating a quantitative description of the environmental situation; determining the cooperative operation mode that the multi-agent cluster should currently adopt based on the quantitative description of the environmental situation; analyzing the individual capabilities and real-time task load of each agent in the multi-agent cluster based on the cooperative operation mode; integrating the results of the individual capability and real-time task load analysis to construct a distributed task knowledge graph for the multi-agent cluster; using the distributed task knowledge graph to drive a deep reinforcement learning network to perform coupled computation of resource allocation and task paths within the cluster; generating preliminary behavioral strategies for each agent in the multi-agent cluster based on the resource allocation and task paths obtained from the coupled computation; performing cluster-level conflict detection and compatibility verification on the preliminary behavioral strategies; iteratively negotiating and adjusting the preliminary behavioral strategies with conflicts based on the results of conflict detection and compatibility verification; and issuing an executable sequence of autonomous action instructions to each agent based on the results of iterative negotiation and adjustment.

[0079] Example 1: See Figure 2 The system senses the external environment and task commands, triggering a synchronization probe of the initial collaborative state of the multi-agent cluster. This step is implemented as follows: Multimodal data of the physical environment is continuously collected through a deployed sensor array. Simultaneously, top-level task command packets are received from the command system. The top-level task command packets are parsed to extract task objectives, constraints, and priority information. Based on the multimodal data and the task objectives, constraints, and priority information, a synchronization probe trigger signal is generated. In response to the synchronization probe trigger signal, a state query request is broadcast to all agents in the multi-agent cluster. Local state information fed back by each agent is collected, including location, energy level, functional module health status, and current load.

[0080] In practical implementation, sensor arrays deployed in the operational area of ​​the intelligent agent cluster continuously collect multimodal data of the physical environment, including visible light image data, infrared thermal imaging data, lidar point cloud data, and environmental sound spectrum data. In the area monitoring example scenario, the sensor array acquires point cloud data containing the outlines of moving objects and spectrum data of environmental background noise; in the fire rescue example scenario, the sensor array acquires thermal imaging data of temperature distribution inside the building and visible light smoke concentration image data. The command system sends top-level task instruction packets to the decision-making center; these top-level task instruction packets are structured data messages.

[0081] In practical implementation, the decision center parses the received top-level task instruction package, extracting task objectives, constraints, and priority information during the parsing process. In the area monitoring example scenario, the parsed task objective is to complete periodic patrols and report abnormal events within the designated area. The constraint is that the intelligent agent cluster must complete the task within a specified runtime, and the priority information requires prioritizing the moving target tracking subtask. In the fire rescue example scenario, the parsed task objective is to locate and rescue trapped personnel within a building. The constraint is to avoid high-temperature and structurally unstable areas, and the priority information requires life detection and rescue channel opening to have the highest execution level.

[0082] In some embodiments, the decision center generates a synchronization detection trigger signal based on continuously collected multimodal data of the physical environment and the parsed task objectives, constraints, and priority information. The logic for generating the synchronization detection trigger signal involves determining a threshold for the amount of multimodal data of the physical environment and assessing the urgency of the task instructions. Its triggering condition is defined by the following formula:

[0083]

[0084] in: The Boolean value representing the synchronous detection trigger signal. For indicator functions, This represents the total volume of physical environment multimodal data received per unit time. It is a preset data volume threshold. This represents the task priority identifier parsed from the top-level task instruction packet. This represents the set of high-priority identifiers that require immediate triggering of synchronization probes. The synchronization probe trigger signal is set when the amount of multimodal data in the physical environment exceeds a threshold and the task is in a high-priority state.

[0085] In practice, in response to the synchronous detection trigger signal, the decision center broadcasts a status query request to all agents in the multi-agent cluster via the cluster communication network. The status query request includes a unified timestamp and query instruction code. Upon receiving the status query request, each agent in the multi-agent cluster encapsulates its local status information according to a preset format and feeds it back. The collected local status information includes each agent's real-time geographic coordinates, remaining battery energy level, health status diagnostic codes for key functional modules such as robotic arms or cameras, and the current load percentage and communication queue depth of the central processing unit.

[0086] In some embodiments, local status information fed back from agents can reveal data differences. In the area monitoring example scenario, agent A's local status information shows that it is located at coordinates (X1, Y1), with an energy level of 85%, the image recognition module is in normal health, and the current load rate is 60%; agent B's local status information shows that it is located at coordinates (X2, Y2), with an energy level of 45%, the communication relay module has a slight attenuation alarm, and the current load rate is 85%. In the fire rescue example scenario, agent C's local status information includes its internal coordinates after entering the building, its energy level has dropped to 65% due to high-intensity computation, the thermal sensing module is in normal health, and the current load rate is 90%; while agent D's local status information shows that it is still waiting at the entrance, with an energy level of 95%, all modules are healthy, and the load rate is 30%.

[0087] Example 2: See Figure 3 Based on the initial cooperative state obtained from synchronous detection, a joint situational assessment of the dynamic environment in which the multi-agent cluster exists is performed to generate a quantitative description of the environmental situation. This step is implemented as follows: The local state information of all collected agents is spatiotemporally aligned and fused to form a global state view of the cluster. The global state view is analyzed to identify the spatial relative relationships and functional dependencies between agents. Feature extraction and threat source identification are performed in parallel on the continuously collected multimodal data of the physical environment. The comprehensive environmental complexity index is calculated by combining the spatial relative relationships and functional dependencies between agents with the threat source identification results. Based on the comprehensive environmental complexity index and task constraints, a quantitative description of the environmental situation is generated, which includes a map of traversable areas, a dynamic obstacle distribution field, and a heatmap of key task points.

[0088] Based on the quantitative description of the environmental situation, the appropriate cooperative operation mode for the multi-agent cluster is determined. This step is implemented as follows: The comprehensive environmental complexity index is read from the quantitative description of the environmental situation. The comprehensive environmental complexity index is compared with several preset complexity threshold ranges. Based on the comparison results, the corresponding cooperative operation mode is selected from a predefined cooperative operation mode library, which includes tight formation mode, regional distributed search mode, hierarchical command mode, and dynamic alliance mode. The selection of the cooperative operation mode determines the granularity and principles of subsequent resource allocation and task planning.

[0089] In practice, the decision center performs spatiotemporal alignment and fusion of the local state information collected from all agents. This local state information includes location, energy level, functional module health status, and current load. The alignment process interpolates and synchronizes the states of all agents based on a unified timestamp. The fusion process generates a cluster-wide global state view containing the real-time states and relationships of all agents. Analyzing this global state view identifies the spatial relative relationships and functional dependencies between agents. Spatial relative relationships are obtained by calculating the Euclidean distance and azimuth between each pair of agents, while functional dependencies are obtained by checking the compatibility of the input / output interfaces marked in the agent capability model library.

[0090] In some embodiments, the data obtained from the cluster global state view analysis exhibits differences across different scenarios. In the regional monitoring example scenario, analysis shows that the spatial relative distance between agent A and agent B is 150 meters, with an azimuth angle of 30 degrees. The functional dependency is that the image recognition module output of agent A can provide a data source for the communication relay module of agent B. In the fire rescue example scenario, analysis shows that the spatial relative distance between agent C and agent D is 50 meters, with an azimuth angle of 120 degrees. The functional dependency is that the demolition module of agent D requires the structural scanning module of agent C to provide a real-time building interior model.

[0091] In practical implementation, the decision center performs feature extraction and threat source identification in parallel on the multimodal physical environment data continuously collected by the sensor array. Feature extraction extracts features such as edge contours, temperature gradients, motion vectors, and acoustic features from the multimodal data. Threat source identification identifies the extracted features based on a predefined classification model, outputting the type and location coordinates of potential threats. The predefined classification model on which threat source identification is based is specifically implemented as an offline configured feature processing and decision module. This module receives standardized feature vectors extracted from the multimodal data, matches and discriminates feature patterns through built-in classification logic, and outputs the category identifier of the potential threat and its precise coordinates in the environment. The predefined nature of this model means that it does not require dynamic adjustment during system operation and can be directly integrated into the joint situation assessment process, ensuring the real-time generation efficiency of the quantitative description of the environmental situation. Combining the spatial relative relationships and functional dependencies between agents and the threat source identification results, an environmental comprehensive complexity index is calculated. The calculation of the environmental comprehensive complexity index considers the complexity of internal cluster collaboration and the dynamics of the external environment, and its calculation formula is expressed as:

[0092]

[0093] in: This represents the overall environmental complexity index. , , The preset weighting coefficients satisfy the following conditions: , The normalized value representing the standard deviation of the spatial distance between all pairs of agents within the cluster. This represents the normalized value indicating the connectivity of nodes in a functional dependency graph. This represents the normalized value of the number of independent threat sources identified by the threat source identification module per unit time. In the area monitoring example scenario, the calculated value is... The value is 0.35, calculated in the fire rescue example scenario. The value is 0.72.

[0094] Optionally, a quantitative description of the environmental situation is generated based on the comprehensive environmental complexity index and task constraints. This quantitative description includes a map of traversable areas, a dynamic obstacle distribution field, and a heatmap of key task points. The traversable area map generates a binary mesh map based on obstacle information from multimodal physical environment data. The dynamic obstacle distribution field is labeled with the movement trends of moving obstacles in vector field form. The heatmap of key task points assigns probability weights to different locations within the operational area according to the task objectives and constraints. In the area monitoring example scenario, the quantitative description of the environmental situation shows a vast traversable area, a small number of moving patrol vehicles as dynamic obstacles, and several entrances and exits as high-heat task key points. In the fire rescue example scenario, the quantitative description of the environmental situation shows a limited traversable area divided by flames and collapsed debris, a spreading fire line as dynamic obstacles, and life signal detection points as high-heat task key points.

[0095] In some embodiments, the decision center determines the appropriate collaborative operation mode for the multi-agent cluster based on the generated quantitative description of the environmental situation. It reads the comprehensive environmental complexity index from the quantitative description and compares it with several preset complexity threshold ranges. These preset complexity threshold ranges are, for example, defined as a low complexity range [0, 0.3), a medium complexity range [0.3, 0.7), and a high complexity range [0.7, 1.0]. Based on the comparison results, the corresponding collaborative operation mode is selected from a predefined collaborative operation mode library, which includes a tight formation mode, a regional distributed search mode, a hierarchical command mode, and a dynamic alliance mode. In the regional monitoring example scenario, the comprehensive environmental complexity index is 0.35, falling within the medium complexity range, triggering the selection of the regional distributed search mode. In the fire rescue example scenario, the comprehensive environmental complexity index is 0.72, falling within the high complexity range, triggering the selection of the dynamic alliance mode.

[0096] It is understandable that the selection of the collaborative operation mode determines the granularity and principles of subsequent resource allocation and task planning. For example, the tight formation mode requires resource allocation to be carried out on a whole-unit basis, and the task planning path is highly consistent; the regional decentralized search mode allows resources to be divided by region, and the task planning path can be independent; the hierarchical command mode introduces a master-slave decision-making structure, and the resource allocation is hierarchical; the dynamic alliance mode supports agents to temporarily form teams based on task requirements, and the resource allocation and task planning are the most flexible.

[0097] Example 3: Based on the cooperative operation mode, analyze the individual capabilities and real-time task load of each agent in the multi-agent cluster. This step is implemented as follows: Determine the set of agent capability dimensions to be evaluated according to the selected cooperative operation mode. Query the capability model library of each agent to obtain its static capability parameters under the capability dimension set. Simultaneously monitor the list of subtasks being executed or queued by each agent and estimate its real-time computing and physical resource utilization. Combine the static capability parameters with the real-time computing and physical resource utilization to calculate the immediate available capability vector of each agent. The immediate available capability vector represents the agent's potential and efficiency in undertaking new tasks at the current moment. Integrate the results of individual capability and real-time task load analysis to construct a distributed task knowledge graph for the multi-agent cluster. This step is implemented as follows: Decompose the task objective into multiple levels of subtask nodes. Associate each subtask node with the capability dimensions and resource requirements required to complete the subtask. Map each agent to an execution entity node in the graph and use its immediate available capability vector as an attribute of the execution entity node. Weighted virtual connections are established between subtask nodes and execution entity nodes based on capability matching and resource sufficiency. The distributed task knowledge graph dynamically reflects the matching relationship network between task structure, resource requirements, and agent capabilities.

[0098] In practical implementation, the decision-making center determines the set of intelligent agent capability dimensions to be evaluated based on the selected collaborative operation mode. In the regional monitoring example scenario, the selected collaborative operation mode is the regional distributed search mode, and the determined capability dimension set includes image recognition resolution, movement speed, communication bandwidth, and battery life. In the fire rescue example scenario, the selected collaborative operation mode is the dynamic alliance mode, and the determined capability dimension set includes thermal detection range, demolition tool power, load-bearing capacity, and environmental tolerance level.

[0099] The system queries the capability model library for each agent to obtain its static capability parameters across the capability dimension set. The agent capability model library is a pre-entered database recording the nominal performance values ​​of each agent across various capability dimensions. In the area monitoring example scenario, agent A's static capability parameters are: image recognition resolution 4K, movement speed 5m / s, communication bandwidth 100Mbps, and battery life 120 minutes; agent B's static capability parameters are: image recognition resolution 1080P, movement speed 3m / s, communication bandwidth 500Mbps, and battery life 90 minutes. In the fire rescue example scenario, agent C's static capability parameters are: thermal detection range 50 meters, demolition tool power 500W, load capacity 20kg, and environmental tolerance level A; agent D's static capability parameters are: thermal detection range 30 meters, demolition tool power 1500W, load capacity 50kg, and environmental tolerance level B.

[0100] In some embodiments, the decision center simultaneously monitors the list of subtasks being executed or queued by each agent, estimating its real-time computing and physical resource utilization. Real-time computing resource utilization is obtained by reading the agent's CPU usage and memory consumption, while physical resource utilization is obtained by assessing the load cycles of actuators such as robotic arms and the energy consumption rate of moving components. In the area monitoring example scenario, agent A's subtask list includes "Patrol Grid 1," with a computing resource utilization of 60% and a physical resource utilization of 40%; agent B's subtask list includes "Data Relay" and "Abnormal Target Tracking," with a computing resource utilization of 85% and a physical resource utilization of 70%. In the fire rescue example scenario, agent C's subtask list includes "Heat Source Scanning," with a computing resource utilization of 90% and a physical resource utilization of 65%; agent D's subtask list is empty, with a computing resource utilization of 30% and a physical resource utilization of 10%.

[0101] In practical implementation, static capability parameters are combined with real-time computing resource utilization and physical resource utilization to calculate the instantaneous available capability vector for each agent. The calculation process of the instantaneous available capability vector involves independent attenuation correction for each capability dimension, and its calculation formula is expressed as follows:

[0102]

[0103] in: Represents the vector of capabilities that are available immediately. This represents a vector of static capability parameters retrieved from the capability model library. This represents element-wise multiplication of vectors. This represents a scalar value indicating real-time calculation of resource utilization. A scalar representing the physical resource utilization rate. and These are the preset attenuation coefficients for the impact of computing resources and physical resources on capabilities. In the area monitoring example scenario, the calculated immediate available capability vector of agent A is [2.0, 4.2, 60, 96], and the immediate available capability vector of agent B is [0.9, 1.8, 75, 54]. In the fire rescue example scenario, the immediate available capability vector of agent C is [7.5, 175, 7, 0.35A], and the immediate available capability vector of agent D is [27, 1275, 45, 0.8B].

[0104] Optionally, the immediate available capability vector represents the agent's potential and efficiency in undertaking new tasks at the current moment. The lower the value of each dimension in the vector, the smaller the agent's current available margin or the more severe the performance degradation in that capability dimension.

[0105] In practical implementation, the results of individual capabilities and real-time task load analysis are integrated to construct a distributed task knowledge graph for multi-agent clusters. The task objective is used as the root node, decomposing it into multiple levels of sub-task nodes. In the area monitoring example scenario, the task objective "complete area monitoring" is decomposed into sub-task nodes such as "patrol zone A," "patrol zone B," "abnormal target identification," and "data feedback." In the fire rescue example scenario, the task objective "locate and rescue trapped personnel" is decomposed into sub-task nodes such as "building structure scanning," "vital sign detection," "obstacle breaching," "opening rescue channels," and "casualty transfer." Each sub-task node is associated with the capability dimensions and resource requirements needed to complete the sub-task. These associations are appended to the sub-task nodes in the form of an attribute list. For example, the capability dimensions associated with the "obstacle breaching" sub-task node are "breaching tool power" and "load-bearing capacity," and the resource requirement is "continuous working time greater than 10 minutes."

[0106] In some embodiments, the decision center maps each agent to an execution entity node in a distributed task knowledge graph, and uses its readily available capability vector as an attribute of the execution entity node. In the distributed task knowledge graph of the area monitoring example scenario, it includes an execution entity node named "Agent A" with the attribute vector [2.0, 4.2, 60, 96]; and an execution entity node named "Agent B" with the attribute vector [0.9, 1.8, 75, 54]. In the distributed task knowledge graph of the fire rescue example scenario, it includes an execution entity node named "Agent C" with the attribute vector [7.5, 175, 7, 0.35A]; and an execution entity node named "Agent D" with the attribute vector [27, 1275, 45, 0.8B].

[0107] Weighted virtual connections are established between subtask nodes and execution entity nodes based on capability matching and resource sufficiency. Capability matching is calculated by comparing the values ​​of the capability dimensions required by the subtask node with the corresponding dimensions of the execution entity node's immediately available capability vector. Resource sufficiency is calculated by determining whether the execution entity node's remaining battery life and other resources meet the resource requirements of the subtask node. The connection weight combines the evaluation results of capability matching and resource sufficiency; a higher weight indicates better suitability for the agent to execute the subtask.

[0108] It can be understood that a distributed task knowledge graph dynamically reflects the matching network between task structure, resource requirements, and agent capabilities. The nodes and connection weights of the knowledge graph are dynamically updated as the agent's state, task progress, and environmental information change.

[0109] See Figure 4This is a radar chart for assessing the capabilities of an agent. It's a multi-dimensional radar chart used to compare the agent's "static capability parameters" with its "immediately available capabilities." Immediately available capabilities (orange) are significantly lower than static capability parameters (blue), indicating that the agent is currently consuming resources (such as performing tasks), leading to a decrease in actual capability. The degree of decrease is relatively consistent across dimensions, reflecting the general impact of resource consumption on the agent's multi-dimensional capabilities. This chart is commonly used in the individual capability analysis phase of multi-agent autonomous decision-making to help determine the agent's potential and efficiency in undertaking new tasks.

[0110] Example 4: A deep reinforcement learning network is driven by a distributed task knowledge graph to couple resource allocation and task path computation within a cluster. This step is implemented as follows: The current state of the distributed task knowledge graph is encoded as the input feature of the deep reinforcement learning network. The deep reinforcement learning network consists of a value network for evaluating the global state value and a policy network for generating specific allocation strategies. Based on the input features and considering the constraints of cooperative operation modes, the policy network outputs two suggestions: first, the probability distribution for assigning execution entity nodes to sub-task nodes; and second, a preliminary task execution path for the assigned agents. The value network then evaluates the long-term benefits of the allocation and path combinations output by the policy network and guides the policy network to optimize its output through network parameter updates.

[0111] Based on the resource allocation and task paths obtained from coupled computation, a preliminary behavioral policy is generated for each agent in the multi-agent cluster. This step is implemented as follows: The allocation results output by the policy network are parsed to determine the set of subtasks assigned to each agent. For each agent and its assigned set of subtasks, the preliminary task execution path planned by the policy network is extracted. The set of subtasks and the preliminary task execution path are integrated and encapsulated into a preliminary behavioral policy package specific to the agent. The preliminary behavioral policy package contains the action sequence, target point, and expected resource consumption that the agent plans to execute over a period of time.

[0112] In practical implementation, the current state of the distributed task knowledge graph is encoded as the input features of the deep reinforcement learning network. The encoding process converts the node attributes, connection weights, and topological relationships in the graph into fixed-dimensional feature vectors. In the regional monitoring example scenario, the input features include the sub-task nodes of "Patrol A" and "Data Feedback" and their required image recognition capability weights, as well as the execution entity nodes of agents A and B and their immediately available capability vectors [2.0, 4.2, 60, 96] and [0.9, 1.8, 75, 54], and also include the matching degree connection weights between the sub-tasks and the execution entities. In the fire rescue example scenario, the input features include the sub-task nodes of "Breaking Through Obstacles" and "Transferring the Injured" and their required capability weights, as well as the execution entity nodes of agents C and D and their immediately available capability vectors [7.5, 175, 7, 0.35A] and [27, 1275, 45, 0.8B] and connection information.

[0113] Deep reinforcement learning networks consist of a value network for evaluating the global state value and a policy network for generating specific allocation strategies. Based on encoded input features, the policy network, considering cooperative operation mode constraints, outputs two suggestions: first, a probability distribution for assigning execution entity nodes to sub-task nodes; and second, a preliminary task execution path for the assigned agents. Cooperative operation mode constraints are input to the policy network as additional condition vectors; for example, the regional distributed search mode corresponds to a constraint vector encouraging load balancing, while the dynamic alliance mode corresponds to a constraint vector encouraging complementary capabilities. The value network then evaluates the long-term benefits of the allocation and path combinations output by the policy network and guides the policy network to optimize its output through network parameter updates. The long-term benefit evaluation is calculated based on preset indicators such as task completion, resource consumption, and time cost.

[0114] In some embodiments, the probability distribution and path suggestions output by the policy network differ across different scenarios. In the area monitoring example scenario, the policy network outputs a probability of 0.8 for assigning agent A to the "Patrol Zone A" subtask and a probability of 0.2 for assigning agent B, and plans a patrol path for agent A passing through path points P1, P2, and P3. In the fire rescue example scenario, the policy network outputs a probability of 0.9 for assigning agent D to the "Breach Obstacles" subtask and a probability of 0.1 for assigning agent C, and plans a path for agent D directly from the entrance to the target point (Xd, Yd). Refer to Table 1, which shows a hypothetical probability distribution segment of the policy network output.

[0115] Table 1: Subtask-Execution Entity Assignment Probability Table

[0116] Subtask Node Execution Entity Node Assignment probability Key points of the initial task execution path Demolishing obstacles Agent D 0.90 Entrance -> Corridor L1 -> Target Point (Xd, Yd) Demolishing obstacles Agent C 0.10 Entrance -> Corridor L2 -> Target Point (Xd, Yd) Wounded personnel transfer Agent D 0.65 Target point (Xd, Yd) -> Safe point S Wounded personnel transfer Agent C 0.35 Target point (Xd, Yd) -> Safe point S

[0117] The policy evaluation function upon which the policy network outputs the allocation probability aims to maximize the global long-term return. Its objective is to optimize the coupling result of resource allocation and path planning. This objective function is expressed as:

[0118]

[0119] in: Indicates the expected cumulative return. Represents the expectation operator. Indicates the decision-making sequence. Indicates the discount factor and , Indicates in step sequence Positive rewards earned as a result of mission progress Indicates in step sequence Costs incurred due to resource consumption It is a regularization coefficient used to balance task rewards and resource consumption costs. The policy network adjusts its parameters through training to maximize... value.

[0120] In practical implementation, based on the resource allocation and task paths obtained from coupled computation, a preliminary behavioral policy is generated for each agent in the multi-agent cluster. The allocation results output by the policy network are analyzed to determine the set of sub-tasks assigned to each agent. In the area monitoring example scenario, the analysis results determine that agent A is assigned to the sub-tasks of "Patrol Zone A" and "Abnormal Target Identification," while agent B is assigned to the sub-task of "Data Feedback." In the fire rescue example scenario, the analysis results determine that agent D is assigned to the sub-tasks of "Obstacle Clearance" and "Casualty Transfer," while agent C is assigned to the sub-task of "Vitality Detection."

[0121] For each agent and its assigned set of subtasks, a policy network is extracted to plan a preliminary task execution path. The preliminary task execution path consists of a series of ordered spatial coordinates and the expected action at each point. The subtask set and the preliminary task execution path are integrated and encapsulated into a preliminary behavior policy package specific to the agent. The preliminary behavior policy package contains the action sequence, target points, and expected resource consumption that the agent plans to execute over a period of time. In the area monitoring example scenario, agent A's preliminary behavior policy package contains the action sequence "move to P1 -> perform scan -> move to P2 -> perform scan -> move to P3 -> perform identification," with target points P1, P2, and P3, and an expected resource consumption of 15 units of energy. In the fire rescue example scenario, agent D's preliminary behavior policy package contains the action sequence "move to (Xd, Yd) -> perform demolition -> prepare to carry -> move to safe point S," with target points (Xd, Yd) and S, and an expected resource consumption of 40 units of energy and 10 units of tool wear.

[0122] Example 5: Cluster-level conflict detection and compatibility verification of preliminary behavioral strategies. This step is implemented as follows: Collect preliminary behavioral strategy packages from all agents. Establish a virtual cluster behavior simulation environment, load the preliminary behavioral strategy packages, and perform time-series simulation. During the simulation, detect whether spatial trajectory conflicts, resource usage conflicts, or target logic conflicts occur between agents. Record all detected conflict types, agent identifiers involved, and the time window in which the conflict occurred. Generate a conflict detection report, which is used to guide the subsequent negotiation and adjustment process.

[0123] Based on the results of conflict detection and compatibility verification, iterative negotiation and adjustment are performed on the initial behavioral strategies with conflicts. This step is implemented as follows: The conflict detection report is read to locate the conflicting agent combinations and the content of the conflict. A rule-based or deep reinforcement learning network-based negotiation adjuster is initiated. The negotiation adjuster takes the initial behavioral strategies of the conflicting agents and the content of the conflict as input, and fine-tunes the behavioral strategy parameters of the conflicting agents while ensuring the overall task progress. The fine-tuned behavioral strategy is then reinjected into a virtual cluster behavior simulation environment for verification. If the conflict is still not completely resolved, a new round of iterative adjustment is initiated until all conflicts are resolved or reduced to an acceptable range. At this point, the final adjusted behavioral strategy is issued as an executable sequence of autonomous action instructions.

[0124] In specific implementation, iterative negotiation and adjustments are made based on the results. Preliminary behavioral policy packages for all agents are collected. These packages contain the action sequences, target points, and expected resource consumption that the agents plan to execute over a period of time. In the area monitoring example scenario, the collected preliminary behavioral policy package for agent A includes an action sequence of moving along the path point sequence P1->P2->P3 and performing a scan; for agent B, it includes an action sequence of stationing in area Z and continuously forwarding data; and for agent C, it includes an action sequence of performing mobile reconnaissance near area Z. In the fire rescue example scenario, the collected preliminary behavioral policy package for agent D includes an action sequence of moving to coordinates (Xd, Yd) and performing demolition; for agent E, it includes an action sequence of moving to coordinates (Xe, Ye) and preparing for transport; and for agent F, it includes an action sequence of moving to coordinates (Xf, Yf) and performing environmental monitoring.

[0125] A virtual cluster behavior simulation environment is established, loading all collected preliminary behavior policy packages and performing time-series simulations. Based on the action sequences, target points, and resource consumption rates defined in each preliminary behavior policy package, the virtual environment simulates the collaborative operation of all agents on a unified timeline. During the simulation, it detects whether spatial trajectory conflicts, resource usage conflicts, or target logic conflicts occur between agents. Spatial trajectory conflict detection is achieved by calculating whether the position coordinates of different agents at the same time are less than a safe distance threshold. Resource usage conflict detection is achieved by checking whether multiple agents plan to exclusively use the same physical resource at the same time. Target logic conflict detection is achieved by analyzing whether the task objectives of different agents are contradictory.

[0126] In some embodiments, the simulation and detection process generates specific conflict data. In the simulation of a regional monitoring example scenario, it was detected that agents B and C planned to simultaneously occupy the same sub-region of region Z within time windows T1 to T2, resulting in spatial trajectory conflicts and communication bandwidth resource usage conflicts. In the simulation of a fire rescue example scenario, it was detected that agents D and E planned to simultaneously pass through the width-limited L channel at time t3, resulting in spatial trajectory conflicts. Simultaneously, the monitoring path of agent F at time t4 overlapped with the safety boundary of agent D's demolition operation. All detected conflict types, involved agent identifiers, and the time windows in which the conflicts occurred were recorded, generating a structured conflict detection report to guide subsequent negotiation and adjustment processes.

[0127] In practical implementation, based on the results of conflict detection and compatibility verification, the initial behavioral strategies with conflicts are iteratively negotiated and adjusted. The conflict detection report is read to locate the conflicting agent combinations and conflict content. A rule-based or deep reinforcement learning network-based negotiation adjuster is initiated. In the fire rescue example scenario, a deep reinforcement learning network-based negotiation adjuster is initiated for multiple detected conflicts. The negotiation adjuster takes the initial behavioral strategies and conflict content of conflicting agents D, E, and F as input. While ensuring the overall task progress, the negotiation adjuster fine-tunes the behavioral strategy parameters of the conflicting parties. For the deep reinforcement learning network-based negotiation adjuster, its network structure takes the conflict state and current behavioral strategy as input and outputs adjustment suggestions for the behavioral strategy parameters. The adjustment process aims to find a strategy that minimizes the overall conflict and policy deviation. Its comprehensive evaluation value function is expressed as:

[0128]

[0129] in: This represents the overall evaluation value. This indicates the total number of collisions detected. Indicates the first An assessment value for the severity of the conflict. Indicates the first A weighting coefficient for the severity of the conflict. This represents the total offset vector of all agents' original behavioral policy parameters due to the adjustment. This represents the norm of the offset vector. It is a preset strategy parameter reference value. It is a penalty coefficient that measures the importance of policy deviation. Deep reinforcement learning networks learn to maximize this through training. Adjustment strategy.

[0130] The fine-tuned behavioral strategy is re-injected into the virtual cluster behavior simulation environment for verification. In the fire rescue example scenario, the adjustment scheme output by the negotiation adjuster is loaded into the simulation environment for re-simulation. If the conflicts are still not completely resolved, a new round of iterative adjustment is initiated until all conflicts are resolved or reduced to an acceptable range. At this point, the final adjusted behavioral strategy is issued as an executable sequence of autonomous action instructions. In some embodiments, after multiple rounds of iterative adjustment, the conflict detection report shows that all conflicts have been eliminated or their severity is below the threshold. The negotiation adjuster outputs the final behavioral strategy parameters and encapsulates them into an instruction sequence. Optionally, the negotiation adjustment process can be configured to set a maximum number of iterations to avoid infinite loops.

[0131] See Figure 5 This is a dual-indicator trend chart of the multi-agent decision-making iteration process. The dual vertical axis line chart shows the relationship between the "number of conflicts" and the "overall evaluation value" during the iteration process. After 2-4 iterations: the number of conflicts surges and then falls back, corresponding to a rapid increase in the overall evaluation value, reflecting the "trial and error" process in strategy adjustment; after 5 iterations: the number of conflicts tends to stabilize (maintaining at a low level), while the overall evaluation value continues to rise, indicating that the strategy has entered the convergence optimization stage; the two show a negative correlation: as conflicts decrease, the overall quality of the decision-making strategy gradually improves. This chart is often used in the conflict negotiation and iterative adjustment stages of multi-agent autonomous decision-making to help evaluate the effectiveness and convergence of strategy optimization.

[0132] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0133] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-agent autonomous decision-making method based on deep reinforcement learning, characterized in that, Includes the following steps: It senses the external environment and task instructions, triggering a synchronous detection of the initial collaborative state of the multi-agent cluster; Based on the initial cooperative state obtained from the synchronous detection, a joint situation assessment is performed on the dynamic environment in which the multi-agent cluster is located, generating a quantitative description of the environmental situation. Based on the quantitative description of the environmental situation, determine the cooperative operation mode that the multi-agent cluster should currently adopt; Based on the cooperative operation mode, the individual capabilities and real-time task load of each agent in the multi-agent cluster are analyzed. By integrating the individual capabilities and the results of real-time task load analysis, a distributed task knowledge graph for the multi-agent cluster is constructed. Using the distributed task knowledge graph, a deep reinforcement learning network is driven to perform coupled computation of resource allocation and task paths within the cluster; Based on the resource allocation and task path obtained from the coupled calculation, a preliminary behavioral strategy is generated for each agent in the multi-agent cluster. Cluster-level conflict detection and compatibility verification are performed on the preliminary behavior strategy. Based on the results of the conflict detection and compatibility verification, the initial behavioral strategies that have conflicts are iteratively negotiated and adjusted. Based on the results of iterative negotiation and adjustment, an executable sequence of autonomous action instructions is issued to each agent.

2. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 1, characterized in that, The process of sensing the external environment and task instructions to trigger the synchronous detection of the initial collaborative state of the multi-agent cluster includes the following steps: Continuously collect multimodal data of the physical environment through a deployed sensor array; Simultaneously, it receives top-level mission instruction packets from the command system; Parse the top-level task instruction package to extract task objectives, constraints, and priority information; Based on the multimodal data and the task objectives, constraints, and priority information, a synchronous detection trigger signal is generated; In response to the synchronization detection trigger signal, a status query request is broadcast to all agents in the multi-agent cluster; Collect local state information from each agent, including location, energy level, health status of functional modules, and current load.

3. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 2, characterized in that, The steps include: Based on the initial cooperative state obtained from the synchronous detection, performing a joint situation assessment of the dynamic environment in which the multi-agent cluster exists, and generating a quantitative description of the environmental situation; The local state information of all the agents collected is spatiotemporally aligned and fused to form a global state view of the cluster; Analyze the global state view of the cluster to identify the spatial relationships and functional dependencies between agents; In parallel, feature extraction and threat source identification are performed on continuously collected multimodal data of the physical environment; The comprehensive environmental complexity index is calculated by combining the spatial relative relationships and functional dependencies between the intelligent agents and the threat source identification results. Based on the comprehensive environmental complexity index and task constraints, a quantitative description of the environmental situation is generated. The quantitative description of the environmental situation includes a map of passable areas, a dynamic obstacle distribution field, and a heat map of key task points.

4. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 3, characterized in that, The step of determining the appropriate cooperative operation mode for the multi-agent cluster based on the quantitative description of the environmental situation includes: Read the comprehensive environmental complexity index from the quantitative description of the environmental situation; The overall environmental complexity index is compared with multiple preset complexity threshold ranges; Based on the comparison results, the corresponding collaborative operation mode is selected from the predefined collaborative operation mode library, which includes tight formation mode, regional decentralized search mode, hierarchical command mode and dynamic alliance mode. The selection of the collaborative operation mode determines the granularity and principles of subsequent resource allocation and task planning.

5. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 4, characterized in that, Based on the cooperative operation mode, the individual capabilities and real-time task load analysis of each agent in the multi-agent cluster includes the following steps: Based on the selected cooperative operation mode, determine the set of agent capability dimensions that need to be evaluated; Query the capability model library for each agent to obtain its static capability parameters under the set of capability dimensions; At the same time, monitor the list of subtasks that each agent is executing or queuing, and estimate its real-time computing and physical resource utilization. The static capability parameters are combined with the real-time computing and physical resource utilization rates to calculate the instantaneous available capability vector for each agent. The instantaneous availability vector represents the agent's potential and efficiency in undertaking new tasks at the current moment.

6. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 5, characterized in that, The steps for integrating the individual capabilities and the results of real-time task load analysis to construct a distributed task knowledge graph for the multi-agent cluster include: The task objective is used as the root node, and multiple levels of sub-task nodes are decomposed into it. Associate each subtask node with the capability dimensions and resource requirements needed to complete the subtask; Each agent is mapped to an execution entity node in the graph, and its readily available capability vector is used as an attribute of the execution entity node; Weighted virtual connections are established between subtask nodes and execution entity nodes based on capability matching and resource sufficiency. The distributed task knowledge graph dynamically reflects the matching relationship network between task structure, resource requirements, and agent capabilities.

7. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 6, characterized in that, The step of using the distributed task knowledge graph to drive a deep reinforcement learning network to perform coupled computation of resource allocation and task paths within the cluster includes: The current state of the distributed task knowledge graph is encoded as the input feature of a deep reinforcement learning network; The deep reinforcement learning network includes a value network for evaluating the value of the global state and a policy network for generating specific allocation policies. Based on the input features and considering the constraints of the cooperative operation mode, the policy network outputs two suggestions: first, the probability distribution of assigning execution entity nodes to sub-task nodes; and second, the preliminary task execution path for the assigned agents. The value network then evaluates the long-term returns of the allocation and path combination of the strategy network output, and guides the strategy network to optimize its output through network parameter updates.

8. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 7, characterized in that, The step of generating a preliminary behavioral strategy for each agent in the multi-agent cluster based on the resource allocation and task path obtained from the coupled calculation includes: Analyze the allocation results output by the policy network to determine the set of subtasks assigned to each agent; For each agent and its assigned set of subtasks, the policy network is extracted as the preliminary task execution path it plans; The subtask set is integrated with the preliminary task execution path and encapsulated into a preliminary behavior strategy package specific to the intelligent agent; The preliminary behavioral strategy includes the sequence of actions, target points, and expected resource consumption that the agent plans to execute over a period of time.

9. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 8, characterized in that, The steps of performing cluster-level conflict detection and compatibility verification on the preliminary behavior strategy include: Collect preliminary behavioral policy packages from all agents; Establish a virtual cluster behavior simulation environment, load the preliminary behavior strategy package, and perform time-series simulation. During the simulation, it is detected whether there are spatial trajectory conflicts, resource usage conflicts, or target logic conflicts between intelligent agents; Record all detected conflict types, the identities of the agents involved, and the time window in which the conflict occurred; A conflict detection report is generated, which is used to guide the subsequent negotiation and adjustment process.

10. The multi-agent autonomous decision-making method based on deep reinforcement learning according to claim 9, characterized in that, Based on the results of the conflict detection and compatibility verification, the preliminary behavioral strategy with conflicts is iteratively negotiated and adjusted, including the following steps: Read the conflict detection report to locate the conflicting agent combinations and the content of the conflict; Initiate a negotiation tuner based on rules or deep reinforcement learning networks; The negotiation adjuster takes the initial behavioral strategies and conflict content of the conflicting agents as input, and fine-tunes the behavioral strategy parameters of the conflicting parties while ensuring the progress of the global task. The fine-tuned behavior strategy was re-injected into the virtual cluster behavior simulation environment for verification. If the conflict is not fully resolved, a new round of iterative adjustments will be initiated until all conflicts are resolved or reduced to an acceptable level. At this point, the final adjusted behavioral strategy will be issued as an executable sequence of autonomous action instructions.

Images