Multi-agent collaborative intelligent orchestration method and device based on incremental reinforcement learning

Abstract

This application provides a multi-agent collaborative intelligent orchestration method and apparatus based on incremental reinforcement learning, relating to the field of reinforcement learning. The method includes: constructing a task graph model based on initial state information and outputting task hierarchical dependencies; constructing a hierarchical incremental reinforcement learning model based on the task hierarchical dependencies and outputting a first agent policy and a first global policy; constructing a local reward mechanism based on the local task completion progress of each unmanned agent and incrementally updating the first agent policy to output a second agent policy; aggregating the update information of the second agent policy and modifying the first global policy to a second global policy for collaborative intelligent orchestration. This application solves the problem that issuing fixed sets of instructions to each unmanned agent in a cluster based on a pre-set task template leads to an imbalance in task load among agents, resulting in decreased collaboration efficiency and a significant reduction in unmanned search and rescue efficiency.

Description

Technical Field

[0001] This application relates to the field of reinforcement learning, and in particular to a multi-agent collaborative intelligent orchestration method and apparatus based on incremental reinforcement learning. Background Technology

[0002] In unmanned swarm operations such as disaster emergency rescue, complex terrain inspection, or joint sea and air search and rescue, each unmanned intelligent agent needs to work together in a dynamically changing environment to complete tasks such as search, identification, calibration, and rescue.

[0003] Existing technologies often use a centralized scheduling center to issue fixed sets of instructions to each unmanned agent in the cluster based on a pre-set task template in order to achieve initial task allocation and path planning. However, this method usually relies on static environmental assumptions and cannot reflect the impact of dynamic environmental changes, communication topology fluctuations, energy consumption and other factors on the agents in real time. Especially in operation scenarios with complex terrain, multiple obstacles or multiple coupled tasks, due to the lag in task status updates, frequent path conflicts and uneven resource allocation, there will be an imbalance in task load among the agents, which will lead to a decrease in the efficiency of cooperation among the agents and thus a significant reduction in the efficiency of unmanned search and rescue.

[0004] Therefore, there is an urgent need for a multi-agent collaborative intelligent orchestration method and device based on incremental reinforcement learning. Summary of the Invention

[0005] This application provides a multi-agent collaborative intelligent orchestration method and device based on incremental reinforcement learning, which solves the problem that issuing fixed sets of instructions to each unmanned agent in the cluster based on a task preset template will cause an imbalance in the task load among the agents, which in turn leads to a decrease in the collaboration efficiency among the agents and a significant reduction in the efficiency of unmanned search and rescue.

[0006] The first aspect of this application provides a multi-agent collaborative intelligent orchestration method based on incremental reinforcement learning. The method includes: acquiring initial state information of an unmanned swarm task environment, including task information and the running state information of each unmanned agent; constructing a task graph model based on the initial state information and outputting task hierarchical dependencies through the task graph model; constructing a hierarchical incremental reinforcement learning model based on the task hierarchical dependencies and outputting a first agent policy and a first global policy through the hierarchical incremental reinforcement learning model; constructing a local reward mechanism based on the local task completion progress of each unmanned agent and incrementally updating the first agent policy based on the local reward mechanism to output a second agent policy; aggregating the update information of the second agent policy and using an attention weighting mechanism to modify the first global policy into a second global policy; and synchronously distributing the second global policy to each unmanned agent for collaborative intelligent orchestration.

[0007] Optionally, initial state information of the unmanned swarm mission environment is obtained, specifically including: constructing mission information based on spatial geographic feature data, communication topology data, and resource distribution data in the unmanned swarm mission environment; the mission information includes mission type information, regional planning information, mission priority information, resource requirement information, and time constraint information; and obtaining operational status information through a multi-source collaborative sensing network, which includes positioning units, sensors, and communication nodes deployed in the unmanned swarm, and the operational status information includes spatial pose information, sensor status information, power level information, and communication topology information.

[0008] Optionally, a task graph model is constructed based on the initial state information, specifically including: constructing task objectives based on the initial state information, including rescue allocation objectives, resource allocation objectives, and collaborative operation objectives; dividing the global task into multiple sub-tasks according to the task process stages based on the task objectives, including the search stage, identification stage, calibration stage, and rescue stage; and constructing a task graph model for each sub-task, where nodes in the task graph model represent task states and edges represent state transition relationships.

[0009] Optionally, a hierarchical incremental reinforcement learning model is constructed based on task-level dependencies, and the first individual policy and the first global policy are output through the hierarchical incremental reinforcement learning model. Specifically, this includes: determining the mapping relationship between the global layer and the individual layer based on task-level dependencies, where the global layer describes the global task state of the unmanned swarm task environment, and the individual layer describes the local task state of each unmanned agent; in the individual layer, an individual policy network is constructed based on the running state information and the action set corresponding to each unmanned agent; the first individual policy is output through the individual policy network to execute each sub-task; in the global layer, a global policy network is constructed based on task information, task-level dependencies, and the execution progress information of each sub-task; the first global policy is output through the global policy network to coordinate the overall task execution order and resource scheduling order of the unmanned swarm.

[0010] Optionally, a local reward mechanism is constructed based on the local task completion progress of each unmanned intelligent agent, and the first agent policy is incrementally updated based on the local reward mechanism to output the second agent policy. Specifically, this includes: controlling each unmanned intelligent agent to execute the first agent policy and outputting the local task completion progress of the unmanned intelligent agent under preset constraints, including time constraints, path feasibility constraints, and task safety constraints; constructing a local reward function using the local task completion progress as feedback data; inputting the output value of the local reward function into the individual policy network to incrementally update the parameters of the first agent policy and outputting the updated and optimized second agent policy.

[0011] Optionally, the update information of the second agent's policy is aggregated, and the first global policy is modified into the second global policy using an attention weighting mechanism. Specifically, this includes: aggregating the update information of each second agent's policy and obtaining the parameter changes and task contribution indicators in the update information; calculating the attention weights of each unmanned agent based on the task contribution indicators, whereby the attention weights are used to characterize the relative importance of each unmanned agent in the global task; weightedly fusing the attention weights and parameter changes to generate a global policy correction amount; incrementally updating the parameters of the first global policy according to the global policy correction amount, and outputting the updated and optimized second global policy.

[0012] Optionally, the second global policy is synchronously distributed to each unmanned intelligent agent for collaborative intelligent orchestration, specifically including: synchronously distributing the second global policy to each unmanned intelligent agent through a cluster communication network, and constructing a policy instruction set based on the corresponding node of each unmanned intelligent agent in the task graph model; and controlling the unmanned intelligent agents to perform collaborative intelligent orchestration according to the policy instruction set.

[0013] A second aspect of this application provides a multi-agent collaborative intelligent orchestration device based on incremental reinforcement learning. The device includes an acquisition module and a processing module, wherein... The acquisition module is used to acquire the initial state information of the unmanned cluster task environment, including task information and the running state information of each unmanned intelligent agent; based on the initial state information, a task graph model is constructed, and the task hierarchical dependency relationship is output through the task graph model.

[0014] The processing module is used to construct a hierarchical incremental reinforcement learning model based on task hierarchy dependencies, and output the first agent policy and the first global policy through the hierarchical incremental reinforcement learning model; construct a local reward mechanism based on the local task completion progress of each unmanned agent, and incrementally update the first agent policy based on the local reward mechanism to output the second agent policy; aggregate the update information of the second agent policy, and use an attention weighting mechanism to correct the first global policy to the second global policy; and synchronously distribute the second global policy to each unmanned agent for collaborative intelligent orchestration.

[0015] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described above.

[0016] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, which is executed by a processor using the method described in any of the foregoing descriptions.

[0017] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Construct a task graph model based on initial state information and output task hierarchy dependencies; construct a hierarchical incremental reinforcement learning model based on task hierarchy dependencies and output the first agent policy and the first global policy; construct a local reward mechanism based on the local task completion progress of each unmanned agent and incrementally update the first agent policy to output the second agent policy; aggregate the update information of the second agent policy and modify the first global policy to the second global policy for collaborative intelligent orchestration. Thus, through the dynamic coupling of task hierarchy dependencies and hierarchical incremental reinforcement learning model, adaptive decision-making and multi-stage task collaboration of unmanned swarms in complex task environments are realized; the task execution strategies and resource allocation states of each unmanned agent are optimized in real time, thereby improving the overall task completion efficiency and environmental adaptability of the swarm, and realizing intelligent collaboration and efficient orchestration of unmanned swarms in dynamic environments.

[0018] 2. By forming a hierarchical structure and execution dependencies for multi-stage tasks, the order and triggering conditions of each sub-task are clarified, providing task logic constraints and state transition basis for subsequent hierarchical incremental reinforcement learning models, and realizing orderly collaboration and dynamic scheduling of unmanned swarms in different task stages.

[0019] 3. Aggregate the update information of each second agent's strategy and obtain the parameter changes and task contribution indicators from the update information; calculate the attention weight of each unmanned agent based on the task contribution indicators. The attention weight is used to characterize the relative importance of each unmanned agent in the global task; weight and fuse the attention weight and parameter changes to generate the global strategy correction amount; incrementally update the parameters of the first global strategy according to the global strategy correction amount, and output the updated and optimized second global strategy. This enables the global strategy to adaptively respond to the learning state and task contribution of each unmanned agent, so that the global strategy can dynamically reflect the execution differences and resource consumption of the cluster at different task stages, thereby improving the global collaborative efficiency and strategy convergence stability of the unmanned cluster in complex task environments. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a multi-agent collaborative intelligent orchestration method based on incremental reinforcement learning provided in an embodiment of this application. Figure 2 This is a schematic diagram of a multi-agent collaborative intelligent orchestration device based on incremental reinforcement learning provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0021] Explanation of reference numerals in the attached figures: 21. Acquisition module; 22. Processing module; 301. Processor; 302. Communication bus; 303. User interface; 304. Network interface; 305. Memory. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0023] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0024] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0025] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0026] Please refer to Figure 1 The flowchart illustrates a multi-agent collaborative intelligent orchestration method based on incremental reinforcement learning provided in this application embodiment. The flowchart mainly includes the following steps: S101 to S106.

[0027] Step S101: Obtain the initial state information of the unmanned cluster task environment.

[0028] Specifically, in unmanned swarm operations such as disaster emergency rescue, complex terrain inspection, or joint sea and air search and rescue, each unmanned intelligent agent needs to collaborate to complete tasks such as search, identification, calibration, and rescue in a dynamically changing environment. Existing technologies mostly use a centralized dispatch center to issue fixed sets of instructions to each unmanned intelligent agent in the swarm based on a pre-set task template to achieve initial task allocation and path planning. However, such methods usually rely on static environmental assumptions and cannot reflect spatial geographical changes, communication topology fluctuations, and differences in energy status in real time, resulting in insufficient coordination, uneven resource allocation, and decreased execution efficiency during task execution.

[0029] Therefore, this application proposes to obtain the initial state information of the unmanned swarm task environment before task execution. The initial state information includes task information and the operating state information of each unmanned intelligent agent, which is used to provide dynamic and accurate environmental and individual basic data support for subsequent task graph model construction and hierarchical incremental reinforcement learning.

[0030] In one possible implementation, step S101 further includes: acquiring initial state information of the unmanned swarm task environment, specifically including: constructing task information based on spatial geographic feature data, communication topology data, and resource distribution data in the unmanned swarm task environment, the task information including task type information, regional planning information, task priority information, resource requirement information, and time constraint information; acquiring operational status information through a multi-source collaborative sensing network, the multi-source collaborative sensing network including positioning units, sensors, and communication nodes deployed in the unmanned swarm, the operational status information including spatial pose information, sensor status information, power level information, and communication topology information.

[0031] Specifically, data initialization is first performed based on the environmental type of the unmanned swarm mission. When the unmanned swarm is deployed in scenarios such as earthquake disaster area search and rescue, search for missing targets at sea, or inspection of high-risk areas, spatial geographic feature data, including terrain elevation differences, obstacle distribution, and surface reflectivity, is acquired through geographic information acquisition devices. Simultaneously, communication topology data is formed through communication node scanning to describe the real-time communication link quality and node connectivity between each unmanned intelligent agent. Then, resource distribution data is extracted through the task planning module to characterize the locations of available energy supply points, sensor relay nodes, and auxiliary equipment on site. The above data are combined to construct task information, where task type information is used to distinguish between search, identification, calibration, or rescue tasks; area planning information is used to define the spatial boundaries of each sub-task; task priority information is used to determine the execution order of different task stages; resource requirement information is used to constrain the minimum configuration of energy and sensor resources; and time constraint information is used to limit the maximum allowable execution time of each stage.

[0032] Subsequently, operational status information is acquired through positioning units, sensors, and communication nodes deployed in the unmanned swarm. The positioning units output spatial pose information to reflect the real-time position and attitude changes of the unmanned intelligent agent; the sensors output sensor status information to describe the working status and reliability of the perception units; the energy monitoring unit collects power level information to reflect sustainable operation capability; and the communication nodes monitor and output communication topology information to indicate current network connectivity and channel quality. After collecting the above data, the time reference and spatial coordinate system are uniformly processed to obtain initial state information of the unmanned swarm task environment, which can be used for subsequent task graph model construction and reinforcement learning strategy training.

[0033] Step S102: Construct a task graph model based on the initial state information, and output the task hierarchy dependency relationship through the task graph model.

[0034] Specifically, the task graph model uses the task objective as the core node and the task stages and their execution order as the hierarchical structure. Directed edges between nodes represent the dependencies and state transition logic between tasks. First, task stages are determined based on task type, region planning, and priority information, such as search, identification, calibration, and rescue. Then, the executing entity and resource constraints for each stage are determined based on the operational status information and reachability analysis of each unmanned intelligent agent. Furthermore, the dependencies, triggering conditions, and data flow relationships between tasks are mapped to hierarchical connections in the task graph model.

[0035] In one possible implementation, step S102 further includes: constructing a task objective based on initial state information, the task objective including rescue allocation objective, resource allocation objective and collaborative operation objective; dividing the global task into multiple sub-tasks according to the task process stages based on the task objective, the task process stages including search stage, identification stage, calibration stage and rescue stage; constructing a task graph model for each sub-task, the nodes of the task graph model representing the task state and the edges of the task graph model representing the state transition relationship.

[0036] Specifically, the task objectives of the unmanned swarm are first constructed by combining the task type, regional planning, and resource requirements in the task information. These objectives include rescue allocation objectives, resource allocation objectives, and collaborative operation objectives. The rescue allocation objective determines the division of rescue tasks and priority response order among different unmanned intelligent agents in disaster relief or emergency search and rescue scenarios. For example, drones are responsible for high-altitude area searches, unmanned vehicles are responsible for ground rescue channel detection, and underwater robots are responsible for target location and retrieval. The resource allocation objective dynamically plans the optimal allocation of energy and computing resources among the unmanned intelligent agents based on energy levels, remaining power, and communication load. The collaborative operation objective ensures that multiple machines avoid path conflicts, reduce perception redundancy, and achieve time coordination during the same task phase, thereby improving the overall task execution efficiency of the swarm.

[0037] After defining the aforementioned task objectives, the global task is divided into four phases based on task logic: search, identification, calibration, and rescue. The search phase rapidly covers the task area to discover potential targets; the identification phase uses visual or multimodal perception to identify target type and status; the calibration phase uses high-precision sensors to locate and correct the target's position and attitude; and the rescue phase executes operational tasks based on the results of the preceding phases, such as resource delivery, target dragging, or signal relay establishment. Each phase has a strict sequential dependency: the calibration phase is triggered after the identification phase is completed, and the rescue phase is executed after the calibration phase is completed.

[0038] For the aforementioned sub-tasks, a task graph model is further constructed. Each node in the model represents a task state, including a task stage identifier, executor number, state parameters, and time stamp. Directed edges between nodes describe state transition relationships, including task completion trigger conditions, resource update conditions, and communication synchronization conditions. For example, when a drone node in the search stage detects target coordinates, it transmits its state to the identification stage node via the "identification trigger edge." After the identification stage is completed, it transmits the state to the calibration stage node via the "location update edge." In this way, the task graph model comprehensively depicts the hierarchical dependencies of the unmanned swarm in a graph structure, providing clear task logic constraints and stage division criteria for the subsequent construction of hierarchical incremental reinforcement learning models.

[0039] Step S103: Construct a hierarchical incremental reinforcement learning model based on the task hierarchy dependency relationship, and output the first volume policy and the first global policy through the hierarchical incremental reinforcement learning model.

[0040] Specifically, based on the task hierarchy dependencies obtained in step S102, a hierarchical incremental reinforcement learning model is constructed to achieve hierarchical decision-making and adaptive collaboration among multiple agents in an unmanned swarm. This model comprises a two-layer structure: a global policy learning unit and individual policy learning units. The global layer is responsible for integrating global task objectives and stage dependencies to generate a swarm-level decision framework, while the individual layer is responsible for generating individual behavioral policies based on the operational status information of each unmanned agent and the characteristics of its corresponding sub-tasks.

[0041] During the model initialization phase, the mapping relationship between the global task layer and the individual task layer is determined based on the task graph model. Subsequently, in the individual layer, an individual policy network is established based on the running status information and sub-task constraints of each unmanned intelligent agent to output the corresponding first agent policy. In the global layer, a global policy network is established based on task information, task hierarchy dependencies, and the initial output of the individual policy network to generate the first global policy for overall task coordination and resource scheduling of the unmanned cluster.

[0042] By using a policy interaction mechanism between the global layer and the individual layer, the reward signal and policy parameters are transmitted bidirectionally, ensuring that the first individual policy and the first global policy remain convergent during the iteration process.

[0043] In one possible implementation, step S103 further includes: determining the mapping relationship between the global layer and the individual layer based on the task hierarchy dependency relationship, wherein the global layer is used to describe the global task state of the unmanned cluster task environment, and the individual layer is used to describe the local task state of each unmanned intelligent agent; in the individual layer, constructing an individual policy network based on the running state information and the action set corresponding to each unmanned intelligent agent; outputting a first agent policy for executing each sub-task through the individual policy network; in the global layer, constructing a global policy network based on task information, task hierarchy dependency relationship, and execution progress information of each sub-task; outputting a first global policy for coordinating the overall task execution order and resource scheduling order of the unmanned cluster through the global policy network.

[0044] Specifically, based on task-level dependencies, a hierarchical incremental reinforcement learning model is constructed to describe the decision-making structure of multi-agent unmanned swarms. This model comprises two coupled hierarchical structures: a global layer and an individual layer. The global layer characterizes the global task state and resource allocation relationships within the entire unmanned swarm task environment, while the individual layer describes the local task state and action decision-making logic of each unmanned agent at different sub-task stages.

[0045] In the model building process, the mapping relationship between the global layer and the individual layers is first determined based on the task hierarchy dependencies. Specifically, the executing entities of each subtask node in the task graph model are mapped to the corresponding unmanned intelligent agents in the individual layers, and the dependency paths between task stages in the task graph model are mapped to the task state transition structure of the global layer. This mapping relationship allows the global layer to dynamically reflect the global task progress by aggregating the state changes of each individual layer, while the individual layers can obtain execution constraints and policy guidance through feedback from the global layer.

[0046] At the individual level, an individual policy network is constructed based on operational status information and the action sets of each unmanned agent. Operational status information includes spatial pose information, sensor status information, battery level information, and communication topology information; the action sets define the decision-making behaviors that each unmanned agent can execute at the corresponding task stage, such as path adjustment, sensor activation, attitude control, and communication switching. The individual policy network uses policy functions... Description of the An unmanned intelligent agent in a local task state Select action The probability distribution of the state. The feature encoding transforms various parameters in the operational state information into state vectors that are input into the network: spatial pose information reflects the relative position and attitude of the unmanned intelligent agent within the task area; sensor state information represents the availability and data integrity of the perception module; battery level information indicates the duration of continuous operation; and communication topology information represents the real-time communication connectivity with other unmanned intelligent agents. After calculation by the neural network, the output is an action probability distribution to generate the first agent policy for executing each subtask.

[0047] At the global layer, a global policy network is constructed based on task information, task hierarchy dependencies, and the execution progress information of each subtask. Task information includes task type, region planning, task priority, resource requirements, and time constraints. Task hierarchy dependencies determine the decision-making order and triggering conditions for the global policy. Subtask execution progress information originates from the state feedback of each individual policy network and is used to characterize the current completion status of the global task. The global policy network uses policy functions... Describe the global task state of the unmanned cluster. Set of actions to be executed The joint probability distribution of , where, This represents the set of parameters for the global policy network. Global task state. Composed of the operational status information and task information of each unmanned intelligent agent, it is used to reflect the overall task execution status; action set This refers to decision-making instructions at the global level, including resource scheduling order, task allocation scheme, and execution phase switching signals.

[0048] The global policy network optimizes its parameters by minimizing a global loss function, which can be defined as: in, Indicates the global layer in the task state Next action The cumulative rewards obtained are used to reflect the overall task execution performance of the unmanned swarm; Indicated in the global policy network Under the defined joint probability distribution, for the task state With action set Expectation calculation; This is a regularization coefficient used to prevent parameter overfitting. A policy interaction mechanism is established between the global layer and the individual layer to achieve bidirectional information transfer. The individual layer aggregates local state codes and task execution results to form a stage-specific state vector, which is then input to the global layer. The global layer uses this information to correct the global policy output, thereby guiding the individual layer to adjust its policy direction in the next decision cycle.

[0049] Step S104: Construct a local reward mechanism based on the local task completion progress of each unmanned intelligent agent, and incrementally update the policy of the first agent based on the local reward mechanism to output the policy of the second agent.

[0050] Specifically, a local reward mechanism is constructed to dynamically optimize individual strategies by considering the task execution performance of each unmanned intelligent agent at different sub-task stages. This mechanism quantifies the local task completion progress of each unmanned intelligent agent, transforming task execution performance into reward signals, which are used to correct the first agent's strategy and generate an optimized second agent's strategy.

[0051] In one possible implementation, step S104 further includes: controlling each unmanned intelligent agent to execute the first agent policy and outputting the local task completion progress of the unmanned intelligent agent under preset constraints, including time constraints, path feasibility constraints, and task safety constraints; constructing a local reward function using the local task completion progress as feedback data; inputting the output value of the local reward function into the individual policy network to incrementally update the parameters of the first agent policy, and outputting the updated and optimized second agent policy.

[0052] Specifically, within the hierarchical incremental reinforcement learning framework, the first volume policy of each unmanned intelligent agent is incrementally updated based on the progress of local task completion, thereby achieving dynamic optimization and continuous convergence of individual behavioral policies. This process includes three stages: local task execution, local reward function construction, and incremental update of individual policy parameters.

[0053] During the local task execution phase, each unmanned intelligent agent executes its current subtask according to the first agent policy output by the individual policy network. The first agent policy is defined by the policy function. It means that, among them, For the first Actions performed by an unmanned intelligent agent This is the operational status information of the unmanned intelligent agent. This is its set of strategy parameters. Operational status information includes spatial pose information, sensor status information, battery level information, and communication topology information. During task execution, the unmanned agent's behavior is monitored in real time based on these status parameters, and the local task completion progress is output under preset constraints. Preset constraints include time constraints, path feasibility constraints, and task safety constraints. Specifically, the time constraint limits the maximum execution time of a task phase; the path feasibility constraint limits the deviation between the unmanned agent's actual movement trajectory and the planned trajectory to within an allowable range; and the task safety constraint limits the collision risk and communication packet loss rate generated during action execution to not exceed a threshold. This local task completion progress is output as a continuous numerical value, denoted as . Used to characterize the unmanned intelligent agent at the current time step The degree to which the task has been accomplished.

[0054] During the local reward function construction phase, the progress of local task completion is considered. As feedback data, a local reward function is established to measure the task completion quality and execution efficiency of the unmanned intelligent agent. The function is defined as follows:

[0055] in, and These are the positive reward weight coefficient and the penalty weight coefficient, respectively; Indicates the progress of a partial task; This represents the constraint cost function, used to comprehensively reflect the consumption level of the unmanned intelligent agent under time, path, and safety constraints. Specifically, ,in, The time constraint cost represents the deviation between the actual execution time and the planned time of the task. The path constraint cost represents the proportion by which the actual path deviates from the planned path. For safety constraints, it represents the statistical value of safety risk indicators (such as collision rate or communication packet loss rate) during task execution; , , These are the weighting coefficients for the constraint costs. Using the aforementioned local reward function, task execution efficiency can be quantitatively evaluated while ensuring safety and feasibility.

[0056] During the individual policy incremental update phase, the output value of the local reward function is... Input the individual policy network, and the parameters of the first individual policy. Incremental updates are performed using the gradient ascent method, with the parameter update rule as follows: in, The learning rate is used to control the step size for each update. This indicates the local reward function with respect to the parameters. The gradient is used to indicate the direction of parameter adjustment. Through this incremental update rule, the policy of the first agent is finely modified in each iteration cycle, enabling the unmanned agent to more efficiently meet time, path, and safety constraints in subsequent task execution.

[0057] After several rounds of updates, the individual policy networks of each unmanned intelligent agent converge to generate the second agent policy. ,in, This represents the set of policy parameters after local reward optimization. The second individual policy significantly improves upon the first individual policy in terms of task completion rate, energy utilization, and path execution stability, providing an optimal individual foundation at the reinforcement learning level for subsequent global policy correction and multi-agent collaborative orchestration.

[0058] As another inventive aspect of this application, the definition and acquisition method of local task completion progress are described in detail below: Local task completion progress refers to a dynamic indicator used in an unmanned swarm task environment to quantify the degree of task execution of a single unmanned intelligent agent within its assigned sub-task phase. It reflects the degree to which the agent achieves the task objective under time, space, and safety constraints. This indicator plays a crucial role in hierarchical incremental reinforcement learning models, serving as the main input variable for the local reward function and guiding the incremental optimization of individual policies.

[0059] The progress of a partial task can be defined as follows: in, Indicates the first An unmanned intelligent agent at any time The progress of local task completion; , and These represent the task completion components based on time constraints, path feasibility constraints, and task safety constraints, respectively. , and These are the weighting coefficients corresponding to the three types of constraint components.

[0060] in, The time taken for the unmanned intelligent agent to actually complete the current subtask. This refers to the planned time. When the actual execution time perfectly matches the planned time, If it exceeds the time limit or is completed ahead of schedule, A value below 1 will be penalized for performance deviation.

[0061] Path feasibility components The degree to which the execution path of an unmanned intelligent agent conforms to the planned path is defined as: in, This represents the cumulative deviation distance between the executed path and the planned path. This represents the total length of the planned path. When the unmanned intelligent agent moves strictly along the planned path... If path deviation or obstacle avoidance maneuvers occur, The corresponding amount will decrease.

[0062] Task security components The degree of stability in maintaining a safe state by an unmanned intelligent agent during task execution is defined as: in, This indicates the intensity of risk events detected during the mission cycle (such as communication packet loss rate, attitude instability, number of collision warnings, etc.). This indicates the set risk threshold. When the unmanned intelligent agent remains in a safe state throughout the entire process... If a risk event occurs, this component value will decrease accordingly.

[0063] Weight parameters , , These are used to balance the impact of different constraints on the overall task progress, and can be adaptively set according to the task type. For example, in complex path planning tasks, they can be used to improve... The weighting of [the task] should be increased in high-risk environmental missions. The weight.

[0064] Step S105: Aggregate the update information of the second volume policy and use an attention weighting mechanism to modify the first global policy into the second global policy.

[0065] Specifically, information is aggregated on the second agent policy generated by each unmanned intelligent agent after performing local incremental updates, so as to achieve dynamic correction of the global policy.

[0066] In one possible implementation, step S105 further includes: aggregating the update information of each second agent's strategy, and obtaining the parameter change amount and task contribution index in the update information; calculating the attention weight of each unmanned agent based on the task contribution index, the attention weight being used to characterize the relative importance of each unmanned agent in the global task; weightedly fusing the attention weight and parameter change amount to generate a global strategy correction amount; incrementally updating the parameters of the first global strategy according to the global strategy correction amount, and outputting the updated and optimized second global strategy.

[0067] Specifically, after optimizing the local policies of each unmanned intelligent agent, the second agent's policy undergoes global-level information aggregation and weight allocation to achieve adaptive correction of the first global policy and output the optimized second global policy. This process includes four stages: update information extraction, attention weight calculation, global policy correction generation, and global parameter incremental update.

[0068] During the information extraction phase, the second agent policy update information of each unmanned intelligent agent is aggregated. The second agent policy is determined by a function. It means that, among them, For the first Actions performed by an unmanned intelligent agent For running status information, This is the parameter set of the individual policy network after incremental optimization. The parameter change is calculated by comparing the parameter differences between the second and first individual policies. This is used to reflect the adjustment range of each unmanned intelligent agent during policy updates. Subsequently, it is based on the local task completion progress of each unmanned intelligent agent during task execution. and local reward function Calculate the contribution index of the task This metric measures the actual contribution of each unmanned intelligent agent to the overall task objective at the current task stage. Task contribution is defined as:

[0069] in, and These are the progress contribution weight and the reward contribution weight, used to balance the impact of task completion and reward on the overall contribution evaluation.

[0070] During the attention weight calculation phase, the attention weight of each unmanned intelligent agent is calculated based on the task contribution index. This weight is used to characterize the relative importance of the unmanned intelligent agent in the global task. The attention weight is obtained through normalization calculation:

[0071] in, The normalization method represents the number of unmanned intelligent agents. This method allows unmanned intelligent agents with higher contributions to receive greater weight in global policy adjustments, thereby strengthening the influence of efficient individuals in cluster decision-making.

[0072] In the global policy correction generation stage, the parameter changes of each unmanned intelligent agent are weighted and fused with the attention weights to generate the global policy correction. Defined as in, It reflects the comprehensive optimization direction of the global policy under the incremental learning of multiple unmanned intelligent agents and is the core input for updating the parameters of the global policy network.

[0073] During the global parameter incremental update phase, the parameters of the first global policy are updated based on the global policy correction amount. The update rules are as follows: in, Indicates time The parameter set of the global policy network. The global learning rate controls the magnitude of parameter updates. This rule causes the global policy to adjust along the direction of comprehensive optimization in each iteration, thereby gradually approaching the global optimum.

[0074] Through the above algorithm, adaptive weight coordination between individual and global parameters can be achieved within the hierarchical incremental reinforcement learning framework. Indicators reflecting the learning and adjustment level of unmanned intelligent agents and their contribution to the task. Attention weight reflects its actual value to the overall goal. The impact of key individuals is adaptively amplified during global adjustments. This is ultimately achieved through global policy adjustments. Incrementally update the parameters of the first global policy and output the updated and optimized second global policy. ,in This represents the set of parameters after attention-weighted correction.

[0075] Step S106: The second global strategy is synchronously distributed to each unmanned intelligent agent for collaborative intelligent orchestration.

[0076] Specifically, a synchronous broadcast channel and an acknowledgment channel are established in the cluster communication network. The parameter set and policy instruction set of the second global policy are encapsulated into a versioned policy package. The policy package contains the parameter identifier of the second global policy, the applicable task process stage range, resource allocation constraints and effective timestamp, and is bound to the one-to-one correspondence with each sub-task node in the task graph model to ensure that the policy is consistent with the execution scenario.

[0077] In one possible implementation, step S106 further includes: synchronously distributing the second global policy to each unmanned intelligent agent through a cluster communication network, and constructing a policy instruction set based on the corresponding node of each unmanned intelligent agent in the task graph model; and controlling the unmanned intelligent agents to perform collaborative intelligent orchestration according to the policy instruction set.

[0078] Specifically, the strategy is synchronously distributed to all unmanned agents via a cluster communication network. First, based on the sub-task nodes corresponding to each unmanned agent in the task graph model, strategy elements matching its task stage are extracted, including task stage identifier, execution priority, resource allocation constraints, and communication coordination parameters. Based on this, a strategy instruction set corresponding to each unmanned agent is constructed. This strategy instruction set is packaged in structured data format, containing a strategy identifier, task objective, allowed action set, energy scheduling rules, and time synchronization information, thereby ensuring that each unmanned agent can accurately parse its role and execution content within the global strategy upon receiving the instruction set.

[0079] During policy distribution, a highly reliable cluster communication network is used to broadcast and synchronize policy packets, and a version number and timestamp mechanism is employed to ensure policy consistency and timeliness. An acknowledgment mechanism is set up so that each unmanned agent sends back an acknowledgment signal after successfully receiving the policy packet; if no acknowledgment is received within a specified time, a retransmission mechanism is triggered or relay forwarding is performed by an edge node to ensure full coverage and synchronization of policy distribution.

[0080] During the strategy execution phase, each unmanned agent updates its local individual strategy network parameters based on the parsed strategy instruction set and initiates task execution according to the scheduling logic set by the global strategy. Under the control of a global clock, the unified strategy activation time enables all unmanned agents to simultaneously enter the new task cooperation mode. During execution, each unmanned agent maintains information sharing and state synchronization between nodes based on the communication and coordination parameters in the strategy instructions. Upon detecting task conflicts or resource contention, dynamic concessions or task switching are performed based on the priorities defined in the global strategy to ensure the stability and efficiency of global task execution.

[0081] Furthermore, during task execution, the operational status information and task feedback data of each unmanned agent are continuously collected, compared with the target constraints of the second global policy, and the collaborative execution effect is evaluated in real time. When environmental changes or local task anomalies are detected, the local policy instruction set can be regenerated and distributed based on the task graph model, achieving rapid adjustment and closed-loop control. Through the above process, the second global policy can be synchronized and executed collaboratively with high precision in the unmanned swarm, thereby realizing dynamic intelligent orchestration and efficient collaboration of multiple agents in complex task environments.

[0082] Please refer to Figure 2 This illustration shows a schematic diagram of a multi-agent collaborative intelligent orchestration device based on incremental reinforcement learning, provided in an embodiment of this application. The device includes an acquisition module 21 and a processing module 22, wherein... The acquisition module 21 is used to acquire the initial state information of the unmanned cluster task environment, including task information and the running state information of each unmanned intelligent agent; based on the initial state information, a task graph model is constructed, and the task hierarchical dependency relationship is output through the task graph model.

[0083] Processing module 22 is used to construct a hierarchical incremental reinforcement learning model based on task hierarchy dependencies, and output the first agent policy and the first global policy through the hierarchical incremental reinforcement learning model; construct a local reward mechanism based on the local task completion progress of each unmanned agent, and incrementally update the first agent policy based on the local reward mechanism to output the second agent policy; aggregate the update information of the second agent policy, and use an attention weighting mechanism to correct the first global policy to the second global policy; and synchronously distribute the second global policy to each unmanned agent for collaborative intelligent orchestration.

[0084] In one possible implementation, the acquisition module 21 is used to acquire initial state information of the unmanned swarm task environment, specifically including: constructing task information based on spatial geographic feature data, communication topology data, and resource distribution data in the unmanned swarm task environment; the task information includes task type information, regional planning information, task priority information, resource requirement information, and time constraint information; and acquiring operational status information through a multi-source collaborative sensing network, which includes positioning units, sensors, and communication nodes deployed in the unmanned swarm, and the operational status information includes spatial pose information, sensor status information, power level information, and communication topology information.

[0085] In one possible implementation, the acquisition module 21 is used to construct a task graph model based on the initial state information, specifically including: constructing task objectives based on the initial state information, the task objectives including rescue allocation objectives, resource allocation objectives, and collaborative operation objectives; dividing the global task into multiple sub-tasks according to the task process stages based on the task objectives, the task process stages including the search stage, the identification stage, the calibration stage, and the rescue stage; constructing a task graph model for each sub-task, where the nodes of the task graph model represent the task state, and the edges of the task graph model represent the state transition relationship.

[0086] In one possible implementation, the processing module 22 is used to construct a hierarchical incremental reinforcement learning model based on task-level dependencies, and output a first individual policy and a first global policy through the hierarchical incremental reinforcement learning model. Specifically, this includes: determining the mapping relationship between the global layer and the individual layer based on the task-level dependencies, where the global layer describes the global task state of the unmanned swarm task environment, and the individual layer describes the local task state of each unmanned agent; in the individual layer, constructing an individual policy network based on the running state information and the action set corresponding to each unmanned agent; outputting the first individual policy for executing each sub-task through the individual policy network; in the global layer, constructing a global policy network based on task information, task-level dependencies, and the execution progress information of each sub-task; and outputting the first global policy for coordinating the overall task execution order and resource scheduling order of the unmanned swarm through the global policy network.

[0087] In one possible implementation, the processing module 22 is used to construct a local reward mechanism based on the local task completion progress of each unmanned intelligent agent, and to incrementally update the first agent policy based on the local reward mechanism to output a second agent policy. Specifically, this includes: controlling each unmanned intelligent agent to execute the first agent policy and outputting the local task completion progress of the unmanned intelligent agent under preset constraints, including time constraints, path feasibility constraints, and task safety constraints; constructing a local reward function using the local task completion progress as feedback data; inputting the output value of the local reward function into the individual policy network to incrementally update the parameters of the first agent policy, and outputting the updated and optimized second agent policy.

[0088] In one possible implementation, the processing module 22 is used to aggregate the update information of the second agent policy and use an attention weighting mechanism to modify the first global policy into the second global policy. Specifically, this includes: aggregating the update information of each second agent policy and obtaining the parameter change amount and task contribution index in the update information; calculating the attention weight of each unmanned agent based on the task contribution index, whereby the attention weight is used to characterize the relative importance of each unmanned agent in the global task; weighting and fusing the attention weight and parameter change amount to generate a global policy correction amount; incrementally updating the parameters of the first global policy according to the global policy correction amount, and outputting the updated and optimized second global policy.

[0089] In one possible implementation, the processing module 22 is used to synchronously distribute the second global policy to each unmanned intelligent agent for collaborative intelligent orchestration, specifically including: synchronously distributing the second global policy to each unmanned intelligent agent through a cluster communication network, and constructing a policy instruction set based on the corresponding node of each unmanned intelligent agent in the task graph model; and controlling the unmanned intelligent agents to perform collaborative intelligent orchestration according to the policy instruction set.

[0090] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0091] This application also provides an electronic device. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: at least one processor 301, at least one communication bus 302, a user interface 303, at least one network interface 304, and a memory 305.

[0092] The communication bus 302 is used to enable communication between these components.

[0093] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0094] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0095] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 305, and by calling data stored in memory 305. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0096] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. (Refer to...) Figure 3 The memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a multi-agent collaborative intelligent orchestration application based on incremental reinforcement learning.

[0097] exist Figure 3 In the illustrated electronic device, the user interface 303 is primarily used to provide an input interface for the user and acquire user input data; while the processor 301 can be used to call the multi-agent collaborative intelligent orchestration application stored in the memory 305 based on incremental reinforcement learning. When executed by one or more processors 301, the electronic device performs one or more of the methods described in the above embodiments. It should be noted that, for the foregoing method embodiments, for the sake of simplicity, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0098] This application also provides a computer-readable storage medium storing instructions. When executed by one or more processors, these instructions cause an electronic device to perform one or more of the methods described in the above embodiments.

[0099] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0100] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0103] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0104] The above description is merely an exemplary embodiment disclosed in this application and should not be construed as limiting the scope of this application. Any equivalent changes and modifications made in accordance with the teachings of this application shall still fall within the scope of this application.

[0105] This application is intended to cover any variations, uses, or adaptations disclosed herein that follow the general principles disclosed herein and include common knowledge or customary technical means in the art that are not described in this application.

Claims

1. A multi-agent collaborative intelligent orchestration method based on incremental reinforcement learning, characterized in that, The method includes: Obtain the initial state information of the unmanned cluster task environment, which includes task information and the running state information of each unmanned intelligent agent; A task graph model is constructed based on the initial state information, and the task hierarchy dependency relationship is output through the task graph model. A hierarchical incremental reinforcement learning model is constructed based on the task hierarchy dependency relationship, and the first volume policy and the first global policy are output through the hierarchical incremental reinforcement learning model. A local reward mechanism is constructed based on the local task completion progress of each unmanned intelligent agent, and the policy of the first individual is incrementally updated based on the local reward mechanism to output the policy of the second agent. The update information of the second policy is aggregated, and the first global policy is modified to the second global policy using an attention weighting mechanism; The second global strategy is synchronously distributed to each of the unmanned intelligent agents for collaborative intelligent orchestration.

2. The method according to claim 1, characterized in that, The acquisition of the initial state information of the unmanned cluster task environment specifically includes: Based on the spatial geographic features, communication topology data, and resource distribution data in the unmanned cluster mission environment, the mission information is constructed. The mission information includes mission type information, regional planning information, mission priority information, resource requirement information, and time constraint information. The operational status information is obtained through a multi-source collaborative sensing network, which includes positioning units, sensors, and communication nodes deployed in an unmanned cluster. The operational status information includes spatial pose information, sensor status information, power level information, and communication topology information.

3. The method according to claim 1, characterized in that, The construction of the task graph model based on the initial state information specifically includes: Based on the initial state information, task objectives are constructed, including rescue allocation objectives, resource allocation objectives, and collaborative operation objectives. Based on the mission objective, the global mission is divided into multiple sub-tasks according to the mission flow stages, which include the search stage, the identification stage, the calibration stage, and the rescue stage. For each of the subtasks, a task graph model is constructed, where nodes in the task graph model represent task states and edges represent state transition relationships.

4. The method according to claim 3, characterized in that, The construction of a hierarchical incremental reinforcement learning model based on the task hierarchy dependency relationship, and the output of a first volume policy and a first global policy through the hierarchical incremental reinforcement learning model, specifically includes: The mapping relationship between the global layer and the individual layer is determined based on the task hierarchy dependency relationship. The global layer is used to describe the global task state of the unmanned cluster task environment, and the individual layer is used to describe the local task state of each unmanned intelligent agent. In the individual layer, an individual policy network is constructed based on the operational status information and the action set corresponding to each unmanned intelligent agent; The individual policy network outputs the first individual policy for executing each of the sub-tasks; In the global layer, a global policy network is constructed based on the task information, the task hierarchy dependencies, and the execution progress information of each subtask. The first global policy is output through the global policy network to coordinate the overall task execution order and resource scheduling order of the unmanned cluster.

5. The method according to claim 4, characterized in that, The process of constructing a local reward mechanism based on the local task completion progress of each unmanned intelligent agent, and incrementally updating the policy of the first individual agent based on the local reward mechanism to output the policy of the second agent, specifically includes: Control each of the unmanned intelligent agents to execute the first individual strategy, and output the local task completion progress of the unmanned intelligent agents under preset constraints, including time constraints, path feasibility constraints, and task safety constraints. A local reward function is constructed using the progress of the local task as feedback data; The output value of the local reward function is input into the individual policy network to incrementally update the parameters of the first individual policy, and the updated and optimized second individual policy is output.

6. The method according to claim 4, characterized in that, The process of aggregating the update information of the second global policy and using an attention-weighted mechanism to modify the first global policy into the second global policy specifically includes: Aggregate the update information of each of the second individual strategies, and obtain the parameter change amount and task contribution index in the update information; The attention weight of each unmanned intelligent agent is calculated based on the task contribution index. The attention weight is used to characterize the relative importance of each unmanned intelligent agent in the global task. The attention weights and parameter changes are weighted and fused to generate a global policy correction value; The parameters of the first global policy are incrementally updated based on the global policy correction amount, and the updated and optimized second global policy is output.

7. The method according to claim 1, characterized in that, The step of synchronously distributing the second global strategy to each of the unmanned intelligent agents for collaborative intelligent orchestration specifically includes: The second global policy is synchronously distributed to each of the unmanned intelligent agents through the cluster communication network, and a policy instruction set is constructed based on the corresponding node of each unmanned intelligent agent in the task graph model. The unmanned intelligent agent is controlled to perform collaborative intelligent orchestration according to the policy instruction set.

8. A multi-agent collaborative intelligent orchestration device based on incremental reinforcement learning, characterized in that, The device includes an acquisition module and a processing module, wherein, The acquisition module is used to acquire the initial state information of the unmanned cluster task environment, which includes task information and the running state information of each unmanned intelligent agent; construct a task graph model based on the initial state information, and output the task hierarchy dependency relationship through the task graph model; The processing module is configured to construct a hierarchical incremental reinforcement learning model based on the task hierarchy dependency relationship, and output a first individual policy and a first global policy through the hierarchical incremental reinforcement learning model; construct a local reward mechanism based on the local task completion progress of each unmanned agent, and incrementally update the first individual policy based on the local reward mechanism to output a second individual policy; aggregate the update information of the second individual policy, and use an attention weighting mechanism to modify the first global policy into a second global policy; and synchronously distribute the second global policy to each unmanned agent for collaborative intelligent orchestration.

9. An electronic device, characterized in that, The device includes a processor, a communication bus, a user interface, a network interface, and a memory. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1 to 7.

Images