An autonomous cooperative control method and device based on multi-agent hierarchical behavior

Abstract

This invention provides an autonomous collaborative control method and apparatus based on multi-agent hierarchical behavior. It constructs a UAV scheduling model, which includes a planning and decision-making layer, a task coordination layer, and a behavior execution layer. The planning and decision-making layer acquires environmental data and generates a total task. The task coordination layer then decomposes the total task into a list of sub-tasks and generates action commands, which are sent to the behavior execution layer. The behavior execution layer combines the action commands with real-time monitored sensor data to generate a control strategy. After optimizing the control strategy, it is executed by the corresponding controlled object, and the execution results are monitored. Feedback adjustments are made to the planning and decision-making layer, the task coordination layer, and / or the behavior execution layer based on the execution results. These steps improve the effective collaboration between the various layers, and the feedback adjustment enhances the adaptability of each layer to the UAV scheduling scenario during the planning process, avoiding rule rigidity.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, and more specifically, relates to an autonomous collaborative control method and device based on the hierarchical behavior of multiple agents. Background Technology

[0002] In the existing technology, drone swarms have been increasingly widely used in life scenarios, such as live shooting of competitions through drone swarms. However, the current planning methods usually use a pre-defined rule system or script to plan, and generate drone behavior through a pre-defined behavior tree or state machine. The problems of the above methods are: (1) the rules are rigid, rely on human prior knowledge, lack online learning and adaptive capabilities, and are difficult to adapt to dynamic and complex on-site environments; (2) cross-level decision-making lacks effective coordination, and it is difficult to achieve efficient integration between different levels.

[0003] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0004] The problem this invention aims to solve is how to improve the adaptability of UAV planning decisions to dynamic field environments when planning and scheduling UAV swarms.

[0005] Firstly, a method for autonomous cooperative control based on multi-agent hierarchical behavior is provided, which constructs a multi-agent hierarchical autonomous cooperative control model. The model includes a planning and decision-making layer, a task coordination layer, and a behavior execution layer, comprising: The planning and decision-making level collects environmental data, makes environmental predictions based on the environmental data, and generates corresponding overall tasks based on the results of the environmental predictions. The task collaboration layer decomposes the total task and preprocesses it to obtain a list of sub-tasks, and generates action instructions based on the list of sub-tasks. The behavior execution layer monitors and acquires sensor data in real time, fuses the sensor data and the action commands to generate a control strategy, filters the control strategy through predefined safety rules to obtain an optimized strategy, drives the corresponding controlled object to execute the optimized strategy, monitors the execution status of the controlled object in real time, and feeds it back to the planning decision layer and the task coordination layer. Feedback adjustment is triggered step by step between the behavior execution layer, the task coordination layer, or the planning decision layer to improve the adaptability of the scheduling strategy to dynamic scenarios.

[0006] Preferably, the step of collecting environmental data through the planning and decision-making layer, making environmental predictions based on the environmental data, and generating a corresponding overall task based on the results of the environmental predictions specifically includes: Environmental features and rules are pre-encapsulated and stored in the form of memory blocks, task decisions are pre-stored, and task decisions are mapped to corresponding memory blocks; Real-time acquisition of environmental data, and selection of suitable memory blocks as the prediction memory blocks corresponding to the environmental data; Obtain the task decision corresponding to the predicted memory block, and use the corresponding task decision as the total task.

[0007] Preferably, the real-time acquisition of environmental data and the selection of suitable memory blocks as prediction memory blocks corresponding to the environmental data specifically includes: Real-time acquisition of environmental data; Extract multiple memory blocks that match the environmental data, and perform weighted fusion based on the environmental data and the matching memory blocks to obtain the confidence score corresponding to each memory block. The expression for the confidence score is: ; in, For confidence level For state similarity, For temporal similarity, For structural similarity, For semantic similarity, These are the weighting coefficients for state similarity. These are the weighting coefficients for temporal similarity. These are the weighting coefficients for structural similarity. The weighting coefficients for semantic similarity; The memory block with the highest confidence level is used as the prediction memory block corresponding to the environmental data.

[0008] Preferably, the task collaboration layer decomposes the total task and preprocesses it to obtain a list of subtasks, specifically including: Pre-construct a unified knowledge graph; In the planning and decision-making layer, semantic association and constraint reasoning of the environmental state are performed based on the unified knowledge graph to generate the constraint conditions corresponding to the overall task; In the task collaboration layer, the total task is decomposed into multiple constrained subtasks according to the task dependencies, temporal constraints and the constraints in the unified knowledge graph through a hierarchical task decomposition model based on temporal constraints. Obtain the priority of each constraint subtask, and sort the constraint subtasks based on the priority to obtain the subtask list.

[0009] Preferably, the step of generating action instructions based on the subtask list specifically includes: Multiple domain agents are set up, each of which is used to handle different types of constrained sub-tasks. Global value information is shared through a shared value network, and the relationship between the domain agents is established through a multi-head attention mechanism to achieve cross-agent collaborative modeling. Generate the corresponding formation strategy based on the constrained subtasks; Assign each constrained subtask and its corresponding formation strategy in the subtask list to the corresponding domain agent; Each domain-specific agent generates intent parameters based on the constrained subtasks and corresponding formation strategies, and generates an executable macro-action sequence based on the intent parameters; The macro action sequence is verified for feasibility. Macro action sequences that do not meet the preset constraints are eliminated, and the verified macro action sequences are output as action commands.

[0010] Preferably, the behavior execution layer monitors and acquires sensor data in real time, and fuses the sensor data and the action commands to generate a control strategy, specifically including: The behavior execution layer receives action instructions from the task coordination layer and parses the action instructions into HRL sub-targets; Real-time monitoring acquires sensor data, and the sensor data is converted into a state vector; Both the HRL sub-objective and the state vector are input into the policy network in the behavior execution layer. The policy network outputs the control policy that best matches the corresponding reward value based on the reward value corresponding to the HRL sub-objective and the state vector.

[0011] Preferably, the step of filtering the control strategy through predefined security rules to obtain an optimized strategy, and driving the corresponding controlled object to execute the optimized strategy, specifically includes: The control strategy is compared with each constraint in the predefined security rules to determine whether the control strategy satisfies each constraint. If the control strategy satisfies all constraints, then the control strategy will be output as the optimized strategy. If the control strategy does not meet all the constraints, a security constraint penalty mechanism is introduced based on the policy network to modify the strategy, and / or an alternative control strategy that meets the constraints is regenerated based on the candidate policy set. If an alternative strategy that satisfies the constraints cannot be generated, an emergency control command is output to override the current control strategy, and the negative feedback information corresponding to the control process is recorded in the empirical data set for updating the policy network.

[0012] Preferably, the real-time monitoring of the execution status of the controlled object and the feedback to the planning decision layer and task coordination layer, triggering feedback adjustments at each level between the behavior execution layer, task coordination layer, or planning decision layer, specifically includes: The execution status of the controlled object is obtained. The behavior execution layer aggregates and compresses the execution status of the controlled object to obtain the execution result, and then uploads the execution result to the task collaboration layer. The task collaboration layer compares the execution trajectory in the execution result with the corresponding planning trajectory in the corresponding constraint subtask to obtain the execution deviation. When the deviation is less than or equal to the preset value, the controlled object continues to execute according to the optimization strategy. When the deviation exceeds the preset value, the control strategy of the behavior execution layer is replanned, and the execution deviation is determined within a preset time to be less than or equal to the preset value. If yes, the feedback adjustment of the behavior execution layer is completed; if not, the task coordination layer is triggered to redistribute the subtask list and determine whether the execution deviation is less than or equal to the preset value within a preset time. If yes, the feedback adjustment of the task coordination layer is completed; if not, the replanning of the overall task of the planning and decision-making layer is triggered; if the execution deviation is determined to be below the preset value within the preset time window after replanning, the feedback adjustment is completed; if the deviation still does not meet the standard after the time window is exceeded, an alarm signal is sent.

[0013] Secondly, an autonomous cooperative control device based on multi-agent hierarchical behavior is provided, comprising at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor to perform the autonomous cooperative control method based on multi-agent hierarchical behavior.

[0014] Thirdly, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors to perform the method described in the first aspect.

[0015] Fourthly, a chip is provided, comprising: a processor and an interface for calling and running a computer program stored in memory, performing the method as described in the first aspect.

[0016] Fifthly, a computer program product containing instructions is provided that, when executed on a computer or processor, causes the computer or processor to perform the method as described in the first aspect.

[0017] In a sixth aspect, an autonomous cooperative control system based on hierarchical behavior of multiple agents is provided, including an autonomous cooperative control device based on hierarchical behavior of multiple agents as in the second aspect, and using an autonomous cooperative control method based on hierarchical behavior of multiple agents as in the first aspect.

[0018] Unlike existing technologies, the present invention has at least the following beneficial effects: A drone scheduling model is constructed, comprising a planning and decision-making layer, a task coordination layer, and a behavior execution layer. The planning and decision-making layer acquires environmental data and generates a total task. The task coordination layer then decomposes the total task into a list of sub-tasks and generates action commands, which are sent to the behavior execution layer. The behavior execution layer combines the action commands with real-time monitored sensor data to generate a control strategy. After optimizing the control strategy, it is executed by the corresponding controlled objects, and the execution results are monitored. Feedback adjustments are made to the planning and decision-making layer, the task coordination layer, and / or the behavior execution layer based on the execution results. These steps improve the effective coordination between the various layers, and the feedback adjustment enhances the adaptability of each layer to the drone scheduling scenario during the planning process, avoiding rule rigidity. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0020] Figure 1 This is a flowchart of an autonomous cooperative control method based on hierarchical behavior of multiple agents provided in an embodiment of the present invention; Figure 2 This is a flowchart of a method for obtaining the total task in an autonomous cooperative control method based on hierarchical behavior of multiple agents provided in an embodiment of the present invention; Figure 3 This is a flowchart of a method for obtaining predictive memory blocks in an autonomous cooperative control method based on hierarchical behavior of multiple agents, provided in an embodiment of the present invention. Figure 4 This is a flowchart of a method for obtaining a subtask list in an autonomous cooperative control method based on hierarchical behavior of multiple agents provided in an embodiment of the present invention; Figure 5 This is a flowchart of a method for obtaining action instructions in an autonomous cooperative control method based on hierarchical behavior of multiple agents provided in an embodiment of the present invention; Figure 6 This is a flowchart of a method for obtaining control strategies in an autonomous cooperative control method based on hierarchical behavior of multiple agents, provided in an embodiment of the present invention. Figure 7 This is a flowchart of a control strategy optimization method in an autonomous cooperative control method based on hierarchical behavior of multiple agents, provided in an embodiment of the present invention. Figure 8 This is a flowchart of a feedback adjustment method in an autonomous cooperative control method based on hierarchical behavior of multiple agents provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of an autonomous collaborative control device based on the hierarchical behavior of multiple agents, provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] Unless the context otherwise requires, throughout the specification and claims, the term "comprising" is interpreted as openly inclusive, meaning "including, but not limited to." In the description of the specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this disclosure. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics mentioned may be included in any suitable manner in any one or more embodiments or examples; that is, although they may be incorporated into embodiments or examples using the above terms for reasons such as order and position, it does not limit them to be incorporated in combination by a single embodiment or example.

[0023] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, for example, the description may use the prefix "A" or "B" to describe the same type of nouns as two independent entities. In this case, the corresponding features defined with "A" and "B" are used only to distinguish between similar entities and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features.

[0024] In the description of this invention, the expression “A and / or B” (where A and B are used to formally represent specific features) will be used. The corresponding expression includes the following three combinations: only A, only B, and a combination of A and B.

[0025] As used in this invention, “about,” “approximately,” or “approximately” includes the stated value and the average value within an acceptable range of deviation from a particular value, wherein the acceptable range of deviation is determined by a person skilled in the art taking into account the measurement under discussion and the error associated with the measurement of the particular quantity (i.e., the limitations of the measurement system).

[0026] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0027] Example 1: This embodiment provides an autonomous cooperative control method based on multi-agent hierarchical behavior, constructing a multi-agent hierarchical autonomous cooperative control model. The model includes a planning and decision-making layer, a task coordination layer, and a behavior execution layer, such as... Figure 1 As shown, the method flow includes: In step 101, the planning decision layer, task coordination layer, and behavior execution layer are initialized.

[0028] The method provided in this embodiment can be applied to scenarios involving the overall planning and control of multiple controlled objects. The controlled objects can be individual units such as drones. The overall task is planned independently based on the site environment or requirements, and the overall task is decomposed into multiple sub-tasks. The corresponding execution path or action is planned according to each sub-task, and the corresponding controlled objects are assigned to execute it to achieve the effect brought about by the overall task. The task planning at each level is adjusted based on the execution status of the controlled objects to ensure the effectiveness of the task planning.

[0029] The planning and decision-making layer is used for simulation and overall task planning. Its main function is macro-situational awareness. Instead of directly controlling the actions of individual controlled objects, the planning and decision-making layer collects global environmental data (such as terrain and weather), uses a built-in cognitive model to predict the development trend of the entire scene, and generates the overall task accordingly.

[0030] The task coordination layer is used to receive the total task from the planning and decision-making layer, break down the total task into multiple sub-tasks and assign them to agents in different domains, and generate action instructions for each sub-task.

[0031] The behavior execution layer is used to receive action instructions from the task coordination layer and, based on the local data (such as environmental data or visual images) collected in real time by its own sensors, jointly generate a control strategy for a single controlled object (such as a single drone), that is, ultimately output a specific control scheme for a single controlled object.

[0032] In this embodiment, the initialization refers to initializing the parameters in the three levels and the shared state between the three levels to complete the startup of the three levels.

[0033] In step 102, environmental data is collected by the planning and decision-making layer, and environmental prediction is performed based on the environmental data. The corresponding total task is generated based on the results of the environmental prediction.

[0034] In this embodiment, the environmental data may include terrain features, weather features, and interactive object features, etc.; the environmental prediction is to predict the future state of the corresponding area and / or the corresponding interactive object based on the terrain features, weather features, and interactive object features. For example, if it is necessary to record the competition site live by a drone swarm, the planning and decision-making layer collects environmental information such as the weather features, competition progress, and the location of each competitor at the competition site, and predicts the next progress of the competition and the possible location of the corresponding competitors, etc., which is the environmental prediction. Then, the corresponding total task is generated based on the environmental prediction results. For example, when the environmental data shows that most competitors are close to the finish line, it is predicted that the competition is likely to end soon. Based on historical sample experience, it is likely that drones need to be called to the finish line to take pictures of the competitors reaching the finish line. Therefore, the total task is generated: call a specified number of drones to different positions at the finish line to take pictures of the competitors reaching the finish line.

[0035] In this embodiment, the planning and decision-making layer includes multiple intelligent agents. Each intelligent agent in the planning and decision-making layer is responsible for collecting environmental data in different aspects. For example, some intelligent agents are responsible for acquiring weather data, some are responsible for acquiring terrain data, some are responsible for acquiring data of interactive objects, and some are responsible for acquiring real-time status data of all controlled objects. By having multiple intelligent agents perform their respective duties, the overall efficiency of environmental data acquisition is improved.

[0036] The aforementioned environmental predictions and overall task generation are both achieved through an integrated multimodal large model. Using the LLM-ACTR framework, multimodal features are extracted from the multimodal input (i.e., environmental data) and input into the ACTR model. After fine-tuning with a predefined dataset, the semantic parsing accuracy and inference reliability can be improved, generating the environmental prediction results and the overall task.

[0037] In step 103, the task collaboration layer decomposes the total task and preprocesses it to obtain a list of subtasks, and generates action instructions based on the list of subtasks.

[0038] Furthermore, in this embodiment, the decomposition of the total task can be as follows: the total task is split into multiple constrained subtasks according to the hierarchical task network. The preprocessing is as follows: during the decomposition process, constraints are imposed according to a predefined knowledge graph to make the various constrained subtasks obtained from the decomposition relatively compliant, thereby improving the adaptability of each constrained subtask. The various constrained subtasks are then sorted according to a pre-set priority optimization rule. The priority optimization rule can be set according to the optimal risk and optimal resource consumption. All constrained subtasks after being constrained and sorted constitute the subtask list.

[0039] In this embodiment, the task collaboration layer is configured with multiple domain agents. Different domain agents are interconnected through a shared value network. Each task in the subtask list is assigned to a different domain agent. The domain agent generates intent parameters according to the corresponding task, generates a macro action sequence according to the intent parameters, and organizes it with macro rules, which is the action instruction.

[0040] In step 104, the behavior execution layer monitors and acquires sensor data in real time, fuses the sensor data and the action instructions to generate a control strategy, filters the control strategy through predefined safety rules to obtain an optimized strategy, drives the corresponding controlled object to execute the optimized strategy, monitors the execution status of the controlled object in real time, and feeds it back to the planning decision layer and the task coordination layer. Feedback adjustment is triggered step by step between the behavior execution layer, the task coordination layer, or the planning decision layer to improve the adaptability of the scheduling strategy to dynamic scenarios.

[0041] Furthermore, in this embodiment, the sensing data is real-time environmental data obtained through real-time monitoring, the control strategy is an instruction directly applied to the controlled object, and the security rules are used to ensure the compliance of the optimization strategy.

[0042] After applying the optimization strategy to each controlled object, the execution results of each controlled object are monitored in real time to adjust the planning decision layer, task coordination layer, and / or behavior execution layer in reverse. This improves the adaptability of each layer to the drone scheduling scenario during the planning process and avoids rule rigidity. The adaptability to the drone scheduling scenario means that, under different application scenarios of drones, the execution effect of drones in the corresponding scenarios is improved through the corresponding scheduling scheme. For example, in the environmental detection scenario, the efficiency of drone scheduling is improved, and in the live sports broadcast scenario, the energy consumption of drone scheduling is reduced.

[0043] In this embodiment, a hierarchical intelligent agent model is constructed, comprising a decision-making and planning layer, a task coordination layer, and a behavior execution layer. The decision-making and planning layer integrates environmental information, task requirements, and resource status to generate global task objectives and planning results. The task coordination layer decomposes the global task into a set of sub-tasks and generates task execution strategies based on task priority, resource constraints, and collaborative relationships. The behavior execution layer combines real-time sensing data with the task execution strategies to generate control instructions, dynamically optimize the execution process, and drive the corresponding controlled objects to complete the task. During task execution, each layer achieves state synchronization and dynamic adjustment through a feedback mechanism. Based on execution results, environmental changes, and abnormal events, task planning, task allocation, and control strategies are adaptively optimized, thus forming a closed loop of planning, coordination, execution, and feedback. This invention improves the coordination efficiency, adaptability, and task execution stability of multi-agent systems in dynamic and complex environments, avoiding the problems of rule rigidity and insufficient environmental adaptability inherent in traditional rule-driven methods.

[0044] Furthermore, in this embodiment, for the planning decision-making layer, since it needs to perform environmental prediction based on environmental data and generate corresponding overall tasks, in order to perform relatively accurate prediction and planning, it needs to conform to the dynamic changes and historical patterns of the corresponding environment. Therefore, this embodiment involves the following design: the planning decision-making layer collects environmental data, performs environmental prediction based on the environmental data, and generates corresponding overall tasks based on the results of the environmental prediction, such as... Figure 2 As shown, the method flow includes: In step 201, environmental features and rules are pre-encapsulated and stored in the form of memory blocks, task decisions are pre-stored, and task decisions are mapped to corresponding memory blocks.

[0045] In this embodiment, the environmental features can be terrain information, weather information, and road information, etc., and the rules can be various stages in a specific application scenario. For example, in a competition scenario, the environmental information can be competition road information, weather wind direction, and the location of the competitors, etc., and the rules can be the competition progress and unexpected situations, etc. In this embodiment, the memory block is a structured data vector stored in a database.

[0046] In step 202, environmental data is acquired in real time, and suitable memory blocks are selected as the prediction memory blocks corresponding to the environmental data.

[0047] In step 203, the task decision corresponding to the predicted memory block is obtained, and the corresponding task decision is used as the total task.

[0048] The planning and decision-making layer comprises multiple detection agents, a memory module, a perception module, and a motion module. Each agent collects environmental data from different dimensions; for example, one agent collects terrain features, another collects weather features, and yet another collects features of interactive objects. Through collaborative work, data collection efficiency is improved. The perception module receives and encodes the data collected by the agents, converting it into characters understandable to the internal system. During this process, an attention mechanism prioritizes relatively important data before transferring the collected data to the memory module.

[0049] The memory module includes declarative memory, procedural memory, and working memory. The declarative memory stores environmental features and rule information in memory blocks to support subsequent rapid retrieval and reasoning. The environmental features and rule information includes static and dynamic knowledge. The static knowledge represents pre-set basic information before system operation, including environmental structure information, agent capability model information, task constraint rule information, behavioral strategy template information, and agent collaboration relationship information. The dynamic knowledge represents information that changes in real time during task execution, including environmental state changes, agent execution state information, and task feedback information. The unified knowledge graph is used to structure and organize static and dynamic knowledge, and uses keywords as indexes for memory block retrieval to achieve rapid retrieval. Based on a multimodal large model-enhanced retrieval mechanism, keywords in the unified knowledge graph are used as search terms to achieve rapid matching and retrieval of memory blocks, thereby reducing retrieval latency to the millisecond level.

[0050] The procedural memory is used to pre-store task decisions and enhance the generation of each task decision through prompts. In this embodiment, a multimodal large model can be used to simulate human cognitive responses to enhance the adaptability of task decisions. The working memory is used as a temporary cache to temporarily cache the environmental data, prediction memory blocks, and the overall task.

[0051] The motion module is used to decode the overall task and transmit it to the task coordination layer, and monitor whether there is any feedback from the execution results of the controlled objects in the lower layer, so as to facilitate feedback adjustment of the planning and decision-making layer.

[0052] In practical applications, when the perception module detects environmental data, it encodes the environmental data and transmits it to the working memory for caching. At the same time, the declarative memory is activated. The declarative memory compares the environmental data with pre-stored memory blocks (i.e., environmental features and rules), and selects the memory block that best matches the current environmental data as the prediction memory block. That is, it predicts the actual situation of the current scene based on the environmental data and caches the prediction memory block in the working memory. At the same time, the procedural memory is activated. Since the procedural memory has pre-stored a large number of task decisions corresponding to memory blocks, the procedural memory calls the matching task decision based on the prediction memory block in the working memory, which is the total task.

[0053] Furthermore, in this embodiment, when environmental data is acquired, a prediction memory block needs to be obtained based on the environmental data. The corresponding design is as follows: Real-time acquisition of environmental data, filtering out suitable memory blocks as the prediction memory block corresponding to the environmental data, such as... Figure 3 As shown, the method flow includes: In step 301, environmental data is acquired in real time.

[0054] In step 302, multiple memory blocks that match the environmental data are extracted, and a weighted fusion is performed based on the environmental data and the matching memory blocks to obtain the confidence level corresponding to the memory block.

[0055] In this embodiment, confidence level includes the following sub-items: state similarity, temporal similarity, structural similarity, and semantic similarity.

[0056] The expression for the confidence level is: ; in, For confidence level , State similarity is used to measure the degree of matching between the current environmental state and historical states; Temporal similarity is used to measure the consistency of trends in environmental state changes; Structural similarity is used to measure the consistency of topological relationships and task dependencies between objects; Semantic similarity is used to measure the consistency between task objectives and behavioral semantics. These are the weighting coefficients for state similarity. These are the weighting coefficients for temporal similarity. These are the weighting coefficients for structural similarity. is the weight coefficient for semantic similarity, used to characterize the degree of influence of different features on the memory block matching result, and satisfies: ; The weighting coefficients can be dynamically adjusted based on historical training results, scenario types, expert experience rules, or online learning results.

[0057] In step 303, the memory block with the highest confidence level is selected as the prediction memory block corresponding to the environmental data.

[0058] In this embodiment, when the prediction memory block corresponding to the environmental data is obtained, the prediction of the on-site situation reflected by the current environmental data is completed. For example, in a competition scenario, when it is detected that all the contestants are located on the podium, it can be predicted that the award ceremony is currently underway. Based on the obtained prediction memory block, the task decision that matches the prediction memory block in advance can be called as the total task, serving as the general strategy for the next step, and sent to the task coordination layer. The task coordination layer then performs further task decomposition and allocation based on the total task.

[0059] In this embodiment, after the task coordination layer obtains the total task from the planning and decision-making layer, since the total task usually cannot be completed directly by a single controlled object, it needs to be decomposed into multiple sub-tasks, and corresponding instructions are generated for each sub-task for subsequent allocation to the corresponding controlled objects for execution. Therefore, this embodiment also involves the following design: the task coordination layer decomposes the total task and preprocesses it to obtain a list of sub-tasks, such as... Figure 4 As shown, the method flow includes the following: In step 401, a unified knowledge graph is pre-constructed.

[0060] In this embodiment, the unified knowledge graph is used to structurally represent environmental information, task information, and constraint information, and serves as a unified cognitive and constraint reasoning hub shared between the planning and decision-making layer and the task collaboration layer. In the planning and decision-making layer, the unified knowledge graph performs semantic mapping and state modeling on environmental data to understand the current environmental state and generate a total task based on the environmental state and constraint information. In the task collaboration layer, the unified knowledge graph performs structured parsing of the total task and decomposes it based on task dependencies, temporal constraints, and resource constraints to generate multiple sub-tasks. The unified knowledge graph facilitates bidirectional information interaction and synchronous updates between the planning and decision-making layer and the task collaboration layer. The planning and decision-making layer writes task generation information to the unified knowledge graph, and the task collaboration layer reads constraint information based on the unified knowledge graph and feeds back the task decomposition results, thus forming a closed-loop collaboration mechanism.

[0061] In step 402, in the planning decision layer, semantic association and constraint reasoning of the environmental state are performed based on the unified knowledge graph to generate the constraint conditions corresponding to the overall task.

[0062] In step 403, in the task collaboration layer, the total task is decomposed into multiple constrained subtasks according to the task dependencies, temporal constraints, and the constraint conditions in the unified knowledge graph through a hierarchical task decomposition model based on temporal constraints.

[0063] In this embodiment, a hierarchical task decomposition model based on temporal constraints is used to structurally model the overall task, converting it into a task representation containing task nodes, task objectives, and constraint information. Based on dynamic constraint information in a unified knowledge graph, constraint parsing is performed on the task representation to determine the dependencies, temporal relationships, and resource constraints between tasks. According to the constraint parsing results, the overall task is decomposed into multiple sets of subtasks that satisfy the constraints. Based on the dependencies and execution priorities between the subtasks, the subtask sets are sorted to obtain a subtask list.

[0064] In this embodiment, taking a competition scenario as an example, when the overall task is: a player is injured during the competition, and the injured player is filmed, the decomposed constraint subtasks may include: assigning at least one drone to film the injured player in close-up, assigning at least two drones to film the injured player in long-range, and assigning at least four drones to film other players who are playing normally.

[0065] The dynamic constraint is that each constraint subtask obtained by decomposition needs to meet the pre-set constraint conditions in the unified knowledge graph to ensure the compliance of the obtained constraint subtasks and increase the flexibility and applicability of decomposition.

[0066] In step 404, the priority of each constraint subtask is obtained, and the constraint subtasks are sorted based on the priority to obtain the subtask list.

[0067] In this embodiment, according to the needs of those skilled in the art, corresponding utility functions can be set from the dimensions of risk, resource consumption, and type weight. The priority of each constraint subtask can be calculated through the utility function, thereby realizing the priority ranking of each constraint subtask. The final output subtask list can be in XML format, including the task ID and priority of each constraint subtask.

[0068] Furthermore, in this embodiment, after obtaining the subtask list, it is necessary to generate corresponding action instructions for each constrained subtask in the subtask list, which are then used to assign the action instructions to the corresponding controlled objects for execution. Therefore, this embodiment also involves the following design: Figure 5 As shown, the method for generating action instructions based on the subtask list includes the following steps: In step 501, multiple domain agents are set up, each of which is used to handle different types of constrained sub-tasks. Global value information is shared through a shared value network, and the relationship between the domain agents is established through a multi-head attention mechanism to achieve cross-agent collaborative modeling.

[0069] In this embodiment, the shared value network is used to perform unified value assessment on the collaborative behavior of multiple domain agents to generate a global value assessment result, which serves as the basis for agent strategy selection and collaborative decision-making. Different domain agents are used to correspond to different task types. By assigning each constraint subtask to the domain agent corresponding to the task type, the corresponding domain agent subsequently generates corresponding intent parameters based on the corresponding constraint subtask.

[0070] In step 502, a corresponding formation strategy is generated based on the constrained subtask.

[0071] In this embodiment, the formation strategy is to determine the formation arrangement of the controlled objects when performing corresponding constrained sub-tasks. It is used to describe the collaborative relationship and execution structure between multiple domain agents. It can improve the collaborative efficiency of the controlled objects through joint reward optimization.

[0072] In step 503, each constrained subtask in the subtask list and its corresponding formation strategy are assigned to the corresponding domain agent.

[0073] In step 504, each domain agent generates intent parameters based on the constrained subtasks and the corresponding formation strategy, and generates an executable macro action sequence based on the intent parameters.

[0074] In this embodiment, each domain agent formulates corresponding decision intentions based on the constrained sub-tasks and the corresponding formation strategies, and generates corresponding intention parameters. These intention parameters are various parameters required to execute the corresponding constrained sub-tasks, such as target coordinates, expected time, required resources, and constraint conditions. It should be noted that the intention parameters are not action instructions that can be directly given to the controlled object for execution. After the domain agent generates the corresponding intention parameters, it is also necessary to generate a series of specific executable low-level actions based on the intention parameters and encapsulate them into executable macro actions, which are the macro action sequences. Furthermore, in the above process, the domain agent also needs to evaluate the execution effect of the generated macro action sequences. This process requires collaboration with other domain agents based on the shared value network to simulate the overall comprehensive effect of the actual execution of each constrained sub-task.

[0075] In step 505, the macro action sequence is verified for feasibility. Macro action sequences that do not meet the preset constraints are eliminated, and the verified macro action sequences are output as action commands.

[0076] In this embodiment, the feasibility verification is to verify the feasibility and completeness of the macro action sequence, and the action instruction is obtained after the verification is completed.

[0077] In summary, in this embodiment, the task coordination layer includes a task decomposition module, a task allocation module, a formation control module, and a macro action generation module, wherein: The task decomposition module is used to decompose the total task into multiple constrained subtasks in the form of dynamic constraints, and sort them by priority to obtain a list of subtasks.

[0078] The task allocation module is used to assign each constrained subtask in the subtask list to the corresponding domain agent. The domain agent formulates the corresponding decision intention based on the constrained subtask and the corresponding formation strategy, and generates the corresponding intention parameters and sends them to the macro action generation module. The macro action generation module generates the macro action sequence. The domain agent also needs to evaluate the execution effect of the generated macro action sequence and take into account the coordination with other domain agents to improve the efficiency of task completion and system robustness.

[0079] The formation control module is used to plan the formation arrangement of the controlled objects when performing corresponding constrained sub-tasks. It can improve the collaborative efficiency of the controlled objects through joint reward optimization.

[0080] The macro-action generation module is used to generate corresponding macro-action sequences based on the intent parameters generated by the domain agent. This process can be as follows: a series of action instructions are generated based on the intent parameters and organized in the form of macro-actions, and high-frequency actions are encapsulated. The high-frequency actions can be path avoidance, etc., and the generation model is used to assist in the generation of actions. At the same time, knowledge graph data is introduced to generate more reasonable actions. The macro-action generation module is also used to verify the feasibility of the macro-action sequence, to verify the feasibility and completeness of the macro-action sequence. After the verification is completed, the action instructions are obtained.

[0081] Furthermore, in this embodiment, after obtaining the action instructions corresponding to each constraint subtask, they can be sent to the behavior execution layer in XML format. The behavior execution layer needs to formulate control strategies for each controlled object based on the action instructions and the current real-time environment. The corresponding design is as follows: The behavior execution layer monitors and acquires sensor data in real time, and fuses the sensor data and the action commands to generate a control strategy, such as... Figure 6 As shown, the method flow includes: In step 601, the behavior execution layer receives action instructions from the task coordination layer and parses the action instructions into Hierarchical Reinforcement Learning (HRL) objectives.

[0082] In this embodiment, the behavior execution layer includes an instruction receiving module, a sensor processing module, and an action execution module. The instruction receiving module is used to obtain action instructions in XML format from the task coordination layer, and to perform structured parsing and integrity verification on the action instructions through a parser. Based on preset rules, the parsed action instructions are constrained and filtered, and the actions that meet the constraints are sorted according to priority rules to obtain an internal action sequence. The internal action sequence serves as a set of sub-targets in the behavior execution layer and is used to drive the hierarchical execution control process.

[0083] In step 602, real-time monitoring acquires sensor data and converts the sensor data into a state vector.

[0084] The sensor processing module is used to process data from various sensors in the entire system, such as radar and cameras. It uses a multimodal fusion mechanism to detect environmental changes. The sensor processing module also needs to verify whether the action command meets the safety distance requirements based on the sensor data. For example, if the sensor data indicates that an obstacle has appeared near the original action command's trajectory, it needs to verify whether the safety distance between the action command and the obstacle can meet the requirements. The verification result is the state vector, which is then sent to the action execution module.

[0085] In step 603, both the HRL sub-objective and the state vector are input into the policy network in the behavior execution layer. The policy network encodes the features of the HRL sub-objective and the state vector respectively, and fuses the encoded features to generate a joint state representation. Based on the joint state representation, the policy network outputs the corresponding action policy distribution and generates multiple candidate control policies according to the preset action space. Combining the reward function corresponding to the HRL sub-objective, each candidate control policy is evaluated, and the expected reward value of each control policy is calculated. The control policy with the largest expected reward value is selected as the current optimal control policy and output to the action execution module for execution.

[0086] Furthermore, in this embodiment, after obtaining the control strategy, it is necessary to perform final verification and optimization of the control strategy to further optimize the execution effect. Therefore, this embodiment also involves the following design: the control strategy is filtered through predefined security rules to obtain an optimized strategy, and the corresponding controlled object is driven to execute the optimized strategy, such as... Figure 7 As shown, the method flow includes the following: In step 701, the control strategy is compared with each constraint in the predefined security rules to determine whether the control strategy meets each constraint.

[0087] In step 702, if the control strategy satisfies all constraints, then the control strategy is output as the optimization strategy.

[0088] In step 703, if the control strategy does not meet all the constraints, a security constraint penalty mechanism is introduced based on the policy network to modify the strategy, and / or an alternative control strategy that meets the constraints is regenerated based on the candidate policy set.

[0089] In step 704, if an alternative strategy that satisfies the constraints cannot be generated, an emergency control command is output to override the current control strategy, and the negative feedback information corresponding to the control process is recorded in the empirical data set for updating the strategy network.

[0090] Specifically, the policy output is constrained and optimized by introducing a safety penalty factor into the policy network input, and / or candidate control policies that meet the constraints are generated based on resampling of the action space. If multiple candidate control policies exist, each candidate control policy is selected based on a preset exploration-utilization balancing policy to obtain the optimal control policy that meets the constraints. If a candidate control policy that meets the constraints cannot be generated, an emergency control command is output to cover the current control policy. Control policies that conflict with safety rules are marked as negative feedback samples and stored in the experience replay pool for policy network updates.

[0091] In this embodiment, the action execution module is used to organize and execute action instructions using a hierarchical reinforcement learning framework, supports real-time adjustment and high-frequency action macro rule encapsulation (such as evasion maneuvers), and coordinates the action execution of each controlled object through joint reward optimization, and generates execution status feedback.

[0092] Furthermore, in this embodiment, after the optimization strategy is assigned to the corresponding controlled object for execution, it is necessary to monitor the execution status of each controlled object to provide feedback and adjustment to the planning decision layer, task coordination layer, and / or behavior execution layer, thereby optimizing the decision-making and planning of the entire task in real time and improving execution effect and efficiency. Therefore, this embodiment also involves the following design: real-time monitoring of the execution status of the controlled object and feedback to the planning decision layer and task coordination layer; the planning decision layer, task coordination layer, and / or behavior execution layer provide feedback and adjustment based on the execution status of the controlled object, such as... Figure 8 As shown, the method flow includes: In step 801, the execution status of the controlled object is obtained. The behavior execution layer aggregates and compresses the execution status of the controlled object to obtain the execution result, and uploads the execution result to the task collaboration layer.

[0093] In step 802, the task collaboration layer compares the execution trajectory in the execution result with the corresponding planning trajectory in the corresponding constraint subtask to obtain the execution deviation.

[0094] The execution deviation includes a weighted result of trajectory deviation, time deviation, and state deviation.

[0095] In step 803, when the deviation is less than or equal to the preset value, the controlled object continues to execute according to the optimization strategy.

[0096] In step 804, when the deviation is greater than the preset value, the replanning of the control strategy of the behavior execution layer is triggered, and it is determined within a preset time whether the execution deviation is less than or equal to the preset value.

[0097] The local strategy replanning includes online correction of the control strategy based on the current state and re-evaluation of the execution deviation within a preset time window; In step 805, if yes, the feedback adjustment of the behavior execution layer is completed; if no, the task coordination layer is triggered to redistribute the subtask list and determine whether the execution deviation is less than or equal to the preset value within a preset time.

[0098] If the execution deviation after replanning is still greater than the preset threshold, the task coordination layer is triggered to reassign subtasks. The subtask reassignment includes adjusting task priority, reallocating execution objects, and updating the subtask execution order. In step 806, if yes, the feedback adjustment of the task coordination layer is completed; if no, the replanning of the overall task of the planning decision layer is triggered; if the execution deviation is determined to be below the preset value within the preset time window after replanning, the feedback adjustment is completed; if the deviation still does not meet the standard after the time window is exceeded, an alarm signal is sent.

[0099] If the execution deviation still does not meet the requirements after task reassignment, the planning decision layer is triggered to perform global task replanning, regenerate the total task and its constraints, and update them synchronously to the task coordination layer. When the deviation decreases to within the preset threshold after replanning at any level, the feedback adjustment is completed; if the deviation constraint is not met after the preset time window is exceeded, an abnormal alarm signal is output.

[0100] It should be noted that if the execution deviation is determined to be below the preset value within the preset time window or number of retries, the feedback adjustment is completed; if the deviation still does not meet the standard after the time window or number of retries, a higher-level replanning mechanism is triggered or an abnormal safety mode is entered.

[0101] Example 2: Based on Example 1, this embodiment provides an implementation process for a specific scenario as follows: S1, System initialization.

[0102] S1.1 Module Loading: Start the planning and decision-making layer, task coordination layer and behavior execution layer, and complete the initialization of the shared state space.

[0103] S1.2 Digital Advisor Configuration: Activate the real-time scene information agent, terrain agent, and interacting object agent, set federated learning parameters, and use the federated averaging (FedAvg) algorithm to achieve multi-agent fusion.

[0104] S1.3 Communication Protocol Initialization: Establish the query-response protocol for Transformer to ensure that the communication latency between layers is less than 200ms.

[0105] S1.4 Loading the Regulations Template Library: Import the initial 10 types of regulations templates for interactive objects, and configure the online learning mechanism to support dynamic updates.

[0106] S1.5 Macro Action Library Settings: Load action sequences for the task collaboration layer and action sequences for the behavior execution layer.

[0107] S1.6 Termination Condition Definition: Set the criteria for achieving the task objective and the resource and time constraints to determine the triggering time for task completion or system replanning.

[0108] S2. Situation assessment and sketching.

[0109] S2.1 Task Reception: The planning and decision-making layer parses external task instructions through a cognitive model, extracts objectives, time windows, and resource constraints, and stores them in working memory.

[0110] S2.2 Multimodal intelligence fusion: The planning and decision-making layer uses convolutional neural network modules to process image data, parse text reports, and uses recurrent neural network modules to analyze time-series data; at the same time, a time attention mechanism is applied to prioritize the processing of the latest intelligence, reducing information processing latency to less than 0.5 seconds.

[0111] S2.3 On-site situation sketch construction: The planning decision layer uses graph neural networks to model the relationship between the controlled object, the interacting object and the terrain, outputs an on-site situation sketch, and marks the data confidence level. When the confidence level is lower than the preset threshold, supplementary reconnaissance is triggered.

[0112] S2.4 Knowledge Graph Update: Through the federated learning mechanism, the real-time scene information agent, terrain agent, and interacting object agent complete data processing locally, and share the model update to the central knowledge graph; the update result is embedded in declarative memory in the form of memory blocks.

[0113] S3. Predicting the intent of the interacting object and selecting the template.

[0114] S3.1 Analysis of the Action Sequence of the Interacting Object: The real-time scene information agent of the planning and decision-making layer processes the trajectory and configuration data of the interacting object, and matches it with the command template library in the declarative memory to generate an intention prediction of the interacting object.

[0115] S3.2 Simulated Sequence Generation: Generative adversarial networks are used to simulate the possible actions of the interacting object, and the generated simulated sequences are stored in declarative memory.

[0116] S3.3, Regulation Template Selection: Based on the Bayesian inference method, a prior distribution is formed by combining historical simulation data. The matching probability of each regulation template is calculated, and the template with the highest probability is selected as the basis for prediction and updated to working memory.

[0117] S3.4 Inference of Unobserved Targets: By combining a probabilistic graphical model with timeline constraints, the location of unobserved or unreported interacted objects is inferred, and the results are embedded into a unified knowledge graph.

[0118] S3.5 Prediction Consistency Verification: The real-time scene information agent, terrain agent, and interacting object agent cross-validate the prediction results to reduce the false positive rate; the verified prediction results are synchronously updated to working memory.

[0119] S4. Action plan generation and task decomposition.

[0120] S4.1 Action Plan Generation: The planning and decision-making layer generates 5-10 candidate plans. Combining the symbolic reasoning mechanism of the Adaptive Control of Thought-Rational (ACT-R) decision matrix, the resource consumption, risks and benefits of each plan are evaluated and stored in process memory in the form of production rules.

[0121] S4.2 Scheme Optimization and Selection: Monte Carlo tree search is used to evaluate the success rate of candidate schemes, select the optimal scheme, and determine the execution order through the priority scheduling mechanism of the ACT-R control component.

[0122] S4.3 Overall Task Decomposition: The overall task is decomposed into subtasks using timeline-enhanced HTN, and task dependencies are modeled using Graph Neural Network (GNN). Task time constraints are set, and the decomposition results (i.e., constrained subtasks) are embedded into a unified knowledge graph.

[0123] S4.4 Task Instruction Transmission: The planning and decision-making layer transmits the subtask list to the task coordination layer via the Transformer protocol; communication latency is controlled within 200ms. Instructions are formatted by the ACT-R motion module into JavaScript Object Notation (JSON) structure to ensure standardization and parsability of cross-layer transmission.

[0124] S5, Task Assignment.

[0125] S5.1 Instruction Reception: The high-level command agent of the task coordination layer receives the list of sub-tasks from the planning and decision-making layer, parses the task priorities and time constraints, and stores the information in the state space of the Leader-Follower Multi-Agent Proximal Policy Optimization (LFMAPPO) framework.

[0126] S5.2 Task Allocation: The high-level command agent coordinates the communication detection agent and the navigation agent based on the LFMAPPO framework, and applies a heterogeneous reward function to optimize task allocation.

[0127] S5.3 Allocation Optimization: Key observation information is shared through a course learning mechanism combined with a multi-head attention mechanism.

[0128] S5.4 Action Instruction Generation: Domain-specific agents load macro action libraries, refine task parameters, and evaluate task execution performance through a shared value network.

[0129] S5.5 Action Command Issuance: The generated task command is sent to the behavior execution layer through the Transformer's communication protocol, with the communication latency controlled within 200ms.

[0130] S6. Action Execution.

[0131] S6.1 Instruction Reception: The behavior execution layer parses the received task instructions, extracts the path and intent parameters, and stores them in the HRL state space.

[0132] S6.2 Sensor Data Fusion: Image data is processed through a Convolutional Neural Network (CNN), and time-series data is analyzed through a Recurrent Neural Network (RNN) to generate high-dimensional state input.

[0133] S6.3 Transfer Learning Application: Load the pre-trained model in the simulation environment to adapt to the current scenario.

[0134] S6.4 Control Policy Generation: Under the HRL framework, an adaptive control policy is generated by the underlying Proximal Policy Optimization (PPO) algorithm, while a greedy policy is applied to balance exploration and exploitation.

[0135] S6.5 Macro Action Execution: Load predefined action sequences to simplify the real-time decision-making process.

[0136] S6.6 Security Constraint Verification: Ensure actions comply with specifications through model checking tools and support multi-platform collaboration through a joint reward mechanism.

[0137] S6.7 Execution Feedback Upload: Real-time feedback of action execution status to the task collaboration layer. When the deviation exceeds the threshold, the task correction and replanning mechanism is triggered.

[0138] S7, Dynamic Monitoring and Replanning.

[0139] S7.1, Start of hibernation mode: The planning decision-making level sets the adaptive hibernation period (default 2-5 minutes, which can be dynamically adjusted according to the on-site scenario) through the ACT-R control component.

[0140] S7.2 Task Progress Monitoring: The system periodically wakes up to obtain the latest data from the digital advisor and updates it through multimodal large language model (MLLM) and embeds a unified knowledge graph.

[0141] S7.3, Abnormal Event Detection: Use the Isolation Forest algorithm to identify sudden events such as the movement of the interacting object and trigger the ACT-R control component to wake up immediately.

[0142] S7.4 Event Impact Analysis: If an abnormal event is detected, rerun steps S2-S4 to update the situation sketch, the predicted intent of the interacting object, and the solution, and simultaneously update the ACT-R working memory.

[0143] S7.5 Replanning Trigger: When the prediction confidence is lower than 0.8 or the plan of the interacting object changes, the planning decision layer updates the GNN subgraph through incremental learning and reruns steps S4.1-S4.4 to realize task replanning. The replanning time is controlled within 0.5 seconds.

[0144] S7.6 Instruction Synchronization Adjustment: Synchronize the updated instructions to the task coordination layer and behavior execution layer through the Transformer's communication protocol, and rerun steps S5-S6.

[0145] S7.7 Execution Feedback Verification: The task collaboration layer analyzes the feedback from the behavior execution layer, models the uncertainty through a Bayesian network, and adjusts the task allocation if the deviation exceeds 5%. The planning decision layer then evaluates whether replanning is necessary.

[0146] S8, Exception Handling.

[0147] S8.1 Handling of missing intelligence: When the data confidence level is below 0.8, the planning and decision-making level suspends task decomposition, triggers supplementary reconnaissance, until the confidence level is restored, and records the status to working memory.

[0148] S8.2 Communication Interruption Handling: If the inter-layer communication delay exceeds 200ms, the task coordination layer and behavior execution layer switch to the local caching strategy and prioritize the execution of macro action sequences.

[0149] S8.3 Execution Deviation Management: When the deviation reported by the behavior execution layer exceeds 5%, the task coordination layer reallocates tasks through LFMAPPO, and the planning and decision-making layer evaluates whether to replan through the ACT-R control component.

[0150] S8.4 Emergency Response: When an emergency is detected, the planning decision layer is immediately awakened through the isolated forest algorithm and runs steps S2-S4 with high priority.

[0151] S8.5 Controlled Object Fault Recovery: If a single controlled object fails, the behavior execution layer reallocates tasks through a joint reward mechanism, the task coordination layer updates instructions, and the knowledge graph of the planning decision layer is updated synchronously.

[0152] S9. Terminate the process.

[0153] S9.1 Task Completion Verification: The planning and decision-making level confirms that the goal has been achieved through cross-validation of multi-source data (confidence level > 0.95), and the ACT-R control component confirms termination.

[0154] S9.2 Termination Condition Check: When resources are exhausted or the task time window is closed, the system stops task execution.

[0155] S9.3, Operation Record Saving: Stores the situation sketch and task execution log, shuts down all agents and communication protocols, and terminates upon task completion.

[0156] Example 3: Based on the autonomous cooperative control method based on multi-agent hierarchical behavior provided in the foregoing embodiments, the present invention also provides an apparatus for implementing the above-mentioned method based on autonomous cooperative control of multi-agent hierarchical behavior, such as... Figure 9 The diagram shown is a schematic representation of the device architecture according to an embodiment of the present invention. The device for autonomous cooperative control based on multi-agent hierarchical behavior in this embodiment includes one or more processors 41 and a memory 42. Figure 9 Take a processor 41 as an example.

[0157] Processor 41 and memory 42 can be connected via a bus or other means. Figure 9 Taking the example of a connection between China and Israel via a bus.

[0158] The memory 42, as a non-volatile computer-readable storage medium for autonomous cooperative control based on multi-agent hierarchical behavior, can be used to store non-volatile software programs and non-volatile computer-executable programs, such as the autonomous cooperative control method based on multi-agent hierarchical behavior in the foregoing embodiments. The processor 41 executes various functional applications and data processing of the autonomous cooperative control device based on multi-agent hierarchical behavior by running the non-volatile software programs, instructions, and modules stored in the memory 42, thereby realizing the autonomous cooperative control method based on multi-agent hierarchical behavior in the foregoing embodiments.

[0159] Memory 42 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 42 may include memory remotely located relative to processor 41, which can be connected to processor 41 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0160] The program instructions / modules are stored in memory 42. When executed by one or more processors 41, they perform the autonomous cooperative control method based on multi-agent hierarchical behavior in the foregoing embodiments, for example, by performing the steps shown in the flowcharts described above.

[0161] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 9One of the processors 41 enables the above-described processors to execute the autonomous cooperative control method based on multi-agent hierarchical behavior in the foregoing embodiments, for example, to perform the steps shown in the various figures described above.

[0162] It is worth noting that the information interaction and execution process between the modules and units in the above-mentioned device and system are based on the same concept as the processing method embodiment of the present invention. For details, please refer to the description in the method embodiment of the present invention, and will not be repeated here.

[0163] Those skilled in the art will understand that all or part of the steps in the various methods of the embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0164] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An autonomous cooperative control method based on multi-agent hierarchical behavior, characterized in that, A multi-agent hierarchical autonomous cooperative control model is constructed, comprising a planning and decision-making layer, a task coordination layer, and a behavior execution layer, including: The planning and decision-making level collects environmental data, makes environmental predictions based on the environmental data, and generates corresponding overall tasks based on the results of the environmental predictions. The task collaboration layer decomposes the total task and preprocesses it to obtain a list of sub-tasks, and generates action instructions based on the list of sub-tasks. The behavior execution layer monitors and acquires sensor data in real time, fuses the sensor data and the action commands to generate a control strategy, filters the control strategy through predefined safety rules to obtain an optimized strategy, drives the corresponding controlled object to execute the optimized strategy, monitors the execution status of the controlled object in real time, and feeds it back to the planning decision layer and the task coordination layer. Feedback adjustment is triggered step by step between the behavior execution layer, the task coordination layer, or the planning decision layer to improve the adaptability of the scheduling strategy to dynamic scenarios.

2. The autonomous cooperative control method based on multi-agent hierarchical behavior according to claim 1, characterized in that, The process of collecting environmental data through a planning and decision-making layer, making environmental predictions based on the environmental data, and generating corresponding overall tasks based on the results of the environmental predictions specifically includes: Environmental features and rules are pre-encapsulated and stored in the form of memory blocks, task decisions are pre-stored, and task decisions are mapped to corresponding memory blocks; Real-time acquisition of environmental data, and selection of suitable memory blocks as the prediction memory blocks corresponding to the environmental data; Obtain the task decision corresponding to the predicted memory block, and use the corresponding task decision as the total task.

3. The autonomous cooperative control method based on multi-agent hierarchical behavior according to claim 2, characterized in that, The real-time acquisition of environmental data and the selection of suitable memory blocks as prediction memory blocks corresponding to the environmental data specifically includes: Real-time acquisition of environmental data; Extract multiple memory blocks that match the environmental data, and perform weighted fusion based on the environmental data and the matching memory blocks to obtain the confidence score corresponding to each memory block. The expression for the confidence score is: ； in, For confidence level, For state similarity, For temporal similarity, For structural similarity, For semantic similarity, These are the weighting coefficients for state similarity. These are the weighting coefficients for temporal similarity. These are the weighting coefficients for structural similarity. The weighting coefficients for semantic similarity; The memory block with the highest confidence level is used as the prediction memory block corresponding to the environmental data.

4. The autonomous cooperative control method based on multi-agent hierarchical behavior according to claim 2, characterized in that, The task coordination layer decomposes the total task and preprocesses it to obtain a list of subtasks, specifically including: Pre-construct a unified knowledge graph; In the planning and decision-making layer, semantic association and constraint reasoning of the environmental state are performed based on the unified knowledge graph to generate the constraint conditions corresponding to the overall task; In the task collaboration layer, the total task is decomposed into multiple constrained subtasks according to the task dependencies, temporal constraints and the constraints in the unified knowledge graph through a hierarchical task decomposition model based on temporal constraints. Obtain the priority of each constraint subtask, and sort the constraint subtasks based on the priority to obtain the subtask list.

5. The autonomous cooperative control method based on multi-agent hierarchical behavior according to claim 4, characterized in that, The step of generating action instructions based on the subtask list specifically includes: Multiple domain agents are set up, each of which is used to handle different types of constrained sub-tasks. Global value information is shared through a shared value network, and the relationship between the domain agents is established through a multi-head attention mechanism to achieve cross-agent collaborative modeling. Generate the corresponding formation strategy based on the constrained subtasks; Assign each constrained subtask and its corresponding formation strategy in the subtask list to the corresponding domain agent; Each domain-specific agent generates intent parameters based on the constrained subtasks and corresponding formation strategies, and generates an executable macro-action sequence based on the intent parameters; The macro action sequence is verified for feasibility. Macro action sequences that do not meet the preset constraints are eliminated, and the verified macro action sequences are output as action commands.

6. The autonomous cooperative control method based on multi-agent hierarchical behavior according to claim 1, characterized in that, The behavior execution layer monitors and acquires sensor data in real time, and fuses the sensor data and the action commands to generate a control strategy, specifically including: The behavior execution layer receives action instructions from the task coordination layer and parses the action instructions into HRL sub-targets; Real-time monitoring acquires sensor data, and the sensor data is converted into a state vector; Both the HRL sub-objective and the state vector are input into the policy network in the behavior execution layer. The policy network outputs the control policy that best matches the corresponding reward value based on the reward value corresponding to the HRL sub-objective and the state vector.

7. The autonomous cooperative control method based on multi-agent hierarchical behavior according to claim 6, characterized in that, The step of filtering the control strategy through predefined security rules to obtain an optimized strategy, and driving the corresponding controlled object to execute the optimized strategy, specifically includes: The control strategy is compared with each constraint in the predefined security rules to determine whether the control strategy satisfies each constraint. If the control strategy satisfies all constraints, then the control strategy will be output as the optimized strategy. If the control strategy does not meet all the constraints, a security constraint penalty mechanism is introduced based on the policy network to modify the strategy, and / or an alternative control strategy that meets the constraints is regenerated based on the candidate policy set. If an alternative strategy that satisfies the constraints cannot be generated, an emergency control command is output to override the current control strategy, and the negative feedback information corresponding to the control process is recorded in the empirical data set for updating the policy network.

8. The autonomous cooperative control method based on multi-agent hierarchical behavior according to claim 1, characterized in that, The real-time monitoring of the execution status of the controlled object and the feedback to the planning decision layer and task coordination layer, triggering feedback adjustments at each level between the behavior execution layer, task coordination layer, or planning decision layer, specifically includes: The execution status of the controlled object is obtained. The behavior execution layer aggregates and compresses the execution status of the controlled object to obtain the execution result, and then uploads the execution result to the task collaboration layer. The task collaboration layer compares the execution trajectory in the execution result with the corresponding planning trajectory in the corresponding constraint subtask to obtain the execution deviation. When the deviation is less than or equal to the preset value, the controlled object continues to execute according to the optimization strategy. When the deviation exceeds the preset value, the control strategy of the behavior execution layer is replanned, and the execution deviation is determined within a preset time to be less than or equal to the preset value. If yes, the feedback adjustment of the behavior execution layer is completed; if not, the task coordination layer is triggered to redistribute the subtask list and determine whether the execution deviation is less than or equal to the preset value within a preset time. If yes, the feedback adjustment of the task coordination layer is completed; if not, the replanning of the overall task of the planning and decision-making layer is triggered; if the execution deviation is determined to be below the preset value within the preset time window after replanning, the feedback adjustment is completed; if the deviation still does not meet the standard after the time window is exceeded, an alarm signal is sent.

9. An autonomous cooperative control device based on hierarchical behavior of multiple agents, characterized in that, The method includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the autonomous cooperative control method based on multi-agent hierarchical behavior as described in any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by one or more processors, implement the autonomous cooperative control method based on multi-agent hierarchical behavior as described in any one of claims 1-8.

Images