Multi-Agent cooperative control method, system and device and storage medium

Abstract

The invention provides a multi-Agent cooperative control method and system, computer equipment and a storage medium, and belongs to the field of agent control, and the method comprises the steps: obtaining user intention data; determining an action parameter corresponding to the intention category based on a predefined intention-to-action mapping rule, and constructing a multi-agent brain control action instruction; obtaining a current environment state, retrieving memory entries in a brain-like memory library according to the current environment state, obtaining adjacent tracks and determining candidate actions; according to the current environment state and the candidate action, an intelligent control action instruction of the multi-agent is constructed, and memory entries associated with states, actions, track fragments and track scores are pre-stored in a brain-like memory library; and performing gating switching according to the brain control activation degree, and performing linear weighting on the intelligent control action instruction and the brain control action instruction to construct a fusion action instruction. According to the method, two action instructions are fused to realize multi-agent control, so that the robustness, the response speed and the environmental adaptability of the strategy are improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent agent control, specifically relating to a multi-agent collaborative control method, system, computer device, and storage medium. Background Technology

[0002] With the development of intelligent autonomous systems, multi-agent systems have been widely used in fields such as unmanned swarm control, disaster emergency response, and intelligent collaborative robots. These systems accomplish complex tasks through the collaboration of multiple agents, demonstrating good capabilities in distributed execution, self-organizing behavior, and swarm intelligence. However, in practical applications, especially under complex environments and dynamic task-driven conditions, existing multi-agent systems still have significant limitations in policy generation, human-machine integration, task collaboration, and security control.

[0003] Traditional multi-agent decision-making systems are typically trained using reinforcement learning methods, employing policy gradient methods such as MADDPG (Multi-Agent Deep Deterministic Policy Gradient). During training, they rely on high-frequency interactions with the environment and sample optimization strategies from an experience pool. For example, the deep reinforcement learning-based UAV swarm collaborative combat decision-making method proposed by Zhao et al. has achieved some success in terms of mission execution efficiency and behavioral diversity. However, existing multi-agent decision-making systems are affected by the quality of online samples, making it difficult to achieve rapid generalization in unknown environments and resulting in poor applicability in complex scenarios. Summary of the Invention

[0004] To address the problem that existing multi-agent decision-making systems rely too heavily on training samples and have poor applicability in complex scenarios, this invention provides a multi-agent collaborative control method, system, computer device, and storage medium.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A multi-agent collaborative control method includes: Acquire user intent data; based on the intent category corresponding to the user intent data, determine the action parameters corresponding to the intent category based on the predefined intent-to-action mapping rules, and construct brain-controlled action instructions for multiple agents; The system obtains the current environmental state; retrieves memory entries from the brain-like memory bank shared by multiple agents based on the current environmental state, obtains nearby trajectories and determines candidate actions; and constructs intelligent control action instructions for multiple agents based on the current environmental state and candidate actions. The brain-like memory bank pre-stores memory entries related to states, actions, trajectory fragments and trajectory scores. Gating is performed based on the brain-controlled activation level in the user intent data. Based on the gating switching results, intelligent control action commands and brain-controlled action commands are linearly weighted to construct fusion action commands for multiple agents.

[0006] Optionally, the multi-agent collaborative control method provided by the present invention further includes: Analyze user intent data and determine the intent category with the highest activation from the set of intent categories; When the difference in activation between the highest activation category and the other categories in the intent category set is greater than the activation threshold, the corresponding action template is called according to the intent category with the highest activation, based on the mapping rule from intent to action, to determine the action parameters. When the difference in activation between the highest activation category and the other multiple categories in the intent category set is less than the activation threshold, the action templates corresponding to the highest activation category and the other multiple categories are fused according to activation weight to determine the action parameters after intent fusion.

[0007] Optionally, the neighboring trajectories are multiple neighboring trajectory segments. The multi-agent cooperative control method provided by this invention further includes: The current environment state is processed by a state encoder to obtain a state embedding representation; In the brain-like memory bank, K-nearest neighbor retrieval is performed on the state embedding representation based on Euclidean distance to obtain multiple neighboring trajectory segments; Candidate actions are determined based on the nearest trajectory segment with the highest trajectory score; The vectors of the current environment state and candidate actions are concatenated by a diffusion model and iteratively denoised to determine the mean value of the generated actions. Based on the mean value of the actions, intelligent control action instructions for multiple agents are constructed.

[0008] Optionally, the multi-agent collaborative control method provided by the present invention further includes: The training data is fed into the untrained diffusion model to predict the action mean. Calculate the behavioral value loss based on the action mean prediction result and the strategy evaluation function. The consistency loss of behavior distribution is calculated based on the prediction results of the action mean and the actions corresponding to the high-expectation trajectories in the brain-like memory bank. The diffusion model is adjusted based on behavioral value loss and behavioral distribution consistency loss.

[0009] Optionally, the user intent data includes brain control activation level, and the current environmental state includes the physical distance between the agent and the human. The multi-agent collaborative control method provided by this invention also includes: When the gating switching result is that the brain-controlled activation level is greater than the activation threshold, the fusion action command is determined according to the brain-controlled action command; When the gating switching result is that the brain control activation degree is less than the activation degree threshold, the weight of the brain control action command is calculated based on the brain control activation degree and the physical distance between the agent and the human. The intelligent control action command and the brain control action command are weighted and fused according to the weight of the brain control action command to obtain the fused action command.

[0010] Optionally, the multi-agent collaborative control method provided by the present invention further includes: Based on fused action instructions, a graph neural network is used to encode and aggregate neighbor messages for multiple agents, and the aggregated neighbor messages are decoded to generate action vectors for multiple agents. In this graph neural network, the nodes are agents, and the features of the nodes include the current environmental state of the agent. The edges of the graph neural network are determined by the spatial proximity relationships between multiple agents.

[0011] Optionally, the multi-agent collaborative control method provided by the present invention further includes: The system judges the action jump and action conflict degree of the intelligent control action command. When the action jump is greater than the preset action change threshold, or the action conflict degree between the brain control action command and the intelligent control action command is greater than the preset action difference threshold, an alternative intelligent control action command is constructed based on the remaining candidate actions.

[0012] The present invention also provides a multi-agent collaborative control system, comprising: The brain control command construction module is used to acquire user intent data; based on the intent category corresponding to the user intent data, it determines the action parameters corresponding to the intent category based on the predefined intent-to-action mapping rules and constructs multi-agent brain control action commands; The intelligent control instruction construction module is used to obtain the current environmental state; retrieve memory entries in the brain-like memory bank shared by multiple agents based on the current environmental state, obtain the adjacent trajectories and determine the candidate actions; construct intelligent control action instructions for multiple agents based on the current environmental state and candidate actions, wherein the brain-like memory bank pre-stores memory entries related to state, action, trajectory fragments and trajectory scores; The instruction fusion module is used to switch gating based on the brain control activation level in the user intent data. Based on the gating switching result, the intelligent control action instruction and the brain control action instruction are linearly weighted to construct a fused action instruction for multiple agents.

[0013] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in a multi-agent cooperative control method.

[0014] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any step of a multi-agent cooperative control method.

[0015] The multi-agent collaborative control method provided by this invention has the following beneficial effects: Because the multi-agent collaborative control method provided by this invention can collect human intentions and construct brain-controlled action instructions, it reduces the dependence of strategy generation on environmental interaction samples, provides clear behavioral guidance in sudden tasks, and significantly improves response efficiency and task adaptability. Furthermore, the method of retrieving candidate actions that match the multi-agent environmental state based on a brain-like memory bank and generating intelligent control action instructions utilizes historical trajectories as behavioral priors to enhance the generalization ability of state matching. It also optimizes the quality of action generation through a trajectory scoring mechanism, enabling rapid policy reconstruction in unknown environments and reducing dependence on online samples. Finally, the fusion of the two types of action instructions achieves multi-agent control, realizing soft collaboration of "intention-driven + autonomous reasoning." When samples are scarce or the environment changes abruptly, brain-controlled intentions compensate for the uncertainty of autonomous strategies, further strengthening the overall stability and generalization ability of the system. In summary, the multi-agent collaborative control method provided by this invention improves the robustness, response speed, and environmental adaptability of the strategy, avoids the dependence on samples and generalization limitations of existing agent control methods, and enhances the applicability of multi-agent collaborative control methods in complex environments. Attached Figure Description

[0016] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a multi-agent collaborative control method provided in an embodiment of the present invention; Figure 2 The system overall architecture diagram provided for embodiments of the present invention; Figure 3 This is a block diagram of a brain control mode provided in an embodiment of the present invention; Figure 4 This is a block diagram of the intelligent control mode provided in an embodiment of the present invention; Figure 5 This is a diagram of a multi-Agent collaborative hybrid decision-making module provided in an embodiment of the present invention; Figure 6 The overall flowchart provided for embodiments of the present invention; Figure 7This is a schematic diagram of the simulation environment provided in the embodiments of the present invention; Figure 8 Performance comparison chart of the algorithms provided in this embodiment of the invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0019] For the value estimation problem in the state-action mapping process of multi-agent systems, value function decomposition-based methods such as QMIX are widely used. This method achieves centralized training and distributed execution by monotonically decomposing the joint value function, and has achieved good results in some collaborative tasks. However, QMIX and its variants still suffer from inaccurate estimation and slow policy convergence when facing out-of-distribution samples or dynamic environmental changes, limiting their application scope in practical multi-agent deployments.

[0020] Diffusion models, as a powerful generative model, are used to generate diverse and stable policy behavior sequences. For example, DOM2 (Diffuser for Offline Multi-Agent Decision Making) proposes to model policy trajectory distribution using a conditional diffusion model and generate actions through a reverse denoising process. This method significantly improves the stability and sample efficiency of offline policy generation. However, current work mainly focuses on single-agent or fully autonomous agent systems and has not yet organically integrated diffusion models with mechanisms such as human intent, hybrid control, and multi-agent graph structure collaborative reasoning.

[0021] Furthermore, in the field of collaborative control, policy modeling based on graph neural networks has also received widespread attention. The graph-based coordination strategy (GCS) proposed by Ruan et al. achieves effective propagation of local information and enhances collaborative consistency by constructing an adjacency graph structure among agents. However, its application is still limited to fully autonomous systems, lacking the ability to fuse and model human intentions, and failing to consider human safety intervention and multi-source fusion scheduling mechanisms in the control chain.

[0022] On the other hand, the development of brain-computer interface technology enables human operators to convey intention commands in real time through electroencephalogram (EEG) signals, which external systems can then parse into structured intention vectors. How to effectively introduce these high-level intention commands into a multi-agent control system and construct a hybrid "brain-controlled + intelligent-controlled" structure has become an important research direction. Wiem Zemzem et al. proposed a preliminary human-machine hybrid method, but this system still mainly relies on manual feedback adjustment and cannot form a universal and efficient intention-driven strategy generation path.

[0023] Furthermore, brain-like memory mechanisms can provide agents with valuable historical trajectory support, offering effective priors for state matching, action reasoning, and behavior retrieval. Research by Gupta et al. in the single-agent domain has shown that memory systems with expectation labels can improve the efficiency and stability of action generation. However, there is currently no systematic solution for introducing brain-like memory mechanisms into multi-agent systems and combining them with diffusion strategy models and intent input to achieve structured behavior support and a closed loop of strategy evolution.

[0024] In terms of security, existing multi-agent systems generally lack human intervention mechanisms for the policy generation and control command execution processes. If a policy deviates or execution malfunctions, the system struggles to promptly halt the control chain, posing potential security risks. Furthermore, there are no robust manual emergency takeover mechanisms in place to handle sudden changes in the task environment or communication anomalies, limiting the system's reliability in critical scenarios.

[0025] In summary, while relevant technical personnel have achieved certain results in multi-agent systems, the following shortcomings exist: First, there is a lack of agent hybrid control mechanisms based on human intention input; second, there is a lack of brain-like memory-driven action reasoning and trajectory support methods; third, existing diffusion models have not yet been integrated with human-machine hybrid control and group collaborative architecture; furthermore, there is a lack of task consistency guarantee mechanisms supported by graph structures among multiple agents in multi-agent systems; and finally, existing multi-agent systems lack human intervention and security takeover mechanisms for the entire control strategy chain.

[0026] To address the shortcomings of existing agent systems, the multi-agent collaborative control method provided by this invention integrates brain-controlled intention input, brain-like memory support, and diffusion strategy learning mechanism, enabling functions such as multi-source control fusion, policy adaptive evolution, and group-consistent collaboration.

[0027] Example 1 This invention provides a multi-agent collaborative control method, specifically as follows: Figure 1 As shown, it includes the following steps: Step 11: Obtain user intention data; based on the intention category corresponding to the user intention data, determine the action parameters corresponding to the intention category according to the predefined mapping rules from intention to action, and construct a brain-controlled action instruction for multiple agents.

[0028] Among them, Step 11 includes: Step 111: Analyze the user intention data and determine the intention category with the highest activation degree from the set of intention categories.

[0029] Step 112: When the difference in activation degree between the category with the highest activation degree in the set of intention categories and the activation degrees of the remaining multiple categories is greater than the activation degree threshold, based on the mapping rules from intention to action, call the corresponding action template according to the intention category with the highest activation degree, and determine the action parameters.

[0030] Step 113: When the difference in activation degree between the category with the highest activation degree in the set of intention categories and the activation degrees of the remaining multiple categories is less than the activation degree threshold, perform weighted fusion of the action templates corresponding to the category with the highest activation degree and the remaining multiple categories according to the activation degree, and determine the action parameters after intention fusion.

[0031] Specifically, in the multi-Agent collaborative control method provided by the present invention, there are simultaneously Agents driven by a brain-computer interface, that is, Agents in the brain-controlled mode; and Agents operating based on autonomous strategies, that is, Agents in the intelligent control mode. Among them, a brain-controlled Agent means that its behavior decision-making is directly guided by the intention vector provided by a human operator through a brain-computer interface, and an intelligent control Agent means an autonomous control unit with environmental perception, autonomous reasoning, and action generation capabilities, which is applicable to typical platforms such as unmanned equipment and robot swarms. In this system, all Agent nodes uniformly perform control output through a policy generation module, and its policy is jointly driven and generated by the environmental state, historical experience, and external Figure 3 agents. The brain-controlled mode means that the system receives the structured intention vector of a human operator in real time through an external brain-computer interface module and converts it into a control action according to a preset mapping relationship; the intelligent control mode means that an Agent generates a control instruction under the guidance of a policy model such as a diffusion model based on environmental observations, historical trajectories, and results of brain-like memory retrieval. The two control modes can operate independently or can be fused and scheduled according to task requirements, supporting online switching and dynamic priority adjustment.

[0032] Considering that multi-Agent systems generally rely on local state input and independent reasoning of policy networks, lacking global control consistency driven by human intentions. In the face of sudden task changes or drastic environmental changes, the policy response is lagging, the behavior adaptability is poor, it is difficult to achieve rapid reconstruction and efficient linkage, and there is a lack of a human safety review mechanism for operators, resulting in potential risks of uncontrollable operations. The multi-Agent architecture provided by the present invention is as Figure 2As shown, firstly, a brain-computer interface-based intention input channel transforms high-level human intentions into system-recognizable vector inputs. Through intention-action template mapping, rapid response control of the brain-controlled agent is achieved, improving task scheduling efficiency and the smoothness of human-computer interaction. Secondly, a brain-like memory module records historical high-value trajectory data, constructing a memory bank in the form of state-action-reward-expectation labels. This supports the agent in performing prior behavioral reasoning through state embedding matching and experience trajectory retrieval, effectively improving strategy quality and generation stability.

[0033] Furthermore, an autonomous policy generation mechanism based on a conditional diffusion model is employed. Using the current state embedding and historical trajectory priors as inputs, a reverse denoising sampling path is used to generate control actions. Simultaneously, a behavioral value evaluation function is introduced as the optimization objective to achieve distributional approximation and value maximization of policy behaviors, enhancing policy diversity, generalization ability, and environmental adaptability. Regarding policy integration, a brain-controlled and intelligent control fusion mechanism is established, introducing adjustable fusion weights to support action-level soft fusion or priority control switching. Combined with a control mode scheduler, parallel operation and load balancing of human and autonomous control are achieved, ensuring optimal allocation of control resources during task execution.

[0034] Simultaneously, a multi-agent collaborative reasoning module based on graph neural networks is constructed. The communication relationships between agents are modeled as an adjacency graph structure. Multi-level information propagation and joint policy updates are performed through graph neural networks, improving the consistency of policies and the efficiency of collaborative coupling within the cluster. This is suitable for stable operation in scenarios with frequent task topology changes or limited communication. Furthermore, to ensure the system's security and controllability in real-world environments, a two-stage manual intervention mechanism is designed: "before policy generation + before control execution." Operators can review, modify, or suspend policy results at critical nodes. An emergency takeover mechanism is also established. In the event of system policy anomalies, control failures, communication drift, or other emergencies, operators can access the system control channel with the highest priority and forcibly take over control of some or all agents, ensuring uninterrupted tasks and system stability.

[0035] In summary, through the synergy of the above modules and the integration of mechanisms, this invention ultimately constructs a multi-agent human-machine hybrid intelligent control system with human brain intention understanding, autonomous strategy generation, cluster strategy collaboration, and human intervention channels. It is widely applicable to intelligent decision-making application scenarios with high dynamics, high risks, and high collaboration requirements, such as brain-controlled unmanned equipment, human-machine hybrid formations, intelligent transportation systems, and emergency response platforms.

[0036] Specifically, in the multi-agent collaborative control method provided by this invention, the acquisition of user intent data and the construction of brain-controlled action commands are as follows: The first step is to construct a multi-agent subsystem under the mind control model. Specifically, based on the current task requirements and system state, the agent set is configured... Select a subset of Agents The mind control model accepts human intentions for control, among which .

[0037] Define intent category C set to connect user intent data obtained from external brain-computer interfaces. With respect to specific intention categories, Each of them Corresponding to a higher-level human intention, for example Advance accordingly, Corresponding to a right turn, Corresponding to the rise, Corresponding collaboration, and the aforementioned set of intent categories is set during the system initialization phase and does not change dynamically over time.

[0038] This design establishes a mapping rule from intention to action, targeting the intention signals parsed by the brain-computer interface, and establishing a mapping mechanism from intention category to physical action parameters. For example, the mapping rule could be a correspondence between intention categories and action parameter modules. Each intent category , Linear velocity, Angular velocity, For the change in height, This represents the yaw angle transformation. The aforementioned mapping rules are then used to train the brain-controlled agent subsystem.

[0039] After the mind-controlled agent subsystem completes training, it receives parsed user intent data in real time through an external brain-computer interface module. The received intent vector is recorded as follows. ,in This represents the analysis result of the human brain's intention at global time t; This represents the activation intensity of the j-th intent category at the current time t; The number of intent categories defined for the system, such as forward, turn, collaborate, request help, etc. During the user intent data acquisition phase, only the intent vector data is received and timestamps are recorded; the intent vector itself is not... Internal features are extracted, parsed, or modified to ensure the timeliness and completeness of the information, which can then be used by subsequent action generation modules.

[0040] Specifically, such as Figure 3 As shown, for each Agent in a mind-control state, Based on the real-time received intent vectors, the intent category with the highest activation level is determined. The specific intent category is determined as shown in formula (1): (1) Then, based on the intention category with the highest activation level, the corresponding action template is invoked according to the pre-set mapping mechanism from intention category to physical action parameters, thereby determining the brain-controlled action instruction. The corresponding action template is shown in formula (2): (2) Furthermore, when the activation levels of multiple intent categories in the parsed intent vector are close, for example, when the difference in activation levels of multiple intent categories is less than a threshold, the multiple intent categories are weighted and fused by activation intensity to obtain a comprehensive brain-controlled action command, as shown in formula (3): (3) in, Assign weights to each of the multiple intent categories. .

[0041] Step 12: Obtain the current environmental state; based on the current environmental state, retrieve memory entries in the brain-like memory bank shared by multiple agents to obtain neighboring trajectories and determine candidate actions; construct intelligent control action instructions for multiple agents based on the current environmental state and candidate actions, wherein the brain-like memory bank pre-stores memory entries related to states, actions, trajectory fragments and trajectory scores.

[0042] Among them, the neighboring trajectory consists of multiple adjacent trajectory segments, and step 12 includes: Step 121: Process the current environment state through a state encoder to obtain a state embedding representation.

[0043] Step 122: In the brain-like memory bank, perform K-nearest neighbor retrieval on the state embedding representation based on Euclidean distance to obtain multiple neighboring trajectory segments.

[0044] Step 123: Determine candidate actions based on the nearest trajectory segment with the highest trajectory score.

[0045] Step 124: Input the training data into the untrained diffusion model to predict the action mean.

[0046] Step 125: Calculate the behavioral value loss based on the action mean prediction result and the policy evaluation function.

[0047] Step 126: Calculate the behavioral distribution consistency loss based on the prediction results of the action mean and the actions corresponding to the high-expectation trajectories in the brain-like memory bank.

[0048] Step 127: Adjust the diffusion model based on behavioral value loss and behavioral distribution consistency loss.

[0049] Step 128: Concatenate the vectors of the current environment state and candidate actions using the diffusion model and iteratively denoise to determine the mean value of the generated actions. Construct intelligent control action instructions for multiple agents based on the mean value of the actions.

[0050] Specifically, in the multi-agent collaborative control method provided by this invention, the acquisition of user intent data and the construction of brain-controlled action commands are as follows: First, the brain-like memory bank M is constructed. The brain-like memory bank stores state-action-reward-trajectory data accumulated during the interaction process. Each memory entry contains a quintuple as shown in formula (4):

[0051] (4) in, Let this be the state of the Agent at time t. The action to be performed in this state. The immediate reward obtained after an action. The next state reached after the action. This is the expected label. The expected label is shown in formula (5):

[0052] (5) Original state The system integrates environmental situational information such as local target distribution, risk areas, and task stage labels for semantic collaboration and differential expression among multiple agents in the state embedding space. The memory module is continuously and dynamically updated during task execution, prioritizing the retention of high-value trajectories to form a historical experience memory bank.

[0053] In the multi-agent cooperative control method provided by this invention, a state encoder is used. State decoder A reward predictor is used to retrieve similar states and corresponding actions from a brain-like memory database. This is achieved through a state encoder. The state vector of the current environment state Mapped to a low-dimensional embedding representation, state embedding This is represented as shown in formula (6):

[0054] (6) Then through the state decoder State reconstruction is performed, and return estimation is performed using a return predictor, as shown in Equations (7) and (8): (7) (8) During the training phase, the optimization objective is to minimize the weighted sum of the reconstruction error and the prediction error, as shown in formula (9): (9) in, This represents the balance coefficient. This embedding is used for subsequent retrieval and policy modeling, combining state compression and semantic expressiveness. After training, as shown... Figure 4 As shown, for multiple agents in the intelligent control mode, via state encoder Extract the current state embedding from the current environmental state. In the brain-like memory bank In the above, the nearest neighbor trajectory is obtained by K-Nearest Neighbor retrieval based on Euclidean distance, as shown in formula (10):

[0055] (10) Subsequently, for each retrieved trajectory segment, a target label is assigned based on the trajectory's expected value. Instant rewards for fragments The scoring is as shown in formula (11): (11) in, and These are the weighting coefficients corresponding to the expected label and the immediate reward, respectively. After each trajectory segment has been scored, candidate actions can be determined in the following way, thereby generating intelligent control action commands:

[0056] On the one hand, reference movements can be selected from candidate movements based on the movement corresponding to the highest-scoring trajectory. And generate intelligent control action commands, as shown in formula (12): (12) in On the other hand, Softmax weighted sampling can be performed based on the scores corresponding to each trajectory segment. The sampling frequency during weighted sampling is calculated as shown in formula (13):

[0057] (13) in, For temperature parameters, the degree of randomness in sampling is controlled. Finally, a reference action is generated based on the selected mode. Furthermore, the retrieved k trajectory actions can be weighted and fused, that is, by using a distance weighting mechanism, higher weights are given to historical actions that are closer to the current state, further improving the rationality of the actions. Here, z is a normalization factor, ensuring that the sum of all weights is 1. The determination of the reference action is shown in formula (14):

[0058] (14) in, An embedding representation of the state in the trajectory. These are distance-weighted coefficients. .

[0059] For example, in the multi-agent collaborative control method provided by this invention, intelligent control action instructions can be generated through a diffusion model. Specifically, the diffusion model not only references the current state embedding but also integrates historical experience and task preference factors, enabling the action generation process to possess dynamic adaptability and consistency with human operational logic. In the diffusion strategy model, the generation of policy actions for each agent is jointly controlled by three types of inputs: the state embedding vector, candidate reference actions, and control preference vector. Among these, the current state embedding... ,in Indicates the current time The original state of the encoder It is a multi-layer neural network with non-linear mapping capabilities. State embedding. Includes environmental perception features, such as location, speed, obstacles, and nearby agents; candidate reference actions. For similarity retrieval and expectation scoring of historical trajectories, it represents the typical action that might be taken in this state in human experience; control preference vector. Expressing the strategic tendency of the current task, such as prioritizing obstacle avoidance for safety, prioritizing flight duration for energy efficiency, or prioritizing smoothness of movement for stability. These three factors are then concatenated and used to construct a conditional vector as shown in formula (15) through a nonlinear mapping function:

[0060] (15) in, This represents vector-level concatenation; MLP stands for Multilayer Perceptron Network. This represents the dimension of the model's conditional vectors. Then, the actual actions are passed through a Markov chain. Adding Gaussian noise generates a series of actions Finally, it is transformed into Gaussian white noise to realize the modeling of the forward diffusion process, as shown in formulas (16) and (17):

[0061] (16) (17) in, It is the first Noisy motion after step diffusion; It represents the total number of diffusion steps, controlling the particle size of the generated particles. No. The diffusion rate of the step is usually assumed to be a scheduling sequence that increases with time. yes The unit covariance matrix represents independent and identically distributed noise. The resulting distribution after diffusion tends towards a standard normal distribution. After that, from Begin by restoring in sequence. Finally, the strategy actions used for execution are obtained, as shown in formulas (18) and (19):

[0062] (18) (19) in, It is the average of the actions predicted by the model, output by the neural network. It is the variance corresponding to the diffusion step. It is the first The conditional vector of the step, with a fixed input. The initial motion is Gaussian noise. These are the training parameters for the diffusion strategy model. This process simulates the conditional distribution of behavioral samples being gradually recovered from noise.

[0063] Furthermore, the above diffusion model can be trained and optimized in the following way: During model training, the system introduces a joint loss function for optimization, which includes two training sub-objectives: First, we introduce the policy evaluation function. Regarding the current state Below the model to generate actions The expected long-term returns are estimated, and the training objective is to maximize the behavioral value maximization term of the generated action, as shown in formula (20): (20) in This is the current state. The policy action vector is generated by the diffusion model. This represents the expected cumulative reward for the action under the current strategy. The optimization objective is to guide the diffusion generation strategy towards high-value behaviors, thereby improving the overall effectiveness of the agent in task execution.

[0064] Secondly, to ensure that the distribution of generated actions is consistent with the distribution of expert behavioral trajectories extracted from brain-like memory, the system introduces a behavior consistency loss based on diffusion modeling. Specifically, based on the Gaussian diffusion framework, the system optimizes the distribution reconstruction capability of generated behavioral trajectories by minimizing the mean square error between model prediction noise and actual noise. This loss is defined as shown in formula (21):

[0065] (twenty one) in, For experts or original actions in memory trajectory, To add in Real Gaussian noise on, For the model to input noisy actions and diffusion step size The following is an estimation of the original noise. : for the first Noise action after step diffusion For the diffusion process The cumulative retention rate of each step. This loss function constrains the model to accurately reconstruct the original behavioral data during the denoising process, improving the modeling accuracy of the diffusion model for trajectory data distribution.

[0066] The overall loss function is shown in Equation (22): (twenty two) in, The Lagrange multiplier represents the balance between the reinforcement learning objective and behavioral consistency, with the training objective being to minimize... That is, to generate high-value strategic actions that do not deviate from the actual distribution of behavior.

[0067] During the deployment phase, the policy execution process includes constructing condition vectors. Initial sampling noise Multi-step noise reduction sampling and strategic actions Output.

[0068] Furthermore, this invention also provides a specific method for updating and optimizing a brain-like memory bank: After the control execution cycle t of multiple agents ends, a complete trajectory segment is collected for each agent in intelligent control mode or fusion mode, as shown in formula (23): (twenty three) in, This is the current state. In order to perform the action, For instant rewards, For the next state, The expected labels are given for the trajectory segments. Then, a trajectory labeling strategy based on distribution statistics is used for automatic labeling, and the expected labels are shown in formula (24):

[0069] (twenty four) in, and These represent the mean and standard deviation of the rewards within the system's sliding window. A threshold parameter for controlling the desired trajectory filtering intensity; This indicates that the current segment possesses high behavioral quality. This label information is used in the expectation scoring module and importance-weighted sampling during training. Furthermore, it can be applied even when the maximum capacity is exceeded. The five-tuple trajectory sets in the brain-like memory bank are eliminated in a timely manner, for example, by comprehensively considering indicators such as trajectory score, update timestamp, and behavioral diversity. High-expectation segments are retained first and used periodically for training data augmentation.

[0070] In addition, the system periodically performs a retraining process, by proportionally sampling data from the memory bank. High-quality trajectory, reconstruct conditional input , and and use the target action Reconstruct the training data pairs and finally re-optimize the loss function. Iteratively update model parameters This enables the retraining of the policy model.

[0071] In summary, through the aforementioned brain-like memory construction, state embedding retrieval, candidate action generation, and diffusion strategy modeling training, the system further realizes the agent's strategy learning under intelligent control mode, generating a multi-agent subsystem with situational awareness, state reasoning, and action decision-making. This subsystem operates independently of the brain-controlled agent, automatically executes tasks, and lays the foundation for human-machine hybrid collaborative decision-making and control command fusion.

[0072] Step 13: Perform gating switching based on the brain control activation level in the user intent data, and linearly weight the intelligent control action command and brain control action command based on the gating switching result to construct a multi-agent fusion action command.

[0073] The current environment state includes the agent and its state embedding vector, and the physical distance to the human. Step 13 includes: Step 131: When the gating switching result is that the brain control activation level is greater than the activation level threshold, determine the fusion action instruction based on the brain control action instruction.

[0074] Step 132: When the gating switching result is that the brain control activation degree is less than the activation degree threshold, calculate the weight of the brain control action command based on the brain control activation degree and the physical distance between the agent and the human; and weight and fuse the intelligent control action command and the brain control action command according to the weight of the brain control action command to obtain the fused action command.

[0075] Specifically, in the multi-agent collaborative control method provided by this invention, through methods such as... Figure 5 The multi-agent system shown integrates intelligent control action commands and brain-controlled action commands.

[0076] Among them, let the system have a total of There are 1 Agent, with the following set of IDs: The system operates in each control cycle. By controlling the scheduler, each Agent: Assigned to The corresponding mind control mode, The corresponding intelligent control mode, and The corresponding fusion mode, and The specific scheduling rules can be set based on dynamic factors such as task priority, operator status, brain control activation level, and strategy uncertainty. Mode switching meets real-time constraints, with a maximum switching delay not exceeding one control cycle, for example, 50ms. The specific implementation process is not limited in this invention and can be defined by those skilled in the art based on actual needs.

[0077] Furthermore, to achieve structural compatibility of control action commands, the system unifies the output of all control channels into a continuous action vector form as shown in formula (25): (25) in Forward linear velocity, The horizontal angular velocity, Vertical climb speed, Align the target at the correct angle. This is the action vector dimension. The output structure remains consistent across different control sources such as brain control, intelligent control, and fusion control, facilitating encapsulation, scheduling, and execution by subsequent modules.

[0078] For agents in fusion mode: The system simultaneously activates both brain-controlled and intelligent-controlled actions, with the brain-controlled action being the most active. It is obtained by converting the collected human intention vector through intention and action mapping rules; while the intelligent control channel action The control action generated by the diffusion strategy model in the current state.

[0079] Subsequently, based on the linear weighted and gating switching fusion mechanism shown in formula (26), the intelligent control action command and the brain control action command are fused to obtain the fused action command: (26) in, The brain control weighting factor represents the strength of the influence of brain control on the final action; the system supports static configuration. Alternatively, it can be calculated dynamically. Furthermore, the fusion instructions are designed with mind control requirements in mind, incorporating a gating mechanism for switching fusion, for example, when mind control activation exceeds a threshold. At this time, only brain-controlled output is used. This mechanism is used to automatically cut off the agent's autonomous policy control when the operator intervention is strong, as shown in formula (27):

[0080] (27) Furthermore, considering that the action instructions of multiple agents in the system are different, the fusion weight adjustment can also be performed, as shown in formula (28): (28) in, The maximum activation component in the intent vector. For diffusion strategy models The output uncertainty of the current action (e.g.) ), This refers to the physical distance or signal quality between the agent and the operator. It is an adjustable parameter. This is a sigmoid function, mapped to [0,1]. This mechanism ensures that mind control suggestions are prioritized when mind control is clearly established but the strategy is unstable.

[0081] In addition, to prevent frequent control switching from causing motion jitter, the system introduces a fused motion cache structure, as shown in formula (29): (29) in, The final output action for the current cycle. As the motion smoothing factor, initially set .

[0082] Based on the above implementation method, after the fusion action instruction is constructed, the multi-Agent collaborative control method provided by the present invention further includes: Step 14: Based on the fused action instructions, the neighbor messages of multiple agents are encoded and aggregated through a graph neural network, and the aggregated neighbor messages are decoded to generate action vectors of multiple agents. The nodes of the graph neural network are agents, and the features of the nodes include the current environmental state of the agent. The edges of the graph neural network are determined by the spatial proximity relationship between multiple agents.

[0083] Specifically, the intelligent agent control method provided by this invention performs collaborative decision-making through a multi-agent system. The multi-agent state diagram modeling is as follows: Assume the system has a total of... There are 1 Agent, with the following set of IDs: The system operates in each control cycle. Construct the structure shown in the diagram below:

[0084] First, stateless graphs The nodes of the graph Each node Represents an Agent. It is a set of edges, dynamically generated by spatial proximity or communication reachability.

[0085] Subsequently, node feature embedding ,in Including Agent status and control mode encoding, For state embedding functions of MLP, GRU, etc. This represents the feature dimension of the initial nodes in the GNN.

[0086] For example, a graph neural network can be a single-layer network, consisting of... Layered graph neural networks complete the information transfer and joint policy calculation between nodes. For the first layer... Each Agent in the layer The embedding vector update is shown in equation (30): (30) in, For the set of neighbors of a node, For attention mechanisms or MLP neighbor message encoding functions, For neighbor message aggregation operators, you can choose from summation (∑), maximum value (max), weighted average, or other differentiable aggregation methods as needed. For node state update functions such as GRU and ReLU-MLP, Features such as relative position and velocity difference, For the first Layer Time Node The embedded state. After... After layer propagation, the final embedding state is obtained. The final strategy action is generated by the decoder. .

[0087] The joint reward function is shown in formula (31): (31) in, For the agent's local rewards such as target proximity, speed efficiency, etc. As a weighting factor for task importance, For map-wide collaborative rewards, The weights are for collaborative items. Action difference penalties can be applied to collaborative items. The training of the graph policy network is based on joint loss, and the loss function is shown in formula (32):

[0088] (32) The first term maximizes the joint reward, and the second term is a KL regularization term, which suppresses policy overfitting and preserves behavioral consistency. For historical snapshots or expert strategies, This is the regularity strength control coefficient.

[0089] During the execution phase, each agent only needs to perceive its local state and communication neighborhood, and independently execute the embedding and decision generation of the graph neural network, without centralized computing, thus improving the system's scalability. Finally, the motion vector is input to the control command encapsulation module to complete the motion issuance and physical execution.

[0090] In the control instruction encapsulation module, for the control cycle The action vector generated in the process: The system encapsulates it into a standard control command frame. The structure is shown in formula (33):

[0091] (33) in, Agent number, Corresponding timestamp, The current control mode is H / S / F. The motion vector after amplitude limiting ensures safe execution. This is for the validation field. The amplitude limiting process is shown in formula (34):

[0092] (34) Based on the above implementation method, after the fusion action instruction is constructed, the multi-Agent collaborative control method provided by the present invention further includes: Step 15: Determine the action jump and action conflict degree of the intelligent control action command. When the action jump is greater than the preset action change threshold, or the action conflict degree between the brain control action command and the intelligent control action command is greater than the preset action difference threshold, construct an alternative intelligent control action command based on the remaining candidate actions.

[0093] Specifically, the multi-agent collaborative control method provided by this invention offers multiple human intervention paths. The first is pre-decision generation human intervention. Specifically, before the system executes policy action generation, i.e. before diffusion model or graph policy network inference, the following intervention entry point is added, as shown in formula (35):

[0094] (35) in Intervention commands issued proactively by the operator. The confidence level of the current model policy is usually expressed as... express, A preset uncertainty threshold is set. Upon triggering, the system pauses the policy generation process, and the operator can perform the following actions: replacement. Actions include uploading a predefined trajectory; locking the current state and remaining stationary; and requesting a switch to mind control mode.

[0095] Furthermore, after the control command is generated, the system performs a second-stage intervention check, as shown in formula (36): (36) in, For action jumps, The threshold for action change. This is a communication signal interruption flag. The difference between mind control intent and intelligent control strategy is significant. After triggering, the system enters a control confirmation waiting state, allowing the operator to: confirm / deny the current control command; modify the control mode; downgrade to "safety hold" mode, etc.

[0096] Furthermore, if any of the following unforeseen conditions occur during system operation, as shown in formula (37), manual takeover shall be initiated immediately: (37) in, To enable environmental perception to detect critical situations such as collisions, stalls, and rollovers, The emergency signal is triggered by the operator through manual switching, high-intensity brain-controlled activation, and dual-channel confirmation. This represents the magnitude of the increase in the activation level of brain-controlled signals. This is the emergency threshold. Perform the following operations: Force change the control mode. Interrupt all strategy module outputs; activate the mind control channel and assign the highest control privileges; record takeover logs for system retraining and other actions.

[0097] In addition, the system sends command frames to various platforms at a high frequency of 50Hz via dedicated communication buses such as UDP / LoRa / serial ports. The receiving end executes and parses the command fields; maps action values ​​to devices such as servos and motor control quantities; updates the status in real time and provides feedback on the execution; and prioritizes actions such as brain-controlled trajectory analysis when taking over.

[0098] Furthermore, the system collects execution feedback data packets. It includes location, speed, execution error and error code, which can be fed back to platforms such as memory bank update, manual confirmation platform, and policy scheduler, thereby executing a feedback loop and security monitoring.

[0099] In summary, this invention proposes a multi-agent hybrid decision-making method that integrates brain-controlled intention input with autonomous strategy generation. The system receives structured intention vectors through an external brain-computer interface module and converts intention categories into action templates through a pre-defined mapping mechanism, thereby enabling real-time human control over some agents. Specifically, by using prior guidance of brain-controlled intentions to guide the decision-making direction, the system significantly reduces strategy search time and improves response efficiency and task adaptability under unexpected tasks.

[0100] This invention designs an action reasoning module supported by a brain-like memory mechanism. The system constructs a state-action-reward-embedding representation based on high-reward historical trajectories, achieves rapid retrieval of candidate trajectories through state matching, and performs expectation scoring and action fusion generation. The brain-like memory mechanism of this invention possesses stronger state generalization and behavior reconfigurability capabilities under multi-agent collaboration, significantly improving the stability and generation quality of the strategy.

[0101] This invention proposes a multi-agent collaborative policy generation mechanism based on a diffusion model. The system uses the current state embedding and historical behavior priors as conditional variables, and generates high-quality control action trajectories through a diffusion-denoising process. By embedding brain-controlled intentions into the conditional space of the diffusion model, this invention achieves unified modeling of "brain-controlled intentions + autonomous preferences," enhancing the diversity, stability, and security controllability of policy generation.

[0102] This invention constructs a state graph structure among multiple agents based on graph neural networks, enabling multi-layer propagation of neighbor information and collaborative policy modeling. By dynamically introducing task state and distance-aware information into the graph structure construction, this invention achieves real-time modeling of collaborative relationships among agents, thereby improving the collaborative consistency and stability of the cluster under conditions of limited communication and partial disconnection.

[0103] This invention introduces a two-stage human intervention mechanism: "before strategy generation + before control execution," supporting operators to review, select, interrupt, or regenerate strategies in real time. This invention not only incorporates brain-controlled weights during strategy generation but also pre-sets a human intervention channel in the strategy execution path, realizing a full-process human participation mechanism from human input and candidate generation to pre-execution confirmation, significantly improving the system's security, fault tolerance, and operational flexibility.

[0104] This invention constructs a multi-agent behavioral feedback closed-loop mechanism. The system incorporates the state transition trajectory from each task into a brain-like memory bank, and combines this with trajectory labels output by a diffusion strategy to achieve dynamic updates to the memory bank and policy retraining. By constructing a dynamic expectation labeling system and a brain-like memory screening mechanism, this invention enhances the expressive and evolutionary update capabilities of policy distribution, thereby supporting the autonomous adaptation and continuous optimization of the multi-agent system during long-term operation.

[0105] The brain-controlled intent input mechanism proposed in this invention possesses high platform versatility and can adapt to the control needs of various types of agents. The system can map structured human intent vectors into standard control commands, deploying them on multi-agent platforms including various unmanned equipment, humanoid robots, intelligent driving vehicles, and manned / unmanned swarms, achieving unified human-machine collaborative control across platforms and heterogeneous systems. This feature significantly enhances the engineering adaptability and practical application capability of this invention in complex system environments, outperforming most existing multi-agent decision-making methods that are limited to homogeneous platform control.

[0106] Example 2 Based on Example 1, such as Figure 6 As shown, this invention, combined with an example of a drone, provides a specific multi-agent control method: First, such as Figure 7 As shown, this embodiment uses Python and CUDA acceleration to build a multi-agent adversarial simulation environment. It uses the ZeroMQ protocol to send JSON data containing information such as the location of the unmanned cluster, health, missile reserves, and off-road vehicle coordinates to the visualization interface, so as to realize real-time interaction between the algorithm and the environment.

[0107] The training task in this embodiment is to completely destroy enemy off-road vehicles while ensuring the survival of all friendly heterogeneous aircraft in a scenario involving a swarm of armed / unmanned vehicles versus off-road vehicles. Specifically, the enemy off-road vehicles are controlled using a preset behavior pattern, while the friendly heterogeneous aircraft are autonomously controlled based on a reinforcement learning algorithm. The number of friendly heterogeneous aircraft is >10, and the number of enemy off-road vehicles is >2.

[0108] Specifically, this embodiment uses a Markov decision process framework to model the behavioral decision-making strategy of each Agent. The state observation of each Agent... For triples ,in: For one's own information; Information on neighboring agents; Target information. Let the location of Agent i be... The speed is The position and velocity information of i itself are represented as shown in formula (38):

[0109] (38) in, This represents the location coordinates of Agent i. Let L represent the velocity of Agent i in the x and y directions, and let L represent the length of the field. This represents the Agent's maximum speed, used for speed normalization. The positions, speeds, relative orientations, and distances of neighboring Agent j are shown in formula (39):

[0110] (39) in, This represents the position coordinates of Agent j. Describe the velocities of Agent j in the x and y directions. This represents the relative orientation angle between Agent i and neighbor j. This represents the Euclidean distance between Agent i and Agent j. This represents the set of adjacent Agents. Finally, the position, velocity, relative orientation, and distance information of the target Agent k are modeled, as shown in formula (40):

[0111] (40) in, This represents the location coordinates of the target Agent k. Describe the velocities of the target agent k in the x and y directions. This represents the relative orientation angle between Agent i and the target Agent k. This represents the distance between Agent i and the target Agent k. This represents the target set observed by Agent i. In this embodiment, the Agent consists of 16 unmanned equipment and 4 off-road vehicles, whose positions are evenly distributed. The map parameters are shown in Table 1.

[0112] Table 1. Example of Map Parameters Furthermore, the initial position of the drone is randomly generated, and the pseudocode for the drone's control rules is shown below: "Obtain the observation function: " Create total_obs[16×118], initial value -2 For each Agent i (0~15): if self.current_states[i] == survive: Get current_position = self.current_position[i] Create current_obs

[118] with an initial value of 0. # Self-state (4 dimensions) current_obs[0:4] = [x / field length, y / field length, vx / maximum speed, vy / maximum speed] idx = 4 # Other drone information (90 dimensions) for j in 0~15: if j != i: Calculate other_position, vx_other, vy_other current_obs[idx:idx+6] = [ x_other / site length, y_other / Field length, vx_other / maximum speed, vy_other / maximum speed, Relative angle / 360° Euclidean distance / field length ] idx += 6 # Ground target information (24-dimensional) for t in 0~3: Get target_pos, target_vx, target_vy current_obs[idx:idx+6] = [ x_target / field length y_target / field length, target_vx / target maximum speed target_vy / target maximum speed, Relative angle / 360° Euclidean distance / field length ] idx += 6 Store current_obs into total_obs[i] Return total_obs Secondly, a multi-Agent subsystem under the mind control model can be constructed in a manner similar to that in Example 1. For each Agent, the reward calculation formula is shown in formula (41):

[0113] (41) in, This represents the reward for Agent i. This represents the current state of Agent i, for example, 0 means alive, 1 means killed by collision with another of its own kind, and 2 means killed by attacking an enemy. This represents the distance penalty coefficient to the target. The reward coefficient representing the change in distance to the target. This represents the distance from Agent i to the target. Indicates the maximum possible distance. This represents the distance from Agent i to the target in the previous instance. If the target collides with Agent i, i.e. Then an additional reward will be added. If Agent i collides with another Agent j, i.e. Then the punishment will end. .in Indicates the collision distance threshold. Indicates the reward for attacking a target. This represents the penalty when a collision occurs. For each Agent... Its state update rule is shown in formula (42):

[0114] (42) in, Indicates the target's remaining health. This represents the collision distance threshold. If all targets are destroyed, the state of all agents is set to "Completed".

[0115] The target velocity update is shown in formula (43): (43) in, , The velocity of the target in the x and y directions; , Let be the change in velocity. The position update is shown in formula (44):

[0116] (44) in, Indicates the location of the target. Indicates the speed of the target. This represents the time step. In this embodiment, the UAV state space is set as a 1×118 tensor, and the meaning of each dimension is shown in Table 2; the action space is a 1×2 table-dimensional tensor or a two-dimensional tensor, and the meaning of each dimension is shown in Table 3.

[0117] Table 2 State Space Table for UAVs / Drones Table 3 Action Space Table for Both Android and Unmanned Aerial Vehicles The pseudocode for the reward function is shown below: # Calculate rewards for each Agent For each Agent i in 0~15: if Agent survives (current_states[i] == 0): # Bind target (drone i corresponds to target i%4) target_idx = i % 4 # Calculate the Euclidean distance to the target d2target = L2 norm of current coordinates and target coordinates # Basic Reward Calculation reward = - (d2target / 1414) * 0.3 # Distance penalty (maximum distance 1414 is the diagonal of the field) reward += (last distance - d2target) * 0.7 # Proximity reward (positive reward for proximity, negative reward for distance) # Target Collision Detection if d2target < 5 meters: # Condition for successful attack print(attack successful message) current_states[i] = 2 # Mark as attack dead state reward += 200 # Major Success Reward dones[i] = True # Terminate the drone # Target Health Management Target health value -1 if the target health value reaches zero: Mark the target as destroyed (target_states[target_idx] = 1) # Friendly collision detection For other drones, j in 0~15: if j ≠ i and j survives: d2other = L2 norm of the two-machine coordinate system if d2other < 5 meters: # Collision condition current_states[i] = 1 # Marked as collision dead reward -= 200 # Major penalty dones[i] = True break # Exit the loop elif status is 1 (friendly fire, death): reward = -2 dones[i] = True” The pseudocode for the overall process of drone training is shown below: “for episode in 1...n_games: obs = env.reset() for t in 1...max_steps: # 1. Select Action (with exploration noise) actions = [μ_i(o_i) + noise for μ_i, o_i in zip(Agents, obs)] # 2. Environmental Interaction next_obs, rewards, dones, _ = env.step(actions) # 3. Storage Experience memory.add(obs, actions, rewards, next_obs, dones) # 4. Training (if steps > warmup_steps) if total_steps>warmup_steps and total_steps % train_interval == 0: batch = memory.sample(batch_size) # Update Critic critic_loss = MSE(Q(obs,actions), r + γ·Q'(next_obs, μ'(next_obs))) # Update Actor actor_loss = -mean(Q(obs, μ(obs))) # Soft update target network # 5. Check termination conditions if all(dones) or t == max_steps: break The next step is to construct a brain-like memory bank and fuse normalized weighted actions, and generate strategies based on a diffusion model. Then, a multi-agent system that fuses brain control and intelligent control is constructed in the manner mentioned in Example 1, thereby generating fused action instructions.

[0118] In addition, for the simulation training needs of drones, a Python script can be run to attempt to start the training environment. If the training environment starts successfully, the Python script will retrieve data via ZeroMQ and generate a visual simulation interface. Figure 8 The figure shows a comparison of the trend of average reward changing with the number of training steps between the method of this invention and two benchmark algorithms. Overall, the method outperforms MADDPG and DQN. After training, the Python script shuts down the simulation environment, terminates training when there is no improvement in the monitored average reward, and saves the trained model.

[0119] Example 3 The present invention also provides a multi-agent collaborative control system, comprising: The brain control command construction module is used to acquire user intent data; based on the intent category corresponding to the user intent data, it determines the action parameters corresponding to the intent category based on the predefined intent-to-action mapping rules and constructs multi-agent brain control action commands; The intelligent control instruction construction module is used to obtain the current environmental state; retrieve memory entries in the brain-like memory bank shared by multiple agents based on the current environmental state, obtain the adjacent trajectories and determine the candidate actions; construct intelligent control action instructions for multiple agents based on the current environmental state and candidate actions, wherein the brain-like memory bank pre-stores memory entries related to state, action, trajectory fragments and trajectory scores; The instruction fusion module is used to switch gating based on the brain control activation level in the user intent data. Based on the gating switching result, the intelligent control action instruction and the brain control action instruction are linearly weighted to construct a fused action instruction for multiple agents.

[0120] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in an embodiment of a multi-agent cooperative control method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0121] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in an embodiment of a multi-agent cooperative control method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0122] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0125] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0126] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A multi-agent collaborative control method, characterized in that, include: Obtain user intent data; Based on the intent category corresponding to the user intent data, the action parameters corresponding to the intent category are determined according to the predefined intent-to-action mapping rules, and multi-agent brain-controlled action instructions are constructed. The current environmental state is obtained; based on the current environmental state, memory entries are retrieved from the brain-like memory bank shared by multiple agents to obtain neighboring trajectories and determine candidate actions; based on the current environmental state and candidate actions, intelligent control action instructions for the multiple agents are constructed, wherein the brain-like memory bank pre-stores the memory entries associated with states, actions, trajectory segments and trajectory scores; Gating is performed based on the brain control activation level in the user intent data. Based on the gating switching result, the intelligent control action command and the brain control action command are linearly weighted to construct the fusion action command of the multi-agent.

2. The multi-agent collaborative control method according to claim 1, characterized in that, Based on the intent category corresponding to the user intent data, the action parameters corresponding to the intent category are determined according to predefined intent-to-action mapping rules, including: Parse the user intent data and determine the intent category with the highest activation rate from the set of intent categories; When the difference in activation between the highest activation category and the other multiple categories in the intent category set is greater than the activation threshold, the corresponding action template is called according to the intent category with the highest activation, based on the mapping rule from intent to action, to determine the action parameters. When the difference in activation between the highest activation category and the other multiple categories in the intent category set is less than the activation threshold, the action templates corresponding to the highest activation category and the other multiple categories are fused according to activation weight to determine the action parameters after intent fusion.

3. The multi-agent collaborative control method according to claim 1, characterized in that, The neighboring trajectories consist of multiple adjacent trajectory segments. Based on the current environmental state, memory entries are retrieved from a brain-like memory bank shared by multiple agents to obtain neighboring trajectories and determine candidate actions, including: The current environment state is processed by a state encoder to obtain a state embedding representation; In the brain-like memory bank, K-nearest neighbor retrieval is performed on the state embedding representation based on Euclidean distance to obtain the multiple neighboring trajectory segments; The candidate action is determined based on the nearest trajectory segment with the highest trajectory score; The vectors of the current environment state and candidate actions are concatenated by a diffusion model and iteratively denoised to determine the mean value of the generated actions. The intelligent control action instructions of the multi-agent are then constructed based on the mean value of the actions.

4. The multi-agent collaborative control method according to claim 3, characterized in that, Before concatenating the vectors of the current environment state and candidate actions using the diffusion model, the process also includes: The training data is fed into the untrained diffusion model to predict the action mean. The behavioral value loss is calculated based on the action mean prediction result and the strategy evaluation function. The consistency loss of behavior distribution is calculated based on the predicted action mean and the actions corresponding to high-expectation trajectories in the brain-like memory bank. The diffusion model is adjusted based on the loss of behavioral value and the loss of consistency in behavioral distribution.

5. The multi-agent collaborative control method according to claim 1, characterized in that, The current environmental state includes the physical distance between the agent and the human. Based on the gating switching result, the intelligent control action command and the brain control action command are linearly weighted to construct the fused action command of the multi-agent, which includes: When the gating switching result is that the brain control activation degree is greater than the activation threshold, the fusion action instruction is determined according to the brain control action instruction; When the gating switching result is that the brain control activation degree is less than the activation degree threshold, the weight of the brain control action command is calculated based on the brain control activation degree and the physical distance between the agent and the human; the intelligent control action command and the brain control action command are weighted and fused according to the weight of the brain control action command to obtain the fused action command.

6. The multi-agent collaborative control method according to claim 1, characterized in that, After constructing a fused action command by linearly weighting and gating the intelligent control action command and brain control action command, the method further includes: Based on the fused action instructions, a graph neural network is used to encode and aggregate neighbor messages for multiple agents, and the aggregated neighbor messages are decoded to generate action vectors for multiple agents. The nodes of the graph neural network are agents, and the features of the nodes include the current environmental state of the agent. The edges of the graph neural network are determined by the spatial proximity relationships between the multiple agents.

7. The multi-agent collaborative control method according to claim 1, characterized in that, After constructing the intelligent control action instructions for the multi-agent based on the current environmental state and candidate actions, the method further includes: The action jump and action conflict degree of the intelligent control action command are judged. When the action jump is greater than the preset action change threshold, or the action conflict degree between the brain control action command and the intelligent control action command is greater than the preset action difference threshold, an alternative intelligent control action command is constructed based on the remaining candidate actions.

8. A multi-agent collaborative control system, characterized in that, include: Mind control command construction module, used to acquire user intent data; Based on the intent category corresponding to the user intent data, the action parameters corresponding to the intent category are determined according to the predefined intent-to-action mapping rules, and the brain-controlled action instructions of the multi-agent are constructed. The intelligent control instruction construction module is used to obtain the current environmental state; retrieve memory entries in a brain-like memory bank shared by multiple agents based on the current environmental state, obtain neighboring trajectories and determine candidate actions; and construct intelligent control action instructions for the multiple agents based on the current environmental state and candidate actions, wherein the brain-like memory bank pre-stores the memory entries associated with states, actions, trajectory segments and trajectory scores. The instruction fusion module is used to perform gating switching based on the brain control activation level in the user intent data, and to linearly weight the intelligent control action instructions and brain control action instructions based on the gating switching results to construct the fused action instructions of the multi-agent.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the multi-agent collaborative control method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to execute the steps of the multi-agent collaborative control method according to any one of claims 1 to 7.

Images