Multi-agent game cooperative control method and system based on cognitive synchronization mechanism
By introducing a cognitive synchronization mechanism, each agent in a multi-agent system can independently understand its own state and local intentions, and perform cognitive synchronization at the team level when needed. This solves the problem of poor collaborative performance in existing technologies and achieves efficient team collaborative control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-03-31
AI Technical Summary
Existing multi-agent systems lack efficient coordination mechanisms, resulting in poor overall system performance. Centralized methods are susceptible to communication quality issues and have poor scalability, while distributed methods struggle to guarantee overall system coordination performance.
A multi-agent game-theoretic collaborative control method based on cognitive synchronization mechanism is introduced. Through individual cognitive encoders and intention projection layers, each agent independently understands its own state and local intentions. Combined with uncertainty assessment and triggering network, the team-level cognition is synchronized on demand to generate action vectors.
It improves the overall collaborative performance of multi-agent systems, avoids excessive reliance on central nodes, reduces communication overhead, and ensures team cognitive consistency and decision-making collaboration at critical moments.
Smart Images
Figure CN121764191A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent agent technology, and more specifically, to a multi-agent game cooperative control method and system based on a cognitive synchronization mechanism. Background Technology
[0002] In the research on control of multi-agent systems, existing technologies mainly develop along two technical routes: centralized planning and distributed control.
[0003] Centralized methods collect global information and perform unified decision-making and planning through a central node, then distribute detailed decision instructions to each agent. While this method has the potential to generate globally optimal solutions, it is susceptible to communication quality issues, has poor scalability, and is prone to single points of failure, making it only suitable for systems with known environments and small scale. Distributed methods, on the other hand, allow each agent to make independent decisions based on local observations and interactions with other agents, offering better robustness and scalability. However, they are prone to getting trapped in local optima and struggle to guarantee the overall cooperative performance of the system. Summary of the Invention
[0004] The problem that this invention aims to solve is the poor overall performance of existing multi-agent systems due to the lack of efficient coordination mechanisms.
[0005] To address the aforementioned problems, in a first aspect, this invention provides a multi-agent game-theoretic cooperative control method based on a cognitive synchronization mechanism, comprising: The local observation and historical action vectors are input into the individual cognitive encoder to obtain the local cognitive state vector; The local cognitive state vector is input into the individual intention projection layer to obtain the local intention vector; The uncertainty of the current moment is determined based on all local intent vectors within a preset time period prior to the current moment; By concatenating uncertainty and environmental change indicators and inputting them into the triggering network, the synchronization trigger probability is obtained. If the probability of synchronous triggering is less than or equal to the preset probability threshold, the team's common intent vector is determined based on the uncertainty at the current moment and the local intent vectors of all agents obtained. If the probability of synchronous triggering is greater than the preset probability threshold, the local intent vectors of all agents are resynchronized and acquired. Based on the uncertainty at the current moment and the reacquired local intent vectors of all agents, the team's common intent vector is updated. The local cognitive state vector and the team's shared intent vector are concatenated and input into the distributed policy network to generate action vectors.
[0006] Optionally, determining the uncertainty of the current moment based on all local intent vectors within a preset time period prior to the current moment includes: Determine the covariance matrix based on all local intent vectors within a preset time period prior to the current moment; By analyzing the trace of the covariance matrix, we can obtain the uncertainty at the current moment.
[0007] Optionally, the steps for determining or updating the team's shared intent vector include: Determine the dynamic bias matrix based on the current uncertainty of the two agents; Based on the local intent vector of each agent, the query vector, key vector, value vector, and dimension of the key vector of a single agent are obtained through linear projection; Stack the query vectors, key vectors, and value vectors of all agents into query matrix, key matrix, and value matrix, respectively; Based on the dimensions of the dynamic bias matrix, query matrix, key matrix, value matrix, and key vector, input to the attention layer to obtain the team's shared intent vector.
[0008] Optionally, the local intent vector is in, This represents the local intent vector of agent i. The weight matrix representing the individual's intention projection layer. This represents the bias vector of the individual's intention projection layer. Represents the local cognitive state vector of agent i; The uncertainty at the current moment is in, Let represent the uncertainty of agent i at the current time t, T represent the preset time before the current time, Cov() represent the covariance matrix, and tr() represent the trace of the covariance matrix. This represents the local intent vector of agent i at the current time t. This represents the local intent vector of agent i at the current time t. Let represent the local intent vector of agent i at the current time t.
[0009] Optionally, the synchronization trigger probability is in, Indicates the probability of synchronous triggering. This represents the weight matrix of the trigger network. This represents the bias vector that triggers the network; This represents the uncertainty of agent i. Indicators representing environmental change.
[0010] Optionally, the dynamic bias of agent i towards agent j is: in, This represents the uncertainty of agent i. λ represents the uncertainty of agent i, and λ represents the learnable scaling parameter of the attention layer; The output matrix of the attention layer is: Where A represents the output matrix of the attention layer, Q represents the query matrix formed by stacking the query vectors of all agents, K represents the key matrix formed by stacking the key vectors of all agents, and V represents the value matrix formed by stacking the value vectors of all agents. B represents the dimension of the key vector, and B represents the dynamic bias matrix of the attention layer. The average value of each row of the output matrix A is taken to obtain the team's shared intent vector.
[0011] Optionally, the action vector is in, This represents the action vector of agent i. Represents a distributed policy network. This represents the local cognitive state vector of agent i. This represents the team's shared intent vector.
[0012] Optionally, the multi-agent game-theoretic cooperative control method based on cognitive synchronization mechanism also includes: Based on the training dataset, the individual cognitive encoder, individual intention projection layer, triggering network, attention layer and distributed policy network are trained. Each data in the training dataset includes global state, global reward, local observation, action vector and global state at the next time step. Based on the comprehensive loss function, the parameters of the individual cognitive encoder, individual intention projection layer, triggering network, attention layer and distributed policy network are updated in reverse until the global reward converges in multiple consecutive training cycles, or the total number of training steps reaches the preset maximum number of steps, thus obtaining the trained individual cognitive encoder, individual intention projection layer, triggering network, attention layer and distributed policy network. The comprehensive loss function includes policy gradient loss and cognitive consistency loss.
[0013] Optionally, the comprehensive loss function for in, Weighting coefficients representing the loss of cognitive consistency; The policy gradient loss for in, This represents the action vector of agent i at the current time t. This represents the global state at the current time t. Denotes the joint dominance function under joint conditions. This represents the current parameters of the distributed policy network. Represents the previous parameters of the distributed policy network; clip() is a function to constrain the numerical range. Represents the joint action of all intelligent agents The probability ratio under the old and new strategies Indicates the limiting parameter. This represents the minimum probability ratio. This represents the maximum value of the probability ratio; The loss of cognitive consistency for in, This represents the local intent vector of agent i. This represents the team's shared intent vector.
[0014] Secondly, the present invention also provides a multi-agent game cooperative control system based on a cognitive synchronization mechanism, comprising: The individual cognitive encoding module is used to input local observation and historical action vectors into the individual cognitive encoder to obtain a local cognitive state vector; it is also used to input the local cognitive state vector into the individual intention projection layer to obtain a local intention vector. The triggering module is used to determine the uncertainty of the current moment based on all local intent vectors within a preset time period before the current moment; it is also used to concatenate the uncertainty and environmental change indicators and input them into the triggering network to obtain the synchronous triggering probability. The attention synchronization module is used to determine the team's common intent vector based on the uncertainty at the current moment and the local intent vectors of all agents when the synchronization trigger probability is less than or equal to a preset probability threshold; it is also used to resynchronize and acquire the local intent vectors of all agents when the synchronization trigger probability is greater than the preset probability threshold, and update the team's common intent vector based on the uncertainty at the current moment and the reacquired local intent vectors of all agents. The decision output module is used to concatenate the local cognitive state vector and the team's common intention vector, and input them into the distributed policy network to generate action vectors.
[0015] This invention provides a multi-agent game-theoretic collaborative control method and system based on a cognitive synchronization mechanism. Compared with existing technologies, it has the following advantages: By introducing individual cognitive encoders and individual intent projection layers, each agent can independently understand its own state and local intent, avoiding excessive reliance on a central node. By introducing uncertainty assessment and triggering networks, the system intelligently determines when team-level cognitive synchronization is needed. This on-demand synchronization mechanism avoids continuous high-frequency communication overhead while ensuring consistency in team cognition at critical moments, effectively improving overall collaborative efficiency and task completion quality. By concatenating local cognitive state vectors and the team's shared intent vector and inputting them into a distributed policy network, each agent, when generating action vectors, considers not only its own local perception but also the team's overall goal. This allows the drone to maintain individual flexibility while ensuring its actions align with the team's collaborative objectives. This mechanism effectively solves the problem of individual decision-making being disconnected from team goals in traditional distributed methods, significantly improving the overall collaborative performance of multi-agent systems. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 A flowchart illustrating a multi-agent game cooperative control method based on a cognitive synchronization mechanism, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a multi-agent game cooperative control system based on a cognitive synchronization mechanism, provided as an embodiment of the present invention. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application are described clearly and completely. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.
[0020] like Figure 1As shown in the figure, an embodiment of this application provides a multi-agent game cooperative control method based on a cognitive synchronization mechanism, comprising: S1: Input the local observation and historical action vectors into the individual cognitive encoder to obtain the local cognitive state vector.
[0021] S2: Input the local cognitive state vector into the individual intention projection layer to obtain the local intention vector.
[0022] S3: Determine the uncertainty of the current moment based on all local intent vectors within a preset time period prior to the current moment.
[0023] S4: Concatenate the uncertainty and environmental change indicators and input them into the triggering network to obtain the synchronization trigger probability.
[0024] Determine whether the probability of synchronous triggering is greater than a preset probability threshold.
[0025] S5: If the synchronous trigger probability is less than or equal to the preset probability threshold, determine the team's common intent vector based on the uncertainty at the current moment and the local intent vectors of all agents obtained.
[0026] S6: If the probability of synchronization triggering is greater than the preset probability threshold, resynchronize and acquire the local intent vectors of all agents. Based on the uncertainty at the current moment and the reacquired local intent vectors of all agents, update the team's common intent vector.
[0027] S7: Concatenate the local cognitive state vector and the team's common intention vector, input them into the distributed policy network, and generate action vectors.
[0028] In this optional embodiment, by introducing individual cognitive encoders and individual intent projection layers, each agent can independently understand its own state and local intent, avoiding excessive reliance on a central node. By introducing uncertainty assessment and triggering networks, the system intelligently determines when team-level cognitive synchronization is needed. This on-demand synchronization mechanism avoids continuous high-frequency communication overhead while ensuring consistency in team cognition at critical moments, thereby effectively improving overall collaborative efficiency and task completion quality. By concatenating the local cognitive state vector and the team's shared intent vector and inputting them into the distributed policy network, the collaborative policy to be executed by the corresponding agent is obtained. When generating action vectors, each agent considers not only its own local perception but also the overall goal of the team. This allows the drone to maintain individual flexibility while ensuring that its actions are consistent with the team's collaborative goals. This mechanism effectively solves the problem of individual decision-making being disconnected from team goals in traditional distributed methods, significantly improving the overall collaborative performance of multi-agent systems. For example, during a search, even if a drone discovers a minor target in a localized area, if the team's collective intention is to prioritize searching another area, the drone can, under the guidance of the policy network, balance individual discovery with team objectives and make a better decision.
[0029] The following is a detailed description of each step.
[0030] S1: Input the local observation and historical action vectors into the individual cognitive encoder to obtain the local cognitive state vector.
[0031] Specifically, each agent, from its own local perspective, forms a personalized cognitive expression of the situation. The module's input is each agent's own local observations and historical action vectors. These input vectors are fed into an individual cognitive encoder composed of fully connected layers and recurrent neural networks (RNNs) to obtain a local cognitive state vector. This vector contains agent i's private, dynamic understanding of the current situation, used for internal reasoning and memory within the agent. The agent's local observations can be raw data acquired through sensors such as cameras, radar, and sonar, including image frames, distance measurements, or sound signals. The historical action vectors can simply record the discrete action sequences performed by the agent over the past N time steps, such as "forward," "turn left," and "grab."
[0032] S2: Input the local cognitive state vector into the individual intention projection layer to obtain the local intention vector.
[0033] Specifically, the local intent vector is The individual's intention projection layer is a fully connected layer. This represents the local intent vector of agent i. The weight matrix representing the individual's intention projection layer. This represents the bias vector of the individual's intention projection layer. The local cognitive state vector of agent i is represented by a local intent vector generated through the hyperbolic tangent activation function tanh. This vector is a low-dimensional dense vector representing the team's strategic focus as perceived unilaterally by the agent. By replacing complex centralized semantic modeling with distributed local encoding, the latency and overhead of the synchronization mechanism are fundamentally reduced.
[0034] S3: Determine the uncertainty of the current moment based on all local intent vectors within a preset time period prior to the current moment.
[0035] This step specifically includes: determining the covariance matrix based on all local intent vectors within a preset time period prior to the current moment; and analyzing the trace of the covariance matrix to obtain the uncertainty at the current moment.
[0036] Specifically, the uncertainty of each agent at the current moment is determined by calculating the trace of the covariance matrix of all intention vectors over a previous historical time period T. The uncertainty at the current moment is... Here, tr() represents the trace of the covariance matrix, which is equal to the sum of all elements on the main diagonal of the matrix. This represents the uncertainty of agent i at the current time t, where T represents the preset time before the current time, and Cov() represents the covariance matrix. This represents the local intent vector of agent i at the current time t. This represents the local intent vector of agent i at the current time t. Let represent the local intent vector of agent i at the current time t.
[0037] S4: Concatenate the uncertainty and environmental change indicators and input them into the triggering network to obtain the synchronization trigger probability.
[0038] Specifically, the environmental change indicator can be a binary flag or a continuous value provided by environmental sensors (e.g., detection of new obstacles, degradation of communication quality, etc.). The triggering judgment for cognitive synchronization is mainly based on the agent's uncertainty and the environmental change indicator. The two vectors are concatenated and input into the triggering network to obtain the synchronization trigger probability. The sigmoid function compresses the output to the interval [0, 1], and the output value is the synchronization trigger probability. , This represents the weight matrix of the trigger network. This represents the bias vector that triggers the network; This represents the uncertainty of agent i. Indicators representing environmental change.
[0039] Further determine whether the probability of synchronous triggering is greater than the preset probability threshold.
[0040] S5: If the synchronous trigger probability is less than or equal to the preset probability threshold, determine the team's common intent vector based on the uncertainty at the current moment and the local intent vectors of all agents obtained.
[0041] Specifically, in this case, the team's shared intent vector can be obtained by simply averaging the local intent vectors of all agents, or by weighted averaging, where the weights can be assigned based on the uncertainty of each agent, for example, agents with lower uncertainty are given higher weights for their intents.
[0042] S6: If the probability of synchronization triggering is greater than the preset probability threshold, resynchronize and acquire the local intent vectors of all agents. Based on the uncertainty at the current moment and the reacquired local intent vectors of all agents, update the team's common intent vector.
[0043] Specifically, resynchronization can be achieved through a broadcast mechanism, requiring all agents to resend their latest local intent vectors. When updating the team's shared intent vector, besides simple averaging or weighted averaging, the determination or updating of the team's shared intent vector can also involve: determining a dynamic bias matrix based on the current uncertainty of the two agents; obtaining the query vector, key vector, value vector, and key vector dimensions of each agent through linear projection based on their local intent vectors; stacking the query vectors, key vectors, and value vectors of all agents into query matrix, key matrix, and value matrix, respectively; and inputting the dimensions of the dynamic bias matrix, query matrix, key matrix, value matrix, and key vector into the attention layer to obtain the team's shared intent vector.
[0044] Receive local intent vectors from all agents via communication. These are projected and stacked into a query matrix Q, a key matrix K, and a value matrix V, respectively, and standard attention weights are calculated. To ensure cognitive consistency among agents, a dynamic bias based on cognitive uncertainty is introduced to adjust the attention of agent i to agent j. The dynamic bias of agent i to agent j is: in, This represents the uncertainty of agent i. Let λ represent the uncertainty of agent i, and λ represent the learnable scaling parameter of the attention layer. This bias term is used to penalize groups of agents with large differences in uncertainty, thus favoring agents with similar cognitive levels to reinforce each other during attention allocation, quickly leveling out the cognitive gap within the team. The dynamic bias matrix... Injected into the attention calculation, the output matrix of the attention layer is Where A represents the output matrix of the attention layer, Q represents the query matrix formed by stacking the query vectors of all agents, K represents the key matrix formed by stacking the key vectors of all agents, and V represents the value matrix formed by stacking the value vectors of all agents. Let B represent the dimension of the key vector, and let B represent the dynamic bias matrix of the attention layer. The softmax function normalizes the matrix row-wise to obtain the attention output matrix. Averaging this matrix row-wise generates a unified team intent vector, which represents the strategic consensus of the entire system at the current moment, guiding subsequent collaborative decisions among agents. Existing collaborative frameworks are inefficient in utilizing communication resources and have low value density of synchronized content, resulting in insufficient adaptability and robustness. By efficiently aggregating high-level intents rather than low-level states and introducing uncertainty-driven dynamic biases, agile and robust cognitive coordination is achieved.
[0045] S7: Concatenate the local cognitive state vector and the team's common intention vector, input them into the distributed policy network, and generate action vectors.
[0046] Specifically, the distributed policy network consists of a parameterized function approximator such as a multilayer perceptron (MLP), convolutional neural network (CNN), or Transformer. Through vector concatenation, the agent's decisions are forced to align with team consensus, eliminating action conflicts at the cognitive level. By deeply coupling shared intent into decision-making, it ensures that the policy is consistent from the outset, improving system robustness. The action vector is... in, This represents the action vector of agent i. Represents a distributed policy network. This represents the local cognitive state vector of agent i. This represents the team's shared intent vector.
[0047] like Figure 2 As shown in the figure, an embodiment of this application provides a multi-agent game cooperative control system based on a cognitive synchronization mechanism, comprising: The individual cognitive encoding module 10 is used to input local observation and historical action vectors into the individual cognitive encoder to obtain a local cognitive state vector; it is also used to input the local cognitive state vector into the individual intention projection layer to obtain a local intention vector.
[0048] The trigger module 20 is used to determine the uncertainty of the current moment based on all local intent vectors within a preset time period before the current moment; it is also used to concatenate the uncertainty and environmental change indicators and input them into the trigger network to obtain the synchronous trigger probability.
[0049] The attention synchronization module 30 is used to determine the team's common intent vector based on the uncertainty at the current moment and the local intent vectors of all agents when the synchronization trigger probability is less than or equal to a preset probability threshold; it is also used to resynchronize and acquire the local intent vectors of all agents when the synchronization trigger probability is greater than the preset probability threshold, and update the team's common intent vector based on the uncertainty at the current moment and the reacquired local intent vectors of all agents.
[0050] The decision output module 40 is used to concatenate the local cognitive state vector and the team's common intention vector, and input them into the distributed policy network to generate action vectors.
[0051] Specifically, the overall architecture is deployed on each agent, forming a self-contained intelligent unit. Through a hierarchical and progressive mechanism design, the inefficiency of collaborative mechanisms in traditional methods is effectively addressed. Due to the collaborative effect of the individual cognitive encoding module and the triggering module, the system can adapt to environmental changes, activating the synchronization mechanism only when necessary, thus significantly reducing communication load. Simultaneously, the attention synchronization module optimizes team intent generation through a dynamic bias matrix and aggregation algorithm, avoiding decision-making biases caused by limited local information in distributed methods. For example, in a drone search and rescue scenario, when sudden environmental changes (such as the appearance of new obstacles) increase uncertainty, if the synchronization trigger probability output by the triggering module exceeds a threshold, the attention synchronization module immediately triggers a global intent update, prompting all drones to quickly adjust their search direction. In a stable environment, the system only needs to determine the team intent based on local data, maintaining efficient and low-cost operation. Through the above technical path, this application significantly improves overall collaborative performance while ensuring the robustness and scalability of the multi-agent system, providing reliable technical support for multi-agent control in complex dynamic environments.
[0052] The multi-agent game cooperative control method based on cognitive synchronization mechanism also includes: Based on the training dataset, the individual cognitive encoder, individual intention projection layer, triggering network, attention layer, and distributed policy network are trained. Each data point in the training dataset includes the global state, global reward, local observation, action vector, and the global state at the next time step.
[0053] Based on the comprehensive loss function, the parameters of the individual cognitive encoder, individual intention projection layer, triggering network, attention layer and distributed policy network are updated in reverse until the global reward converges in multiple consecutive training cycles, or the total number of training steps reaches the preset maximum number of steps, thus obtaining the trained individual cognitive encoder, individual intention projection layer, triggering network, attention layer and distributed policy network. The comprehensive loss function includes policy gradient loss and cognitive consistency loss.
[0054] Specifically, cognitive synchronization loss is used to guide individual intentions to align with shared intentions, thereby training a game-theoretic cooperative policy network for each agent. Reinforcement learning is primarily reflected in the learning and training of each agent's cognitive alignment, game-theoretic cooperative control strategy, and cognitive synchronization triggering network. During the training phase, multiple agent replicas are run in parallel in a simulation environment, with each agent executing its current policy and collecting experience data. Where s is the global state and r is the global reward. For local observation, For action vectors, This represents the global state at the next time step. For each sample, the forward propagation calculations of multiple modules in the system are performed sequentially: in, This represents the local cognitive state vector of agent i at the current time t. For the local observations of agent i at the current time t, This represents the historical action vector of agent i at historical time t-1. This represents the local intent vector of agent i at the current time t. This represents the query vector of agent i at the current time t. This represents the key vector of agent i at the current time t. This represents the value vector of agent i at the current time t. , and These are the row vectors that make up matrices Q, K, and V, respectively. This represents the uncertainty of agent i at the current time t. This represents the uncertainty of agent j at the current time t. This represents the team's shared intention vector at the current time t. This indicates that the average value of A is calculated row by row. This represents the action vector of agent i at the current time t. This represents an indicator of environmental change at the current time t. This represents the probability of synchronization triggering of agent i at the current time t.
[0055] The parameters of the network within all modules are updated synchronously by optimizing a comprehensive loss function that incorporates the contributions of all agents. This loss function consists of two core components: policy gradient loss and cognitive consistency loss. For the policy gradient loss, the Proximal Policy Optimization (PPO) algorithm is used to update the distributed policy network of the distributed decision output module. for in, This represents the action vector of agent i at the current time t. This represents the global state at the current time t. This represents the current parameters of the distributed policy network. Represents the previous parameters of the distributed policy network; clip() is a function to constrain the numerical range. Represents the joint action of all intelligent agents The probability ratio under the old and new strategies Indicates the limiting parameter. This represents the minimum probability ratio. This represents the maximum value of the probability ratio. The joint advantage function under the joint state is represented by the generalized advantage estimation (GAE) method.
[0056] Loss of cognitive consistency for in, This represents the local intent vector of agent i. This represents the team's shared intent vector.
[0057] Comprehensive loss function The weighted sum of the two: in, The weights representing the cognitive consistency loss are typically set to 0.1-1.0. Network parameters within all modules are trained synchronously by calculating the gradient of the comprehensive loss function with respect to all parameters. When the global reward... If the system fails to improve significantly (i.e., converges) within multiple consecutive evaluation periods, or if the total number of training steps reaches the preset maximum number of steps, training is stopped, and the trained network is output. During the execution phase, multiple modules are completely replicated to each agent, and the game-theoretic collaborative control of the entire multi-agent system is completed through the architectural model composed of the execution modules.
[0058] Example 1: Cooperative reconnaissance mission by drone swarm Multiple drones need to traverse complex areas and conduct reconnaissance of multiple random targets. Each drone acts as an intelligent agent. The application process is as follows: In the initial phase, each UAV acquires local observation vectors based on its onboard sensors. The process involves several stages. The first stage includes information such as terrain, weather, and suspicious signals within the field of view. Next, each UAV's individual cognitive encoding module generates local cognitive states based on local observations, such as "Target A area has a high threat level" or "Our UAV 2 has low battery," as well as local intent information such as "Recommend searching eastward" or "Anomaly detected to the north." Subsequently, all UAVs exchange all local intents, and through an attention synchronization module, these intents are merged into a unified team common intent vector. This vector might be decoded as: "Current strategic focus: Concentrate forces to verify the anomaly to the north, and disperse some forces to expand the search area to the east." All UAVs then begin operations based on this consensus. Each UAV's decision output module, based on the latest team common intent vector and its own local intent, employs game-theoretic collaborative control strategies to output specific flight control commands, automatically forming collaborative tactics such as encirclement and relay tracking without ground station intervention. If unexpected events occur during reconnaissance, such as target discovery or target loss, the cognitive uncertainty of the associated UAVs will significantly increase, triggering a new round of cognitive synchronization. All UAVs then obtain a new common intent through communication.
[0059] The proposed solution has extremely low latency from target discovery to full cluster response, far faster than traditional methods that require sending instructions back to the data center for redistribution, enabling real-time cognitive synchronization. It only exchanges lightweight intent vectors at critical moments, rather than continuously transmitting video streams, greatly saving bandwidth and improving communication efficiency. Even if some drones lose contact, the remaining units can still autonomously coordinate based on the latest consensus to continue completing the task, increasing system robustness.
[0060] Example 2: Cooperative Collision Avoidance and Mission Takeover in On-Orbit Services of Spacecraft Clusters In the space environment, a cluster of service spacecraft is performing on-orbit servicing missions for a space facility, such as collaboratively capturing a failed satellite. They need to avoid each other and space debris with extreme precision, and quickly take over the mission in the event of a unit failure. The application process is as follows: Each service spacecraft continuously acquires observational information such as its own position, velocity, and attitude, as well as space debris warnings, through inter-satellite links and onboard sensors. Its individual cognitive encoding module outputs a local intent vector containing its own motion intentions and risk predictions. The attention synchronization module merges all local intent vectors to generate a high-precision team shared intent vector, which encodes the cluster's overall safe maneuver range and the optimal path sequence for collaborative acquisition, ensuring collision-free operation at the millisecond and millimeter levels. Each service spacecraft's distributed decision output module, based on the team shared intent vector and its own local intent vector, outputs thruster ignition pulse sequences and robotic arm control commands, achieving precise and safe movements. In the face of sudden collision risks, unit failures, or a surge in uncertainty or cognitive inconsistencies, cognitive synchronization is triggered, and a new team shared intent vector is reached again through the attention synchronization module.
[0061] This application's solution ensures absolute consistency of safety paths and task steps across all units during extremely high-risk missions through cognitive-level consensus and intent synchronization, fundamentally preventing collisions caused by misunderstandings. Faced with sudden threats or unit failures, the system can reorganize itself through event-triggered cognitive reconstruction, enhancing its adaptability and robustness. In-orbit mission decisions must be completed within an extremely short timeframe; this solution's distributed synchronization mechanism avoids the lengthy delays associated with transmitting data back to Earth, thus resolving real-time issues.
[0062] In summary, compared with existing technologies, it has the following beneficial effects: 1. In terms of collaborative efficiency, this application achieves a leap from state consistency to cognitive consistency, fundamentally improving the consistency of decision-making from a system architecture perspective. In existing technologies, agents make decisions under conditions of incomplete information and cognitive isolation, which is the root cause of low collaborative efficiency and decision conflicts. This application introduces a cognitive synchronization layer, providing all agents with a unified consensus, thereby naturally guiding each agent towards a common goal, significantly reducing the possibility of policy conflicts, and improving the system's success rate and task completion rate in complex tasks.
[0063] 2. In terms of communication efficiency, this application maximizes value density and achieves on-demand allocation, significantly reducing bandwidth requirements. It exchanges high-value-density intent vectors, rather than raw observation data or low-level action commands. Combined with an event-triggered mechanism, communication occurs only when there is cognitive inconsistency or sudden environmental changes. Compared to traditional methods that require continuous exchange of state information or decision results, this application significantly reduces the system's communication load and computational overhead, making the method particularly suitable for harsh environments with limited communication bandwidth, such as aerospace and underwater applications.
[0064] 3. Enhanced system adaptability to uncertainty and dynamic changes. Existing methods suffer from drastic performance degradation when some agents fail to perceive changes or when the environment undergoes sudden shifts. This application's fully distributed architecture eliminates the risk of single points of failure, and the event-triggered mechanism enables the system to proactively perceive and respond to uncertainty. In particular, when individual agents fail or their information is interfered with, the attention synchronization module automatically reduces their weights, allowing the system to quickly reconstruct consensus based on information from healthy agents, forming a unified and accurate understanding of environmental changes, thus exhibiting stronger robustness.
[0065] 4. It provides a cross-domain, universal framework. The "cognitive encoding-synchronization-decision" architecture proposed in this application strips away the details of specific application scenarios, with its core focus on addressing the cognitive alignment problem between intelligent agents. Therefore, this framework can not only be applied to drone formations and intelligent vehicles, but is also naturally suitable for complex systems with extremely high requirements for autonomous collaboration, such as spacecraft swarms and robot teams, providing a novel and universal technical path for solving the collaborative control challenges in such systems.
[0066] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0067] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A multi-agent game cooperative control method based on a cognitive synchronization mechanism, characterized in that, The method comprises the following steps: inputting the local observation and the historical action vector into an individual cognitive encoder to obtain a local cognitive state vector; inputting the local cognitive state vector into an individual intention projection layer to obtain a local intention vector; determining uncertainty at the current moment according to all local intention vectors within a preset time length before the current moment; concatenating the uncertainty and an environmental change indicator and inputting them into a trigger network to obtain a synchronous trigger probability; if the synchronous trigger probability is less than or equal to a preset probability threshold, determining a team common intention vector according to the uncertainty at the current moment and the obtained local intention vectors of all agents; if the synchronous trigger probability is greater than the preset probability threshold, re-synchronizing the local intention vectors of all agents, updating the team common intention vector according to the uncertainty at the current moment and the re-synchronized local intention vectors of all agents; concatenating the local cognitive state vector and the team common intention vector and inputting them into a distributed strategy network to generate an action vector.
2. The multi-agent game collaborative control method based on cognitive synchronization mechanism according to claim 1, characterized in that, The step of determining the uncertainty at the current moment according to all local intention vectors within a preset time length before the current moment comprises the following steps: determining a covariance matrix according to all local intention vectors within a preset time length before the current moment; analyzing the trace of the covariance matrix to obtain the uncertainty at the current moment.
3. The multi-agent game collaborative control method based on cognitive synchronization mechanism according to claim 1, characterized in that, The step of determining or updating the team common intention vector comprises the following steps: determining a dynamic bias matrix according to the uncertainty at the current moment of the two agents; determining a query vector, a key vector, a value vector and the dimension of the key vector of a single agent through linear projection according to the local intention vector of each agent; stacking the query vectors, the key vectors and the value vectors of all agents into a query matrix, a key matrix and a value matrix respectively; inputting the dynamic bias matrix, the query matrix, the key matrix, the value matrix and the dimension of the key vector into an attention layer to obtain the team common intention vector.
4. The multi-agent game collaborative control method based on cognitive synchronization mechanism according to claim 2, characterized in that, The local intention vector is wherein, represents a local intention vector of the agent i, represents a weight matrix of the individual intention projection layer, represents a bias vector of the individual intention projection layer, represents a local cognitive state vector of the agent i; The uncertainty at the current moment is wherein, represents the uncertainty of the agent i at the current time t, T represents a preset time length before the current time, Cov() represents a covariance matrix, and tr() represents a trace of the covariance matrix, represents the local intention vector of the agent i at the current time t, represents the local intention vector of the agent i at the current time t, represents the local intention vector of the agent i at the current time t.
5. The multi-agent game collaborative control method based on cognitive synchronization mechanism according to claim 1, characterized in that, The synchronous trigger probability is wherein, denotes the probability of a synchronous trigger, denotes a weight matrix of the trigger network, denotes a bias vector of the trigger network; denotes the uncertainty of the agent i, denotes an environmental change indicator.
6. The multi-agent game collaborative control method based on cognitive synchronization mechanism according to claim 3, characterized in that, The dynamic bias of the agent i to the agent j is wherein, denotes the uncertainty of agent i, denotes the uncertainty of agent i, and λ denotes a learnable scaling parameter for the attention layer; The output matrix of the attention layer is wherein A represents an output matrix of the attention layer, Q represents a query matrix stacked by query vectors of all agents, K represents a key matrix stacked by key vectors of all agents, and V represents a value matrix stacked by value vectors of all agents, denotes the dimension of the key vector, and B represents a dynamic bias matrix of the attention layer. The team common intention vector is obtained by averaging each row of the output matrix A.
7. The multi-agent game collaborative control method based on cognitive synchronization mechanism according to claim 1, characterized in that, The action vector is wherein, represents an action vector of the agent i, represents a distributed policy network, represents a local cognitive state vector of the agent i, represents a team common intention vector.
8. The multi-agent game collaborative control method based on cognitive synchronization mechanism according to claim 1, characterized in that, The method further comprises the following steps: training the individual cognitive encoder, the individual intention projection layer, the trigger network, the attention layer and the distributed strategy network according to a training data set, wherein each data in the training data set comprises a global state, a global reward, a local observation, an action vector and a global state at the next moment; updating the parameters of the individual cognitive encoder, the individual intention projection layer, the trigger network, the attention layer and the distributed strategy network in a reverse direction according to a comprehensive loss function until the global reward converges in continuous multiple training periods or the total training step number reaches a preset maximum step number, so as to obtain the trained individual cognitive encoder, the individual intention projection layer, the trigger network, the attention layer and the distributed strategy network, wherein the comprehensive loss function comprises a strategy gradient loss and a cognitive consistency loss.
9. The multi-agent game collaborative control method based on cognitive synchronization mechanism according to claim 1, characterized in that, The integrated loss function is wherein, represents a weight coefficient of the cognitive consistency loss; the policy gradient loss for wherein, represents the action vector of the agent i at the current time t, represents the global state at the current time t, represents the joint advantage function under the joint state, represents the current parameter of the distributed policy network, represents the previous parameter of the distributed policy network; clip() numerical range constraint function, represents the joint action of all agents the probability ratio under the new and old policies, represents the limit parameter, represents the minimum value of the probability ratio, represents the maximum value of the probability ratio; The cognitive consistency loss To wherein, represents the local intention vector of the agent i, represents the team common intention vector.
10. A multi-agent game cooperative control system based on a cognitive synchronization mechanism, characterized in that, The method comprises the following steps: The individual cognitive coding module is configured to input the local observation and the historical action vector into an individual cognitive encoder to obtain a local cognitive state vector; and configured to input the local cognitive state vector into an individual intention projection layer to obtain a local intention vector. The trigger module is configured to determine uncertainty at the current time according to all local intention vectors within a preset time length before the current time; and configured to splice the uncertainty and an environment change index and input them into a trigger network to obtain a synchronization trigger probability. The attention synchronization module is configured to determine a team common intention vector according to the uncertainty at the current time and the obtained local intention vectors of all agents if the synchronization trigger probability is less than or equal to a preset probability threshold; and configured to re-synchronize the local intention vectors of all agents, update the team common intention vector according to the uncertainty at the current time and the re-synchronized local intention vectors of all agents if the synchronization trigger probability is greater than the preset probability threshold. The decision output module is configured to splice the local cognitive state vector and the team common intention vector and input them into a distributed strategy network to generate an action vector.