A Multi-Agent Interaction Intent Understanding and Cooperative Control Method Based on a Large Model

Abstract

This invention relates to the field of large model-driven inference technology, specifically disclosing a method for understanding and coordinating the interaction intentions of multi-agent systems based on large models. The method includes the following steps: S1, perception of environmental and multi-source heterogeneous interaction information; S2, hierarchical intention understanding driven by the large model; S3, generation of intention-guided collaborative strategies; and S4, action execution and closed-loop online optimization. Through end-to-end technological innovation, this invention significantly improves the accuracy of intention understanding, efficiency of collaborative control, and scene adaptability of multi-agent systems, providing core technical support for multi-agent collaborative applications in complex scenarios. It possesses extremely high engineering application value and industrial promotion potential.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large model driven reasoning, in particular to a multi-agent interaction intention understanding and collaborative control method based on a large model. BACKGROUND

[0002] With the development of artificial intelligence and autonomous control technology, multi-agent systems are increasingly widely used in automatic driving vehicle team collaboration, unmanned aerial vehicle cluster formation, distributed robot rescue and other fields. The core requirement of such systems is to accurately understand the neighborhood intention through real-time interaction perception between agents, and generate globally collaborative control actions; for example, in automatic driving, the rear vehicle needs to recognize the front vehicle's lane changing intention to adjust the following distance, the unmanned aerial vehicle cluster needs to maintain the formation through collaborative intention, and the rescue robot needs to allocate the search area based on the target intention of the partner. However, existing multi-agent collaboration technology still has the following key limitations, which are difficult to meet the actual needs of complex scenarios:

[0003] 1. Existing technology does not sufficiently fuse multi-modal data of environmental state information (such as position, obstacle) and neighborhood interaction information (such as communication instructions, action feedback), and the intention recognition lacks a hierarchical structure:

[0004] Only focusing on a single behavior intention (such as left turn and acceleration), ignoring the target intention (such as whether the unmanned aerial vehicle is detecting obstacles or supporting partners) and the collaborative intention (which agents need to cooperate with), leading to misjudgment of intention; for example, in automatic driving, the rear vehicle mistakenly judges the front vehicle's "temporary lane changing to avoid obstacles" intention as a regular overtaking, causing the risk of following too close;

[0005] The fusion method of discrete data and continuous data is simple, and the semantic association of multi-modal data is not captured, for example, in robot rescue, the "infrared vital sign signal" cannot be associated with the partner's "request for support" instruction, missing the rescue opportunity.

[0006] 2. Existing collaborative control strategies mostly rely on preset rules or single environmental rewards, without considering the intentions of neighborhood agents in the strategy generation logic, leading to disconnection between intention and action collaboration:

[0007] Strategy generation only considers environmental feedback (such as whether the target point is reached and whether the obstacle is collided), ignoring the intention Figure 1 consistency; for example, in the unmanned aerial vehicle cluster, a certain unmanned aerial vehicle does not coordinate with the partner's intention to maintain a triangular formation, and accelerates alone, causing the formation to collapse;

[0008] The reward function design is one-sided, only quantifying environmental interaction benefits (such as travel distance, search area), without measuring the matching degree of action and intention, leading to the direction of strategy update deviating from the actual collaboration demand, for example, the rescue robot prioritizes completing the search area indicator, rather than prioritizing rescuing the injured with the partner.

[0009] 3. Current general-purpose language models are not customized and optimized for multi-agent interaction scenarios, making them difficult to apply directly:

[0010] Lacking end-to-end reasoning capabilities for observation, intent, and action, it is unable to infer potential intents from historical observation sequences (such as neighborhood actions over the past 5 steps). For example, it is unable to predict lane-changing intents from observations of the autonomous vehicle in front slightly slowing down and activating its turn signal.

[0011] The model has redundant parameters and high inference latency. The hundreds of billions of parameters of the general large language model are difficult to adapt to the edge devices of multi-agent systems (such as vehicle controllers and drone embedded modules). In real-time interactive scenarios, the inference latency often exceeds 200ms, far exceeding the clinical requirement of response within 50ms.

[0012] 4. There are two major problems with the closed-loop optimization of the existing system:

[0013] The experience pool uses a simple first-in-first-out elimination strategy, which does not prioritize high-value data (such as high-quality data). Figure 1 Prioritizing the sampling of data (such as data with consistent rewards and rare intent samples) leads to low model update efficiency; for example, if a rare emergency braking intent data is eliminated too early, the model's ability to respond to sudden risks cannot be improved.

[0014] The large language model and the collaborative policy network are not updated in sync. For example, the intent understanding model has been fine-tuned with new data, but the policy network is still updated based on old intent data, which leads to the contradiction that the intent understanding is accurate but the policy execution is biased. Summary of the Invention

[0015] To address the shortcomings of existing technologies, this invention provides a method for understanding and coordinating the interaction intentions of multiple agents based on a large model, thus solving the problems mentioned in the background technology.

[0016] To achieve the above objectives, the present invention provides the following technical solution: a method for understanding and coordinating the interaction intentions of multiple agents based on a large model, comprising the following steps:

[0017] S1. Environmental and Multi-Source Heterogeneous Interaction Information Perception: Multiple agents collect two types of information in real time through their own onboard sensors and distributed communication links between agents, forming local observation sequences for each agent. ;

[0018] First, environmental status information, which includes at least the agent's own position, speed, task progress, and the distribution of environmental obstacles;

[0019] Second, neighborhood interaction information, which includes at least communication instructions from neighborhood agents, feedback on executed actions, and task collaboration requests; among which... , Represents intelligent agents At time step Single-step observation data, To observe the size of the historical window, and Adjust dynamically based on task complexity;

[0020] S2. Understanding Hierarchical Intent Driven by Large Models: This involves interpreting the local observation sequences obtained in step S1. The input is fed into a pre-trained and task-fine-tuned large language model, which serves as the core inference engine and outputs information about other agents in the neighborhood. Hierarchical Intent Understanding Results The hierarchical intent understanding result contains at least three types of sub-intents;

[0021] The target intent used to characterize the ultimate goal of an intelligent agent. ;

[0022] Behavioral intentions used to predict future actions ;

[0023] Collaboration intent used to quantify collaboration needs ,Right now ;

[0024] S3. Generation of Intent-Guided Collaborative Strategies: Based on the Hierarchical Intent Understanding Results of Step S2 Each agent invokes a centralized training-distributed execution architecture cooperative policy network to generate cooperative control actions that satisfy the global task objective. The parameter updates of the cooperative policy network are guided by a composite reward function, which simultaneously incorporates environmental interaction rewards and intention rewards. Figure 1 Consistent rewards;

[0025] S4. Action Execution and Closed-Loop Online Optimization: Each agent executes the collaborative control actions generated in step S3. The new environmental state after the action is executed, the feedback intention of the neighboring intelligent agent, and the reward value are combined into closed-loop data and fed back to the large language model and cooperative policy network.

[0026] Among them, the large language model fine-tunes the intent reasoning parameters based on closed-loop data, and the collaborative policy network updates the control policy based on closed-loop data, realizing an end-to-end closed loop of intent understanding, action generation and feedback optimization.

[0027] Preferably, the large language model in step S2 adopts an encoder-decoder architecture, and the specific implementation process is as follows:

[0028] Encoder encoding process: The encoder adopts a 6-12 layer Transformer encoder structure, which encodes the local observation sequence of the input. Context encoding is performed; firstly, discrete observations are transformed into 128-512 dimensional embedding vectors through an embedding layer, and continuous observations are mapped to feature vectors of the same dimension through a normalization layer. Then, a position encoding layer is used to supplement the temporal dimension information, and finally, a context-aware hidden state sequence is output.

[0029] ;

[0030] in, For intelligent agents At time step The hidden state vector has dimensions of 256-1024, and the number of trainable parameters increases linearly with the number of Transformer layers. It is a hidden state sequence The basic building block of a sequence is a single-time-step hidden state in the sequence. For time steps, the range of values ​​is: It covers all moments within the historical window;

[0031] Decoder decoding process: The decoder adopts a 6-12 layer Transformer decoder structure, based on the hidden state sequence. Generate hierarchical intent understanding results During the decoding process, the generation task distinguishes between target intent, behavioral intent, and collaborative intent through intent type masks.

[0032] The Transformer encoder / decoder has 4-8 attention heads, the Feed-Forward network has 1024-2048 hidden layers, and the GELU activation function is used.

[0033] Preferably, the target intent in step S2 For one A 3D probability vector is used to represent the agent. The final task objective estimate is obtained from the decoder's hidden state using the following formula:

[0034] ;

[0035] in, For intelligent agents For intelligent agents At time step The target intent probability vector, with dimension . The first vector The value of each element represents the agent. Select the The probability of a target class is such that the sum of all elements is 1. The trainable weight matrix for decoding the target intent, with dimension . This is used to map hidden states to the target category dimension; Let be the hidden state vector output by the decoder at the target intent marker location, with dimension . These are the key contextual features extracted by the decoder for the target intent task; The trainable bias vector for decoding the target intent, with dimension . This is used to compensate for the offset of the linear transformation; for An activation function is used to map the result of a linear transformation to a probability distribution.

[0036] Preferably, the behavioral intent in step S2 A probability distribution of an action sequence is used to predict the agent. In the future The potential action sequence within a step is calculated as follows:

[0037] First, predict the agent. In the future step( The probability distribution of actions:

[0038] ;

[0039] Then calculate the future using the chain rule. Joint probability distribution of the action sequence of the steps:

[0040] ;

[0041] in, For intelligent agents In the future The behavioral intention of a step, i.e., the sequence of actions ; For intelligent agents In the future The action of a step, with values ​​ranging from the action space. ; The conditional probability distribution of actions based on the previous hidden state has a dimension of . , Size of the action space; For the decoder to predict the first The hidden state vector for each action has a dimension of . Inherited from the previous Contextual information about the step action; The trainable weight matrix for decoding behavioral intent, with dimension . ; The trainable bias vector for decoding behavioral intent, with dimension . ; For the future The joint probability of a sequence of actions measures the reliability of the action sequence; a higher value indicates that the sequence is more trusted by the agent. implement.

[0042] Preferably, the collaborative intent in step S2 For one Attention weight vector, For intelligent agents The number of neighboring agents, used to represent the number of agents. To achieve the goal The intensity of cooperation with neighboring agents required in future actions is calculated through an intentional attention mechanism:

[0043] ;

[0044] ;

[0045] in, For intelligent agents For intelligent agents At time step The collaboration intent weight, with a value range of [value range missing]. The larger the value, the more intelligent the agent. Need to work with intelligent agents The higher the intensity of cooperation; For intelligent agents The set of neighborhood intelligent agents is determined by the communication radius or task relevance. This is the attention scoring function used to calculate the agent's performance. and The degree of collaboration is expressed as a scalar. The intelligent agent output by the decoder The collaborative intent hidden state vector, with dimension . It integrates collaborative needs information that combines target intent and behavioral intent; For intelligent agents The collaborative intent shared through the communication link hides the state vector, with a dimension of [missing information]. ; The trainable weight matrix for collaborative intent attention has dimensions of . It is used to learn the collaborative relationship features between different intelligent agents; The transpose of the vector, with dimension , is used to perform inner product operations between vectors.

[0046] Preferably, the collaborative strategy network in step S3 is a centralized training-distributed execution architecture, and the specific implementation process is as follows:

[0047] The network input layer takes an agent as its input. Local observation sequence Hierarchical intent understanding results with all neighboring agents The two types of inputs are fused into a single dimension through a splicing layer. The input feature vector, For single-step observation dimension, For hierarchical intent dimensions;

[0048] The network hidden layers employ 2-4 layers of fully connected networks, with hidden layer dimensions ranging from 512 to 2048. The activation function used is the LeakyReLU function, which is employed to extract non-linear collaborative associations of input features.

[0049] The network output layer outputs intelligent agents. At time step Coordinated control actions If it is a discrete action space, the output is the action probability distribution; if it is a continuous action space, the output is the action value.

[0050] Parameter update rules, parameters of the cooperative strategy network By maximizing the cumulative expected reward Updated, the formula is:

[0051] ;

[0052] in, Let be the objective function of the policy network, i.e., the cumulative expected reward; In strategy The expected value of a reward is used to measure the average level of the reward. This represents the maximum number of time steps for the task. This is a discount factor, with a value range of [0.9, 0.99], used to weigh the importance of current rewards against future rewards; The reward for environmental interaction is calculated based on the task objective; For the purpose Figure 1 Consistent rewards are used to measure the degree of alignment between actions and intentions; This is the reward balance coefficient, with a value range of [0.3, 0.7], used to adjust the weight of environmental rewards and intention rewards.

[0053] Preferably, the intention in step S3 Figure 1 Consistent reward The following formula is used to simultaneously encourage action and one's own intention. Figure 1 Matching actions with collaboration needs:

[0054] ;

[0055] in, For the purpose Figure 1 Consistent reward, with a value range of [0,1]. The larger the value, the higher the degree of matching between the action and the intention. As the reward weighting coefficient, satisfying ,and ; for Divergence, used to measure the performance of an agent Predicted action distribution Distribution of actual actions The difference, the range of values ​​is The smaller the difference, the better. The larger; For intelligent agents The probability distribution of the actual action sequence is obtained from the statistics of the action execution log; For intelligent agents For intelligent agents The weight of the intention to collaborate; The action similarity function is calculated using cosine similarity, and the formula is as follows: The range of values ​​is The larger the value, the higher the degree of cooperation and matching between the two actions; for Norms are used to calculate the magnitude of a vector. For current intelligent agents The execution action, For neighborhood intelligent agents Synchronous actions.

[0056] Preferably, the online optimization in step S4 specifically includes two parts: fine-tuning of the large language model and updating the collaborative strategy network, as follows:

[0057] A closed-loop experience pool is constructed by storing the closed-loop data after the actions are executed into the experience pool, with a capacity of [missing information]. The system employs a first-in-first-out (FIFO) strategy to eliminate expired data, while also introducing a priority experience replay mechanism to assign higher sampling weights to high-reward data.

[0058] For fine-tuning the large language model, after accumulating 100-1000 closed-loop data points, 32-128 data points are randomly sampled from the experience pool as fine-tuning samples; the fine-tuning loss function is the intention prediction cross-entropy loss. :

[0059] ;

[0060] in, For hierarchical intent Cross-entropy with true intention, Loss balance coefficient Fine-tuning was performed using the AdamW optimizer with a learning rate of [missing information]. Only the parameters of the top 2-4 layers of the large language model are updated to avoid destroying the underlying pre-trained features;

[0061] Cooperative strategy network update, using the PPO algorithm to update network parameters. Each iteration samples 256-1024 empirical data points, iterating 3-10 times; during updates, the objective function is clipped to ensure the stability of the policy update, the formula being:

[0062] ;

[0063] in, The strategy parameters before the update. For the dominant function, This is the editing coefficient. .

[0064] Preferably, before step S1, a pre-training step of the large language model is included. The pre-training process focuses on intent reasoning ability and action chain reasoning ability, as detailed below:

[0065] Pre-training dataset construction: The dataset contains three classes of data, and the total size is not less than [amount missing]. The sample data consists of: 1) multi-agent collaborative dialogue text; 2) task instruction-action log pairing data; and 3) intent annotation data.

[0066] The pre-training task design employs multi-task joint pre-training, simultaneously optimizing three objective functions:

[0067] Masked Language Modeling (MLM): Randomly mask 15% of the tokens in the text, train the model to predict the masked tokens, and use cross-entropy as the loss function;

[0068] Next Sentence Prediction (NSP): Input two sentences, train the model to determine whether the next sentence is a reasonable continuation of the previous sentence, and use the loss function as binary cross-entropy.

[0069] Action Chain Prediction (ACP): Input the task instruction and the previous T-1 steps of action, train the model to predict the action chain... For each step of the action, the loss function is either the action category cross-entropy or the MSE loss.

[0070] Pre-training parameter settings: Fine-tuned based on an open-source large language model, with 5-20 training epochs, batch size of 32-128, and a learning rate of [missing information]. The optimizer used is AdamW, with a weight decay factor of . This ensures that the model initially possesses the ability to reason about intent and predict actions in multi-agent scenarios.

[0071] The preferred method for understanding and coordinating multi-agent interactions based on a large model is adapted to three typical multi-agent collaborative tasks, and is further modified for different tasks based on specific scenarios:

[0072] Autonomous driving fleet cooperative control: In this scenario, the environmental state information in step S1 is supplemented with traffic light status, lane line position, and the speed of the vehicle in front, and the neighborhood interaction information is supplemented with fleet communication protocol frames; the target intent in step S2... The categories are lane keeping, lane changing, overtaking, and emergency braking, with behavioral intent. The action space includes acceleration, steering, and braking;

[0073] Drone swarm formation flight: In this scenario, the environmental status information in step S1 is supplemented with altitude, heading angle, battery level and airspace obstacles, and the neighborhood interaction information is supplemented with millimeter-wave radar data between drones; the cooperative strategy network output layer in step S3 adds formation constraints.

[0074] Distributed robot collaborative search and rescue: In this scenario, the environmental status information in step S1 is supplemented with vital signs and terrain slope, and the neighborhood interaction information is supplemented with the shared data of the rescue target location; the online optimization in step S4 increases the rescue priority weight.

[0075] This invention addresses the limitations of existing technologies through end-to-end innovation in "multimodal perception, hierarchical intent understanding, intent-guided collaboration, and closed-loop optimization," and its beneficial effects are as follows:

[0076] 1. Relying on the Transformer encoder, deep semantic fusion of discrete communication commands and continuous sensor data is achieved to generate context-aware hidden state sequences; the decoder independently generates three types of sub-intents—target, behavior, and cooperation—through "intent type mask," accurately identifying the task objectives, action planning, and cooperation needs of neighboring intelligent agents, and avoiding cooperation mismatch caused by intent cognition bias.

[0077] 2. The collaborative strategy network takes hierarchical intent as its core input, combining "environmental interaction and intent" Figure 1 The "coherent" composite reward function updates parameters to ensure synchronized collaborative actions (such as the following vehicle dynamically adjusting its deceleration based on the lane-changing intention of the preceding vehicle in autonomous driving); it adopts a "centralized training-distributed execution" architecture to optimize the global task objectives while reducing the computational pressure on edge devices and meeting the low-latency requirements of real-time interaction scenarios.

[0078] 3. By using "multi-task pre-training (MLM, NSP, ACP) and top-level parameter fine-tuning", the large language model is equipped with the ability to reason about "observation, intent and action", reducing parameter redundancy to adapt to edge devices; the pre-trained model comes with general logic for multi-agent interaction, and only needs to supplement scene-specific data (such as V2X protocol frames, vital signs signals) and fine-tune a few parameters to achieve scene transfer, reducing transfer costs.

[0079] 4. The experience pool introduces priority experience replay, focusing on high-value key data (such as sudden obstacles and emergency rescue data) to improve model update efficiency; the large language model and collaborative strategy network are updated synchronously based on the same closed-loop data to avoid "accurate understanding of intent but deviation in strategy execution" and ensure stable system operation in dynamic scenarios (such as the rapid response of drone swarms to airspace obstacles).

[0080] 5. For three types of scenarios—autonomous driving, drone swarms, and rescue robots—adaptation is achieved by supplementing dedicated perception data, customizing intent categories, and optimizing strategy constraints: autonomous driving enhances fleet communication and action space adaptation, drone swarms enhance formation constraints, and rescue robots enhance vital sign perception and rescue priority, thus solving the problem of traditional "one-size-fits-all" adaptation.

[0081] In summary, this invention significantly improves the intent understanding accuracy, collaborative control efficiency, and scenario adaptability of multi-agent systems through end-to-end technological innovation, providing core technical support for multi-agent collaborative applications in complex scenarios and possessing extremely high engineering application value and industrial promotion potential. Attached Figure Description

[0082] Figure 1 This is a schematic diagram of the process of a multi-agent interaction intent understanding and collaborative control method based on a large model according to the present invention;

[0083] Figure 2 This is a block diagram illustrating the principle of the centralized training-distributed execution architecture of this invention. Detailed Implementation

[0084] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] like Figure 1 As shown, this invention provides a technical solution: a method for understanding and coordinating the interaction intentions of multiple agents based on a large model, comprising the following steps:

[0086] S1. Environmental and Multi-Source Heterogeneous Interaction Information Perception: Multiple agents collect two types of information in real time through their own onboard sensors (such as positioning sensors, image sensors, and force sensors) and distributed communication links between agents, forming local observation sequences for each agent. ;

[0087] First, environmental state information, including at least the agent's own position, speed, task progress, and the distribution of environmental obstacles; second, neighborhood interaction information, including at least communication commands, feedback on executed actions, and task cooperation requests from neighboring agents; among these... , Represents intelligent agents At time step Single-step observation data, To observe the size of the historical window, and Adjust dynamically based on task complexity;

[0088] S2. Understanding Hierarchical Intent Driven by Large Models: This involves interpreting the local observation sequences obtained in step S1. The input is fed into a pre-trained and task-fine-tuned large language model, which serves as the core inference engine and outputs information about other agents in the neighborhood. Hierarchical Intent Understanding Results The hierarchical intent understanding result contains at least three types of sub-intents;

[0089] The target intent used to characterize the ultimate goal of an intelligent agent. Behavioral intentions used to predict future actions Collaboration intent used to quantify collaboration needs ,Right now ;

[0090] S3. Generation of Intent-Guided Collaborative Strategies: Based on the Hierarchical Intent Understanding Results of Step S2 Each agent invokes a centralized training-distributed execution architecture cooperative policy network to generate cooperative control actions that satisfy global task objectives (such as formation control and cooperative search). The parameter updates of the cooperative policy network are guided by a composite reward function, which simultaneously incorporates environmental interaction rewards and intention rewards. Figure 1 Consistent rewards;

[0091] S4. Action Execution and Closed-Loop Online Optimization: Each agent executes the collaborative control actions generated in step S3. The data will be composed of the new environmental state after the action is executed, the feedback intention of the neighboring agents, and the reward value, forming a closed-loop data. The data is fed back to the large language model and the collaborative policy network. The large language model fine-tunes the intent inference parameters based on the closed-loop data, and the collaborative policy network updates the control policy based on the closed-loop data, thus realizing an end-to-end closed loop of intent understanding, action generation, and feedback optimization.

[0092] More specifically, the large language model in step S2 adopts an encoder-decoder architecture, and the specific implementation process is as follows:

[0093] Encoder encoding process: The encoder adopts a 6-12 layer Transformer encoder structure, which encodes the local observation sequence of the input. Context encoding is performed; firstly, discrete observations (such as communication commands) are transformed into 128-512 dimensional embedding vectors through an embedding layer, and continuous observations (such as position and velocity) are mapped to feature vectors of the same dimension through a normalization layer. Then, a position encoding layer is used to supplement the temporal dimension information, and finally, a context-aware hidden state sequence is output.

[0094] ;

[0095] in, For intelligent agents At time step The hidden state vector has dimensions of 256-1024, and the number of trainable parameters increases linearly with the number of Transformer layers. It is a hidden state sequence The basic building block of a sequence is a single-time-step hidden state in the sequence. For time steps, the range of values ​​is: It covers all moments within the historical window;

[0096] Decoder decoding process: The decoder adopts a 6-12 layer Transformer decoder structure, based on the hidden state sequence. Generate hierarchical intent understanding results During the decoding process, the generation tasks of target intent, behavioral intent, and collaborative intent are distinguished by intent type masking; that is, when decoding the target intent, the markers of behavioral and collaborative intents are masked, and when decoding the behavioral intent, the markers of target and collaborative intents are masked, to ensure the independence and accuracy of each sub-intent generation.

[0097] The Transformer encoder / decoder has 4-8 attention heads, the Feed-Forward network has 1024-2048 hidden layers, and the GELU activation function is used.

[0098] To prevent interference between target intent, behavioral intent, and collaborative intent during the generation process, an intent type mask (IntentTypeMask) is introduced during decoding. The specific operation is as follows:

[0099] Target Intent Decoding: In generating target intent At that time, the tokens related to behavioral intent and collaborative intent in the decoder input are masked, and only features related to the target task (such as the agent) are retained. (Task objective description)

[0100] Decoding behavioral intent: In generating behavioral intent At the same time, by masking out the markers related to target intent and collaborative intent, we can focus on the correlation features between historical action sequences and future action predictions;

[0101] Decoding collaborative intents: In generating collaborative intents At the same time, by masking out the tags related to the target intent and behavioral intent, only the characteristics of the cooperation needs between neighboring intelligent agents (such as the intelligent agents) are retained. and (Communication and cooperation instructions).

[0102] Hierarchical Intent Output: Through the above decoding process, the decoder finally outputs the hierarchical intent understanding result. ,in:

[0103] Target Intent for dimensional probability vector ( (Number of task target categories), used to characterize the intelligent agent. The ultimate goal of the task; behavioral intention For the future The joint probability distribution of step action sequences is used to predict the agent. Potential actions; intention to cooperate for dimensional attention weight vector ( (Number of agents in the neighborhood), used to quantify agents. The strength of cooperation with neighboring intelligent agents.

[0104] During the encoder preprocessing stage, it is necessary to strictly ensure that the discrete embedding vector and the continuous feature vector have the same dimension (128-512 dimensions) to avoid dimension mismatch problems in the subsequent encoding process. The number of layers of the Transformer encoder / decoder can be adjusted according to the real-time requirements of the application scenario. For example, if autonomous driving requires low latency response, a 6-layer structure can be used; if the drone swarm has high accuracy requirements, a 12-layer structure can be used.

[0105] More specifically, the target intent in step S2 For one dimensional probability vector ( For example, in a collaborative search task, the number of task target categories is [number of categories]. (corresponding to "Search Region A", "Search Region B", and "Supported Friendly Agents"), used to represent agents. The final task objective estimate is obtained from the decoder's hidden state using the following formula:

[0106] ;

[0107] The formula is essentially a two-step process involving linear transformation and probability mapping: first, through the weight matrix... With bias vector Map the hidden state to the target category dimension, and then... The function is transformed into a probability distribution, ensuring that the sum of the probabilities of all targets is 1.

[0108] in, For intelligent agents For intelligent agents At time step The target intent probability vector, with dimension . The first vector The value of each element represents the agent. Select the The probability of a target class is such that the sum of all elements is 1. The trainable weight matrix for decoding the target intent, with dimension . ( The hidden state dimension for the decoder, i.e. ), used to map hidden states to the target category dimension; Let be the hidden state vector output by the decoder at the target intent marker location, with dimension . These are the key contextual features extracted by the decoder for the target intent task; The trainable bias vector for decoding the target intent, with dimension . This is used to compensate for the offset of the linear transformation; for An activation function is used to map the result of a linear transformation to a probability distribution.

[0109] The constraints are as follows:

[0110] Probabilistic characteristics: for The vector, the sum of all its elements must be 1, to ensure that the probability of agent j choosing each target can be directly quantified;

[0111] Dimensional Relationships: Dimensions Strict matching required Dimensions and Dimensions To avoid dimensional mismatch in linear transformations;

[0112] The possible values ​​of: (Decoder hidden state dimension) is linked to the number of Transformer layers in the encoder-decoder architecture in step S2, ranging from 256 to 1024. The more layers, the more complex the task. The larger the possible value, the richer the feature representation.

[0113] Target Intent Its function: to quantify the agent through probability. The goal selection bias avoids the "either / or" goal misjudgment in traditional methods (such as in autonomous driving, distinguishing between...). The difference in objectives between "temporary obstacle avoidance" and "regular lane changing"); intelligent agents pass clear After achieving the ultimate goal, one can adjust their own strategies accordingly (such as...). When the objective is "support friendly forces", Can be given priority (Open up a path); The number of target categories can be dynamically adjusted according to the task (e.g., in a drone swarm). It corresponds to "reconnaissance", "formation", "material transportation" and "return", and has strong scene adaptability.

[0114] More specifically, the behavioral intent in step S2 A probability distribution of an action sequence is used to predict the agent. In the future The potential action sequence within a step is calculated as follows:

[0115] First, predict the agent. In the future step( The probability distribution of actions:

[0116] ;

[0117] Then calculate the future using the chain rule. Joint probability distribution of the action sequence of the steps:

[0118] ;

[0119] in, For intelligent agents In the future The behavioral intention of a step, i.e., the sequence of actions ; For intelligent agents In the future The action of a step, with values ​​ranging from the action space. ( ); The conditional probability distribution of actions based on the previous hidden state has a dimension of . , Size of the action space; For the decoder to predict the first The hidden state vector for each action has a dimension of . Inherited from the previous Contextual information about the step action; The trainable weight matrix for decoding behavioral intent, with dimension . ; The trainable bias vector for decoding behavioral intent, with dimension . ; For the future The joint probability of a sequence of actions measures the reliability of the action sequence; a higher value indicates that the sequence is more trusted by the agent. implement.

[0120] behavioral intention It is a key component of large-scale model-driven hierarchical intent understanding, representing the probability distribution of an agent's action sequence within future steps. Unlike traditional solutions that only predict single actions, it focuses on "multi-step action trajectory prediction," providing "neighborhood action predictability" support for multi-agent collaboration; and solving the problem of collaboration mismatch caused by sudden changes in the actions of neighboring agents (such as in autonomous driving, where the following vehicle predicts the lane-changing action of the vehicle in front to adjust the following distance in advance, or in drone swarms, where the flight trajectory of companions is predicted to maintain formation).

[0121] Through formula Realizing the future The probability prediction of movement has two main functions:

[0122] Quantifying the probability of an action occurring: using The function transforms the linear transformation result into a probability distribution (the sum of the probabilities of all actions is 1), allowing direct determination of the likelihood of each action occurring. For example, in an autonomous driving scenario, if the probability of a "right turn" action is 0.8 and the probability of a "straight ahead" action is 0.2, it indicates that the agent... There is an 80% probability that the next step will be a right turn, providing clear action prediction basis for neighboring intelligent agents (such as the car behind);

[0123] Capturing temporal correlation features: relying on hidden state vectors Before inheritance The step action context (such as the features of the first two steps "turn on the turn signal and decelerate") ensures that the predicted single-step action conforms to the temporal logic; it avoids meaningless action predictions such as "accelerate, emergency brake and accelerate", and ensures the rationality of the action sequence.

[0124] Through the chain rule formula Calculating the Future The joint probability of step action sequences has the following function:

[0125] Measuring the credibility of an action sequence: joint probability The value ranges from [0,1]. A larger value indicates that the action sequence is more likely to be detected by the agent. Execution; for example, in a drone swarm scenario, the joint probability of the action sequence "ascend 50 meters, maintain heading, and make a slight adjustment to the left by 10°" is 0.75, indicating that the formation's trajectory adjustment is highly reliable, and neighboring drones can adjust their own positions in advance based on this to avoid collisions;

[0126] Supporting collaborative strategy decision-making: "Multi-step action prediction" is transformed into quantitative indicators (joint probabilities), providing explicit input for the collaborative strategy generation in step S3; for example, in a rescue robot scenario, if the companion agent... The probability of the sequence "approaching vital signs and releasing the rescue ladder" is 0.9. The current intelligent agent can prioritize moving to this area to cooperate with the rescue and improve the efficiency of coordination.

[0127] behavioral intention The design can be dynamically adjusted according to the scene, and the core adaptive functions are:

[0128] Flexible expansion of motion space: motion space Action types can be added or removed according to scenario requirements; in autonomous driving scenarios The drone scenario can be expanded to It meets the action prediction requirements of different multi-agent tasks.

[0129] Dynamic adjustment of prediction step size: prediction step size It can be adjusted according to the complexity of the task; suitable for simple scenarios (such as indoor robot collaboration). (Predicting the next 3 steps), complex scenarios (such as long-range drone formations) (Predicting the next 8 steps), balancing prediction accuracy and real-time performance (too long a step size will increase inference delay, while too short a step size will result in insufficient prediction).

[0130] More specifically, the collaborative intent in step S2 For one Attention weight vector, For intelligent agents The number of neighboring agents, used to represent the number of agents. To achieve the goal The intensity of cooperation with neighboring agents required in future actions is calculated through an intentional attention mechanism:

[0131] ;

[0132] ;

[0133] in, For intelligent agents For intelligent agents At time step The collaboration intent weight, with a value range of [value range missing]. The larger the value, the more intelligent the agent. Need to work with intelligent agents The higher the intensity of cooperation; For intelligent agents The set of neighboring intelligent agents is determined by the communication radius or task relevance (e.g., in a drone swarm). Distance (All drones with a range of less than 500 meters) This is the attention scoring function used to calculate the agent's performance. and The degree of collaboration is expressed as a scalar. The intelligent agent output by the decoder The collaborative intent hidden state vector, with dimension . It integrates collaborative needs information that combines target intent and behavioral intent; For intelligent agents The collaborative intent shared through the communication link hides the state vector, with a dimension of [missing information]. ; The trainable weight matrix for collaborative intent attention has dimensions of . It is used to learn the collaborative relationship features between different intelligent agents; The transpose of the vector, with dimension , is used to perform inner product operations between vectors.

[0134] Collaborative Intent It is a key sub-intention for understanding the hierarchical intent of the large model, providing intelligence for the agent. against Neighboring intelligent agents Dimensional attention weight vector; its core value is to integrate the agent's attention weight vector. To achieve the goal The “fuzzy collaboration requirements” are transformed into “quantified weights” to clarify the collaboration objects and intensity, thus solving the pain points of “unclear collaboration objects and no basis for intensity” in multi-agent collaboration (such as clear collaboration between three close-range drones to maintain formation, and weak collaboration with long-range drones).

[0135] Through the formula of intentional attention mechanism accomplish:

[0136] Output interpretable weights: The value ranges from [0,1], and the value directly corresponds to the cooperation strength (e.g., 0.8 is high-intensity cooperation, which requires cooperation in moving the wounded; 0.1 is weak cooperation, which only shares location), providing clear cooperation priorities for neighboring agents;

[0137] Filtering core collaborators: Automatically highlighting highly correlated agents (such as drones) through normalized summation. Of the five neighborhoods, only of Da (The rest <0.1), to avoid indiscriminate collaboration and waste of resources.

[0138] Through the scoring function accomplish:

[0139] Integrating multi-dimensional needs: relying on the fusion of target intent (e.g., "rescue the wounded") and behavioral intent (e.g., "approach the target area") hidden state vectors. Capture supply and demand relationships (such as...) "Fire extinguishing" and "Delivering fire extinguishing agents" ensures coordinated matching;

[0140] Learning contextual features: trainable matrix (D×D dimension) adaptive learning of scene patterns (autonomous driving focuses on vehicle speed and lane association; drones focus on altitude and heading association), making the association calculation fit the actual needs.

[0141] The neighborhood set is updated dynamically: Determined by communication radius / task relevance (e.g., drones) of For drones with a range of less than 500 meters, when the intelligent agent enters or leaves the range or the task-related relationships change, and Real-time updates prevent ineffective collaboration.

[0142] Weights are adjusted in real time to reflect the target intent. Changes (such as switching from "searching area A" to "supporting area B") or adjustments to the needs of neighboring areas (such as...) When the battery is low and tasks need to be shared, the hidden state vector is updated synchronously. Adjust intensity in real time to adapt to dynamic scenarios.

[0143] More specifically, such as Figure 2 As shown, the collaborative policy network in step S3 adopts a centralized training-distributed execution architecture, and the specific implementation process is as follows:

[0144] The network input layer (used for data fusion) takes an intelligent agent as its input. Local observation sequence Hierarchical intent understanding results with all neighboring agents The two types of inputs are fused into a single dimension through a splicing layer. The input feature vector, For single-step observation dimension, For hierarchical intent dimensions;

[0145] The hidden layers (used for feature extraction) employ 2-4 layers of fully connected networks, with hidden layer dimensions ranging from 512 to 2048. The activation function used is the LeakyReLU function (with a negative slope of 0.01), which is used to extract non-linear collaborative associations of input features.

[0146] The network output layer (used for action generation) outputs an intelligent agent. At time step Coordinated control actions If it is a discrete action space, the output is the action probability distribution; if it is a continuous action space, the output is the action value (such as speed, angle).

[0147] Parameter update rules, parameters of the cooperative strategy network By maximizing the cumulative expected reward Updated, the formula is:

[0148] ;

[0149] in, Let be the objective function of the policy network, i.e., the cumulative expected reward; In strategy (Parameters are) The mathematical expectation under the strategy is used to measure the average level of reward; The maximum number of time steps for a task (e.g., in a collaborative rescue mission). (corresponding to 1 hour). This is a discount factor, with a value range of [0.9, 0.99], used to weigh the importance of current rewards against future rewards; The reward for environmental interaction is calculated based on the task objective (e.g., in autonomous driving). (The sum of the distance penalty to the target lane and the safe distance bonus to the vehicle in front). For the purpose Figure 1 Consistent rewards are used to measure the degree of alignment between actions and intentions; This is the reward balance coefficient, with a value range of [0.3, 0.7], used to adjust the weight of environmental rewards and intention rewards.

[0150] The collaborative strategy network is the carrier of "intent-guided collaboration". It adopts a centralized training-distributed execution structure. It optimizes the global collaborative goals (such as drone formation and overall vehicle safety) through centralized training, and reduces the computing pressure on edge devices (vehicle controllers, drone modules) by relying on distributed execution, so as to meet the real-time response requirements and solve the pain points of "insufficient global optimization" or "excessive edge latency" of traditional strategies.

[0151] Network input layer: Integrating the intelligent agent "Local observation sequence" (Self-position, obstacles, etc.) and "hierarchical intent of neighborhood agents" (Goals, behaviors, and collaborative intentions) are combined and integrated to produce output. 3D feature vectors. Breaking down the data silos between "observation" and "intent", the network can simultaneously grasp both "its own environmental state" and "neighborhood cooperation needs" (such as in autonomous driving, the following vehicle combines its own speed with the lane-changing intention of the vehicle in front), providing comprehensive input for subsequent strategy generation.

[0152] Network Hidden Layers: A 2-4 layer fully connected network (hidden layer dimensions 512-2048) and the LeakyReLU activation function (negative slope 0.01) are used to extract "non-linear cooperative associations" from the input features. This captures complex cooperative patterns among multiple agents (such as drones). The steering needs to match the neighborhood. Altitude changes, rescue robots The search path needs to be coordinated (The range of vital signs detection) avoids the collaborative logic that traditional linear models cannot depict, and improves strategy adaptability.

[0153] Network output layer: Outputs intelligent agents At time step Coordinated control actions It outputs the probability distribution of actions in discrete space (such as turning left or going straight) and the specific action values ​​in continuous space (such as speed or turning angle). It directly provides executable cooperative instructions to the agent (such as "ascend 5 meters, turn 10°" for drones and "decelerate 2 m / s" for vehicles), ensuring the strategy is implemented and compatible with action requirements in different scenarios, offering high flexibility.

[0154] Parameter update: By maximizing the cumulative expected reward Update network parameters .

[0155] Balancing multi-dimensional rewards: (Environmental incentives, such as safe distance and task progress) ensure that actions comply with environmental constraints. (meaning Figure 1Consistent rewards ensure that actions match the intentions of the neighborhood, and α (0.3-0.7) adjusts the weights of the two to avoid "only focusing on the environment and ignoring cooperation" (such as drones only focusing on obstacle avoidance and disrupting formation).

[0156] Balancing short-term and long-term benefits: γ (0.9-0.99) weighs current rewards (such as immediate obstacle avoidance) against future rewards (such as long-term formation stability), ensuring that the strategy is not limited to short-term interests and improving overall coordination efficiency.

[0157] The collaborative strategy network transforms hierarchical intents into executable collaborative actions through a complete process of "data fusion, feature extraction, action generation, and reward update," ensuring both global mission objectives (such as rescue efficiency and formation safety) and adapting to the real-time requirements of edge devices.

[0158] More specifically, the meaning of step S3 Figure 1 Consistent reward The following formula is used to simultaneously encourage action and one's own intention. Figure 1 Matching actions with collaboration needs:

[0159] ;

[0160] in, For the purpose Figure 1 Consistent reward, with a value range of [0,1]. The larger the value, the higher the degree of matching between the action and the intention. As the reward weighting coefficient, satisfying ,and ; for Divergence, used to measure the performance of an agent Predicted action distribution Distribution of actual actions The difference, the range of values ​​is The smaller the difference, the better. The larger; For intelligent agents The probability distribution of the actual action sequence is obtained from the statistics of the action execution log; For intelligent agents For intelligent agents The collaboration intent weight (from the collaboration intent result of step S2); The action similarity function is calculated using cosine similarity, and the formula is as follows: The range of values ​​is The larger the value, the higher the degree of cooperation and matching between the two actions; for Norms are used to calculate the magnitude of a vector. For current intelligent agents The execution action, For neighborhood intelligent agents Synchronous actions.

[0161] meaning Figure 1 Consistent reward It is a key guide for updating the parameters of the collaborative strategy network, with a value range of [0,1] (the larger the value, the higher the matching degree between action and intention). Its value lies in simultaneously encouraging "the agent's own actions and intentions..." Figure 1 The "matching" and "cooperation needs with neighboring intelligent agents" solve the problem of "collaboration disconnect" caused by traditional rewards that only focus on environmental feedback and ignore intention association (such as drones only focusing on obstacle avoidance and breaking away from formation).

[0162] Part One: Ensure your movements align with your intention:

[0163] Function: Measure the agent's performance using KL divergence. Predicting action distribution "and the distribution of actual actions" The smaller the difference, the better. The larger the value, the higher the reward.

[0164] Function: To constrain the agent's actions to conform to the predicted intent (e.g. in autonomous driving, if the predicted action sequence is "decelerate → turn right", but the actual action is "accelerate → go straight", the KL divergence increases and the reward decreases), to avoid the agent's actions from deviating from the intent, and to ensure the consistency of individual behavior;

[0165] Weight: Values ​​range from [0.4, 0.6], and... The sum is 1, balancing the reward weights of "matching one's own intentions" and "matching collaboration needs".

[0166] Part Two: Ensure actions match collaboration requirements:

[0167] Function: Combine neighborhood intelligent agents right Collaboration intention weight (From S2, the larger the value, the higher the collaboration requirement), and and action similarity (Cosine similarity, the larger the value, the higher the collaboration matching degree), calculate the collaboration matching reward;

[0168] Function: To drive intelligent agents Actions that align with the needs of neighborhood collaboration (such as drones) right The collaboration weight is 0.8. The movements need to be adjusted to maintain a similar formation trajectory to k; if the movements differ significantly, Reduce rewards and penalties to avoid "going it alone" from disrupting overall collaboration;

[0169] Weight: Values ​​range from [0.4, 0.6], ensuring that the proportion of collaboration needs in the rewards is reasonable and not dominated by one's own intentions.

[0170] meaning Figure 1 Consistent reward By employing a dual-dimensional design of "matching one's own intent with matching the needs of collaboration," the system provides precise guidance for collaborative strategy networks. This ensures that individual agent actions do not deviate from their own intents while also promoting their adaptation to the collaborative needs of their neighborhood.

[0171] More specifically, the online optimization in step S4 includes two parts: fine-tuning of the large language model and updating the collaborative strategy network, as follows:

[0172] The closed-loop experience pool is constructed by storing closed-loop data after the actions are executed. Stored in the experience pool, the experience pool capacity is The system employs a first-in-first-out (FIFO) strategy to eliminate expired data, while also introducing a priority experience replay mechanism to assign higher sampling weights to high-reward data.

[0173] For fine-tuning the large language model, after accumulating 100-1000 closed-loop data points, 32-128 data points are randomly sampled from the experience pool as fine-tuning samples; the fine-tuning loss function is the intention prediction cross-entropy loss. :

[0174] ;

[0175] in, For hierarchical intent Cross-entropy with the true intent (derived from manual annotation or task logs), Loss balance coefficient Fine-tuning was performed using the AdamW optimizer with a learning rate of [missing information]. Only the parameters of the top 2-4 layers of the large language model are updated to avoid destroying the underlying pre-trained features;

[0176] Cooperative strategy network update, using the PPO algorithm to update network parameters. Each iteration samples 256-1024 empirical data points, iterating 3-10 times; during updates, the objective function is clipped to ensure the stability of the policy update, the formula being:

[0177] ;

[0178] in, The strategy parameters before the update. This is the advantage function (used to measure the relative value of an action). Editing coefficient .

[0179] Online optimization is a key link in the "dynamic adaptation to scenarios" of multi-agent systems. By "storing data in a closed-loop experience pool and updating the dual models synchronously", it solves the pain points of "wasting high-value data and asynchronous model updates" in traditional optimization, ensuring that the intent understanding accuracy of the large language model and the action generation capability of the collaborative strategy network are continuously improved, and adapting to complex scenario changes (such as sudden obstacles and adjustment of rescue priorities).

[0180] Closed-loop experience pool function: Stores closed-loop data after an action is executed. ,capacity It adopts a "first-in, first-out" and "priority experience replay" approach; it both eliminates expired data and prioritizes high-reward data (such as high-interest data). Figure 1 Data on consistency and emergency rescue is given high sampling weight;

[0181] Purpose: To prevent high-value data (such as rare emergency braking intention data) from being prematurely eliminated, allowing subsequent model updates to focus on "key scenario samples" and improve optimization efficiency (such as in rescue scenarios, prioritizing the use of closed-loop data that reveals vital signs to update the data and strengthen rescue strategies).

[0182] Large language model fine-tuning function: For every 100-1000 closed-loop data points accumulated, sample 32-128 samples and use... ( In order to predict cross-entropy, The loss function is AdamW optimizer (learning rate 1×10⁻⁻⁶). 5 -5×10⁻ 5 Fine-tune the parameters of the top 2-4 layers of the model;

[0183] Function: Through Narrow the gap between "predicted intent" and "real intent" (e.g., correcting the problem of misjudging "temporary obstacle avoidance and lane change" as "regular overtaking"); update only the top-level parameters to avoid destroying the underlying general features, while allowing the model to adapt to scene details (e.g., learning the intent association of V2X protocol frames in autonomous driving).

[0184] Cooperative strategy network update function: Updates parameters θ using the PPO algorithm, sampling 256-1024 data points each time, iterating 3-10 times. ( For old parameters, For the dominant function, Ensure stable updates;

[0185] Purpose: To prevent excessively large policy updates in the editing objective function (e.g., avoiding a sudden shift from "slow following" to "high-speed overtaking"), ensuring action continuity; through Focus on "high relative value actions" (such as actions with high collaboration matching degree) to make strategy updates more in line with the overall collaboration needs.

[0186] More specifically, prior to step S1, there is a pre-training step for the large language model. The pre-training process focuses on intent reasoning ability and action chain reasoning ability, as detailed below:

[0187] Pre-training dataset construction: The dataset contains three classes of data, and the total size is not less than [amount missing]. Sample data includes: 1) Multi-agent collaborative dialogue text (e.g., "Agent A: Requesting support for area B | Agent B: Already en route, expected arrival in 5 minutes"); 2) Task instruction-action log pairing data (e.g., "Instruction: Drive around obstacles | Action log: Turn left 10° → Accelerate 5m / s → Turn right 5°"); 3) Intent annotation data (e.g., "Action sequence: Approach fire source → Release fire extinguishing agent | Annotation intent: Target intent = Extinguish fire, Collaborative intent = No collaboration required").

[0188] The pre-training task design employs multi-task joint pre-training, simultaneously optimizing three objective functions:

[0189] Masked Language Modeling (MLM): Randomly masking 15% of the tokens in the text (e.g., "Agent A:") Supporting region B), the model is trained to predict the mask token, and the loss function is cross-entropy;

[0190] Next Sentence Prediction (NSP): Input two sentences (e.g., "Agent A: Request support" and "Agent B: Refuse support"), train the model to determine whether the next sentence is a reasonable follow-up to the previous sentence, and use the loss function as binary cross-entropy.

[0191] Action Chain Prediction (ACP): Input the task command and the previous T-1 steps of action (e.g., "Command: Formation flight, Action 1: Agent A position (10,20), Action 2: Agent B position (15,20)"). Train the model to predict the action chain. For each step of the action, the loss function is either the action category cross-entropy (for discrete actions) or the MSE loss (for continuous actions).

[0192] Pre-training parameter settings: Fine-tuned based on open-source large language models (such as Llama2, ChatGLM), with 5-20 training epochs, batch size of 32-128, and learning rate. The optimizer used is AdamW, with a weight decay factor of . This ensures that the model initially possesses the ability to reason about intent and predict actions in multi-agent scenarios.

[0193] More specifically, the multi-agent interaction intent understanding and collaborative control method based on large models is adapted to three typical multi-agent collaborative tasks, and is scenario-specifically adjusted for different tasks:

[0194] Autonomous driving fleet cooperative control: In this scenario, the environmental state information in step S1 is supplemented with traffic light status, lane line position, and the speed of the vehicle in front; the neighborhood interaction information is supplemented with fleet communication protocol frames (such as V2X protocol); the target intent in step S2... The categories are lane keeping, lane changing, overtaking, and emergency braking, with behavioral intent. The action space is acceleration (-2~5m / s²), steering (-30~30°) and braking (0~100%).

[0195] Drone swarm formation flight: In this scenario, the environmental state information in step S1 is supplemented with altitude, heading angle, battery level and airspace obstacles, and the neighborhood interaction information is supplemented with millimeter-wave radar data between drones; the cooperative strategy network output layer in step S3 adds formation constraint terms (such as maintaining a triangular formation, the distance error between agents does not exceed 5 meters).

[0196] Distributed robot collaborative search and rescue: In this scenario, the environmental state information in step S1 is supplemented with vital signs signals (such as infrared detection results) and terrain slope, and the neighborhood interaction information is supplemented with the shared data of the rescue target location; the online optimization in step S4 increases the rescue priority weight (such as when vital signs are found, the intent understanding and control strategy for the area is updated first).

[0197] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A large model-based multi-agent interaction intention understanding and collaborative control method, characterized in that, The method comprises the following steps: S1, environment and multi-source heterogeneous interaction information perception: multiple agents collect two types of information in real time through their own sensors and distributed communication links between agents, and form local observation sequences of each agent ; One is the environmental state information, which at least contains the agent's own position, speed, task progress and environmental obstacle distribution; The second is neighborhood interaction information, at least including communication instructions of neighborhood agents, feedback of executed actions, and task cooperation requests; wherein, , represents an agent at a time step of single-step observation data, is an observation history window size, and , dynamically adjusted according to task complexity; S2, hierarchical intention understanding driven by a large model: input the local observation sequence obtained in step S1 into a pre-trained and task-fine-tuned large language model as a core reasoning engine, and output a hierarchical intention understanding result of other agents in the neighborhood ; the hierarchical intention understanding result contains at least three types of sub-intentions;​​ Target intent for characterizing an agent's final goal ; Behavioral intent for predicting future actions ; Collaboration intent for quantifying collaboration needs i.e. ; S3, collaborative policy generation guided by intention: based on the hierarchical intention understanding result of step S2 Each agent calls a collaborative policy network of a centralized training-distributed execution architecture to generate collaborative control actions that meet the global task goal The parameter update of the collaborative policy network is guided by a composite reward function that contains both environment interaction rewards and intention consistency rewards S4, action execution and closed-loop online optimization: each agent executes the cooperative control action generated in step S3 and the new environment state after action execution, the neighborhood agent feedback intention and the reward value form a closed-loop data, which is fed back to the large language model and the cooperative strategy network; The large language model refines the intention reasoning parameters based on the closed-loop data, and the collaborative strategy network updates the control strategy based on the closed-loop data, realizing the end-to-end closed loop of intention understanding, action generation and feedback optimization.

2. The method of claim 1, wherein, The large language model in step S2 adopts an encoder-decoder architecture, and the specific implementation process is as follows: Encoder encoding process: the encoder adopts a 6-12 layer Transformer encoder structure to encode the input local observation sequence Context encoding; first, the discrete observation is converted into an embedding vector of 128-512 dimensions through an embedding layer, the continuous observation is mapped to a feature vector of the same dimension through a normalization layer, and the time dimension information is supplemented through a position encoding layer, and finally the context-aware hidden state sequence is output: ； wherein, is an agent at time step is a hidden state vector of dimension 256-1024, the number of trainable parameters increases linearly with the number of Transformer layers, is a sequence of hidden states is a basic building block of the sequence, being a single time step hidden state in the sequence, is a time step, taking values in the range covering all time instances within the history window; Decoder decoding process: the decoder adopts a 6-12 layer Transformer decoder structure, based on the hidden state sequence Generating hierarchical intent understanding results ; distinguish the generation tasks of target intent, action intent and collaborative intent through intent type mask in the decoding process; The number of attention heads of the Transformer encoder / decoder is 4-8, the hidden layer dimension of the Feed-Forward network is 1024-2048, and the activation function adopts the GELU function.

3. The method of claim 2, wherein, Target intent in step S2 For one A probability vector representing the final task goal estimate of the agent is decoded from the decoder hidden state by the following equation: ； in, For intelligent agents For intelligent agents At time step The target intent probability vector, with dimension . The first vector The value of each element represents the agent. Select the The probability of a target class is such that the sum of all elements is 1. The trainable weight matrix for decoding the target intent, with dimension . This is used to map hidden states to the target category dimension; Let be the hidden state vector output by the decoder at the target intent marker location, with dimension . These are the key contextual features extracted by the decoder for the target intent task; The trainable bias vector for decoding the target intent, with dimension . This is used to compensate for the offset of the linear transformation; for An activation function is used to map the result of a linear transformation to a probability distribution.

4. The method of claim 3, wherein, The behavior intention in step S2 A probability distribution for a sequence of actions for predicting an agent In the future The potential sequence of actions within a step, the specific calculation process is as follows: First, predict the agent. In the future step( The probability distribution of actions: ； The future is calculated again by the chain rule The joint probability distribution of the sequence of actions ； wherein, is the agent in the future is the behavior intention of the agent in the future step, i.e., the action sequence ; is the agent in the future is the action of the agent in the future step, taking values in the action space ; is the action conditional probability distribution based on the hidden state of the previous step, with dimension , is the size of the action space; is the hidden state vector of the decoder when predicting the th action, with dimension , which inherits the context information of the previous action; is the trainable weight matrix for the behavior intention decoding, with dimension ; is the trainable bias vector for the behavior intention decoding, with dimension ; is the joint probability of the action sequence in the future steps, which is used to measure the credibility of the action sequence, and the larger the value, the more likely the sequence is to be executed by the agent .

5. The method of claim 4, wherein, Collaboration intention in step S2 For one Attention weight vector, For intelligent agents The number of neighboring agents, used to represent the number of agents. To achieve the goal The intensity of cooperation with neighboring agents required in future actions is calculated through an intentional attention mechanism: ； ； in, For intelligent agents For intelligent agents At time step The collaboration intent weight, with a value range of [value range missing]. The larger the value, the more intelligent the agent. Need to work with intelligent agents The higher the intensity of cooperation; For intelligent agents The set of neighborhood intelligent agents is determined by the communication radius or task relevance. This is the attention scoring function used to calculate the agent's performance. and The degree of collaboration is expressed as a scalar. The intelligent agent output by the decoder The collaborative intent hidden state vector, with dimension . It integrates collaborative needs information that combines target intent and behavioral intent; For intelligent agents The collaborative intent shared through the communication link hides the state vector, with a dimension of [missing information]. ; The trainable weight matrix for collaborative intent attention has dimensions of . It is used to learn the collaborative relationship features between different intelligent agents; The transpose of the vector, with dimension , is used to perform inner product operations between vectors.

6. The method of claim 5, wherein, The collaborative strategy network in step S3 is a centralized training-distributed execution architecture, and the specific implementation process is as follows: network input layer, input is the agent local observation sequence hierarchical intention understanding result with all neighboring agents , the two types of inputs are fused into an input feature vector with a dimension of by a concatenation layer, is the single-step observation dimension, is the hierarchical intention dimension; The network hidden layer adopts 2-4 layers of fully connected network, the hidden layer dimension is 512-2048, and the activation function adopts the LeakyReLU function, which is used to extract the nonlinear cooperative correlation of input features; network output layer, outputting for the agent at time step a coordinated control action , outputting an action probability distribution if the action space is discrete or an action value if the action space is continuous; parameters of the policy network by maximizing the cumulative expected reward update, the formula is: ； wherein, is the objective function of the policy network, i.e., the cumulative expected reward; is the mathematical expectation under the policy , which is used to measure the average level of rewards; is the maximum time step of the task; is the discount factor, which is in the range of [0.9, 0.99], used to weigh the importance of current rewards and future rewards; is the environment interaction reward, calculated by the task goal; is the intention consistency reward, used to measure the matching degree of actions and intentions; is the reward balance coefficient, in the range of [0.3, 0.7], used to adjust the weight of environment rewards and intention rewards.

7. The method of claim 6, wherein, The intention consistency reward in step S3 This is calculated by the following formula to encourage both actions that are consistent with the agent's own intentions and actions that match the cooperative requirements: ； in, This is a reward for consistency between intent and action, with a value range of [0,1]. A larger value indicates a higher degree of matching between the action and the intent. As the reward weighting coefficient, satisfying ,and ; for Divergence, used to measure the performance of an agent Predicted action distribution Distribution of actual actions The difference, the range of values ​​is The smaller the difference, the better. The larger; For intelligent agents The probability distribution of the actual action sequence is obtained from the statistics of the action execution log; For intelligent agents For intelligent agents The weight of the intention to collaborate; The action similarity function is calculated using cosine similarity, and the formula is as follows: The range of values ​​is The larger the value, the higher the degree of cooperation and matching between the two actions; for Norms are used to calculate the magnitude of a vector. For current intelligent agents The execution action, For neighborhood intelligent agents Synchronous actions.

8. The method of claim 7, wherein, The online optimization in step S4 specifically includes two parts of large language model fine-tuning and collaborative strategy network updating, and the process is as follows: The closed-loop experience pool is constructed, and the closed-loop data after the action is performed is stored in the experience pool, and the capacity of the experience pool is , the advanced and outgoing strategy is adopted to eliminate the expired data, and the priority experience playback mechanism is introduced to give higher sampling weight to the high reward data. Large language model fine-tuning, after accumulating 100-1000 closed-loop data, randomly sample 32-128 data from the experience pool as fine-tuning samples; the fine-tuning loss function is the intent prediction cross-entropy loss : ； wherein, is a hierarchical intention cross-entropy with the real intention, is a loss balance coefficient ; AdamW optimizer is used for fine-tuning, and the learning rate is , only updating the top 2-4 layer parameters of the large language model to avoid destroying the bottom pre-training features; The network parameters are updated by using a PPO algorithm 256-1024 pieces of experience data are sampled each time, and 3-10 rounds of iterations are performed; the stability of the policy update is ensured by clipping the objective function during the update, and the formula is: ； wherein, a cooperative control action for a global task objective, a local observation sequence for each agent, other agents a hierarchical intention understanding result, policy parameters before update, a advantage function, a clipping coefficient, .

9. The method of claim 8, wherein, Before step S1, there is also a pre-training step for the large language model, and the pre-training process takes the intention reasoning ability and action chain reasoning ability as the core target, and the specific process is as follows: Pre-training dataset construction: the dataset contains three types of data, and the total size is not less than Bar samples: one is multi-agent collaborative dialogue text; two is task instruction-action log pairing data; three is intent annotation data; The pre-training task design adopts multi-task joint pre-training, and simultaneously optimizes three objective functions: Masked language modeling (MLM): randomly mask 15% of the tokens in the text, train the model to predict the masked tokens, and the loss function is cross entropy; Next sentence prediction (NSP): input two sentences, train the model to determine whether the second sentence is a reasonable continuation of the first sentence, and the loss function is binary cross entropy; Action chain prediction ACP: input task instruction and previous T-1 step action, train model to predict the T-th step action action, and the loss function is action category cross-entropy or MSE loss. Pre-training parameter settings: based on open source large language model for fine tuning, training rounds are 5-20 rounds, batch size is 32-128, learning rate is , the optimizer uses AdamW, the weight decay coefficient is , to ensure that the model has the ability of intention reasoning and action prediction in multi-agent scenarios.

10. The method of claim 9, wherein, The multi-agent interaction intention understanding and collaborative control method based on large model is suitable for three typical multi-agent collaborative tasks, and is adjusted according to different tasks: Autonomous vehicle platoon cooperative control: In this scenario, the environment state information in step S1 is supplemented by traffic signal light status, lane line position, and front vehicle speed, and the neighborhood interaction information is supplemented by vehicle platoon communication protocol frames; the target intention in step S2 is to keep the lane, change the lane, overtake, and emergency brake The action space of the behavior intention is acceleration, steering, and braking. Unmanned aerial vehicle cluster formation flight: in this scene, the environmental state information in step S1 is supplemented with height, heading angle, battery level and airspace obstacle, and the neighborhood interaction information is supplemented with millimeter wave radar data between unmanned aerial vehicles; the output layer of the collaborative strategy network in step S3 adds formation shape constraint term; Distributed robot cooperative search and rescue: in this scene, the environmental state information in step S1 is supplemented with vital sign signal and terrain slope, and the neighborhood interaction information is supplemented with rescue target position sharing data; the online optimization in step S4 increases the rescue priority weight.

Images