Method and system for optimizing communication between unmanned aerial vehicle clusters

Abstract

The invention discloses an optimization method and system for communication between unmanned aerial vehicle clusters, and belongs to the technical field of unmanned cluster communication. In order to solve the problems of cluster communication bandwidth limitation and information cooperation efficiency, a graph modeling method and an adaptive attention mechanism are mainly adopted, and a learnable cluster communication graph is constructed and a communication structure is dynamically optimized. According to the method, through end-to-end joint optimization of the communication graph and the architecture parameters, the data transmission path between the nodes is optimized, the communication bandwidth occupation is reduced, and the key information transmission efficiency is improved. And meanwhile, the communication graph and the model parameters are synchronously optimized by adopting a double-layer optimization method, so that the collaborative decision-making capability of the cluster in a limited communication environment is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of UAV swarm communication, and particularly relates to a method and system for optimizing communication between UAV swarms. Background Art

[0002] In many artificial intelligence applications, the collaborative cooperation of UAV swarms is crucial for the successful achievement of goals. To enhance the coordination among nodes in a UAV swarm, a distributed communication framework is usually adopted. However, sharing information among all UAV swarm nodes consumes a large amount of resources, while using a manually predefined communication architecture restricts the communication capabilities between nodes, thus suppressing the collaboration potential. For this reason, a method for optimizing communication between UAV swarms is proposed. The problem is formulated as a task of determining a communication graph. By continuously relaxing the graph representation and combining an attention mechanism, the swarm communication graph is optimized, enabling UAVs to develop more coordinated and complex strategies.

[0003] When the number of swarm nodes is large, it becomes difficult to distinguish valuable information for collaborative decision-making from the globally shared data, and communication may interfere with collaborative learning. In recent years, methods for dynamically adjusting the communication graph during the inference process have attracted wide attention. However, this method assumes that all agents always need to communicate with some other agents, which requires complex scheduling algorithms and thus leads to waste of bandwidth. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem of swarm communication learning, and to propose a method and system for optimizing communication between UAV swarms, optimize the communication efficiency between swarms, reduce data transmission delay and bandwidth occupancy, and realize efficient data transmission path planning and improve the swarm communication performance by constructing a communication graph, introducing an adaptive attention mechanism to calculate communication weights, and adopting an end-to-end joint optimization method to dynamically adjust the communication architecture.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] A method for optimizing communication between UAV swarms includes the following steps:

[0007] 1) Obtain the observation information of the UAV swarm and construct a swarm communication graph, which includes nodes, edges, a node feature matrix, an adjacency matrix, a sparsity hyperparameter of the adjacency matrix, and a degree matrix. Among them, the nodes represent UAVs, the edges represent the communication relationships between UAVs, the node feature matrix includes the observation information of the UAVs, the adjacency matrix represents the communication weights between UAVs, the sparsity hyperparameter represents the number of other UAVs communicating with the current UAV, and the degree matrix represents the communication connection situation of the UAVs;

[0008] 2) Construct an optimized network for cluster communication, including an encoder and a decoder; input the node feature matrix and the adjacency matrix into the encoder for encoding to generate an encoded observation vector; input the encoded observation vector together with the historical action information of the node into the decoder to predict the next action distribution of the UAV.

[0009] 3) Train the optimized network for cluster communication to optimize the adjacency matrix and the parameters of the encoder and decoder.

[0010] 4) Update the cluster communication graph based on the optimized adjacency matrix, and use the optimized network for cluster communication to process the updated cluster communication graph, where the current observation information and historical action data are input to predict the next action distribution of the UAV.

[0011] Further, the steps of encoding by the encoder in step 2) include:

[0012] Normalize the observation information of the UAV in the node feature vector so that the tensor mean of the observation information is 0 and the standard deviation is 1.

[0013] Input the normalized observation information into the self-attention network of the encoder to perform splitting of the query vector, key vector, and value vector.

[0014] Calculate the similarity between the current node and neighbor nodes according to the query vector and key vector.

[0015] Calculate the implicit attention score according to the similarity between the current node and neighbor nodes.

[0016] On the basis of the implicit attention score, add explicit edge information and the temporal causal relationship between nodes to obtain the attention score of the adjacency matrix.

[0017] Perform masking processing on the attention score of the adjacency matrix.

[0018] Perform residual processing and normalization processing on the masked attention score and the normalized observation information to obtain the updated attention score.

[0019] Input the updated attention score into a two-layer graph convolutional neural network, and combine the neighbor matrix and degree matrix to capture the topological structure characteristics of the cluster communication graph and the relationship between nodes to obtain the encoded observation information.

[0020] Further, the steps of prediction by the decoder in step 2) include:

[0021] Perform non-linear transformation and normalization processing on the actions of the nodes through a multi-layer perceptron.

[0022] The standardized actions and inundation are input into the relational attention network of the decoder for calculation. The self-attention network and cross-attention network are used to calculate the action distribution, and then the final action distribution is generated through two layers of multi-layer perceptrons.

[0023] Furthermore, in step 3), training the cluster communication optimization network includes training the encoder. The steps of training the encoder include:

[0024] Introduce an additional projection layer to enable the encoder to estimate the value function of the observed information;

[0025] Extract the observations and corresponding actions at multiple time steps from the trajectory data of the UAV cluster, and calculate the immediate reward after performing the actions at each time step;

[0026] Use the target network to estimate the value of the next time step, use the main network to estimate the value of the current time step, and calculate the difference between the two, which is used as part of the loss function;

[0027] Optimize the parameters of the encoder by minimizing this error.

[0028] Furthermore, when training the encoder, a regularization term is added to prevent overfitting by constraining the causal matrix learned by the encoder through the Frobenius norm.

[0029] Furthermore, in step 3), training the cluster communication optimization network includes training the decoder. The steps of training the decoder include:

[0030] Calculate the ratio factor based on the current policy and historical policy to measure the change between the old and new policies;

[0031] Calculate the advantage function to evaluate the superiority of the current action;

[0032] Calculate the loss function. By comparing the unclipped advantage term with the clipped advantage term, select the smaller value to limit the policy update amplitude;

[0033] Optimize the parameters of the decoder by minimizing the proximal policy optimization loss function.

[0034] Furthermore, the steps of optimizing the adjacency matrix and the parameters of the encoder and decoder in step 3) include:

[0035] Relax the adjacency matrix to a continuous matrix to enable differentiable updates, and use an alternating update method to optimize the encoder parameters, decoder parameters, and adjacency matrix respectively;

[0036] Sample the adjacency matrix using the Gumbel-Max Trick method, and approximate the discrete operation through continuous parameterization so that it can participate in the gradient backpropagation;

[0037] Expand the Gumbel - Max Trick method to extend the one - hot representation to a multi - hot representation to meet the constraint that each node communicates with only a finite number of other nodes;

[0038] Optimize and update the adjacency matrix based on the objective of minimizing the loss function on the validation set.

[0039] Furthermore, the steps of updating the cluster communication graph based on the optimized adjacency matrix in step 4) include:

[0040] Each node accesses its local observations and actions and receives additional information passed by other nodes through communication;

[0041] Calculate the connection relationship between each node and other nodes through the optimized adjacency matrix to update the cluster communication graph.

[0042] Furthermore, the steps of predicting the next - step action distribution of the UAV in step 4) include:

[0043] Generate actions in an autoregressive manner, that is, use the current action as the input of the decoder to predict the next - step action;

[0044] Starting from the initial action in sequence, gradually generate the actions at each time step until the final action sequence is generated.

[0045] A communication optimization system for UAV clusters includes:

[0046] A cluster communication graph for obtaining the observation information of the UAV cluster and constructing the cluster communication graph;

[0047] A cluster communication optimization network module, including an encoder and a decoder, for inputting the node feature matrix and the adjacency matrix into the encoder for encoding to generate an encoded observation vector; then inputting the encoded observation vector together with the historical action information of the node into the decoder to predict the next - step action distribution of the UAV;

[0048] A training module for training the cluster communication optimization network, optimizing the parameters of the adjacency matrix, the encoder, and the decoder, and updating the cluster communication graph based on the optimized adjacency matrix.

[0049] The beneficial effects obtained by the present invention are as follows:

[0050] 1. The present invention proposes a learnable communication graph construction method, models the communication relationship between nodes as an optimizable adjacency matrix, dynamically adjusts the communication graph structure through a two - layer optimization strategy, and under limited communication bandwidth, realizes efficient communication relationship management by sparsifying the adjacency matrix to meet the requirements of different task scenarios.

[0051] 2. The present invention uses the multi-hot Gumbel-Max Trick method to continuously represent the discrete communication graph adjacency matrix, enabling it to participate in backpropagation of gradients. By combining sparsity constraints, end-to-end optimization is achieved, solving the problem of optimizing discrete variables, and at the same time achieving global performance improvement during the process of optimizing the communication graph and node strategies.

[0052] 3. The present invention introduces a causal matrix in communication optimization to clarify the causal relationship constraints between nodes, restrict the calculation range of communication attention, and only allow nodes to interact with causally related nodes. By reducing unnecessary communication, it reduces the interference and bandwidth occupation of redundant message passing on the network, and at the same time ensures the directionality and rationality of message passing, promoting more stable and efficient collaborative strategy learning.

[0053] 4. Through the end-to-end gradient descent optimization framework, the optimization of the communication graph in the present invention does not depend on discrete candidate set search, improving the optimization efficiency compared with traditional methods, avoiding the computational overhead brought by discrete search, and enhancing the adaptability of communication structure learning.

[0054] 5. The present invention adopts a learning-based communication architecture. The local observations and actions of each agent can be directly transmitted to specific agents. By combining attention units, the weights of received messages are dynamically allocated, achieving efficient information screening and transmission from the perspective of graph modeling, and enjoying the guarantee of monotonic performance improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flowchart of the steps of a method for optimizing communication between UAV swarms according to the present invention.

[0056] Figure 2 It is a network architecture diagram of the cluster communication optimization according to the present invention.

[0057] Figure 3 It is a schematic diagram of the update of the cluster communication optimization network according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] To make the technical features and advantages or technical effects in the above technical solutions of the present invention more obvious and understandable, the following is a detailed description through embodiments.

[0059] This embodiment specifically provides an optimization method for communication between UAV clusters. This method regards the communication architecture between UAV cluster nodes as a learnable graph, and transforms the cluster communication learning problem into the task of determining the communication graph. By introducing graph representation to structurally characterize the cluster, and combining the attention mechanism in the graph modeling framework, it adaptively learns and dynamically adjusts the data transmission path between clusters, reduces the occupancy of communication bandwidth, improves the transmission efficiency of key information, and finally realizes the end-to-end joint optimization of the communication graph and architecture parameters. Among them, the communication graph structure is learned through a two-layer optimization method, effectively realizing the simultaneous optimization of the communication graph and model parameters.

[0060] The main process of this method is as Figure 1 shown, including the following three parts:

[0061] I. Construct a cluster communication graph

[0062] For the UAV cluster, construct a cluster communication graph G=(V, E, X, α, S, D) to describe the communication relationship between UAVs.

[0063] Among them, the node set V={v1, v2,..., v n} represents the UAVs in the UAV cluster, and the node v i represents the i-th UAV, and n represents the total number of UAVs. The edge set E={e 1→2 , e 2→3 ,..., e n-1→n} is used to describe the communication relationship between UAVs, where e i→j represents the edge between nodes v i and v j , that is, there is a communication connection between UAVs. The node feature matrix X represents the set of observation information of nodes, X={o1, o2,..., o n}, where o i represents the observation information of node v i , and M represents the number of features. Introduce a learnable adjacency matrix to represent the communication weight between nodes. Due to the communication bandwidth limitation, each node in the cluster can only communicate with some other nodes, and this limitation is represented by the sparsity hyperparameter S of the adjacency matrix. Define the degree matrix to describe the connection situation of each node in the communication graph.

[0064] II. Construct a cluster communication optimization network

[0065] To optimize the communication and decision-making processes of an unmanned aerial vehicle (UAV) swarm, a communication optimization network for UAV swarms is constructed. This network takes the observation information of UAV swarm nodes as input and generates action outputs for UAV swarm nodes. By simultaneously optimizing the cooperation strategies of nodes and the swarm communication graph, the network enables dynamic adjustment of the communication structure, thereby achieving efficient information cooperation and decision-making in a constrained communication environment.

[0066] As Figure 2 shown, the swarm communication optimization network consists of an encoder and a decoder. The encoder is responsible for mapping the observation information of UAV swarm nodes into hidden layer representations to extract rich semantic features and provide support for subsequent processing. The decoder then generates the action sequences of nodes based on the output of the encoder and the information in the swarm communication graph.

[0067] 1. Encoder part:

[0068] Input the node feature matrix X, the adjacency matrix α, and the degree matrix D into the encoder. Through the encoder, the original observation information of the nodes is transformed into a higher-dimensional abstract representation to capture the individual characteristics and interaction relationships of the nodes.

[0069] Standardize the observation information in the node feature matrix X so that the tensor mean of the original observation information is 0 and the standard deviation is 1, in order to improve the stability of training and accelerate convergence.

[0070] For the observation information o i of the current UAV node v i , the standardization process can be expressed by the following formula:

[0071]

[0072] where represents the standardized observation information, and LayerNorm(·) represents the standardization layer.

[0073] Input the standardized observation information into the self-attention network to perform the splitting of the query vector, key vector, and value vector, which can be expressed by the following formula:

[0074]

[0075] where q i , k i , v i represent the query vector, key vector, and value vector respectively, and W q , W k , W v are learnable weight matrices.

[0076] Calculate the current node v i and neighboring node v jThe similarity can be expressed by the following formula:

[0077] s i,j = τ(q i , k j )

[0078] where τ(·) is a function for calculating the similarity score, and k j is the key vector of the neighboring drone node j; the similarity s i,j is the hidden layer information of the edge e i→j , which is used to measure the degree of attention of node v i to the information sent to node v j . Based on this, in the calculation of the implicit attention score, the similarity s i,j is used as the core input.

[0079] To identify the edges that contribute the most to the final performance, based on the implicit attention score, explicit edge information and temporal causality are added for enhancement, which can be expressed by the following formula:

[0080]

[0081] where ξ represents the calculation of the implicit attention score; j represents a node in the drone cluster; n represents the number of nodes in the drone cluster; r *→* represents the explicit edge information obtained by processing the adjacency matrix α through the embedding layer, which consists of communication intensity, node distance, and interaction frequency; g *→* represents the temporal causality between nodes, which is represented by the causality matrix Γ; the causality matrix Γ consists of 0 and 1, indicating whether there is temporal causality between two nodes in the drone cluster.

[0082] The attention score s i of the adjacency matrix α is masked by M using the explicit edge information and the causality matrix Γ to ensure that only node pairs represented as connected in α can transmit information to each other. Specifically, if there is no edge connection between two nodes, the corresponding attention score will be masked (i.e., set to negative infinity, -∞), and the weights of these scores will become 0 in subsequent calculations and will not affect the result, which can be expressed by the following formula:

[0083]

[0084] The tensor s i obtained from the above processing is subjected to residual processing to retain the original information, prevent excessive changes, and perform normalization processing, which can be expressed by the following formula:

[0085]

[0086] A two - layer graph convolutional neural network (GCN) is used to capture the topological structure characteristics of the cluster communication graph and the relationships between nodes. The input of the first layer is s′ i , and the output of the second layer is The update process can be expressed by the following formula:

[0087]

[0088] where σ is a non - linear activation function, W (1) and W (2) are the learnable weight matrices of the first - layer and second - layer graph convolutional neural networks respectively. Dropout means randomly discarding the outputs of some neurons to avoid over - dependence on certain neurons, thereby improving the generalization ability of the model.

[0089] The encoded observation information not only captures the information of the UAV cluster nodes but also reflects the deep dependence relationships brought about by node - to - node communication.

[0090] 2. Decoder part

[0091] The decoder takes the historical action sequence a 0:m-1 , m = 1, …, n and the output of the encoder as inputs. The decoder adopts a relationship enhancement mechanism, calculates the attention between the encoded actions and the observation representation, and constrains the attention calculation to be only between the current node v i and its previous node v j (j < i).

[0092] The action tensor a i in the node historical action sequence is input into the multi - layer perceptron of the decoder to enhance the representation of the tensor and perform normalization, which can be expressed by the following formula:

[0093]

[0094] where GELU represents a non - linear activation function; fc0 represents the multi - layer perceptron.

[0095] The normalized action and the mask M are input into the relational attention network for calculation, where the relational attention network consists of a self - attention network layer and a cross - attention network layer. The calculation of the relational self - attention network can be expressed by the following formula:

[0096]

[0097] where, W‘ k ,W‘ vThey respectively represent the weight matrices of the query vector, key vector, and value vector in the decoder self-attention network.

[0098] The output h of the self-attention network i is input into the cross-attention network, which can be expressed by the following formula:

[0099]

[0100] where W″ k , W″ v respectively represent the weight matrices of the query vector, key vector, and value vector in the cross-attention network, and fc1 represents a multi-layer perceptron.

[0101] Finally, the action distribution is obtained through two layers of multi-layer perceptrons which can be expressed by the following formula:

[0102]

[0103] where fc2 represents the first layer of multi-layer perceptron; fc3 represents the second layer of multi-layer perceptron.

[0104] III. Training and Inference of the Cluster Communication Optimization Network

[0105] 1. Training Phase

[0106] In the training phase, the encoder and decoder of the cluster communication optimization network are trained, and the adjacency matrix is optimized in an end-to-end manner to enable the dynamic update of the cluster communication graph to adapt to different task requirements, as Figure 3 shown.

[0107] When training the encoder, the encoder is regarded as a critic and an additional projection layer is introduced to estimate the value function. The optimization objective is as follows:

[0108]

[0109] where φ represents the encoder parameters; T represents the total number of time steps of a trajectory; r(o t , a t ) represents the immediate reward obtained by the node according to the current environment after executing the action at time step t, o t represents the observation at time step t, a t represents the action at time step t; γ ∈ [0, 1] represents the discount factor; represents the value function estimated by the target network, which is used to estimate the value of the next time step, is the parameter of the target network, which is the exponential moving average of the main network parameter φ here, representing the encoded observation of node v m at time step t + 1; Represents the value function estimated by the main network, which is used for value estimation at the current time step. Represents node v m The encoded observation of node v at time step t; λ represents the regularization coefficient, which is used to control the impact of the causal matrix constraint on the total loss; ‖·‖ F Represents the Frobenius norm, which is used to calculate the difference between matrices.

[0110] When training the decoder, the decoder uses the Proximal Policy Optimization (PPO) algorithm to optimize the action output of nodes in the UAV swarm. The optimization objective is:

[0111]

[0112] where θ represents the decoder parameters; Represents the unclipped term of the advantage function, Represents the clipped term of the advantage function. The loss of the proximal optimization algorithm takes the minimum of the above two to balance policy improvement and stability; Represents the new policy, Represents the old policy, which is used to measure the relative change between the new and old policies; is the joint advantage function, which is used to evaluate the advantage of an action in the current state.

[0113] Use a two-layer optimization method to simultaneously optimize the adjacency matrix α and the encoder parameters v and θ. Among them, α is used as the upper-layer variable, while φ and θ are used as the lower-layer variables. The optimization objective can be expressed as follows:

[0114]

[0115] |α| ≤ S × N 2

[0116] where the loss function L = L Encoder (φ) + L Decoder (θ). Different online return data are used in the training stage L train and the validation stage L val |α| represents the number of connected edges; S represents the hyperparameter that controls the sparsity of the adjacency matrix, which determines the communication limit range of each node; N represents the dimension of the adjacency matrix.

[0117] Since the computational cost of simultaneously optimizing the adjacency matrix and parameters is high, the cost of directly calculating the architecture gradient is too high, and each value of the adjacency matrix α is a discrete binary value {0, 1}. Therefore, an alternating update method is used to relax α into a continuous matrix, so as to achieve differentiable update. The optimization process is as follows:

[0118]

[0119]

[0120] Among them, ω represents the learning rate of internal optimization; η represents the learning rate of upper-layer variables. This method updates φ and θ through single-step training, approximately solving φ(α) and θ(α), and avoiding the high computational cost brought by fully solving the internal optimization problem.

[0121] To optimize the binarization of the adjacency matrix α, the existing Gumble-Max Trick method is used to sample it, and the discrete operation is approximated by continuous parameterization. This method utilizes the characteristics of the Gumble distribution to generate sampling values from the discrete distribution, enabling the sampling of the adjacency matrix to participate in the gradient backpropagation and realizing differentiable optimization during training.

[0122] To satisfy the constraint of the adjacency matrix α (that is, each node can only communicate with a limited number of other nodes), the Gumble-Max Trick method is extended, and the one-hot representation (selecting a single-node connection) is extended to a multi-hot representation (selecting multiple nodes k = S×N connections), thereby supporting many-to-many communication between nodes, which can be expressed by the following formula:

[0123] e i = k hot (k - argmax[Softmax(α ij + t j , for j = 1,..., n)])

[0124] Among them, α ij represents the element in the adjacency matrix, reflecting the importance of communication between nodes; t j is the noise sampled from the Gumble(0, 1) distribution; represents the edge connected to the node; Softmax represents the non-linear activation function; k - argmax represents selecting the top k maximum values; k hot represents satisfying the constraint of the connection quantity.

[0125] 2. Inference stage

[0126] In the inference stage, each node can access its local observations and actions, and receive additional information transmitted by other nodes through communication, which can be expressed by the following formula:

[0127] e i = k hot (k - argmax[α ij , for j = 1,..., n])

[0128] In addition, the inference process generates actions in an autoregressive manner, that is, the current action α mAs the decoder input to generate the next action α m+1 Starting from the initial action α0, generate iteratively in sequence until the final action α n-1 .

[0129] By restricting communication usage, nodes in the UAV swarm can still effectively select actions under restricted communication conditions. Compared with a fully connected communication graph, this method significantly reduces the communication cost and computational overhead of the swarm while ensuring efficient information collaboration and decision-making capabilities.

[0130] Although the present invention has been disclosed above by way of embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent replacements made by those of ordinary skill in the art to the technical solutions of the present invention shall all be covered within the protection scope of the present invention. The protection scope of the present invention shall be subject to what is defined by the claims.

Claims

1. An optimization method for communication between UAV swarms, characterized in that, The steps include the following: 1) Obtain the observation information of the UAV cluster and construct a cluster communication graph, which includes nodes, edges, a node feature matrix, an adjacency matrix, a sparsity hyperparameter of the adjacency matrix, and a degree matrix. Here, the nodes represent UAVs, the edges represent the communication relationships between UAVs, the node feature matrix includes the observation information of the UAVs, the adjacency matrix represents the communication weights between UAVs, the sparsity hyperparameter represents the number of other UAVs communicating with the current UAV, and the degree matrix represents the communication connection situation of the UAVs; 2) Construct a cluster communication optimization network, including an encoder and a decoder; input the node feature matrix and the adjacency matrix into the encoder for encoding to generate an encoded observation vector; input the encoded observation vector together with the historical action information of the nodes into the decoder to predict the next action distribution of the UAVs; 3) Train the cluster communication optimization network to optimize the adjacency matrix and the parameters of the encoder and decoder; 4) Update the cluster communication graph based on the optimized adjacency matrix, and use the optimized cluster communication optimization network to process the updated cluster communication graph, where the current observation information and historical action data are input to predict the next action distribution of the UAVs.

2. The method according to claim 1, characterized in that, The steps of encoding by the encoder in step 2) include: Perform normalization processing on the observation information of the UAVs in the node feature vector so that the tensor mean of the observation information is 0 and the standard deviation is 1; Input the normalized observation information into the self-attention network of the encoder to perform splitting of query vectors, key vectors, and value vectors; Calculate the similarity between the current node and neighbor nodes according to the query vector and key vector; Calculate the implicit attention score according to the similarity between the current node and neighbor nodes; On the basis of the implicit attention score, add explicit edge information and the temporal causal relationship between nodes to obtain the attention score of the adjacency matrix; Perform masking processing on the attention score of the adjacency matrix; Perform residual processing and normalization processing on the masked attention score and the normalized observation information to obtain the updated attention score; Input the updated attention score into a two-layer graph convolutional neural network, and combine it with the neighbor matrix and degree matrix to capture the topological structure characteristics of the cluster communication graph and the relationships between nodes to obtain the encoded observation information.

3. The method according to claim 1, characterized in that, The steps of prediction by the decoder in step 2) include: Perform nonlinear transformation and normalization processing on the actions of the nodes through a multi-layer perceptron; Input the normalized actions and masks into the relational attention network of the decoder for calculation, calculate the action distribution using a self-attention network and a cross-attention network, and then generate the final action distribution through two layers of multi-layer perceptrons.

4. The method according to claim 1, wherein Training the cluster communication optimization network in step 3) includes training the encoder. The steps of training the encoder include: Introduce an additional projection layer so that the encoder can estimate the value function of the observation information; Extract observations and corresponding actions at multiple time steps from the trajectory data of the UAV cluster, and calculate the immediate reward after performing the actions at each time step; Use the target network to estimate the value of the next time step, use the main network to estimate the value of the current time step, and calculate the difference between the two, and use this difference as part of the loss function; Optimize the parameters of the encoder by minimizing this error.

5. The method according to claim 4, wherein When training the encoder, a regularization term is also added to constrain the causal matrix learned by the encoder through the Frobenius norm to prevent overfitting.

6. The method according to claim 1, characterized in that, In step 3), training the cluster communication optimization network includes training the decoder. The steps of training the decoder include: Calculate the ratio factor based on the current policy and the historical policy to measure the change between the new and old policies. Calculate the advantage function to evaluate the superiority of the current action. Calculate the loss function. By comparing the unclipped advantage term with the clipped advantage term, select the smaller value to limit the policy update amplitude. Optimize the parameters of the decoder by minimizing the proximal policy optimization loss function.

7. The method according to claim 1, wherein In step 3), the steps of optimizing the adjacency matrix and the parameters of the encoder and decoder include: Relax the adjacency matrix into a continuous matrix to enable differentiable updates, and use an alternating update method to optimize the encoder parameters, decoder parameters, and adjacency matrix respectively. Use the Gumbel-Max Trick method to sample the adjacency matrix and approximate the discrete operation through continuous parameterization so that it can participate in the gradient backpropagation. Extend the Gumbel-Max Trick method to expand the one-hot representation to a multi-hot representation to meet the constraint that each node communicates with only a limited number of other nodes. Optimize and update the adjacency matrix based on the objective of minimizing the loss function on the validation set.

8. The method according to claim 1, wherein In step 4), the steps of updating the cluster communication graph based on the optimized adjacency matrix include: Each node accesses its local observations and actions and receives additional information transmitted by other nodes through communication. Calculate the connection relationship between each node and other nodes through the optimized adjacency matrix to update the cluster communication graph.

9. The method according to claim 1, wherein In step 4), the steps of predicting the next action distribution of the UAV include: Generate actions in an autoregressive manner, that is, use the current action as the input of the decoder to predict the next action. Starting from the initial action in sequence, gradually generate the actions at each time step until the final action sequence is generated.

10. An inter - UAV swarm communication optimization system for implementing the method according to any one of claims 1 - 9, characterized in that, Include: A cluster communication graph for obtaining the observation information of the UAV cluster and constructing the cluster communication graph. A cluster communication optimization network module, including an encoder and a decoder, for inputting the node feature matrix and the adjacency matrix into the encoder for encoding to generate an encoded observation vector; then inputting the encoded observation vector together with the historical action information of the node into the decoder to predict the next action distribution of the UAV. A training module for training the cluster communication optimization network, optimizing the adjacency matrix and the parameters of the encoder and decoder, and updating the cluster communication graph based on the optimized adjacency matrix.

Images