Multi-unmanned aerial vehicle air combat advantage competition strategy generation method based on graph attention network

Abstract

The invention discloses a multi-unmanned aerial vehicle air combat advantage competition strategy generation method based on a graph attention network. The method comprises the following steps: a, representing air combat observation data as a topological graph; b, according to the topological graph, adopting a graph attention neural network as a strategy model, and combining priori knowledge to obtain an abstract relation graph representing an air combat relation; and c, carrying out maneuvering decision making according to the abstract relation graph. According to the invention, a graph attention network and a method for establishing an abstract relation graph are adopted to learn a strategy which is more beneficial to cooperation; and the air combat relationship and communication among the unmanned aerial vehicles are enhanced, so that the finally output actions achieve effective cooperation.

Description

Technical Field

[0001] The present invention relates to the field of multi-UAV autonomous collaborative decision-making and control technology, and in particular to a multi-UAV air combat advantage competition strategy generation method based on a graph attention network. Background Art

[0002] In the military field, drones are widely used in reconnaissance, surveillance, intelligence collection and strike missions. It is difficult for a single drone to effectively perform diverse tasks in a complex combat environment. Therefore, it is particularly important to study multi-drone coordination methods to adapt to changing battlefield needs. In the context of military intelligence, the application of drones has gradually shifted from single-machine systems to multi-machine systems and drone clusters, focusing on multi-drone intelligent combat and improving battlefield advantages through group intelligence.

[0003] Under the complex conditions of strong confrontation and uncertainty, the battlefield situation changes rapidly. Without the support of a stable communication link, it is difficult for human-in-the-loop control and decision-making to meet the requirements of real-time decision-making. Therefore, drones need to have autonomous decision-making capabilities. Multi-UAV visual air combat confrontation is a highly dynamic confrontation environment. Under the conditions of strong electromagnetic interference and limited communication, it is difficult for the central control unit to achieve multi-UAV visual air combat decision-making to meet the requirements of real-time decision-making. If the central control unit is responsible for situation fusion processing and distributes information to other drones, the communication requirements between drones will be higher. In addition, it is difficult for a centrally controlled multi-UAV system to continue to fight after the central node is attacked. Decentralization is a more effective implementation method. UAVs make distributed decisions through their own perception and communication with neighboring friendly parties, improve their autonomous decision-making capabilities, and can collaborate with friendly parties to generate new behavioral strategies to effectively fight the enemy. However, decentralized decision-making lacks the coordination of the central control unit. Distributed decision-making methods are needed to ensure that each drone makes reasonable decisions based on a full understanding of the situation to achieve coordination with friendly parties.

[0004] Traditional air combat strategies usually rely on expert knowledge to build a library of maneuver and engagement rules, and use heuristic algorithms and game theory to generate strategies. However, the mathematical model of game theory is difficult to construct, and the sufficiency and necessity of its solution are difficult to prove. Expert systems and heuristic algorithms rely on artificially designed rules. When the number of intelligent agents increases, it becomes very difficult to design complete rules or scalable algorithms. In addition, the generalization ability of these rules is limited, and it is difficult to achieve the optimal. In the process of confrontation, the strategies of both sides continue to evolve and constrain each other. There is not necessarily a single strategy that can completely suppress the other side.

[0005] At present, using neural networks to fit the strategies of UAVs by training neural network models and learning the ideal strategies through reinforcement learning methods has become a research hotspot. Many researchers regard each UAV as an independent individual, import the observed air combat time series data into the recurrent neural network to output decision actions, ignoring the air combat relationship and communication between each UAV, so the final output action is difficult to achieve effective cooperation. At present, some scholars divide air supremacy into three levels: air superiority, air domination and air hegemony. In multi-aircraft intelligent air combat, given that it is difficult to achieve absolute air supremacy, it is usually more practical to win air supremacy in key battlefields and key stages than to win absolute air supremacy. Therefore, here we focus on the problem of obtaining air superiority, that is, obtaining a relative advantage over the enemy and having the initiative in combat, and design a multi-aircraft air combat decision-making method for air superiority. Summary of the invention

[0006] The present invention aims to provide a multi-UAV air combat advantage competition strategy generation method based on graph attention network, which represents the air combat observation data as a topological graph structure, uses graph attention network to replace traditional neural network as a strategy model, combines prior knowledge to derive an abstract relationship graph representing the air combat relationship, and then makes maneuver decisions based on the abstract relationship graph. The MAPPO algorithm is used in training, and through the centralized training and decentralized execution (CTDE) framework, the UAV makes decisions based on local observations and local communications, without the need to obtain global status through global communication, and has good real-time and robustness.

[0007] The technical solution adopted by the present invention is specifically as follows:

[0008] The method for generating multi-UAV air combat advantage competition strategy based on graph attention network includes the following steps:

[0009] a. Represent the air combat observation data as a topological graph;

[0010] b. Based on the topological graph, a graph attention neural network is used as a strategy model, and an abstract relationship graph representing air combat relationships is obtained in combination with prior knowledge;

[0011] c. Make flexible decisions based on abstract relationship diagrams.

[0012] Preferably, in step a, the method for constructing the topology map is as follows:

[0013] Taking drones as nodes, the nodes represent the drones’ own information as node attributes, and the information between drones as edge attributes, thus constructing an undirected topological graph.

[0014] In step b, the abstract relationship diagram is obtained by the following method:

[0015] The topology diagram Find the subset of friendly relationship nodes and the subset of enemy relationship nodes respectively, and use the graph attention network to obtain the most important friendly features and the most important enemy characteristics Then, the unique hot encoding of the most important friendly ID and enemy ID at the previous moment and the own ID and the vertex information are imported into the multi-layer perceptron to obtain the most important enemy ID and the most important friendly ID at the current moment. Then, through partial active communication, the ID of the most important friendly drone that regards itself as the most important and the ID of the most important enemy drone of its most important friendly drone are obtained to establish an abstract relationship graph.

[0016] The step c of making a maneuver decision based on the abstract relationship diagram includes:

[0017] Abstract relationship diagram The potential combat intention features are obtained through the graph attention network, and then the average pooled features of the potential combat intention of the friendly party are obtained through the communication mechanism. Then the distance to the center of the occupied area is calculated, and all features are input into the gated recurrent unit to obtain the fused potential intention features. Then, through multi-agent joint decision-making, the potential fused intention features are passed through a three-head multi-layer perceptron to obtain the output of maneuvering actions, target selection and shooting instructions.

[0018] Further preferably, the method for constructing the topological map in step a comprises the following sub-steps:

[0019] a.1. Construct a simulation environment based on actual air combat scenarios: A cluster of fixed-wing drones with isomorphic and symmetrical capabilities carrying limited missiles conducts operations in a certain three-dimensional airspace with the goal of occupying a certain area and blocking the enemy agent outside the area, using the control quantity {δ a ,δ e ,δ r ,th i ,shot} respectively control the control surface, throttle and shooting command of the UAV, and define the environmental constraint model, the individual constraint model of the UAV and the countermeasure impact rules;

[0020] a.2. Model the multi-UAV air combat problem as a hidden Markov decision process, described by the tuple (S, A, O, R, γ), where S represents all states of the environment, A represents the set of executable actions, O represents the observation information obtained by each UAV, R represents the immediate reward for the UAV to perform o in state s, and γ is the discount factor for future rewards;

[0021] a.3. Represent the state of the drone as a mathematical topological graph structure, and define the scenario abstract graph as G s =(V,E), where V is the vertex set, used to represent the information of the drone itself; E is the edge set, used to represent the relationship between drones.

[0022] Preferably, step b comprises:

[0023] The abstract scene diagram of drone i at time t The elements in are represented as in is the serial number of the friendly drone at time t, is the number of the enemy drone at time t. They are in, They represent the state information of drone i at time t, the information of the launched missile and the information of the incoming missile, respectively. They represent the relative height, relative speed, relative distance, azimuth and angle of UAV i and UAV j at time t respectively;

[0024] Add the distance d between the drone and the center of the occupied area to the state space, and the observable variables of drone i are

[0025] Preferably, in step c:

[0026] The actions of the agent are divided into maneuver decision actions, target selection actions and shooting actions, which are represented by A = {a1, a2, a3}; the control maneuver actions are 14 basic maneuver actions defined by NASA, and the basic maneuver actions are implemented by calling the rule library; the target selection action is one-hot encoded according to the number of targets, and the shooting action is a Boolean value;

[0027] The reward function is the scenario reward r e and key event rewards r s , the scenario reward is a sparse reward, which is given when air superiority is achieved, that is, when our drone is in the occupied area and has an advantage in the number of aircraft; the key event reward is given when the enemy aircraft is destroyed and / or the enemy aircraft is searched and / or the missile guidance is completed, as shown in the following formula:

[0028]

[0029] Among them, r s is the total reward at step t, e t is the key event, w is the reward scale weight of the corresponding event, and λ is the linear annealing factor, which is linearly annealed to 0 after 100 iterations.

[0030] Further preferably, step c comprises the following sub-steps:

[0031] c.1. Use a two-layer graph attention network to reason about friendly and enemy relationships respectively. For friendly relationships, first obtain the spatial features of drone i and calculate the attention coefficient between drone i and j:

[0032]

[0033] Where ‖ is the concatenation operator, T represents transposition, is the attention coefficient of drone j to i at time t, is a trainable weight matrix; is the weight vector transpose of a single-layer neural network, and LeakyReLU() is the activation function;

[0034] c.2. The UAV transmits its intention to the friendly UAV and obtains the final fusion information through the communication mechanism. in, is the fusion information obtained by UAV i at time t, f(g) is the gated recurrent unit, d is the distance between UAV i and the center of the occupied area, For the received communication intention, Where N c Assemble for communication drones;

[0035] c.3. Multi-agent joint decision-making is as follows: The fusion information Import the three-head neural network with softmax activation function and directly output the decision action of drone i.

[0036]

[0037] Where MH(g) is a three-head neural network operator with softmax activation function, They are respectively the maneuver decision action, target selection action and shooting action of UAV i.

[0038] Furthermore, this method for generating multi-UAV air combat advantage competition strategies based on graph attention networks also includes air superiority competition training based on a multi-agent proximal strategy optimization algorithm in a deep reinforcement learning method.

[0039] Preferably, the air superiority competition training based on the MAPPO deep reinforcement learning algorithm includes the following steps:

[0040] Step 1: Build a Critic network for each UAV’s policy network Actor;

[0041] Step 2: Collect the tracks collected by the Actor network Put it into the experience replay pool, and each iteration is based on calculate in

[0042] in accordance with calculate in is the output of the critic network, Stored in the experience replay pool;

[0043] Step 3: Extract group B data from the experience replay pool each time the network is updated. Calculate the gradients of Actor and Critic separately Update the Actor network and Critic network through the Adam optimizer.

[0044] The beneficial effects of the present invention are as follows:

[0045] 1. The present invention represents the relationship between drones with a topological graph, and adopts a graph attention network and a method of establishing an abstract relationship graph to learn a strategy that is more conducive to cooperation; it strengthens the air combat relationship and communication between drones, so that the final output action achieves effective cooperation.

[0046] 2. Aiming at the field of air superiority competition, the present invention proposes an autonomous generation method of multi-aircraft air combat decision-making strategies based on multi-agent reinforcement learning method, which has good learning efficiency and robustness.

[0047] 3. The multi-agent proximal strategy optimization algorithm adopted by the present invention improves the real-time performance of multi-aircraft air combat strategy through central training and decentralized execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Schematic diagram of the strategy model structure based on graph neural network.

[0049] Figure 2 Schematic diagram of the MAPPO-based training method. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0051] Example 1

[0052] This embodiment discloses a method for generating a multi-UAV air combat advantage competition strategy based on a graph attention network, specifically comprising:

[0053] 1. System architecture of air combat decision model based on graph attention network

[0054] The system architecture is a multi-aircraft air combat strategy generation model based on graph attention network, which is mainly divided into three parts: air combat relationship reasoning, potential combat intention fusion and multi-agent joint decision-making.

[0055] like Figure 1 As shown in the figure, the system architecture is in the air combat relationship reasoning part, which represents the drone's own information as the attribute of the node, and the information between drones as the attribute of the edge, thus constructing an undirected topological graph. For this topological graph, we find the subset of friendly nodes and the subset of enemy nodes, and use GAT to get the most important friendly features. and the most important enemy characteristics Then, the unique hot encoding of the most important friendly ID and enemy ID at the previous moment and your own ID and your own vertex information are imported into the Multi-Layer Perceptron (MLP) to obtain the most important enemy ID and the most important friendly ID at the current moment. Then, through partial active communication, the ID of the most important enemy drone that regards you as the most important friendly drone and your most important friendly drone is obtained to establish an abstract relationship graph (ARG). Complete the air combat relationship reasoning; in the potential combat intention fusion part, The potential combat intention features are obtained through GAT, and then the average pooled features of the potential combat intentions of the friendly forces are obtained through the communication mechanism. Then the distance to the center of the occupied area is calculated, and all features are input into the gated recurrent unit (GRU) to obtain the fused potential intention features. The multi-agent joint decision-making optimization part uses the potentially fused intention features through a three-head MLP to obtain the output of maneuvering actions, target selection, and shooting instructions.

[0056] 2. Establishment of Hidden Markov Decision Model for Multi-aircraft Air Combat and Definition of Parameters

[0057] 2.1. First, a simulation environment is constructed based on the actual air combat scenario: a cluster of fixed-wing drones with isomorphic and symmetrical capabilities carrying limited missiles is deployed in a certain three-dimensional airspace to occupy a certain area and block the enemy agent outside the area. a ,δ e ,δ r ,th i ,shot} respectively control the control surface, throttle and shooting command of the UAV, and define the environmental constraint model, the UAV individual constraint model and the countermeasure impact rules respectively.

[0058] 2.2. The multi-UAV air combat problem is modeled as a hidden Markov decision process, which is described by the tuple (S, A, O, R, γ), where S represents all states of the environment, A represents the set of executable actions, O represents the observation information obtained by each UAV, R represents the immediate reward for the UAV to execute o in state s, and γ is the discount factor for future rewards.

[0059] 2.3. Represent the state of the drone as a mathematical topological graph structure and define the Situation Abstraction Graph (SAG) as G s =(V,E), where V is the vertex set, which is used to represent the information of the drone itself; E is the edge set, which represents the relationship between drones.

[0060] 2.4. UAV i at time t The elements in can be represented as in is the serial number of the friendly drone at time t, is the number of the enemy drone at time t. They are in, They represent the state information of UAV i at time t, the launched missile information and the incoming missile information respectively. They represent the relative height, relative speed, relative distance, azimuth and angle of UAV i and UAV j at time t respectively.

[0061] 2.5. The drone needs to consider its positional relationship with the occupied area. Here, the distance d from the center of the occupied area is used and added to the state space. Therefore, the observable variable of drone i is

[0062] 2.6. The actions of the agent are divided into maneuver decision actions, target selection actions and shooting actions, which can be expressed as A = {a1, a2, a3}. The control maneuver actions are designed as the 14 basic maneuvers defined by NASA, which can be implemented by calling the rule library; the target selection action is one-hot encoded according to the number of targets, and the shooting action is a Boolean value.

[0063] 2.7. Design the reward function as the scenario reward r e and key event rewards r s The scenario reward is a sparse reward, which is given when air superiority is achieved, that is, when our drone is in the occupied area and has an advantage in the number of aircraft, we get a reward as shown in Table 1; the key event reward is given when the enemy aircraft is destroyed, when the enemy aircraft is searched, when the missile guidance is completed, etc., and its mathematical expression is rs is the total reward at step t, e t is the key event, w is the reward scale weight of the corresponding event, and λ is the linear annealing factor, which is linearly annealed to 0 after 100 iterations.

[0064] Table 1 Key air combat event reward shaping

[0065]

[0066]

[0067] 3. Air combat relationship reasoning method based on graph attention network

[0068] 3.1. Use two layers of GAT to reason about friendly and enemy relations. For friendly relations, first obtain the spatial features of drone i and calculate the attention coefficient between drone i and j:

[0069]

[0070] Where ‖ is the concatenation operator, T represents transposition, is the attention coefficient of drone j to i at time t, is a trainable weight matrix; is the weight vector transpose of a single-layer neural network, and LeakyReLU() is the activation function. The attention coefficients between other drones and enemy drones are also calculated using this method.

[0071] 3.2. Aggregate the features from friendly and opponent respectively. The output of the graph attention layer of drone i is in and is the spatial characteristics of UAV i with respect to friendly and adversary. δ(·) is a nonlinear function; and is a trainable weight matrix.

[0072] 3.3. Using time features to improve the accuracy of situation estimation. In order to capture the time features, the one-hot encoding at time t-1 is embedded into a compact vector, indicating the previous combat relationship: in represents the time characteristics of the drone at time t, OHE() is the one-hot encoder function, and f i t-1 is the most important friendly force of drone i at time t-1, and represents the most important opponent at time t-1. Therefore, the spatiotemporal characteristics of drone i at time t can be expressed as: Among them, f() is a 2-layer neural network.

[0073] 3.4. Importing the spatiotemporal features into a two-headed neural network with a softmax activation function can obtain the most important friendly drone and the most important enemy drone for drone i at time t.

[0074] 3.5. Using the partial active communication method, let the drones send their own air combat relationships to the corresponding friendly drones through unidirectional communication, so as to construct an ARG. ARG is a directed topological graph structure. The out-degree of its vertices represents the most important friendly and enemy drones, and the in-degree represents the friendly drone that is considered the most important.

[0075] 4. Latent Intention Fusion Method

[0076] 4.1. As shown below, ARG is imported into GAT to obtain the combat intention of drone i at time t.

[0077]

[0078] in, is the attention coefficient of drone i and drone j at time t:

[0079]

[0080] 4.2. In order to strengthen cooperation, drones can convey their intentions to friendly drones and obtain the final fusion information through the communication mechanism. in, is the fusion information obtained by UAV i at time t, f() is GRU, d is the distance between UAV i and the center of the occupied area, For the received communication intention, Where N c Assemble for communication drones.

[0081] 5. Multi-agent joint decision-making optimization module

[0082] This module is responsible for making the final decision based on the fusion information. Import into the three-head neural network with softmax activation function and directly output the decision action of drone i. Where MH() is a three-headed neural network operator with a softmax activation function. They are respectively the maneuver decision action, target selection action and shooting action of UAV i. The action finally transmitted to the environment is

[0083] Example 2

[0084] like Figure 2As shown, based on Example 1, this example discloses an air superiority competition training strategy based on the MAPPO deep reinforcement learning algorithm as follows:

[0085] 1. Build a Critic network for each drone’s policy network Actor.

[0086] 2. Collect the tracks of the Actor network Put it into the experience replay pool, and each iteration is based on calculate in in accordance with calculate in is the output of the critic network, Stored in the experience replay pool.

[0087] 3. Each time the network is updated, group B data is extracted from the experience replay pool. Calculate the gradients of Actor and Critic separately The Actor network and Critic network are then updated through the Adam optimizer.

[0088] The present invention establishes a strategy generation model for drone clusters under the node topological relationship of changing battlefield situations based on the air combat relationship reasoning method of graph attention network.

[0089] The present invention integrates domain expert knowledge to design an abstract relationship diagram, thereby increasing the accuracy of potential intention prediction of multi-aircraft air combat, thereby improving the winning rate of the generated strategy.

[0090] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, technicians familiar with the field may make various corresponding changes and deformations based on the present invention, but these corresponding changes and deformations should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for generating multi-UAV air combat advantage competition strategies based on graph attention network, characterized in that: The following steps are involved: a. Represent the air combat observation data as a topological graph; b. Based on the topological graph, a graph attention neural network is used as a strategy model, and an abstract relationship graph representing air combat relationships is obtained in combination with prior knowledge; c. Make flexible decisions based on abstract relationship diagrams.

2. The method for generating a multi-UAV air combat advantage competition strategy based on a graph attention network according to claim 1 is characterized in that: In step a, the method for constructing the topology map is as follows: Taking drones as nodes, the nodes represent the drones’ own information as node attributes, and the information between drones as edge attributes, thus constructing an undirected topological graph. In step b, the abstract relationship diagram is obtained by the following method: The topology diagram Find the subset of friendly relationship nodes and the subset of enemy relationship nodes respectively, and use the graph attention network to obtain the most important friendly features and the most important enemy characteristics Then, the unique hot encoding of the most important friendly ID and enemy ID at the previous moment and the own ID and the vertex information are imported into the multi-layer perceptron to obtain the most important enemy ID and the most important friendly ID at the current moment. Then, through partial active communication, the ID of the most important friendly drone that regards itself as the most important and the ID of the most important enemy drone of its most important friendly drone are obtained to establish an abstract relationship graph. The step c of making a maneuver decision based on the abstract relationship diagram includes: Abstract relationship diagram The potential combat intention features are obtained through the graph attention network, and then the average pooled features of the potential combat intention of the friendly party are obtained through the communication mechanism. Then the distance to the center of the occupied area is calculated, and all features are input into the gated recurrent unit to obtain the fused potential intention features. Then, through multi-agent joint decision-making, the potential fused intention features are passed through a three-head multi-layer perceptron to obtain the output of maneuvering actions, target selection and shooting instructions.

3. The method for generating a multi-UAV air combat advantage competition strategy based on a graph attention network according to claim 2 is characterized in that: The method for constructing the topology map in step a includes the following sub-steps: a.

1. Construct a simulation environment based on actual air combat scenarios: A cluster of fixed-wing drones with isomorphic and symmetrical capabilities carrying limited missiles conducts operations in a certain three-dimensional airspace with the goal of occupying a certain area and blocking the enemy agent outside the area, with the control quantity {δ a ,δ e ,δ r ,th i ,shot} respectively control the control surface, throttle and shooting command of the UAV, and define the environmental constraint model, the individual constraint model of the UAV and the countermeasure impact rules; a.

2. Model the multi-UAV air combat problem as a hidden Markov decision process, described by the tuple (S, A, O, R, γ), where S represents all states of the environment, A represents the set of executable actions, O represents the observation information obtained by each UAV, R represents the immediate reward for the UAV to perform o in state s, and γ is the discount factor for future rewards; a.

3. Represent the state of the drone as a mathematical topological graph structure, and define the scenario abstract graph as G s =(V,E), where V is the vertex set, used to represent the information of the drone itself; E is the edge set, used to represent the relationship between drones.

4. The method for generating a multi-UAV air combat advantage competition strategy based on a graph attention network according to claim 3 is characterized in that: Step b includes: The abstract scene graph of drone i at time t The elements in are represented as in is the serial number of the friendly drone at time t, is the number of the enemy drone at time t. They are in, They represent the state information of drone i at time t, the information of the launched missile and the information of the incoming missile, respectively. They represent the relative height, relative speed, relative distance, azimuth and angle of UAV i and UAV j at time t respectively; Add the distance d between the drone and the center of the occupied area to the state space, and the observable variables of drone i are 5. The method for generating a multi-UAV air combat advantage competition strategy based on a graph attention network according to claim 4 is characterized in that: In step c: The actions of the agent are divided into maneuver decision actions, target selection actions and shooting actions, which are represented by A = {a1, a2, a3}; the control maneuver actions are 14 basic maneuver actions defined by NASA, and the basic maneuver actions are implemented by calling the rule library; the target selection action is one-hot encoded according to the number of targets, and the shooting action is a Boolean value; The reward function is the scenario reward r e and key event rewards r s , the scenario reward is a sparse reward, which is given when air superiority is achieved, that is, when our drone is in the occupied area and has an advantage in the number of aircraft; the key event reward is given when the enemy aircraft is destroyed and / or the enemy aircraft is searched and / or the missile guidance is completed, as shown in the following formula: Among them, r s is the total reward at step t, e t is the key event, w is the reward scale weight of the corresponding event, and λ is the linear annealing factor, which is linearly annealed to 0 after 100 iterations.

6. The method for generating a multi-UAV air combat advantage competition strategy based on a graph attention network according to claim 5 is characterized in that: Step c includes the following sub-steps: c.

1. Use a two-layer graph attention network to reason about friendly and enemy relationships respectively. For friendly relationships, first obtain the spatial features of drone i and calculate the attention coefficient between drone i and j: Where ‖ is the concatenation operator, T represents transposition, is the attention coefficient of drone j to i at time t, is a trainable weight matrix; is the weight vector transpose of a single-layer neural network, and LeakyReLU() is the activation function; c.

2. The UAV transmits its intention to the friendly UAV and obtains the final fusion information through the communication mechanism. in, is the fusion information obtained by UAV i at time t, f(g) is the gated recurrent unit, d is the distance between UAV i and the center of the occupied area, For the received communication intention, Where N c Assemble for communication drones; c.

3. Multi-agent joint decision-making is as follows: The fusion information Import the three-head neural network with softmax activation function and directly output the decision action of drone i. Where MH(g) is a three-head neural network operator with softmax activation function, They are respectively the maneuver decision action, target selection action and shooting action of UAV i.

7. The method for generating a multi-UAV air combat advantage competition strategy based on a graph attention network according to any one of claims 1 to 6, characterized in that: It also includes air superiority competition training based on the multi-agent proximal strategy optimization algorithm in deep reinforcement learning method.

8. The method for generating a multi-UAV air combat advantage competition strategy based on a graph attention network according to claim 7 is characterized in that: The air superiority competition training based on the MAPPO deep reinforcement learning algorithm includes the following steps: Step 1: Build a Critic network for each UAV’s policy network Actor; Step 2: Collect the tracks collected by the Actor network Put it into the experience replay pool, and each iteration is based on calculate in in accordance with calculate in is the output of the critic network, Stored in the experience replay pool; Step 3: Extract group B data from the experience replay pool each time the network is updated. Calculate the gradients of Actor and Critic separately Update the Actor network and Critic network through the Adam optimizer.

Images