An unmanned aerial vehicle air combat decision method and system thereof

Abstract

The application belongs to the technical field of unmanned aerial vehicle air combat decision, and particularly relates to an unmanned aerial vehicle air combat decision method and system, wherein the unmanned aerial vehicle air combat decision method comprises the following steps: abstracting an air combat unmanned aerial vehicle situation, and constructing an unmanned aerial vehicle air combat situation graph; dividing the unmanned aerial vehicle air combat situation graph into multiple unmanned aerial vehicle air combat relationship graphs; performing feature extraction on each unmanned aerial vehicle air combat relationship graph, fusing features extracted from each unmanned aerial vehicle air combat relationship graph to obtain unmanned aerial vehicle air combat fusion features; and generating a friendly unmanned aerial vehicle air combat strategy based on the unmanned aerial vehicle air combat fusion features.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned aerial vehicle air combat decision-making, and particularly relates to an unmanned aerial vehicle air combat decision-making method and system. BACKGROUND

[0002] In large-scale unmanned aerial vehicle air combat, the unmanned aerial vehicle autonomously makes combat decision, breaks away from the dependence on pilots, and can break through the limit of flight operation. At present, the unmanned aerial vehicle autonomously makes combat decision mostly by a rule-based expert system method, a probability model / fuzzy logic and computing intelligence hybrid method, a machine learning and deep reinforcement learning method. The data amount to be processed is large, the efficiency is low, the learning difficulty is great, and the effect is not ideal in large-scale unmanned aerial vehicle combat.

[0003] The present application is proposed in view of the above technical defects.

[0004] It should be noted that the disclosure of the above background art is only used to assist in understanding the inventive concept and technical solutions of the present application, and it does not necessarily belong to the prior art of the present patent application. In the absence of explicit evidence that the above content has been disclosed on the filing date of the present application, the above background art should not be used to evaluate the novelty and inventiveness of the present application. SUMMARY

[0005] The purpose of the present application is to provide an unmanned aerial vehicle air combat decision-making method and system to overcome or alleviate at least one aspect of the technical defects known to exist.

[0006] The technical solution of the present application is:

[0007] In one aspect, an unmanned aerial vehicle air combat decision-making method is provided, comprising:

[0008] abstracting an air combat unmanned aerial vehicle situation, and constructing an unmanned aerial vehicle air combat situation graph;

[0009] dividing the unmanned aerial vehicle air combat situation graph into a plurality of unmanned aerial vehicle air combat relationship graphs;

[0010] extracting features from each unmanned aerial vehicle air combat relationship graph, fusing the features extracted from each unmanned aerial vehicle air combat relationship graph, and obtaining unmanned aerial vehicle air combat fusion features;

[0011] generating a friendly unmanned aerial vehicle air combat strategy based on the unmanned aerial vehicle air combat fusion features.

[0012] According to at least one embodiment of the present application, in the above unmanned aerial vehicle air combat decision-making method, the abstracting of the air combat unmanned aerial vehicle situation and the constructing of the unmanned aerial vehicle air combat situation graph are specifically:

[0013] The aerial combat unmanned aerial vehicle is taken as a node, and node information includes state information of the corresponding aerial combat unmanned aerial vehicle and state information of a strike object carried by the aerial combat unmanned aerial vehicle.

[0014] A connection is made between nodes corresponding to two aerial combat unmanned aerial vehicles capable of acquiring state information of each other, and an aerial combat situation map of unmanned aerial vehicles is constructed.

[0015] According to at least one embodiment of the present application, in the unmanned aerial vehicle aerial combat decision method, the state information of the aerial combat unmanned aerial vehicle includes position, speed, and yaw angle of the aerial combat unmanned aerial vehicle.

[0016] The state information of the strike object carried by the aerial combat unmanned aerial vehicle includes strike speed and survival state of the strike object carried by the aerial combat unmanned aerial vehicle.

[0017] According to at least one embodiment of the present application, in the unmanned aerial vehicle aerial combat decision method, the aerial combat situation map of unmanned aerial vehicles is divided into a plurality of aerial combat relationship graphs of unmanned aerial vehicles, and specifically:

[0018] An upper limit of the number of aerial combat relationship graphs of unmanned aerial vehicles is set.

[0019] According to the aerial combat situation map of unmanned aerial vehicles, a first neural network is used to configure a probability of each node appearing in each aerial combat relationship graph of unmanned aerial vehicles, and node information in each aerial combat relationship graph of unmanned aerial vehicles and a connection line are obtained.

[0020] According to at least one embodiment of the present application, in the unmanned aerial vehicle aerial combat decision method, the upper limit of the number of aerial combat relationship graphs of unmanned aerial vehicles is specifically set as one half of the number of aerial combat unmanned aerial vehicles.

[0021] According to at least one embodiment of the present application, in the unmanned aerial vehicle aerial combat decision method, features of each aerial combat relationship graph of unmanned aerial vehicles are extracted, and features extracted from each aerial combat relationship graph of unmanned aerial vehicles are fused to obtain aerial combat fusion features of unmanned aerial vehicles, and specifically:

[0022] A second neural network is used to extract features from each aerial combat relationship graph of unmanned aerial vehicles, and features extracted from each aerial combat relationship graph of unmanned aerial vehicles are fused to obtain aerial combat fusion features of unmanned aerial vehicles.

[0023] According to at least one embodiment of the present application, in the unmanned aerial vehicle aerial combat decision method, the aerial combat strategy of the own unmanned aerial vehicle is generated based on the aerial combat fusion features of unmanned aerial vehicles, and specifically:

[0024] A third neural network is used to generate the aerial combat strategy of the own unmanned aerial vehicle based on the aerial combat fusion features of unmanned aerial vehicles.

[0025] According to at least one embodiment of the present application, the unmanned aerial combat decision method described above further comprises:

[0026] The own unmanned aerial combat strategy is interacted with the simulation environment, and samples of the own unmanned aerial combat strategy are collected.

[0027] The samples of the own unmanned aerial combat strategy are used to train the neural network in the unmanned aerial combat decision method.

[0028] According to at least one embodiment of the present application, in the unmanned aerial combat decision method described above, the samples of the own unmanned aerial combat strategy are used to train the neural network in the unmanned aerial combat decision method, specifically:

[0029] The samples of the own unmanned aerial combat strategy are used to train the neural network in the unmanned aerial combat decision method based on a reinforcement learning algorithm optimized based on a near-source strategy.

[0030] In another aspect, an unmanned aerial combat decision system is provided, comprising:

[0031] An unmanned aerial combat situation map construction module abstracts an aerial combat unmanned aerial vehicle situation and constructs an unmanned aerial combat situation map.

[0032] An unmanned aerial combat relationship graph division module divides the unmanned aerial combat situation map into a plurality of unmanned aerial combat relationship graphs.

[0033] An unmanned aerial combat fusion feature extraction module extracts features from each unmanned aerial combat relationship graph, fuses the features extracted from each unmanned aerial combat relationship graph, and obtains unmanned aerial combat fusion features.

[0034] An own unmanned aerial combat strategy generation module generates an own unmanned aerial combat strategy based on the unmanned aerial combat fusion features.

[0035] An own unmanned aerial combat strategy sample collection module interacts the own unmanned aerial combat strategy with the simulation environment, and collects samples of the own unmanned aerial combat strategy.

[0036] An unmanned aerial combat decision neural network training module uses the samples of the own unmanned aerial combat strategy to train the neural network in the unmanned aerial combat decision method. BRIEF DESCRIPTION OF DRAWINGS

[0037] Fig. 1 is a schematic diagram of the unmanned aerial combat decision method provided by the embodiments of the present application;

[0038] Fig. 2 FIG. 1 is a schematic diagram of a UAV air combat decision system provided by an embodiment of the present application.

[0039] In order to better illustrate the embodiments, some components in the drawings can be omitted, enlarged or reduced, and the size of the actual product is not represented. In addition, the drawings are only used for illustrative description and cannot be understood as a limitation of the present application. DETAILED DESCRIPTION

[0040] In order to make the technical solutions of the present application and the advantages thereof clearer, the technical solutions of the present application will be further clearly and completely described below with reference to the drawings. It should be understood that the specific embodiments described herein are only some embodiments of the present application, which are used to explain the present application, but not to limit the present application. It should be noted that, in order to facilitate the description, only parts related to the present application are shown in the drawings, and other related parts can be referred to the general design. In the case of no conflict, the embodiments in the present application and the technical features in the embodiments can be combined to obtain new embodiments.

[0041] In addition, unless otherwise defined, the technical terms or scientific terms used in the present application description should be the general meaning understood by the general technical personnel in the field of the present application. The words indicating the direction or position relationship such as "upper", "lower", "left", "right", "center", "vertical", "horizontal", "inner", "outer" and the like used in the present application description are only used to indicate the relative direction or position relationship, and not to imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and the relative position relationship may also change accordingly when the absolute position of the described object changes, therefore, it cannot be understood as a limitation of the present application. The "first", "second", "third" and the like used in the present application description are only for the purpose of description, and are used to distinguish different components, and cannot be understood as indicating or implying relative importance. The "one", "an" or "the" and the like used in the present application description should not be understood as an absolute limitation on the quantity, but should be understood as the existence of at least one. The "include" or "contain" and the like used in the present application description means that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, and other elements or objects are not excluded.

[0042] In addition, it should be noted that, unless otherwise defined and limited, the "installation", "connection", "connection" and the like used in the present application description should be understood in a broad sense, for example, the connection can be fixed connection, or detachable connection, or integrally connected; can be mechanical connection, or electrical connection; can be directly connected, or indirectly connected through intermediate medium, or the connection between two elements, the person skilled in the art can understand the specific meaning of the present application according to the specific circumstances.

[0043] The accompanying drawings are incorporated in and constitute a part of this specification. Figs. 1-2 The application is described in further detail below.

[0044] In one aspect, a method for unmanned aerial vehicle air combat decision-making is provided, comprising:

[0045] Abstracting the air combat unmanned aerial vehicle situation, and constructing an unmanned aerial vehicle air combat situation graph;

[0046] Dividing the unmanned aerial vehicle air combat situation graph into a plurality of unmanned aerial vehicle air combat relationship graphs;

[0047] Extracting features from each unmanned aerial vehicle air combat relationship graph, fusing the features extracted from each unmanned aerial vehicle air combat relationship graph, and obtaining unmanned aerial vehicle air combat fusion features;

[0048] Generating a friendly unmanned aerial vehicle air combat strategy based on the unmanned aerial vehicle air combat fusion features.

[0049] For the unmanned aerial vehicle air combat decision-making method disclosed in the above embodiments, those skilled in the art can understand that the design abstracts the air combat unmanned aerial vehicle situation, constructs an unmanned aerial vehicle air combat situation graph, and then divides the unmanned aerial vehicle air combat situation graph into a plurality of unmanned aerial vehicle air combat relationship graphs, extracts features of each unmanned aerial vehicle air combat relationship graph, fuses the features, and obtains unmanned aerial vehicle air combat fusion features, so that a friendly unmanned aerial vehicle air combat strategy can be generated based on the unmanned aerial vehicle air combat fusion features, which can effectively reduce the amount of data processing, improve efficiency, and reduce the difficulty of learning the friendly unmanned aerial vehicle air combat strategy, and achieve ideal results in large-scale unmanned aerial vehicle combat.

[0050] In some optional embodiments, in the unmanned aerial vehicle air combat decision-making method described above, the abstracting of the air combat unmanned aerial vehicle situation and the construction of the unmanned aerial vehicle air combat situation graph are specifically as follows:

[0051] Taking the air combat unmanned aerial vehicle as a node, the node information includes state information of the corresponding air combat unmanned aerial vehicle and state information of a strike object carried by the air combat unmanned aerial vehicle;

[0052] Connecting between nodes corresponding to two air combat unmanned aerial vehicles that can obtain each other's state information, and constructing the unmanned aerial vehicle air combat situation graph.

[0053] In some optional embodiments, in the unmanned aerial vehicle air combat decision-making method described above, the state information of the air combat unmanned aerial vehicle includes the position, speed, and yaw angle of the air combat unmanned aerial vehicle.

[0054] The state information of the strike object carried by the air combat unmanned aerial vehicle includes the strike speed and survival state of the strike object carried by the air combat unmanned aerial vehicle.

[0055] In some optional embodiments of the unmanned aerial combat decision-making method described above, the unmanned aerial combat situation graph is divided into a plurality of unmanned aerial combat relationship graphs, specifically:

[0056] An upper limit of the number of unmanned aerial combat relationship graphs is set.

[0057] According to the unmanned aerial combat situation graph, a first neural network is used to configure the probability of each node appearing in each unmanned aerial combat relationship graph, and node information and connections in each unmanned aerial combat relationship graph are obtained. Specifically, a corresponding subgraph division algorithm can be used, which can be designed and selected by a person skilled in the art when applying the technical solutions disclosed in the present application according to actual conditions.

[0058] According to the fact that a pair of aircrafts is the smallest cooperative unit in aerial combat, each aerial combat unmanned aerial vehicle of ours needs to take into account at least one teammate or one opponent. In N-to-N aerial combat, there are N! possible combinations. When the scale of aerial combat increases sharply, the number of combinations will increase dramatically. In the unmanned aerial combat decision-making method disclosed in the above embodiments, an upper limit of the number of unmanned aerial combat relationship graphs is set, and a first neural network is used to configure the probability of each node appearing in each unmanned aerial combat relationship graph, so that the problem of dividing the combat relationship graph is converted into the problem of selecting the probability of each node appearing in each unmanned aerial combat relationship graph. The upper limit of the output dimension can be fixed, which can cover all combinations and adapt to the dynamic changes of unmanned aerial combat relationship graphs and nodes in them.

[0059] In some optional embodiments of the unmanned aerial combat decision-making method described above, the upper limit of the number of unmanned aerial combat relationship graphs is set, specifically, half of the number of aerial combat unmanned aerial vehicles is used as the upper limit of the number of unmanned aerial combat relationship graphs.

[0060] In some optional embodiments of the unmanned aerial combat decision-making method described above, the features of each unmanned aerial combat relationship graph are extracted, and the extracted features of each unmanned aerial combat relationship graph are fused to obtain unmanned aerial combat fusion features, specifically:

[0061] A second neural network is used to extract features of each unmanned aerial combat relationship graph, and the extracted features of each unmanned aerial combat relationship graph are fused to obtain unmanned aerial combat fusion features. Specifically, a graph convolution network can be used, which can be designed and selected by a person skilled in the art when applying the technical solutions disclosed in the present application according to actual conditions.

[0062] The graph convolution network is one of important methods capable of realizing spatial information aggregation in a graph, and the graph convolution network can reliably realize feature extraction and fusion of each unmanned aerial vehicle air combat relationship graph.

[0063] The most important thing in deep learning is to learn features, and with the increase of network layers, the features are more and more abstract, and then used for the final task. For a graph task, it is expected that a deep model can learn more abstract features from the most initial features of the graph, such as learning high-level features of a node, which integrates the features of other nodes in the graph according to the graph structure, and then uses the features for subsequent tasks.

[0064] In the graph convolution network, the Laplacian matrix of the graph is often used as a fusion matrix to aggregate adjacent node features to complete node representation update. In order to ensure that the self information is not lost in the message passing process, a self connection is added to each node in the graph to introduce the influence of the node on itself, solve the self passing problem, and a unit matrix can be directly added to the original graph adjacency matrix to obtain a new adjacency matrix, as follows:

[0065]

[0066] wherein,

[0067] H is a new node feature;

[0068] D is a degree matrix;

[0069] A is an original graph adjacency matrix;

[0070] W is a trainable parameter matrix.

[0071] In some optional embodiments, in the unmanned aerial vehicle air combat decision method described above, the unmanned aerial vehicle air combat fusion feature is used to generate a self unmanned aerial vehicle air combat strategy, specifically:

[0072] The unmanned aerial vehicle air combat fusion feature is used to generate a self unmanned aerial vehicle air combat strategy by a third neural network, including a maneuvering behavior and a striking behavior, and a corresponding strategy generation network can be used, which can be designed and selected by a person skilled in the art according to the actual situation when the technical solution disclosed in the present application is applied.

[0073] In some optional embodiments, the unmanned aerial vehicle air combat decision method described above further includes:

[0074] The self unmanned aerial vehicle air combat strategy is interacted with a simulation environment to collect samples of the self unmanned aerial vehicle air combat strategy.

[0075] The sample of the air combat strategy of the unmanned aerial vehicle of the own side is used to train the neural network in the unmanned aerial vehicle air combat decision method, and after the training is completed, the unmanned aerial vehicle air combat decision method is applied to the unmanned aerial vehicle air combat decision.

[0076] In some optional embodiments, in the unmanned aerial vehicle air combat decision method, the sample of the air combat strategy of the unmanned aerial vehicle of the own side is used to train the neural network in the unmanned aerial vehicle air combat decision method, and the training is specifically as follows:

[0077] The sample of the air combat strategy of the unmanned aerial vehicle of the own side is used to train the neural network in the unmanned aerial vehicle air combat decision method based on the reinforcement learning algorithm optimized based on the near source end strategy, and the optimization objective function is defined as follows:

[0078]

[0079] Wherein,

[0080] S t is the state at time t;

[0081] a t is the action taken in the state S t ;

[0082] theta is the network parameter of the new strategy;

[0083] theta' is the network parameter of the old strategy;

[0084] A θ (S t ,a t ) is the advantage of the state action pair (S t ,a t ) when the strategy parameter is theta.

[0085] In order to ensure a certain exploratory nature and let the intelligent agent learn a more compact relationship and filter out unimportant relationships, information entropy can be introduced as part of the optimization objective function, and the total optimization objective function can be written as:

[0086]

[0087] On the other hand, a unmanned aerial vehicle air combat decision system is provided, comprising:

[0088] The unmanned aerial vehicle air combat situation map construction module abstracts the air combat unmanned aerial vehicle situation and constructs the unmanned aerial vehicle air combat situation map;

[0089] The unmanned aerial vehicle air combat relationship graph division module divides the unmanned aerial vehicle air combat situation map into multiple unmanned aerial vehicle air combat relationship graphs;

[0090] The unmanned aerial combat fusion feature extraction module extracts features from each unmanned aerial combat relationship graph, fuses the features extracted from each unmanned aerial combat relationship graph, and obtains unmanned aerial combat fusion features.

[0091] The own unmanned aerial combat strategy generation module generates the own unmanned aerial combat strategy based on the unmanned aerial combat fusion features.

[0092] The own unmanned aerial combat strategy sample collection module interacts the own unmanned aerial combat strategy with the simulation environment, and collects samples of the own unmanned aerial combat strategy.

[0093] The unmanned aerial combat decision neural network training module trains the neural network in the unmanned aerial combat decision method by using the samples of the own unmanned aerial combat strategy.

[0094] For the unmanned aerial combat decision system disclosed in the above embodiments, since it corresponds to the unmanned aerial combat decision method disclosed in the above embodiments, the description is relatively simple, and the specific related parts can be referred to the related description in the part of the unmanned aerial combat decision method. The technical effects can also be referred to the technical effects of the related part of the unmanned aerial combat decision method, which will not be described here.

[0095] In addition, those skilled in the art should also realize that each module of the unmanned aerial combat decision system disclosed in the embodiments can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the embodiments in the present application are generally described in terms of functions. Whether the functions are performed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can choose different methods to implement the described functions for each specific application and its actual constraints, but such implementation should not be considered beyond the scope of the present application.

[0096] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other.

[0097] So far, the technical solution of the present application has been described in combination with the preferred embodiments shown in the drawings. Those skilled in the art should understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without deviating from the principles of the present application, those skilled in the art can make equivalent changes or replacements to the related technical features. The technical solutions after the changes or replacements will fall within the protection scope of the present application.

Claims

1. A method for aerial combat decision-making for unmanned aerial vehicles (UAVs), characterized in that, include: Abstracting the aerial combat situation of drones, and constructing an aerial combat situation map of drones; The aerial combat situation map of drones is divided into multiple aerial combat relationship maps of drones; Feature extraction is performed on the aerial combat relationship maps of each UAV, and the extracted features are fused to obtain the UAV aerial combat fusion feature. Based on the characteristics of UAV aerial combat fusion, generate your own UAV aerial combat strategy; Interact with the simulation environment to collect samples of the player's own UAV air combat strategy; Using samples of our own UAV air combat strategies, we train the neural network in the UAV air combat decision-making method. The process involves extracting features from the aerial combat relationship maps of each UAV, and then fusing the extracted features to obtain fused UAV aerial combat features. Specifically: The second neural network is used to extract features from the aerial adversarial relationship graphs of each UAV, and the features extracted from the aerial adversarial relationship graphs of each UAV are fused to obtain the UAV aerial adversarial fusion features. The second neural network adopts a graph convolutional network. In the graph convolutional network, the Laplacian matrix of the graph is used as the fusion matrix to aggregate the features of adjacent nodes to complete the node representation update. Self-connection is added to each node in the graph to introduce the influence of the node itself on itself, solve the self-propagation problem, and the new adjacency matrix is ​​obtained by directly adding the identity matrix to the original graph adjacency matrix. The process of training the neural network in the UAV air combat decision-making method using samples of the player's own UAV air combat strategy is as follows: Using samples of our own UAV air combat strategies, a reinforcement learning algorithm based on near-source policy optimization is used to train the neural network in the UAV air combat decision-making method. The optimization objective function is defined as: in, S t The state at time t; a t For state S t The following actions were taken; θ represents the network parameters of the new strategy; θ′ represents the network parameters of the old strategy; A θ (S t ,a t ) represents a state-action pair (S) with policy parameter θ. t ,a t Advantages of ) Introducing information entropy as part of the optimization objective function, the overall optimization objective function is: The aerial combat situation map of drones is divided into multiple aerial combat relationship maps of drones, specifically as follows: Set an upper limit on the number of drone aerial combat relationship diagrams; Based on the UAV aerial combat situation map, the probability of each node appearing in each UAV aerial combat relationship map is configured using the first neural network, thereby obtaining the node information and its connections in each UAV aerial combat relationship map.

2. The UAV aerial combat decision-making method according to claim 1, characterized in that, The abstraction of the aerial combat situation of UAVs and the construction of an aerial combat situation map of UAVs are specifically as follows: Using aerial combat drones as nodes, node information includes the status information of the corresponding aerial combat drone and the status information of the target it carries. By connecting the nodes corresponding to two aerial combat drones that can obtain each other's status information, an aerial combat situation map of drones is constructed.

3. The UAV aerial combat decision-making method according to claim 2, characterized in that, The status information of the aerial combat drone includes the drone's position, speed, and yaw angle; The status information of the aerial combat drone carrying the strike object includes the strike speed and survival status of the strike object carried by the aerial combat drone.

4. The UAV aerial combat decision-making method according to claim 3, characterized in that, The upper limit for the number of drone aerial combat relationship diagrams is specifically set at half the number of drones involved in aerial combat.

5. The UAV aerial combat decision-making method according to claim 4, characterized in that, The method for generating an air combat strategy for one's own drones based on the fusion characteristics of drone air combat is as follows: Based on the fusion characteristics of UAV aerial combat, a third neural network is used to generate an aerial combat strategy for one's own UAV.

6. A UAV aerial combat decision-making system for implementing the UAV aerial combat decision-making method of claim 5, characterized in that, include: The UAV aerial combat situation map construction module abstracts the UAV aerial combat situation and constructs an UAV aerial combat situation map. The UAV aerial combat relationship diagram division module divides the UAV aerial combat situation map into multiple UAV aerial combat relationship diagrams. The UAV aerial combat fusion feature extraction module extracts features from the aerial combat relationship diagrams of various UAVs and fuses the extracted features to obtain the UAV aerial combat fusion features. The module for generating air combat strategies for friendly drones generates air combat strategies for friendly drones based on the fusion characteristics of air combat strategies for drones. The module for collecting samples of the air combat strategies of the user's UAVs interacts with the simulation environment to collect samples of the user's air combat strategies. The training module for the UAV air combat decision neural network uses samples of the UAV air combat strategy to train the neural network in the UAV air combat decision method. The process involves extracting features from the aerial combat relationship maps of each UAV, and then fusing the extracted features to obtain fused UAV aerial combat features. Specifically: The second neural network is used to extract features from the aerial adversarial relationship graphs of each UAV, and the features extracted from the aerial adversarial relationship graphs of each UAV are fused to obtain the UAV aerial adversarial fusion features. The second neural network adopts a graph convolutional network. In the graph convolutional network, the Laplacian matrix of the graph is used as the fusion matrix to aggregate the features of adjacent nodes to complete the node representation update. Self-connection is added to each node in the graph to introduce the influence of the node itself on itself, solve the self-propagation problem, and the new adjacency matrix is ​​obtained by directly adding the identity matrix to the original graph adjacency matrix. The process of training the neural network in the UAV air combat decision-making method using samples of the player's own UAV air combat strategy is as follows: Using samples of our own UAV air combat strategies, a reinforcement learning algorithm based on near-source policy optimization is used to train the neural network in the UAV air combat decision-making method. The optimization objective function is defined as: in, S t The state at time t; a t For state S t The following actions were taken; θ represents the network parameters of the new strategy; θ′ represents the network parameters of the old strategy; A θ (S t ,a t ) represents a state-action pair (S) with policy parameter θ. t ,a t The advantages of this approach are: Information entropy is introduced as part of the optimization objective function, and the overall optimization objective function is:

Images