Low-altitude airspace unmanned aerial vehicle formation autonomous collision avoidance decision-making method

Abstract

The invention provides a low-altitude airspace unmanned aerial vehicle formation autonomous collision avoidance decision-making method comprising the steps that an autonomous collision avoidance decision-making model of an unmanned aerial vehicle is constructed, and the autonomous collision avoidance decision-making model is composed of an entity behavior prediction module, an observation information reconstruction module and a decision-making generation module; training the autonomous collision avoidance decision model, and configuring a trained autonomous collision avoidance decision model for each unmanned aerial vehicle in the unmanned aerial vehicle formation; each unmanned aerial vehicle in the unmanned aerial vehicle formation obtains environment state information in a sensing range of the unmanned aerial vehicle in real time, and the environment state information comprises motion entity state information; and based on the motion entity state information, an entity behavior prediction module, an observation information reconstruction module and a decision generation module are sequentially utilized to obtain a specific collision avoidance action of each unmanned aerial vehicle. According to the autonomous collision avoidance method, a traditional decision method based on MARL and an entity behavior prediction technology based on spatial-temporal feature extraction are combined, and early perception and avoidance of sudden collision threats can be realized.

Description

Technical Field

[0001] This invention relates to the field of autonomous collision avoidance decision-making technology for unmanned aerial vehicles (UAVs), and specifically to an autonomous collision avoidance decision-making method for UAV formations in low-altitude airspace. Background Technology

[0002] With the rise of the "low-altitude economy," low-altitude airspace is gradually opening up, leading to a significant increase in low-altitude flight activities. The mixed operation of unmanned aerial vehicles (UAVs) with other types of aircraft will become the norm for low-altitude traffic in the future. However, the current air traffic management system for low-altitude traffic is still incomplete, resulting in frequent UAV collisions and unauthorized flights. To ensure the safe and stable operation of UAV swarms in low-altitude environments, providing them with an efficient and robust autonomous collision avoidance decision-making method is crucial.

[0003] Breakthroughs in artificial intelligence technology have injected new momentum into the research of collision avoidance decision-making methods. Decision-making methods based on multi-agent reinforcement learning (MARL) have gradually become a research focus. These methods rely on the "perception-cognition-decision" loop established through continuous interaction between agents and the environment, achieving end-to-end autonomous decision-making. In particular, the multi-agent deep deterministic policy gradient (MADDPG) algorithm, based on the centralized training-decentralized execution (CTDE) paradigm, maintains the flexibility of distributed decision-making while addressing environmental non-stationarity issues by evaluating global information introduced into the network, effectively coordinating the strategies among formation members. Therefore, it is suitable for autonomous collision avoidance decision-making in UAV formations. However, entities such as "black-flying" aircraft in low-altitude environments exhibit unexpected behaviors (such as sudden turns or accelerations), potentially causing sudden collision threats. In current MARL methods, UAVs rely solely on currently perceived environmental state information for decision-making, making it difficult to accurately understand the complex behavioral patterns of these entities. This results in a certain lag in responding to sudden collision threats, and the accumulation of this lag will affect the accuracy and robustness of UAV decision-making. Therefore, how to accurately predict entity behavior and use precisely predicted future entity state information to assist MARL decision-making, thereby achieving early perception and avoidance of sudden collision threats, has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the present invention provides an autonomous collision avoidance decision-making method for low-altitude airspace UAV formations to solve the problems of poor decision-making accuracy in the prior art, which relies solely on the currently perceived environmental state information for UAVs to make decisions.

[0005] This invention provides a behavior recognition method based on single-view UAV swarm trajectory, including:

[0006] S1: Construct an autonomous collision avoidance decision model for UAVs, wherein the autonomous collision avoidance decision model consists of an entity behavior prediction module, an observation information reconstruction module, and a decision generation module;

[0007] The entity behavior prediction module is used to perform the following steps:

[0008] S101: Construct a spatiotemporal situation map based on the spatial situation map of the historical time step and the spatial situation map of the current time step, and use a graph convolutional neural network to extract the spatial features of each spatial situation map in the spatiotemporal situation map;

[0009] S102: After the spatial features of each spatial situation map, residual connections are used and stacked along the time dimension to obtain the residual spatial feature matrix;

[0010] S103: The residual space feature matrix is ​​transformed into a transformation matrix through dimensionality transformation, and a multi-scale temporal convolutional network is used to extract the temporal features of the transformation matrix to obtain a spatiotemporal feature matrix that contains both temporal and spatial features.

[0011] S104: Input the spatiotemporal feature matrix into a fully connected layer and a long short-term memory network layer to predict the future state information of the entity;

[0012] The observation information reconstruction module is used to flatten the predicted future state information of the entity and splice it with the environmental state information observed by each UAV in the UAV formation at the current time step to obtain the reconstructed observation information of each UAV.

[0013] The decision generation module is constructed from a multilayer perceptron and is used to obtain the collision avoidance action of the UAV at the current time step based on the reconstructed observation information.

[0014] S2: Train the autonomous collision avoidance decision model based on the MADDPG algorithm, and configure a trained autonomous collision avoidance decision model for each drone in the drone formation;

[0015] S3: Each drone in the drone formation acquires environmental state information within its perception range in real time and uses it as its raw observation information. The environmental state information includes the state information of moving entities.

[0016] S4: Based on the state information of the moving entity, use the entity behavior prediction module in the autonomous collision avoidance decision model to predict the future state information of the moving entity in the environment;

[0017] S5: Based on the future state information of the entity, the observation information of each UAV in the UAV formation is reconstructed using the observation information reconstruction module in the autonomous collision avoidance decision model;

[0018] S6: Based on the reconstructed observation information of each UAV, the specific collision avoidance actions of each UAV are obtained using the decision generation module in the autonomous collision avoidance decision model.

[0019] Preferably, in S101, the time step The spatial situation map is represented as , Represents a set of nodes. Let represent a set of edges, where nodes are moving entities in the environment, node features are the state information of the moving entities, and edges are the spatial interaction relationships between entities;

[0020] The spatiotemporal situation diagram is obtained by stacking the spatial situation diagram of the current time step and the spatial situation diagrams of the past T-1 time steps along the time dimension. The spatiotemporal situation diagram is denoted as... In the formula, Indicates time step Spatial situation map, This indicates the number of time steps contained in the spatiotemporal situation map.

[0021] Further preferably, in S101, the graph convolution formula for the (l+1)th layer of the graph convolutional neural network is as follows:

[0022] ;

[0023] In the formula, , The weighted adjacency matrix representing the spatial situation map. Represents the identity matrix. for The degree matrix, and They represent the first time. Layer and first The spatial feature matrix extracted after layer graph convolution calculation. Indicates the first The learnable weight matrix of the layer. Let LeakyReLU be the activation function, where the weighted adjacency matrix is... elements in The formula is as follows:

[0024] ;

[0025] In the formula, Representing entities With entity The Euclidean distance between them This indicates the maximum observation distance of the drone.

[0026] Further optimization, in S102, the formula for calculating the residual space feature matrix obtained after residual connection and stacking along the time dimension is as follows:

[0027] ;

[0028] In the formula, This represents the residual space feature matrix obtained after residual concatenation and stacking along the time dimension. Representing a spatiotemporal situation map The original node feature matrix, This indicates the number of time steps contained in the spatiotemporal situation diagram. Indicates the number of nodes. Represents the dimension of node features. It is a spatial feature matrix obtained by stacking the spatial features extracted at each time step along the time dimension.

[0029] Further preferably, in S103, the multi-scale temporal convolutional network includes two temporal convolutional networks with different kernel sizes and dilation rates, used to extract the temporal features of the transformation matrix from two different scales, respectively, to obtain two spatiotemporal feature matrices that simultaneously contain temporal and spatial features. and Then, a gated feature fusion mechanism is used to combine the two spatiotemporal feature matrices. and By fusing these features, a spatiotemporal feature matrix that simultaneously contains both temporal and spatial characteristics is obtained.

[0030] Further optimization, in S2, the training method of the autonomous collision avoidance decision model is as follows:

[0031] S21: Construct and initialize the network structure required for the MADDPG algorithm. The network structure includes an Actor network, a Target Actor network, a Critic network, and a Target Critic network. The Actor network and the Target Actor network are both autonomous collision avoidance decision models. The Critic network and the Target Critic network are both constructed using MLP, and their network structures are completely identical. The inputs of the Critic network and the Target Critic network include environmental state information observed by all UAVs, action information of all UAVs, and one-hot encoded information of the ID of a single UAV in the UAV formation. The output of the Critic network and the Target Critic network is a Q-value, which is used to evaluate the quality of the collision avoidance action taken by the single UAV under the current environmental state.

[0032] S22: At each time step, the UAV formation interacts with the training environment once and obtains a set of empirical data, which is stored in the experience replay pool. A set of time steps is defined for each interaction before updating the model parameters. Each set of empirical data includes the observation set of the UAV formation at the current time step. The constructed spatiotemporal situation map Predicted future state information of entities A set of actions performed by drone formations Rewards and feedback obtained from the environment And the observation set of the drone formation at the next time step ;

[0033] S23: When in the update node, sample a batch of experience data from the experience replay pool as training samples for training, and update the network parameters of the Actor network, Target Actor network, Critic network, and Target Critic network.

[0034] S24: Repeat the network parameter update process of S23 until the set maximum number of time steps is reached to obtain the trained autonomous collision avoidance decision model.

[0035] Further optimization, in S23, the Actor network parameters are updated in the following way:

[0036] ;

[0037] In the formula, Represents an Actor network. Represents the network parameters of the Actor network. This represents the learning rate of the Actor network. The formula for the policy gradient of an Actor network;

[0038] The parameters of the Critic network are updated by minimizing the following loss function:

[0039] ;

[0040] In the formula, This represents the network parameters of the Critic network. This represents the reward feedback obtained from the environment. This indicates the observation set for the next time step of the drone formation. express one-hot encoding, ,in, express The reward obtained through interaction with the environment, where γ represents the discount factor. Indicates the Target Critic network. Represents the Target Actor network. Indicates the next time step Reconstructed observation information;

[0041] Both the Target Actor network and the Target Critic network use a soft update method for parameter updates, as follows:

[0042] ;

[0043] ;

[0044] In the formula, Indicates the soft update coefficient. This represents the network parameters of the Target Critic network. This represents the network parameters of the TargetActor network.

[0045] Further optimization, in S3, the environmental state information includes the state information of moving entities, the position of static obstacles, and the distance between the UAV and the static obstacles and moving entities.

[0046] The autonomous collision avoidance decision-making method for low-altitude UAV formations provided by this invention combines traditional MARL-based decision-making methods with entity behavior prediction technology based on spatiotemporal feature extraction. This enables UAVs to make decisions based on the environmental state information observed at the current time step, while also incorporating forward-looking future entity state information, thereby proactively sensing and responding to potential future collision threats. Entity behavior prediction is achieved through spatiotemporal feature extraction. In terms of spatial feature extraction, the situation of moving entities at each time step is modeled as a spatial situation map, which clearly represents the spatial interaction relationships between entities. A graph convolutional neural network effectively extracts its spatial features. For temporal feature extraction, a multi-scale temporal convolutional network is used to extract temporal features in the temporal dimension with receptive fields of different scales, thus avoiding a decrease in prediction accuracy due to the loss of local information. Finally, fully connected layers and LSTM layers are used to predict the future state information of entities. The input comprehensively considers the spatial features of the entity situation at each time step and the temporal features of the entity situation's evolution over time, thus enabling more accurate prediction of the future state information of moving entities. Attached Figure Description

[0047] Figure 1 The flowchart shows the autonomous collision avoidance decision-making method for low-altitude airspace UAV formations provided by this invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Entity behavior in low-altitude environments exhibits strong temporal dependence and spatial coupling, forming complex spatiotemporal interactions between entities. However, mainstream methods based on RNNs and Transformers can only capture the temporal dependence of environmental state information in the time dimension, neglecting the spatial interactions between entities. Therefore, it is necessary to comprehensively consider both temporal and spatial characteristics to improve prediction accuracy. Subsequently, the accurate prediction of future entity state information can assist MARL decision-making, enabling early perception and avoidance of sudden collision threats.

[0050] like Figure 1 As shown, this invention provides an autonomous collision avoidance decision-making method for low-altitude airspace UAV formations, comprising the following steps:

[0051] S1: Construct an autonomous collision avoidance decision model for UAVs, wherein the autonomous collision avoidance decision model consists of an entity behavior prediction module, an observation information reconstruction module, and a decision generation module;

[0052] The entity behavior prediction module is used to perform the following steps:

[0053] S101: Construct a spatiotemporal situation map based on the spatial situation map of the historical time step and the spatial situation map of the current time step, and use a graph convolutional neural network (GCN) to extract the spatial features of each spatial situation map in the spatiotemporal situation map;

[0054] Among them, the spatial situation map of historical time steps can be stored in the historical spatial situation map cache area; time steps The spatial situation map is represented as , Represents a set of nodes. Let represent a set of edges, where nodes are moving entities in the environment, and node features are the state information of the moving entities (the state information includes the entity's position, velocity, attitude angle, etc.), and edges are the spatial interaction relationships between entities.

[0055] The spatiotemporal situation diagram is obtained by stacking the spatial situation diagram of the current time step with the spatial situation diagrams of the past T-1 time steps along the time dimension. The spatiotemporal situation diagram is denoted as... In the formula, Indicates time step Spatial situation map, This indicates the number of time steps contained in the spatiotemporal situation diagram;

[0056] The graph convolution formula for the (l+1)th layer of the graph convolutional neural network is as follows:

[0057] ;

[0058] In the formula, , The weighted adjacency matrix representing the spatial situation map. Represents the identity matrix. for The degree matrix, where the diagonal elements are the degrees of the nodes. Used to normalize the adjacency matrix to accelerate the learning process of GCN. and They represent the first time. Layer and first The spatial feature matrix extracted after layer graph convolution calculation. Indicates the first The learnable weight matrix of layer +1, Let LeakyReLU be the activation function, where the weighted adjacency matrix is... elements in The formula is as follows:

[0059] ;

[0060] In the formula, Representing entities With entity The Euclidean distance between them This represents the maximum observation distance of the drone; the above formula is used to reflect: when the entity With entity When the Euclidean distance between the two entities is less than or equal to the maximum observation distance of the UAV, there is spatial interaction between the two entities. The closer they are, the higher the degree of spatial interaction.

[0061] S102: After the spatial features of each spatial situation map, residual connections are used and stacked along the time dimension to obtain the residual spatial feature matrix. :

[0062] ;

[0063] In the formula, This represents the residual space feature matrix obtained after residual concatenation and stacking along the time dimension. Representing a spatiotemporal situation map The node feature matrix, This indicates the number of time steps contained in the spatiotemporal situation diagram. Indicates the number of nodes. Represents the dimension of node features. It is a spatial feature matrix obtained by stacking the spatial features extracted at each time step along the time dimension;

[0064] Among these features, residual connections can prevent information loss due to excessive network depth.

[0065] S103: Transform the residual space feature matrix through dimensionality transformation. Transformation into a transformation matrix The transformation matrix is ​​extracted using a multi-scale temporal convolutional network (MS-TCN). The temporal characteristics are used to obtain a spatiotemporal feature matrix that simultaneously contains both temporal and spatial features. ;

[0066] The multi-scale temporal convolutional network includes two temporal convolutional networks with different kernel sizes and dilation rates. ), used to extract the transformation matrix from two different scales respectively. The temporal characteristics are used to obtain two spatiotemporal feature matrices that simultaneously contain both temporal and spatial features. and Then, a gated feature fusion mechanism is used to combine the two spatiotemporal feature matrices. and By fusing the two features, a spatiotemporal feature matrix that simultaneously contains both temporal and spatial features is obtained. ;

[0067] Among them, for the first in MS-TCN indivual The time feature extraction process is represented as follows:

[0068] ;

[0069] In the formula, Indicates the first indivual , Represents the transformation matrix Through the first indivual The resulting spatiotemporal feature matrix contains both temporal and spatial features.

[0070] The feature fusion formula of the gated feature fusion mechanism is as follows:

[0071] ;

[0072] In the formula, This represents the spatiotemporal feature matrix obtained after feature fusion. This represents element-wise multiplication. The weight matrix represents the feature fusion. The calculation method is as follows:

[0073] ;

[0074] In the formula, This represents the sigmoid function. and Represents the learnable parameters. Indicates the bias term;

[0075] S104: The spatiotemporal feature matrix... The input is passed to a fully connected (FC) layer and a long short-term memory (LSTM) network layer to predict the future state information of the entity:

[0076] ;

[0077] In the formula, This represents the predicted future state information of an entity. Indicates the number of nodes. Indicates the number of future time steps. Dimensions representing information about the future state of an entity;

[0078] The observation information reconstruction module is used to perform the following steps:

[0079] The predicted future state information of the entities is flattened and then stitched together with the environmental state information observed by each drone in the drone formation at the current time step to obtain the reconstructed observation information for each drone. Specifically, for the first drone in the drone formation... drone Its reconstructed observation information The method to obtain it is as follows:

[0080] ;

[0081] In the formula, This indicates a splicing operation. Indicates the flattening operation. Indicates the first drone The original environmental state information observed at the current time step;

[0082] The decision generation module is constructed from a multilayer perceptron (MLP) for the first stage of a UAV formation. drone The input to this module is the reconstructed observation information. The output is the collision avoidance action of the drone at the current time step. :

[0083] ;

[0084] S2: Train the autonomous collision avoidance decision model based on the MADDPG algorithm, and configure a trained autonomous collision avoidance decision model for each drone in the drone formation;

[0085] The training method for the autonomous collision avoidance decision-making model is as follows:

[0086] S21: Construct and initialize the network structure required for the MADDPG algorithm. The network structure includes an Actor network, a Target Actor network, a Critic network, and a Target Critic network. The Actor network and the Target Actor network are both autonomous collision avoidance decision models. The Critic network and the Target Critic network are both constructed using MLP, and their network structures are completely identical. The inputs of the Critic network and the Target Critic network include environmental state information observed by all UAVs, action information of all UAVs, and one-hot encoded information of the ID of a single UAV in the UAV formation. The output of the Critic network and the Target Critic network is a Q-value, which is used to evaluate the quality of the collision avoidance action taken by the single UAV under the current environmental state.

[0087] S22: At each time step, the drone formation interacts with the training environment once and obtains a set of experience data, which is stored in the experience replay pool. The model parameters are updated once every number of time steps after each interaction.

[0088] The experience replay pool is used to store and sample empirical data for model training. Each set of empirical data stored in the experience replay pool includes the observation set of the UAV formation at the current time step. The constructed spatiotemporal situation map Predicted future state information of entities A set of actions performed by drone formations Rewards and feedback obtained from the environment And the observation set of the drone formation at the next time step ;

[0089] S23: When in the update node, sample a batch of experience data from the experience replay pool as training samples for training, and update the network parameters of the Actor network, Target Actor network, Critic network, and Target Critic network.

[0090] The Actor network parameters are updated as follows:

[0091] ;

[0092] In the formula, Represents an Actor network. Represents the network parameters of the Actor network. This represents the learning rate of the Actor network. This represents the policy gradient formula for the Actor network. The specific calculation method is as follows:

[0093] ;

[0094] In the formula, This indicates the calculation of mathematical expectation. This indicates the number of drones in the drone formation, and D represents the experience replay pool. These represent the set of UAV observations and the set of actions sampled from the experience replay pool, respectively. Represents the Critic network;

[0095] The parameters of the Critic network are updated by minimizing the following loss function:

[0096] ;

[0097] In the formula, This represents the network parameters of the Critic network. This represents the reward feedback obtained from the environment. This indicates the observation set for the next time step of the drone formation. express one-hot encoding, ,in, express The reward obtained through interaction with the environment, where γ represents the discount factor. Indicates the Target Critic network. Represents the Target Actor network. Indicates the next time step Reconstructed observation information;

[0098] Both the Target Actor network and the Target Critic network use a soft update method for parameter updates. The specific parameter update method is as follows:

[0099] ;

[0100] ;

[0101] In the formula, Indicates the soft update coefficient. This represents the network parameters of the Target Critic network. This represents the network parameters of the TargetActor network;

[0102] S24: Repeat the network parameter update process of S23 until the set maximum number of time steps is reached to obtain the trained autonomous collision avoidance decision model;

[0103] S3: Each drone in the drone formation acquires environmental state information within its perception range in real time and uses it as its raw observation information. The environmental state information includes the state information of moving entities.

[0104] The environmental status information within the perception range of each drone in the drone formation can be obtained through onboard sensors on the drone; the environmental status information includes the status information of moving entities (including position, speed, attitude angle, etc.), the position of static obstacles, and the distance between the drone and the static obstacle or moving entity.

[0105] S4: Based on the state information of the moving entity, use the entity behavior prediction module in the autonomous collision avoidance decision model to predict the future state information of the moving entity in the environment;

[0106] S5: Based on the future state information of the entity, the observation information of each UAV in the UAV formation is reconstructed using the observation information reconstruction module in the autonomous collision avoidance decision model;

[0107] S6: Based on the reconstructed observation information of each UAV, the specific collision avoidance actions of each UAV are obtained using the decision generation module in the autonomous collision avoidance decision model.

[0108] This autonomous collision avoidance decision-making method for low-altitude UAV formations integrates entity behavior prediction technology based on spatiotemporal feature extraction, building upon the multi-agent reinforcement learning (MARL) decision-making approach. In addition to the environmental state information observed by the UAV at the current time step, it incorporates forward-looking future entity state information for decision-making, thereby anticipating potential future collision threats and learning effective avoidance strategies through training. For entity behavior prediction, spatial feature extraction is achieved through a graph convolutional neural network (GCN). The situation of moving entities at each time step is modeled as a spatial situation map, clearly representing the spatial interaction relationships between entities. The weighted adjacency matrix required by GCN is constructed based on the distance between entities, clearly representing different degrees of spatial interaction between entities. Temporal feature extraction is achieved through a multi-scale temporal convolutional network (MS-TCN). MS-TCN can extract temporal features in the temporal dimension with receptive fields of different scales, avoiding the decrease in prediction accuracy caused by the loss of local information. The spatial features of the entity situation at each time step and the temporal features of the entity situation evolving over time enable more accurate prediction of the future state information of moving entities. Prediction is achieved through fully connected layers and LSTM layers. The autonomous collision avoidance decision model is trained based on the MADDPG algorithm. Its centralized training and decentralized execution paradigm maintains the flexibility of distributed decision-making. At the same time, by introducing global information into the Critic network during training, the autonomous collision avoidance decision model learns to coordinate the strategies among formation members, thus avoiding conflicts between the UAV and the decisions of other formation members when the UAV makes decisions using the trained autonomous collision avoidance decision model.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for autonomous collision avoidance decision-making in low-altitude airspace unmanned aerial vehicle (UAV) formations, characterized in that, include: S1: Construct an autonomous collision avoidance decision model for UAVs, wherein the autonomous collision avoidance decision model consists of an entity behavior prediction module, an observation information reconstruction module, and a decision generation module; The entity behavior prediction module is used to perform the following steps: S101: Construct a spatiotemporal situation map based on the spatial situation map of the historical time step and the spatial situation map of the current time step, and use a graph convolutional neural network to extract the spatial features of each spatial situation map in the spatiotemporal situation map; S102: After the spatial features of each spatial situation map, residual connections are used and stacked along the time dimension to obtain the residual spatial feature matrix; S103: The residual space feature matrix is ​​transformed into a transformation matrix through dimensionality transformation, and a multi-scale temporal convolutional network is used to extract the temporal features of the transformation matrix to obtain a spatiotemporal feature matrix that contains both temporal and spatial features. S104: Input the spatiotemporal feature matrix into a fully connected layer and a long short-term memory network layer to predict the future state information of the entity; The observation information reconstruction module is used to flatten the predicted future state information of the entity and splice it with the environmental state information observed by each UAV in the UAV formation at the current time step to obtain the reconstructed observation information of each UAV. The decision generation module is constructed from a multilayer perceptron and is used to obtain the collision avoidance action of the UAV at the current time step based on the reconstructed observation information. S2: Train the autonomous collision avoidance decision model based on the MADDPG algorithm, and configure a trained autonomous collision avoidance decision model for each drone in the drone formation; S3: Each drone in the drone formation acquires environmental state information within its perception range in real time and uses it as its raw observation information. The environmental state information includes the state information of moving entities. S4: Based on the state information of the moving entity, use the entity behavior prediction module in the autonomous collision avoidance decision model to predict the future state information of the moving entity in the environment; S5: Based on the future state information of the entity, the observation information of each UAV in the UAV formation is reconstructed using the observation information reconstruction module in the autonomous collision avoidance decision model; S6: Based on the reconstructed observation information of each UAV, the specific collision avoidance actions of each UAV are obtained using the decision generation module in the autonomous collision avoidance decision model.

2. The autonomous collision avoidance decision-making method for low-altitude airspace UAV formations according to claim 1, characterized in that, In S101, the time step The spatial situation map is represented as , Represents a set of nodes. Let represent a set of edges, where nodes are moving entities in the environment, node features are the state information of the moving entities, and edges are the spatial interaction relationships between entities; The spatiotemporal situation diagram is obtained by stacking the spatial situation diagram of the current time step and the spatial situation diagrams of the past T-1 time steps along the time dimension. The spatiotemporal situation diagram is denoted as... In the formula, Indicates time step Spatial situation map, This indicates the number of time steps contained in the spatiotemporal situation map.

3. The autonomous collision avoidance decision-making method for low-altitude airspace UAV formations according to claim 1, characterized in that, In S101, the graph convolution formula for the (l+1)th layer of the graph convolutional neural network is as follows: ； In the formula, , The weighted adjacency matrix representing the spatial situation map. Represents the identity matrix. for The degree matrix, and They represent the first time. Layer and first The spatial feature matrix extracted after layer graph convolution calculation. Indicates the first The learnable weight matrix of the layer. Let LeakyReLU be the activation function, where the weighted adjacency matrix is... elements in The formula is as follows: ； In the formula, Representing entities With entity The Euclidean distance between them This indicates the maximum observation distance of the drone.

4. The autonomous collision avoidance decision-making method for low-altitude airspace UAV formations according to claim 1, characterized in that, In S102, the formula for calculating the residual space feature matrix obtained after residual connection and stacking along the time dimension is as follows: ； In the formula, This represents the residual space feature matrix obtained after residual concatenation and stacking along the time dimension. Representing a spatiotemporal situation map The original node feature matrix, This indicates the number of time steps contained in the spatiotemporal situation diagram. Indicates the number of nodes. Represents the dimension of node features. It is a spatial feature matrix obtained by stacking the spatial features extracted at each time step along the time dimension.

5. The autonomous collision avoidance decision-making method for low-altitude airspace UAV formations according to claim 1, characterized in that, In S103, the multi-scale temporal convolutional network includes two temporal convolutional networks with different kernel sizes and dilation rates, used to extract the temporal features of the transformation matrix from two different scales, respectively, to obtain two spatiotemporal feature matrices that simultaneously contain temporal and spatial features. and Then, a gated feature fusion mechanism is used to combine the two spatiotemporal feature matrices. and By fusing these features, a spatiotemporal feature matrix that simultaneously contains both temporal and spatial characteristics is obtained.

6. The autonomous collision avoidance decision-making method for low-altitude airspace UAV formations according to claim 1, characterized in that, In S2, the training method for the autonomous collision avoidance decision model is as follows: S21: Construct and initialize the network structure required for the MADDPG algorithm. The network structure includes an Actor network, a Target Actor network, a Critic network, and a Target Critic network. The Actor network and the Target Actor network are both autonomous collision avoidance decision models. The Critic network and the Target Critic network are both constructed using MLP, and their network structures are completely identical. The inputs of the Critic network and the Target Critic network include environmental state information observed by all UAVs, action information of all UAVs, and one-hot encoded information of the ID of a single UAV in the UAV formation. The output of the Critic network and the Target Critic network is a Q-value, which is used to evaluate the quality of the collision avoidance action taken by the single UAV under the current environmental state. S22: At each time step, the UAV formation interacts with the training environment once and obtains a set of empirical data, which is stored in the experience replay pool. A set of time steps is defined for each interaction before updating the model parameters. Each set of empirical data includes the observation set of the UAV formation at the current time step. The constructed spatiotemporal situation map Predicted future state information of entities A set of actions performed by drone formations Reward feedback obtained from the environment And the observation set of the drone formation at the next time step ; S23: When in the update node, sample a batch of experience data from the experience replay pool as training samples for training, and update the network parameters of the Actor network, Target Actor network, Critic network, and Target Critic network. S24: Repeat the network parameter update process of S23 until the set maximum number of time steps is reached to obtain the trained autonomous collision avoidance decision model.

7. The autonomous collision avoidance decision-making method for low-altitude airspace UAV formations according to claim 6, characterized in that, In S23, the Actor network parameters are updated as follows: ； In the formula, Represents an Actor network. Represents the network parameters of the Actor network. This represents the learning rate of the Actor network. The formula for the policy gradient of an Actor network; The parameters of the Critic network are updated by minimizing the following loss function: ； In the formula, This represents the network parameters of the Critic network. This represents the reward feedback obtained from the environment. This indicates the observation set for the next time step of the drone formation. express one-hot encoding, ,in, express The reward obtained through interaction with the environment, where γ represents the discount factor. Indicates the Target Critic network. Represents the Target Actor network. Indicates the next time step Reconstructed observation information; Both the Target Actor network and the Target Critic network use a soft update method for parameter updates, as follows: ； ； In the formula, Indicates the soft update coefficient. This represents the network parameters of the Target Critic network. This represents the network parameters of the Target Actor network.

8. The autonomous collision avoidance decision-making method for low-altitude airspace UAV formations according to claim 1, characterized in that, In S3, environmental status information includes the status information of moving entities, the position of static obstacles, and the distance between the drone and static obstacles and moving entities.

Images