Aircraft 4D trajectory prediction method and system based on double-attention space-time diagram convolution

Abstract

The invention discloses an aircraft 4D trajectory prediction method and system based on double-attention space-time diagram convolution, and belongs to the technical field of intelligent time sequence prediction. According to the method, the self-attention mechanism is utilized to reconstruct the predefined adjacent matrix, and optimization is continuously carried out along with network training so as to improve generalization of the network. The problem that the distance-based predefined adjacency matrix cannot fully reflect the potential relationship between the aircrafts in the dynamic scene is effectively solved. On the basis of a learnable adjacency matrix based on a self-attention mechanism, the correlation between nodes is further aggregated by using the graph attention network, and additional feature extraction is performed on the space-time diagram based on the powerful global information aggregation capability of the graph attention network, so that the model can capture more potential relationships between the nodes, and the accuracy of feature extraction is improved. And the problem that the correlation between the nodes cannot be effectively aggregated by the traditional graph convolution operation is improved. And the prediction accuracy of the future trajectory of the aircraft is further improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence time series prediction technology, and in particular to a method and system for aircraft 4D trajectory prediction based on dual-attention spatiotemporal graph convolution. Background Art

[0002] ICAO released the sixth edition of the Global Air Navigation Plan in 2019, which clearly stated that "Trajectory Based Operation" (TBO) will be the overall integration and ultimate goal of various leads in the "Aviation System Block Upgrade", and plans to promote and apply it globally after 2031. The transition of air traffic management from "Clearance Based Operation" (CBO) to TBO has brought new challenges to the 4D trajectory prediction function and performance. As the core technology for tasks such as conflict detection and resolution, abnormal aircraft behavior monitoring, dense flight area control, and arrival and departure sorting, how to accurately and efficiently predict future trajectories is one of the main difficulties in current research. The Federal Aviation Administration of the United States defines 4D trajectory as a four-dimensional description of the flight path of an aircraft, including longitude, latitude, altitude and time. Due to airspace congestion, bad weather, emergencies and other reasons, the actual flight process of an aircraft sometimes does not follow the pre-planned route. These emergencies reduce the predictability of air traffic, increase the difficulty of air traffic management, and seriously hinder the efficient operation of the air traffic management system. If the future trajectory of an aircraft can be predicted in advance, it will help managers make decisions in advance to deal with emergencies, thereby ensuring the safety and efficiency of the flight. Therefore, it is very necessary to predict the 4D trajectory of the aircraft.

[0003] In the technical solution disclosed in the document "Terminal Area Aircraft Track Prediction Based on Attention Seq2Seq Model [J]. Science, Technology and Engineering, 2024, 24(09): 3882-3895", an attention mechanism and an exponential decay sampling method in teacher supervision are introduced to propose an airport terminal area track prediction model based on a sequence-to-sequence framework. The sequence-to-sequence framework realizes multi-step prediction, the attention mechanism improves the prediction accuracy of the decoder, and the exponential decay sampling method accelerates the convergence of the model in the training phase, which improves the generalization of the prediction model to a certain extent. However, this method does not solve the problem that the spatial correlation between aircraft is difficult to capture, and cannot be actually applied in complex scenarios.

[0004] A data-driven 4D trajectory prediction model is proposed in the paper "Ma L, Meng X, Wu Z. Data-Driven 4D Trajectory Prediction Model Using Attention-TCN-GRU [J]. Aerospace, 2024, 11 (4): 313.". First, a spatial gap filling method is proposed to reconstruct the trajectory of each aircraft, thereby generating a high-quality trajectory dataset with consistent time intervals and no noise. Then, a hybrid neural network based on the seq2seq model is designed, named Attention-TCN-GRU, which includes an encoding module for extracting features from historical trajectory data, an attention module for obtaining multi-level periodicity in flight history trajectories, and a decoding module for recursively generating predicted trajectory sequences. The model can capture long-term and short-term correlations and repeatability between trajectories, thereby improving the accuracy of 4D trajectory prediction. However, this method focuses on the trajectory prediction of a single aircraft and is difficult to apply to the task of simultaneously predicting the future trajectories of multiple aircraft.

[0005] In the patent application with application number “CN202311236417.3” and application name “An intention-based aircraft trajectory prediction method”, the inventor proposed an intention-based aircraft trajectory prediction method, which first preprocesses the original aircraft data, including data standardization, generating time series data, and constructing a data set. Then, the Res-GRU model is used to predict the last frame of the subsequent prediction sequence based on the above preprocessed time series data, and linearly interpolates with the last frame of the previous input sequence to obtain the intention prediction trajectory. Finally, the preprocessed time series data is input into the sparse self-attention spatiotemporal Transformer neural network for prediction to obtain the spatiotemporal prediction trajectory, and residual connection is performed with the above intention prediction trajectory to obtain the final trajectory prediction sequence. However, the recurrent neural network structure used in this application has low computational efficiency, which is not conducive to application in real air traffic management systems.

[0006] In addition, in the patent application with application number "CN202310031668.1" and application name "Aircraft trajectory prediction method, storage medium and device based on attention mechanism", the inventor proposed an aircraft trajectory prediction method based on attention mechanism. First, through the embedding method, position coding is introduced to convert the input time series sequence into a feature vector and input it into the codec model. In the encoder, the self-attention mechanism is used to weight the feature vector, and the weighted feature vector is passed through a feedforward neural network and then input into the decoder. Subsequently, another part of the same format data of the same batch of the time series sequence and the data encoded by the encoder are input into the decoder for decoding. Finally, all the time series sequence prediction results are output at once through a fully connected layer. However, this application also focuses on the trajectory prediction of a single aircraft, which is difficult to apply in multi-aircraft scenarios.

[0007] In summary, the above research and patent applications have achieved good results in predicting aircraft trajectories, but there are still two key problems: First, the current aircraft trajectory prediction method lacks feature mining of the interaction relationship between aircraft, which is very limited in complex mission scenarios. Second, the current trajectory prediction method does not capture the spatiotemporal characteristics of aircraft trajectory data well, and has poor performance in large-scale and long-term trajectory prediction. Summary of the invention

[0008] The present application provides a method and system for predicting 4D trajectory of an aircraft based on dual-attention spatiotemporal graph convolution, which reconstructs the features of the predefined adjacency matrix in the traditional graph network model through self-attention operation, and can better capture the implicit complex relationships between nodes in the spatiotemporal graph, so as to effectively improve the ability to predict the future trajectory of the aircraft.

[0009] In a first aspect, the present application provides a method for predicting 4D trajectory of an aircraft based on dual-attention spatiotemporal graph convolution, the method comprising the following steps:

[0010] Step 1: Construct a spatiotemporal graph based on the aircraft trajectory data. The spatiotemporal graph is a sequence of spatial graphs at multiple moments, where each node in the spatial graph represents an aircraft, and the node feature of each node is used to standardize the flight status feature information of the corresponding aircraft; the edge between nodes represents the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t; for each spatial graph, based on the edge between nodes, the N×N dimensional adjacency matrix A of each spatial graph can be obtained t , where N represents the number of aircraft;

[0011] Step 2: Reconstruct the adjacency matrix of each spatial graph based on the self-attention network to obtain the reconstructed adjacency matrix A′ t ;

[0012] Step 3: Extract features of the spatiotemporal graph based on the spatiotemporal graph convolutional network and graph attention network;

[0013] The spatial graph feature V is obtained based on the node features of all nodes in each spatial graph. t , splicing the spatial graph features V of the same spatial graph (i.e., at the same time) t And reconstruct the adjacency matrix A′ t Obtaining a first fused initial feature of each spatial graph; stacking the first fused initial features of all spatial graphs and using them as input data of a spatiotemporal graph convolutional network, and obtaining a first extracted feature based on the output features of the spatiotemporal graph convolutional network;

[0014] Concatenate the spatial graph features V of the same spatial graph t And the normalized adjacency matrix Obtaining the second fused initial features of each spatial graph, stacking the second fused initial features of all spatial graphs and using them as input data of the graph attention network, and obtaining the second extracted features based on the output features of the graph attention network;

[0015] Then, the first extracted feature and the second extracted feature are added and converged to obtain the extracted feature of the space-time graph;

[0016] Step 4: Input the extracted features of the space-time graph into the trajectory prediction network to obtain the predicted trajectory of each aircraft.

[0017] Furthermore, in step 1, the flight status characteristic information of the flying device includes but is not limited to: the position, speed, heading, model, body size, etc. of the aircraft.

[0018] Furthermore, in step 1, a kernel function is used to characterize the mutual influence degree between the two aircrafts. Preferably, the index factors of the kernel function for measuring the mutual influence degree between the two aircrafts include the direction, distance and relative speed between the aircrafts.

[0019] It should be noted that in this application, the normalized adjacency matrix In step 1, the adjacency matrix A can be t It can be obtained by performing normalization processing, or it can be obtained by performing normalization processing in step 2.

[0020] Furthermore, Among them, the auxiliary matrix I is the identity matrix, is the normalized adjacency matrix The diagonal node degree matrix of .

[0021] Furthermore, step 2 specifically includes:

[0022] Based on the trainable query weight matrix W q , key weight matrix Wk , value weight matrix W v The normalized adjacency matrix The corresponding rows of the linear transformation are transformed to generate the query vector of node i The key vector of node j Sum value vector in and The adjacency matrix The i-th and j-th rows of , where i, j = 1, ..., N;

[0023] According to the formula Calculate the attention score e ij , and normalize it to get the attention weight α i j ;

[0024] The attention weight α ij With the value vector v j att Perform weighted summation to obtain the reconstruction feature of node i

[0025] The reconstruction features h of all nodes i ’ are concatenated to obtain the reconstructed adjacency matrix sequence A’.

[0026] Furthermore, in step 3, a multi-head graph attention network is used to extract features from the second fused initial features to obtain second extracted features.

[0027] Furthermore, in step 4, the trajectory prediction network is used to output and determine the trajectory distribution mean and trajectory distribution covariance matrix of the aircraft.

[0028] Furthermore, in step 4, the trajectory prediction network adopts a time extrapolation convolutional network.

[0029] Furthermore, in this application, when jointly training the self-attention network, spatiotemporal graph convolution network, graph attention network and trajectory prediction network for reconstructing the adjacency matrix of each spatial graph, the loss function used is:

[0030]

[0031] Among them, W loss are all learnable parameters of the model (i.e., the trajectory prediction model composed of the self-attention network, spatiotemporal graph convolutional network, graph attention network, and trajectory prediction network), T P To predict the length of the time series, represents the trajectory of the i-th aircraft at time t, is the mean value of the trajectory distribution of the i-th aircraft at time t, is the trajectory distribution covariance matrix of the i-th aircraft at time t.

[0032] The second aspect of the present application provides an aircraft 4D trajectory prediction system based on dual-attention spatiotemporal graph convolution, including a sensor, a data transmission channel, a data preprocessing module and a trajectory prediction model;

[0033] Among them, the sensor is used to collect the aircraft trajectory data in real time and transmit it to the data preprocessing module through the data transmission channel;

[0034] The trajectory prediction model includes: (trained) self-attention network (adjacency matrix reconstruction module based on self-attention mechanism), spatiotemporal graph convolution network, graph attention network and trajectory prediction network;

[0035] The data preprocessing module is used to construct a space-time graph based on the aircraft trajectory data, extract the adjacency matrix sequence A of the space-time graph, and normalize it to obtain a normalized adjacency matrix sequence and will The adjacency matrix reconstruction module based on the self-attention mechanism is fed into the trajectory prediction model; and the reconstructed adjacency matrix sequence A′ output by splicing the spatiotemporal graph features and the adjacency matrix reconstruction module based on the self-attention mechanism is obtained to obtain the first fused initial features and fed into the spatiotemporal graph convolutional network of the trajectory prediction model; splicing the spatiotemporal graph features and the adjacency matrix sequence The second fused initial features are obtained and sent to the graph attention network of the trajectory prediction model;

[0036] The adjacency matrix reconstruction module based on the self-attention mechanism of the trajectory prediction model is used to reconstruct the adjacency matrix, the spatiotemporal graph convolutional network is used to extract the first extraction feature; the graph attention network is used to extract the second extraction feature; based on the splicing layer, the first extraction feature and the second extraction feature are added and converged to obtain the extraction feature of the spatiotemporal graph and sent to the trajectory prediction network to output the predicted trajectory of each aircraft;

[0037] The spatiotemporal graph constructed by the data preprocessing module is a sequence of spatial graphs at multiple moments, in which each node of the spatial graph represents an aircraft, and the node feature of each node is used to standardize the flight status feature information of the corresponding aircraft; the edge between nodes represents the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t; for each spatial graph, based on the edge between nodes, the N×N dimensional adjacency matrix A of each spatial graph can be obtained t , where N is the number of aircraft. Therefore, by stacking the adjacency matrix A at all times t The adjacency matrix sequence A of the space-time graph can be obtained.

[0038] Furthermore, the trajectory prediction system of the present application also includes a management terminal, which is used to perform intelligent approach sorting, abnormal behavior detection and / or AI-assisted decision-making based on the predicted trajectory of each aircraft output by the trajectory prediction model.

[0039] The technical solution provided by this application brings at least the following beneficial effects:

[0040] (1) This application proposes a learnable adjacency matrix based on a self-attention mechanism. To address the problem that the predefined adjacency matrix cannot effectively capture the characteristics of the potential relationship between nodes, this application uses the self-attention mechanism to reconstruct the predefined adjacency matrix and continuously optimizes it as the network is trained to improve its generalization. This effectively solves the problem that the predefined adjacency matrix based on distance cannot fully reflect the potential relationship between aircraft in dynamic scenes.

[0041] (2) This application proposes a new aircraft trajectory prediction method based on dual-attention spatiotemporal graph convolution. Based on the learnable adjacency matrix based on the self-attention mechanism, the graph attention network is used to further aggregate the correlation between nodes. Based on the powerful global information aggregation ability of the graph attention network, additional features are extracted from the spatiotemporal graph, so that the model can capture more potential connections between nodes, improving the problem that traditional graph convolution operations cannot effectively aggregate the correlation between nodes. The accuracy of the aircraft's future trajectory prediction is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0043] Figure 1 This is the overall framework diagram of the aircraft 4D trajectory prediction method based on dual-attention spatiotemporal graph convolution proposed in this application.

[0044] Figure 2 This is the adjacency matrix reconstruction module based on the self-attention mechanism in this application.

[0045] Figure 3 These are the spatiotemporal graph convolutional network and graph attention network modules in this application.

[0046] Figure 4 This is a schematic diagram of the structure of the aircraft 4D trajectory prediction system based on dual-attention spatiotemporal graph convolution proposed in this application.

[0047] Figure 5 This is a visual comparison result of trajectory prediction in the embodiment of this application. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions of the embodiments of the present application will be described in detail and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the embodiments described with reference to the drawings are exemplary and are intended to be used to explain the present application, and cannot be understood as limiting the present application.

[0049] See also Figure 1 , an embodiment of the present application provides an aircraft 4D trajectory prediction method based on dual-attention spatiotemporal graph convolution, which reconstructs the features of the adjacency matrix predefined in the traditional graph network model through self-attention operation, and uses the graph attention network to perform additional feature extraction on the spatiotemporal graph, so that the model can capture more potential connections between nodes and obtain a more accurate prediction of the future trajectory of the aircraft.

[0050] In one embodiment, the aircraft 4D trajectory prediction method based on dual-attention spatiotemporal graph convolution proposed in this application includes the following steps:

[0051] Step 1: Construct a space-time graph based on the aircraft trajectory data. The specific steps are as follows:

[0052] 1.1) Define the spatial graph:

[0053] Define the spatial graph at a certain time t as G t =(V t ,E t ), which contains two elements, representing the node V t and edge E t .in It's G t The set of all node vectors in , N is the number of aircraft in the scene, and each element in the set It contains the characteristics of the i-th aircraft at time t, that is, the node characteristics of each node are used to standardize the flight status characteristic information of the corresponding aircraft, which includes the position, speed, heading, model, fuselage size and other information of the aircraft, which can be expressed as:

[0054]

[0055] in, represents the position information of the i-th aircraft at time t, represents the speed information of the i-th aircraft at time t, Represents the heading information of the i-th aircraft at time t.

[0056] Above It's G t The set of all edges in , each element in the set Characterizes the node and nodes The connection relationship between them can be regarded as the degree of mutual influence between the two aircraft, where i and j are the aircraft numbers respectively and t is the time.

[0057] In one embodiment, the aircraft feature is the position information of the aircraft, which can be expressed as:

[0058]

[0059] in, represents the longitude of the i-th aircraft at time t, represents the latitude of the i-th aircraft at time t, represents the altitude of the i-th aircraft at time t.

[0060] 1.2) Construct the adjacency matrix:

[0061] Using kernel function To measure the interaction between the two aircraft, Kernel Function Factors such as the direction, distance, and relative speed between aircraft are taken into consideration.

[0062] In one embodiment, the kernel function Specifically, it can be expressed as follows:

[0063]

[0064] in, This means that the closer the distance between two aircraft is, the greater the impact they have on each other, and vice versa.

[0065] According to E t The adjacency matrix A t The construction is as follows:

[0066]

[0067] Among them, A t is an N×N matrix, reflecting G t The mutual influence relationship between all nodes in the scene, N is the number of aircraft in the scene.

[0068] Finally, the adjacency matrix A t Normalization is performed, and the specific formula is as follows:

[0069]

[0070] Among them, the auxiliary matrix I is the identity matrix, is the normalized adjacency matrix The diagonal node degree matrix of .

[0071] 1.3) Constructing a space-time graph:

[0072] All G at time t∈{1…T} t Stacked together, we get the final space-time map in E can be Characterize, N represents the number of aircraft.

[0073] In one embodiment, the time length is 8, then at time t∈{1…8}, the time-space diagram in E can be Characterize, N represents the number of aircraft.

[0074] Step 2: Reconstruct the adjacency matrix based on self-attention operation. The specific steps are as follows:

[0075] 2.1) Linear transformation to obtain the eigenvector:

[0076] For the adjacency matrix constructed in 1.2) Using three different trainable weight matrices W q , W k , W v Perform a linear transformation to generate the corresponding eigenvector q i att , k j att and v j att , as shown below:

[0077]

[0078] in, and The adjacency matrix The i-th and j-th rows represent the connection relationship between nodes i and j in the graph.

[0079] 2.2) Calculate the attention score e ij , and normalize it (such as through the Softmax function) to obtain the attention weight α ij :

[0080]

[0081] Among them, q i att is the query vector of node i, k j att is the key vector of node j, e ij It reflects the strength of the association between node i and node j.

[0082] 2.3) Reconstruct the adjacency matrix:

[0083] The attention weight α ij With the feature vector v j att Perform weighted summation to obtain the feature h after node i is reconstructed i i , as follows:

[0084]

[0085] Finally, the features of all nodes are concatenated to obtain the reconstructed adjacency matrix A′. A′ can dynamically adjust the relationship weights based on global information, rather than being limited to predefined local connections. This enables the model to better capture the implicit complex relationships between nodes and improve the ability to predict the future trajectory of the aircraft. Figure 2 A schematic diagram of the processing process of the adjacency matrix reconstruction module based on the self-attention mechanism provided in this embodiment.

[0086] Step 3: Use spatiotemporal graph convolutional network operations and graph attention network operations to extract features. The specific steps are as follows:

[0087] 3.1) Spatiotemporal graph convolutional network operations:

[0088] Use the reconstructed adjacency matrix A′ in 2.3) to perform spatiotemporal graph convolution operations.

[0089] Define the characteristics of all aircraft in the lth network layer at time t as V t (l) , V t The specific definition is consistent with that in 1.1). This application uses V (l) Indicates V t (l) Stacking in the time dimension. Finally, the operation of each network layer in the spatiotemporal graph convolutional network is realized by the following formula:

[0090] V (l+1) =σ(A′V (l) W stgcn (l) ) (10)

[0091] Among them, W stgcn (l) is the trainable network parameter of the lth layer of the spatiotemporal graph convolutional network, and σ(·) is the activation function. Finally, the embedding result of the spatiotemporal graph convolutional network output is expressed as in Preset value for the dimension of the embedding result.

[0092] In one embodiment, the activation function σ(·) uses the ReLU activation function. Finally, the embedding result output by the spatiotemporal graph convolutional network is expressed as Where 9 is the dimension of the embedding result, 20 is the timestamp, and N is the number of aircraft in the scene.

[0093] 3.2) Graph Attention Network Operation:

[0094] The graph attention network dynamically assigns different weights to each neighbor node through the self-attention mechanism, allowing the model to capture more potential connections between nodes. First, the graph attention coefficient is calculated using the following formula:

[0095]

[0096] in, is the attention coefficient from node j to node i, N i represents the set of neighbor nodes of node i. The input feature of the network is v = {v1, v2, ... v N}, N represents the number of nodes. GAT is the linear transformation weight matrix applied to each node, and a is the weight vector, which can map the input to the feature space. Finally, the LeakyReLU activation function is used to provide nonlinear output and the SoftMax function is used for normalization. This application uses a multi-head graph attention network, so the output features of node i are as follows:

[0097]

[0098] in, is the normalized attention coefficient of the kth attention head, W GAT k is the linear transformation weight matrix of the kth attention head, v j is the input feature of the jth node, and σ(·) is the activation function.

[0099] That is, the graph attention network dynamically assigns different weights to each neighbor node through the self-attention mechanism, so that the model can capture more potential connections between nodes. First, the graph attention coefficient is calculated by formula (11): It represents the attention coefficient from node j to node i. The input feature of the graph attention network is v = {v1, v2, …v N}, N represents the number of nodes. In this embodiment, a multi-head graph attention network is used with 4 heads, so the output feature v of node i is i ′ is as follows:

[0100]

[0101] in, is the normalized attention coefficient of the kth attention head, W GAT k is the linear transformation weight matrix of the kth attention head, v j is the input feature of the j-th node, σ(·) is the activation function, and || represents the concatenation operation.

[0102] 3.3) Aggregate the output features in 3.1) and 3.2) to obtain the final output result V out , as shown below:

[0103]

[0104] Among them, STGCN(·) and GAT(·) represent the spatiotemporal graph convolution network operation and graph attention network operation respectively, V represents the input node vector, and A′ represent the normalized adjacency matrix and the reconstructed adjacency matrix, respectively.

[0105] That is, the output features of the spatiotemporal graph convolutional network and the output features of the graph attention network are added and converged to obtain the final output result. Figure 3 The spatiotemporal graph convolutional network and graph attention network modules provided for this embodiment.

[0106] Step 4: Define the network training loss function and predict the trajectory. The specific steps are as follows:

[0107] 4.1) Predicted trajectory:

[0108] This application uses the probability distribution of the aircraft's future trajectory as the prediction result. This application assumes that the aircraft's trajectory at the future time t Satisfies the ternary Gaussian distribution, that is, in is the mean value of the trajectory distribution of the i-th aircraft at time t, is the trajectory distribution covariance matrix of the i-th aircraft at time t.

[0109] Therefore, the output of the predicted trajectory is the mean of the ternary Gaussian distribution And the covariance matrix The elements in , where t and i are the predicted time and aircraft number respectively. Since the output of the network cannot always satisfy the positive definiteness of the covariance matrix, the present invention outputs a lower triangular matrix with positive diagonals and through To construct the covariance matrix

[0110]

[0111] in, for The transpose of .

[0112] Finally, this application calculates the trajectory prediction results through a time extrapolation convolutional network.

[0113] In one embodiment, in order to realize the training of the trajectory prediction model composed of the self-attention network (i.e., the adjacency matrix reconstruction module based on the self-attention mechanism) for reconstructing the adjacency matrix of each spatial graph, the spatiotemporal graph convolution network, the graph attention network and the trajectory prediction network, the network model parameters are trained by minimizing the negative log-likelihood loss, which are specifically defined as follows:

[0114]

[0115] Among them, W loss are all learnable parameters of the trajectory prediction model, T P is the length of the predicted time series.

[0116] In one embodiment, setting T P =12.

[0117] On the other hand, the embodiment of the present application also provides an aircraft 4D trajectory prediction system based on dual attention spatiotemporal graph convolution, such as Figure 4 As shown, it includes sensors, data transmission channels, data preprocessing modules, trajectory prediction models and management terminals; wherein the trajectory prediction model includes: a trained adjacency matrix reconstruction module based on the self-attention mechanism, a spatiotemporal graph convolutional network, a graph attention network and a trajectory prediction network. In this prediction system, sensors are used to collect aircraft trajectory data in real time and transmit it to the data preprocessing module through a data transmission channel. The sensors include radar, laser, image and other sensors, and the data transmission channels include satellites, ADS-B, 4G / 5G and the like. The data preprocessing module is used to construct a spatiotemporal graph based on the aircraft trajectory data, extract the adjacency matrix sequence A of the spatiotemporal graph, and normalize it to obtain a normalized adjacency matrix sequence and will The adjacency matrix reconstruction module based on the self-attention mechanism is fed into the trajectory prediction model; and the reconstructed adjacency matrix sequence A′ output by splicing the spatiotemporal graph features and the adjacency matrix reconstruction module based on the self-attention mechanism is obtained to obtain the first fused initial features and fed into the spatiotemporal graph convolutional network of the trajectory prediction model; splicing the spatiotemporal graph features and the adjacency matrix sequence The second fused initial feature is obtained and sent to the graph attention network of the trajectory prediction model; the adjacency matrix reconstruction module based on the self-attention mechanism of the trajectory prediction model is used to reconstruct the adjacency matrix, and the spatiotemporal graph convolutional network is used to extract the first extracted feature; the graph attention network is used to extract the second extracted feature; based on the splicing layer, the first extracted feature and the second extracted feature are added and converged to obtain the extracted feature of the spatiotemporal graph and sent to the trajectory prediction network to output the predicted trajectory of each aircraft; wherein, the spatiotemporal graph constructed by the data preprocessing module is a sequence of spatial graphs at multiple moments, wherein each node of the spatial graph represents an aircraft, and the node feature of each node is used to standardize the flight status feature information of the corresponding aircraft; the edge between the nodes represents the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t; for each spatial graph, based on the edge between the nodes, the N×N dimensional adjacency matrix A of each spatial graph can be obtained t , where N is the number of aircraft. Therefore, by stacking the adjacency matrix A at all times t The adjacency matrix sequence A of the space-time graph can be obtained. The management terminal is used to perform intelligent approach sorting, abnormal behavior detection and / or AI-assisted decision-making based on the predicted trajectory of each aircraft output by the trajectory prediction model.

[0118] In order to further verify the prediction performance of the method / system proposed in this application, simulation verification is performed based on two data sets. The two data sets used are made based on real open source ADS-B data of the airspace within a radius of 50km centered on two large airports, and two dense airspaces with a radius of 100km above Europe and North America. The data set has a total of 45,463 trajectory data, each of which includes five pieces of information: time, aircraft number, aircraft longitude, aircraft latitude, and aircraft altitude. The time interval between two trajectory data of each aircraft is 10s.

[0119] This embodiment uses the average displacement error (ADE) and the final displacement error (FDE) to quantitatively evaluate the performance of the proposed method. The specific formula is as follows:

[0120]

[0121] in and are the actual horizontal or vertical trajectory of the aircraft and the horizontal or vertical trajectory predicted by the model, respectively. p is the predicted time range, and N is the number of aircraft. The average displacement error shows the average prediction performance of the model, and the final displacement error shows the end-point prediction performance of the model. The smaller these two indicators are, the better the prediction performance of the model.

[0122] In this embodiment, the model is trained in batches of 128, using the ReLU activation function. The training optimizer selects Adam, the initial learning rate is 0.001, and when the training rounds reach 200 rounds, the learning rate decays to 0.0002, and finally a total of 400 rounds are trained. In the specific implementation process, the model predicts the trajectory distribution of the next 120 seconds based on the observation state of the past 80 seconds.

[0123] This embodiment is compared with the existing LSTM, S-LSTM, ST-GAT and S-STGCNN methods, and the comparison indicators used are the above-mentioned average displacement error and final displacement error. The final comparison results are shown in Table 1, where the value on the left represents the average displacement error and the value on the right represents the final displacement error. The comparison results show that the prediction indicators of this embodiment on both data sets have reached the best, indicating that the performance of this embodiment is better.

[0124] This embodiment also visualizes the aircraft trajectory prediction results and compares them with the visualization results of the S-STGCNN method. Figure 5 As shown. (5a) and (5b) are the horizontal and vertical prediction visualization results of this embodiment, respectively, and (5c) and (5d) are the horizontal and vertical prediction visualization results of the S-STGCNN method, respectively. It can be seen from the visualization results that the prediction of the future trajectory of the aircraft by this embodiment is very close to the actual future trajectory, but the S-STGCNN method has a large deviation when predicting aircraft No. 1 and aircraft No. 2, which shows that the prediction performance of this embodiment is better.

[0125] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

[0126] Table 1 Comparison of trajectory prediction performance of this embodiment with other methods

[0127]

[0128] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0129] In addition, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include at least one of the features.

[0130] Any process or method description described in a flowchart or otherwise in this specification may be understood to represent a module, fragment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0131] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0132] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. Aircraft 4D trajectory prediction method based on dual-attention spatiotemporal graph convolution, characterized by: The following steps are involved: Step 1: Construct a spatiotemporal graph based on the aircraft trajectory data. The spatiotemporal graph is a sequence of spatial graphs at multiple moments, where each node in the spatial graph represents an aircraft, and the node feature of each node is used to standardize the flight status feature information of the corresponding aircraft; the edge between nodes represents the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t; for each spatial graph, based on the edge between nodes, the N×N dimensional adjacency matrix A of each spatial graph can be obtained t , where N represents the number of aircraft; Step 2: Reconstruct the adjacency matrix of each spatial graph based on the self-attention network to obtain the reconstructed adjacency matrix A′ t ; Step 3: Extract features of the spatiotemporal graph based on the spatiotemporal graph convolutional network and graph attention network; Based on the node features of all nodes in each spatial graph, the spatial graph feature V is obtained t , splicing the spatial graph features V of the same spatial graph t And reconstruct the adjacency matrix A′ t Obtaining a first fused initial feature of each spatial graph; stacking the first fused initial features of all spatial graphs and using them as input data of a spatiotemporal graph convolutional network, and obtaining a first extracted feature based on the output features of the spatiotemporal graph convolutional network; Concatenate the spatial graph features V of the same spatial graph t And the normalized adjacency matrix Obtaining the second fused initial features of each spatial graph, stacking the second fused initial features of all spatial graphs and using them as input data of the graph attention network, and obtaining the second extracted features based on the output features of the graph attention network; Then, the first extracted feature and the second extracted feature are added and converged to obtain the extracted feature of the space-time graph; Step 4: Input the extracted features of the space-time graph into the trajectory prediction network to obtain the predicted trajectory of each aircraft.

2. The method according to claim 1, characterized in that In step 1, the flight status characteristic information of the flying device includes: the position, speed, heading, model and / or body size of the aircraft.

3. The method according to claim 1, characterized in that In step 1, a kernel function is used to characterize the degree of mutual influence between the two aircraft.

4. The method according to claim 1, characterized in that Normalized adjacency matrix Among them, the auxiliary matrix I is the identity matrix, is the normalized adjacency matrix The diagonal node degree matrix of .

5. The method according to claim 1, characterized in that Step 2 specifically includes: Based on the trainable query weight matrix W q , key weight matrix W k , value weight matrix W v The normalized adjacency matrix The corresponding rows of the linear transformation are transformed to generate the query vector of node i The key vector of node j Sum value vector in and The adjacency matrix The i-th and j-th rows of , where i, j = 1, ..., N; According to the formula Calculate the attention score e ij , and normalize it to get the attention weight α ij ; The attention weight α ij With the value vector v j att Perform weighted summation to obtain the reconstruction feature of node i Reconstructed features h′ for all nodes i The reconstructed adjacency matrix sequence A′ is obtained by splicing.

6. The method according to claim 1, characterized in that In step 3, a multi-head graph attention network is used to extract the second fused initial features to obtain the second extracted features.

7. The method according to claim 1, characterized in that In step 4, the trajectory prediction network is used to output the trajectory distribution mean and trajectory distribution covariance matrix of the aircraft.

8. The method according to claim 1, characterized in that In step 4, the trajectory prediction network uses a time-extrapolated convolutional network.

9. The method according to claim 7 or 9, characterized in that When jointly training the self-attention network, spatiotemporal graph convolutional network, graph attention network, and trajectory prediction network to reconstruct the adjacency matrix of each spatial graph, the loss function used is: Among them, W loss are all learnable parameters of the trajectory prediction model based on the self-attention network, spatiotemporal graph convolutional network, graph attention network and trajectory prediction network, T P To predict the length of the time series, represents the trajectory of the i-th aircraft at time t, is the mean value of the trajectory distribution of the i-th aircraft at time t, is the trajectory distribution covariance matrix of the i-th aircraft at time t.

10. Aircraft 4D trajectory prediction system based on dual-attention spatiotemporal graph convolution, characterized by: It includes sensors, data transmission channels, data preprocessing modules and trajectory prediction models; Among them, the sensor is used to collect the aircraft trajectory data in real time and transmit it to the data preprocessing module through the data transmission channel; The trajectory prediction model includes an adjacency matrix reconstruction module based on the self-attention mechanism, a spatiotemporal graph convolutional network, a graph attention network, and a trajectory prediction network; The data preprocessing module is used to construct a space-time graph based on the aircraft trajectory data, extract the adjacency matrix sequence A of the space-time graph, and normalize it to obtain a normalized adjacency matrix sequence and will The adjacency matrix reconstruction module based on the self-attention mechanism is fed into the trajectory prediction model; and the reconstructed adjacency matrix sequence A′ output by splicing the spatiotemporal graph features and the adjacency matrix reconstruction module based on the self-attention mechanism is obtained to obtain the first fused initial features and fed into the spatiotemporal graph convolutional network of the trajectory prediction model; splicing the spatiotemporal graph features and the adjacency matrix sequence The second fused initial features are obtained and sent to the graph attention network of the trajectory prediction model; The adjacency matrix reconstruction module based on the self-attention mechanism of the trajectory prediction model is used to reconstruct the adjacency matrix, the spatiotemporal graph convolutional network is used to extract the first extraction feature; the graph attention network is used to extract the second extraction feature; based on the splicing layer, the first extraction feature and the second extraction feature are added and converged to obtain the extraction feature of the spatiotemporal graph and sent to the trajectory prediction network to output the predicted trajectory of each aircraft; The spatiotemporal graph constructed by the data preprocessing module is a sequence of spatial graphs at multiple moments, in which each node of the spatial graph represents an aircraft, and the node feature of each node is used to standardize the flight status feature information of the corresponding aircraft; the edge between nodes represents the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t; for each spatial graph, based on the edge between nodes, the N×N dimensional adjacency matrix A of each spatial graph can be obtained t , where N represents the number of aircraft.

Images