Aircraft 4D trajectory prediction method and system based on double-attention spatio-temporal graph convolution

Abstract

The application discloses a kind of based on dual attention space-time graph convolution aircraft 4D trajectory prediction method and system, belong to intelligent time series prediction technical field.The present application reconstructs predefined adjacency matrix using self-attention mechanism, and constantly optimizes with the training of network to improve its generalizability.This effectively solves the problem that the predefined adjacency matrix based on distance cannot fully reflect the potential relationship between aircrafts in dynamic scene.On the basis of learnable adjacency matrix based on self-attention mechanism, the present application further aggregates the correlation between nodes using graph attention network, based on the powerful global information convergence capability of graph attention network, additional feature extraction is carried out on space-time graph, so that the model can capture more potential links between nodes, improve the problem that traditional graph convolution operation cannot effectively aggregate the correlation between nodes.Further improve the accuracy of aircraft future trajectory prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence time series prediction, in particular to a 4D trajectory prediction method and system for aircraft based on double attention spatiotemporal graph convolution. BACKGROUND

[0002] The International Civil Aviation Organization released the 6th edition of the Global Air Navigation Plan in 2019, which clearly states that "Trajectory Based Operation (TBO)" will be the overall integration and final implementation goal of various types of lead lines in the "Aviation System Block Upgrade", and plans to be applied globally after 2031. The transition of air traffic management from "Clearance Based Operation (CBO)" to TBO brings new challenges to the performance of 4D trajectory prediction. As the core technology for tasks such as conflict detection and resolution, aircraft abnormal behavior monitoring, dense flight area control, arrival and departure sequencing, how to accurately and efficiently predict future trajectories is one of the main difficulties in current research. The Federal Aviation Administration of the United States defines 4D trajectory as a four-dimensional description of the flight path of an aircraft, including longitude, latitude, altitude, and time. Due to airspace congestion, bad weather, emergency accidents, and other reasons, the actual flight process of an aircraft sometimes does not follow the previously planned route. These unexpected situations reduce the predictability of air traffic and increase the difficulty of air traffic management, seriously hindering the efficient operation of the air traffic management system. If the future trajectory of an aircraft can be predicted in advance, it will be beneficial for managers to make decisions in advance to deal with unexpected situations, thereby ensuring the safety and efficiency of flights, so it is necessary to predict the 4D trajectory of an aircraft.

[0003] In the technical solution disclosed in the literature "Terminal Area Aircraft Trajectory Prediction Based on Attention Seq2Seq Model[J]. Science Technology and Engineering, 2024, 24(09): 3882-3895", an attention mechanism and an exponential decay sampling method in teacher supervision are introduced, and an airport terminal area trajectory prediction model based on a sequence-to-sequence framework is proposed. The sequence-to-sequence framework realizes multi-step prediction, the attention mechanism improves the prediction accuracy of the decoder, and the exponential decay sampling method accelerates the convergence of the model in the training stage, which improves the generalization of the prediction model to a certain extent. However, this method does not solve the problem of difficulty in capturing spatial correlation between aircrafts, and cannot be practically applied in complex scenarios.

[0004] A data-driven 4D trajectory prediction model is proposed in the paper "Ma L, Meng X, Wu Z. Data-Driven 4D Trajectory Prediction Model Using Attention-TCN-GRU[J]. Aerospace, 2024, 11(4): 313.". First, a spatial gap filling method is proposed to reconstruct the trajectory of each aircraft, thereby generating a high-quality trajectory dataset with consistent time intervals. Subsequently, a hybrid neural network based on the seq2seq model is designed, named Attention-TCN-GRU, which includes an encoding module for extracting features from historical trajectory data, an attention module for obtaining multi-level periodicity in flight history trajectories, and a decoding module for recursively generating predicted trajectory sequences. The model can capture long-term and short-term correlations and repetitiveness between trajectories, thereby improving the accuracy of 4D trajectory prediction. However, this method focuses on the trajectory prediction of a single aircraft and is difficult to apply to the task of simultaneously predicting the future trajectories of multiple aircraft.

[0005] In the patent application with application number “CN202311236417.3” and application name “A method for aircraft trajectory prediction based on intention”, the inventor proposed an aircraft trajectory prediction method based on intention. The method first preprocesses the original aircraft data, including steps such as data standardization, generating time series data, and constructing a data set. Subsequently, the Res-GRU model is used to predict the last frame of the subsequent prediction sequence based on the above-mentioned preprocessed time series data, and linearly interpolates with the last frame of the previous input sequence to obtain the intention prediction trajectory. Finally, the preprocessed time series data is input into the sparse self-attention spatiotemporal Transformer neural network for prediction to obtain the spatiotemporal prediction trajectory, and residual connection is performed with the above-mentioned intention prediction trajectory to obtain the final trajectory prediction sequence. However, the recurrent neural network structure used in this application has low computational efficiency, which is not conducive to application in real air traffic management systems.

[0006] In addition, in the patent application with application number "CN202310031668.1" and application name "Aircraft trajectory prediction method, storage medium and device based on attention mechanism", the inventor proposed an aircraft trajectory prediction method based on the attention mechanism. First, through the embedding method, position encoding is introduced to convert the input time series sequence into a feature vector and input it into the codec model. In the encoder, the self-attention mechanism is used to weight the feature vector, and the weighted feature vector is passed through a feedforward neural network and then input into the decoder. Subsequently, another part of the same format data of the same batch of the time series sequence and the data encoded by the encoder are input into the decoder for decoding. Finally, all the time series sequence prediction results are output at once through a fully connected layer. However, this application also focuses on the trajectory prediction of a single aircraft, which is difficult to apply to multi-aircraft scenarios.

[0007] In summary, the aforementioned research and patent applications have achieved promising results in predicting aircraft trajectories. However, two key issues remain: First, current trajectory prediction methods lack the ability to mine the features that influence the interactions between aircraft, resulting in significant limitations in complex mission scenarios. Second, current trajectory prediction methods do not adequately capture the spatiotemporal characteristics of aircraft trajectory data, resulting in poor performance in predicting trajectories over large areas and over long periods of time. Summary of the Invention

[0008] The present application provides a method and system for predicting the 4D trajectory of an aircraft based on dual-attention spatiotemporal graph convolution. It reconstructs the features of the predefined adjacency matrix in the traditional graph network model through self-attention operation, which can better capture the implicit complex relationships between nodes in the spatiotemporal graph, thereby effectively improving the ability to predict the future trajectory of the aircraft.

[0009] In a first aspect, the present application provides a method for predicting 4D aircraft trajectories based on dual-attention spatiotemporal graph convolution, the method comprising the following steps:

[0010] Step 1: Construct a spatiotemporal graph based on the aircraft trajectory data. The spatiotemporal graph is a sequence of spatial graphs at multiple moments. Each node in the spatial graph represents an aircraft, and the node feature of each node is used to standardize the flight status feature information of the corresponding aircraft. The edges between nodes represent the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t. For each spatial graph, based on the edges between nodes, the N×N dimensional adjacency matrix A of each spatial graph can be obtained. t , where N represents the number of aircraft;

[0011] Step 2: Reconstruct the adjacency matrix of each spatial graph based on the self-attention network to obtain the reconstructed adjacency matrix A′ t ;

[0012] Step 3: Extract features from the spatiotemporal graph based on the spatiotemporal graph convolutional network and graph attention network;

[0013] The spatial graph feature V is obtained based on the node features of all nodes in each spatial graph t , splicing the spatial graph features V of the same spatial graph (i.e., at the same time) t and reconstruct the adjacency matrix A′ t Obtaining a first fused initial feature of each spatial graph; stacking the first fused initial features of all spatial graphs and using them as input data of a spatiotemporal graph convolutional network, and obtaining a first extracted feature based on the output features of the spatiotemporal graph convolutional network;

[0014] Splicing the spatial graph features V of the same spatial graph t And the normalized adjacency matrix Obtain the second fused initial features of each spatial graph, stack the second fused initial features of all spatial graphs and use them as input data of the graph attention network, and obtain the second extracted features based on the output features of the graph attention network;

[0015] Then, the first extracted feature and the second extracted feature are added together to obtain the extracted feature of the spatiotemporal graph;

[0016] Step 4: Input the extracted features of the spatiotemporal graph into the trajectory prediction network to obtain the predicted trajectory of each aircraft.

[0017] Furthermore, in step 1, the flight status characteristic information of the aircraft includes but is not limited to: the position, speed, heading, model, body size, etc. of the aircraft.

[0018] Furthermore, in step 1, a kernel function is used to characterize the degree of mutual influence between the two aircraft. Preferably, the index factors of the kernel function for measuring the degree of mutual influence between the two aircraft include the direction, distance and relative speed between the aircraft.

[0019] It should be noted that in this application, the normalized adjacency matrix In step 1, the adjacency matrix A can be t It can be obtained by performing normalization processing, or it can be obtained by performing normalization processing in step 2.

[0020] Further, Among them, the auxiliary matrix I is the identity matrix, is the normalized adjacency matrix The diagonal node degree matrix of .

[0021] Furthermore, step 2 specifically includes:

[0022] Based on the trainable query weight matrix W q , key weight matrix Wk , value weight matrix W v The normalized adjacency matrix Perform linear transformation on the corresponding rows to generate the query vector of node i The key vector of node j Sum value vector in and The adjacency matrix The i-th and j-th rows of , where i, j = 1, ..., N;

[0023] According to the formula Calculate the attention score e ij , and normalize it to get the attention weight α i j ;

[0024] The attention weight α ij With the value vector v j att Perform weighted summation to obtain the reconstructed features of node i

[0025] Reconstructed features h for all nodes i ’ are concatenated to obtain the reconstructed adjacency matrix sequence A’.

[0026] Furthermore, in step 3, a multi-head graph attention network is used to extract features from the second fused initial features to obtain second extracted features.

[0027] Furthermore, in step 4, the trajectory prediction network is used to output the trajectory distribution mean and trajectory distribution covariance matrix of the aircraft.

[0028] Furthermore, in step 4, the trajectory prediction network adopts a time extrapolation convolutional network.

[0029] Furthermore, in this application, when jointly training the self-attention network, spatiotemporal graph convolution network, graph attention network, and trajectory prediction network to reconstruct the adjacency matrix of each spatial graph, the loss function used is:

[0030]

[0031] Among them, W loss are all learnable parameters of the model (i.e., the trajectory prediction model composed of the self-attention network, spatiotemporal graph convolutional network, graph attention network, and trajectory prediction network), T P To predict the length of the time series, represents the trajectory of the i-th aircraft at time t, is the mean value of the trajectory distribution of the i-th aircraft at time t, is the covariance matrix of the trajectory distribution of the i-th aircraft at time t.

[0032] The second aspect of the present application provides an aircraft 4D trajectory prediction system based on dual-attention spatiotemporal graph convolution, including a sensor, a data transmission channel, a data preprocessing module and a trajectory prediction model;

[0033] Among them, the sensor is used to collect the aircraft trajectory data in real time and transmit it to the data preprocessing module through the data transmission channel;

[0034] The trajectory prediction model includes: a (trained) self-attention network (an adjacency matrix reconstruction module based on the self-attention mechanism), a spatiotemporal graph convolutional network, a graph attention network, and a trajectory prediction network;

[0035] The data preprocessing module is used to construct a space-time graph based on the aircraft trajectory data, extract the adjacency matrix sequence A of the space-time graph, and normalize it to obtain the normalized adjacency matrix sequence and will The adjacency matrix reconstruction module based on the self-attention mechanism is fed into the trajectory prediction model; and the reconstructed adjacency matrix sequence A′ output by the spatiotemporal graph features and the adjacency matrix reconstruction module based on the self-attention mechanism is obtained by splicing the spatiotemporal graph features and the adjacency matrix sequence A′ to obtain the first fusion initial feature and feed it into the spatiotemporal graph convolutional network of the trajectory prediction model; splicing the spatiotemporal graph features and the adjacency matrix sequence Get the second fused initial features and send them into the graph attention network of the trajectory prediction model;

[0036] The trajectory prediction model's self-attention-based adjacency matrix reconstruction module is used to reconstruct the adjacency matrix. The spatiotemporal graph convolutional network is used to extract the first extracted features. The graph attention network is used to extract the second extracted features. The first extracted features and the second extracted features are added together based on the concatenation layer to obtain the extracted features of the spatiotemporal graph and fed into the trajectory prediction network to output the predicted trajectory of each aircraft.

[0037] The spatiotemporal graph constructed by the data preprocessing module is a sequence of spatial graphs at multiple moments, where each node in the spatial graph represents an aircraft, and the node features of each node are used to standardize the flight status feature information of the corresponding aircraft; the edges between nodes represent the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t; for each spatial graph, based on the edges between nodes, the N×N dimensional adjacency matrix A of each spatial graph can be obtained t , where N represents the number of aircraft. Therefore, by stacking the adjacency matrix A at all times t The adjacency matrix sequence A of the space-time graph can be obtained.

[0038] Furthermore, the trajectory prediction system of the present application also includes a management terminal for performing intelligent approach sorting, abnormal behavior detection and / or AI-assisted decision-making based on the predicted trajectory of each aircraft output by the trajectory prediction model.

[0039] The technical solution provided by this application brings at least the following beneficial effects:

[0040] (1) This application proposes a learnable adjacency matrix based on a self-attention mechanism. To address the problem that predefined adjacency matrices cannot effectively capture the potential relationship characteristics between nodes, this application uses the self-attention mechanism to reconstruct the predefined adjacency matrix and continuously optimizes it as the network is trained to improve its generalization. This effectively solves the problem that predefined distance-based adjacency matrices cannot fully reflect the potential relationships between aircraft in dynamic scenes.

[0041] (2) This application proposes a new aircraft trajectory prediction method based on dual-attention spatiotemporal graph convolution. Based on the learnable adjacency matrix based on the self-attention mechanism, the graph attention network is used to further aggregate the correlation between nodes. Based on the powerful global information aggregation ability of the graph attention network, additional feature extraction is performed on the spatiotemporal graph, enabling the model to capture more potential connections between nodes, improving the problem that traditional graph convolution operations cannot effectively aggregate the correlation between nodes. This further improves the accuracy of the aircraft's future trajectory prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0043] Figure 1 This is the overall framework diagram of the aircraft 4D trajectory prediction method based on dual-attention spatiotemporal graph convolution proposed in this application.

[0044] Figure 2 This is the adjacency matrix reconstruction module based on the self-attention mechanism in this application.

[0045] Figure 3 These are the spatiotemporal graph convolutional network and graph attention network modules in this application.

[0046] Figure 4 Schematic diagram of the structure of the aircraft 4D trajectory prediction system based on dual-attention spatiotemporal graph convolution proposed in this application.

[0047] Figure 5 This is the visual comparison result of trajectory prediction in the embodiment of this application. DETAILED DESCRIPTION

[0048] In order to enable people skilled in the art to better understand the technical solutions in this specification, the technical solutions of the embodiments of the present application will be described in detail and completely below in combination with the drawings in the embodiments of the present application. Obviously, the embodiments described with reference to the drawings are exemplary and are intended to be used to explain the present application, and cannot be understood as limiting the present application.

[0049] See also Figure 1 , an embodiment of the present application provides an aircraft 4D trajectory prediction method based on dual-attention spatiotemporal graph convolution, which reconstructs the features of the predefined adjacency matrix in the traditional graph network model through self-attention operation, and uses the graph attention network to perform additional feature extraction on the spatiotemporal graph, so that the model can capture more potential connections between nodes and obtain a more accurate future prediction trajectory of the aircraft.

[0050] In one embodiment, the method for aircraft 4D trajectory prediction based on dual-attention spatiotemporal graph convolution proposed in this application includes the following steps:

[0051] Step 1: Construct a space-time graph based on the aircraft trajectory data. The specific steps are as follows:

[0052] 1.1) Define the spatial graph:

[0053] Define the spatial graph at a certain time t as G t =(V t ,E t ), which contains two elements, representing the node V t and edge E t .in It's G t The set of all node vectors in the scene, N is the number of aircraft in the scene, and each element in the set Contains the characteristics of the i-th aircraft at time t, that is, the node characteristics of each node are used to standardize the flight status characteristic information of the corresponding aircraft, which includes various information such as the aircraft's position, speed, heading, model, and fuselage size, which can be expressed as:

[0054]

[0055] in, represents the position information of the i-th aircraft at time t, represents the speed information of the i-th aircraft at time t, Indicates the heading information of the i-th aircraft at time t.

[0056] above It's G t The set of all edges in , each element in the set Represents the node and nodes This connection relationship can be regarded as the degree of mutual influence between the two aircraft, where i and j are the aircraft numbers respectively, and t is the time.

[0057] In one embodiment, the aircraft feature is the position information of the aircraft, which can be expressed as:

[0058]

[0059] in, represents the longitude of the i-th aircraft at time t, represents the latitude of the i-th aircraft at time t, represents the altitude of the i-th aircraft at time t.

[0060] 1.2) Construct the adjacency matrix:

[0061] Using kernel function To measure the interaction between the two aircraft, Kernel Function Factors such as the orientation, distance, and relative speed between aircraft are taken into consideration.

[0062] In one embodiment, the kernel function Specifically, it can be expressed as follows:

[0063]

[0064] in, This means that the closer the distance between two aircraft is, the greater the impact they have on each other, and vice versa.

[0065] According to E t The adjacency matrix A t The construction is as follows:

[0066]

[0067] Among them, A t It is an N×N matrix, reflecting G t The mutual influence relationship between all nodes in the scene, N is the number of aircraft in the scene.

[0068] Finally, the adjacency matrix A t Perform normalization processing, the specific formula is as follows:

[0069]

[0070] Among them, the auxiliary matrix I is the identity matrix, is the normalized adjacency matrix The diagonal node degree matrix of .

[0071] 1.3) Constructing a space-time graph:

[0072] All G at time t∈{1…T} t Stacked together to get the final space-time map in E can be obtained by Characterize, N represents the number of aircraft.

[0073] In one embodiment, the time length is 8, then at time t∈{1…8}, the time-space diagram in E can be obtained by Characterize, N represents the number of aircraft.

[0074] Step 2: Reconstruct the adjacency matrix based on self-attention operation. The specific steps are as follows:

[0075] 2.1) Linear transformation to obtain eigenvectors:

[0076] For the adjacency matrix constructed in 1.2) Using three different trainable weight matrices W q 、W k 、W v Perform linear transformation to generate the corresponding eigenvector q i att , k j att and v j att , as shown below:

[0077]

[0078] in, and The adjacency matrix The i-th and j-th rows represent the connection relationship between nodes i and j in the graph.

[0079] 2.2) Calculate the attention score e ij , and normalize it (such as through the Softmax function) to obtain the attention weight α ij :

[0080]

[0081] Among them, q i att is the query vector of node i, k j att is the key vector of node j, e ij It reflects the strength of the association between node i and node j.

[0082] 2.3) Reconstruct the adjacency matrix:

[0083] The attention weight α ij is multiplied with the feature vector v j . att A weighted sum is performed to obtain the reconstructed feature h i of node i. i The specific process is as follows:

[0084]

[0085] Finally, the features of all nodes are spliced to obtain the reconstructed adjacency matrix A'. A' can dynamically adjust the relationship weight based on global information, rather than being limited to pre-defined local connections. This enables the model to better capture the complex relationships between nodes and improve the prediction ability of the future trajectory of the aircraft. Figure 2 The processing process diagram of the adjacency matrix reconstruction module based on the self-attention mechanism provided in the embodiment is shown in the following figure.

[0086] Step 3: Feature extraction is performed using the spatio-temporal graph convolution network operation and the graph attention network operation, and the specific steps are as follows:

[0087] 3.1) Spatio-temporal graph convolution network operation:

[0088] The reconstructed adjacency matrix A' in 2.3) is used for spatio-temporal graph convolution operation.

[0089] The features of all aircraft at time t in the l-th network layer are defined as V t (l) . t The specific definition of V (l) is consistent with that in 1.1), and the present application uses V t to represent the stacking in the time dimension. (l) Finally, the operation of each network layer in the spatio-temporal graph convolution network is realized by the following formula:

[0090] V (l+1) = σ(A'V (l) W stgcn (l) ) (10)

[0091] where W stgcn (l) is the trainable network parameter of the l-th layer of the spatio-temporal graph convolution network, and σ(·) is the activation function. Finally, the embedding result output by the spatio-temporal graph convolution network is represented as where the dimension of the embedding result is , which is a preset value.

[0092] In one embodiment, the activation function σ(·) uses the ReLU activation function. Finally, the embedding result output by the spatiotemporal graph convolutional network is expressed as Where 9 is the dimension of the embedding result, 20 is the timestamp, and N is the number of aircraft in the scene.

[0093] 3.2) Graph Attention Network Operation:

[0094] The graph attention network dynamically assigns different weights to each neighbor node through the self-attention mechanism, allowing the model to capture more potential connections between nodes. First, the graph attention coefficient is calculated using the following formula:

[0095]

[0096] in, is the attention coefficient from node j to node i, N i Represents the set of neighbor nodes of node i. The input feature of the network is v={v1,v2,…v N}, N represents the number of nodes. GAT is the linear transformation weight matrix applied to each node, and a is the weight vector that maps the input to the feature space. Finally, the LeakyReLU activation function is used to provide nonlinear output and the SoftMax function is used for normalization. This application uses a multi-head graph attention network, so the output features of node i are as follows:

[0097]

[0098] in, is the normalized attention coefficient of the kth attention head, W GAT k is the linear transformation weight matrix of the kth attention head, v j is the input feature of the j-th node, and σ(·) is the activation function.

[0099] That is, the graph attention network dynamically assigns different weights to each neighbor node through the self-attention mechanism, so that the model can capture more potential connections between nodes. First, the graph attention coefficient is calculated by formula (11) It represents the attention coefficient from node j to node i. The input feature of the graph attention network is v = {v1, v2, ... v N}, N represents the number of nodes. In this embodiment, a multi-head graph attention network is used with 4 heads, so the output feature v of node i is i ′ is as follows:

[0100]

[0101] in, is the normalized attention coefficient of the kth attention head, W GAT k is the linear transformation weight matrix of the kth attention head, v j is the input feature of the j-th node, σ(·) is the activation function, and || represents the concatenation operation.

[0102] 3.3) Aggregate the output features in 3.1) and 3.2) to obtain the final output result V out , as shown below:

[0103]

[0104] Among them, STGCN(·) and GAT(·) represent the spatiotemporal graph convolutional network operation and graph attention network operation respectively, V represents the input node vector, and A′ represent the normalized adjacency matrix and the reconstructed adjacency matrix, respectively.

[0105] That is, the output features of the spatiotemporal graph convolutional network and the output features of the graph attention network are added and converged to obtain the final output result. Figure 3 The spatiotemporal graph convolutional network and graph attention network modules provided for this embodiment.

[0106] Step 4: Define the network training loss function and predict the trajectory. The specific steps are as follows:

[0107] 4.1) Predicted trajectory:

[0108] This application uses the probability distribution of the aircraft's future trajectory as the prediction result. This application assumes that the aircraft's trajectory at the future time t Satisfies the ternary Gaussian distribution, that is, in is the mean value of the trajectory distribution of the i-th aircraft at time t, is the covariance matrix of the trajectory distribution of the i-th aircraft at time t.

[0109] Therefore, the output of the predicted trajectory is the mean of the ternary Gaussian distribution and the covariance matrix The elements in , where t and i are the predicted time and aircraft number respectively. Since the output of the network cannot always satisfy the positive definiteness of the covariance matrix, the present invention outputs a lower triangular matrix with positive diagonals and through To construct the covariance matrix

[0110]

[0111] in, for The transpose of .

[0112] Finally, this application calculates the trajectory prediction results through a time-extrapolated convolutional network.

[0113] In one embodiment, in order to train the trajectory prediction model composed of the self-attention network (i.e., the adjacency matrix reconstruction module based on the self-attention mechanism) that reconstructs the adjacency matrix of each spatial graph, the spatiotemporal graph convolutional network, the graph attention network, and the trajectory prediction network, the network model parameters are trained by minimizing the negative log-likelihood loss, which is specifically defined as follows:

[0114]

[0115] Among them, W loss are all learnable parameters of the trajectory prediction model, T P is the length of the predicted time series.

[0116] In one embodiment, setting T P =12.

[0117] On the other hand, the embodiment of the present application also provides an aircraft 4D trajectory prediction system based on dual attention spatiotemporal graph convolution, such as Figure 4 As shown, it includes sensors, data transmission channels, data preprocessing modules, trajectory prediction models and management terminals; wherein, the trajectory prediction model includes: a trained adjacency matrix reconstruction module based on the self-attention mechanism, a spatiotemporal graph convolutional network, a graph attention network and a trajectory prediction network. In this prediction system, sensors are used to collect aircraft trajectory data in real time and transmit it to the data preprocessing module through a data transmission channel. The sensors include radar, laser, image and other sensors, and the data transmission channels include satellites, ADS-B, 4G / 5G, etc. The data preprocessing module is used to construct a spatiotemporal graph based on the aircraft trajectory data, extract the adjacency matrix sequence A of the spatiotemporal graph, and perform normalization processing on it to obtain a normalized adjacency matrix sequence and will The adjacency matrix reconstruction module based on the self-attention mechanism is fed into the trajectory prediction model; and the reconstructed adjacency matrix sequence A′ output by the spatiotemporal graph features and the adjacency matrix reconstruction module based on the self-attention mechanism is obtained by splicing the spatiotemporal graph features and the adjacency matrix sequence A′ to obtain the first fusion initial feature and feed it into the spatiotemporal graph convolutional network of the trajectory prediction model; splicing the spatiotemporal graph features and the adjacency matrix sequence The second fused initial features are obtained and sent to the graph attention network of the trajectory prediction model; the adjacency matrix reconstruction module based on the self-attention mechanism of the trajectory prediction model is used to reconstruct the adjacency matrix, and the spatiotemporal graph convolutional network is used to extract the first extracted features; the graph attention network is used to extract the second extracted features; based on the splicing layer, the first extracted features and the second extracted features are added and converged to obtain the extracted features of the spatiotemporal graph and sent to the trajectory prediction network to output the predicted trajectory of each aircraft; wherein, the spatiotemporal graph constructed by the data preprocessing module is a sequence of spatial graphs at multiple moments, wherein each node of the spatial graph represents an aircraft, and the node features of each node are used to standardize the flight status feature information of the corresponding aircraft; the edges between nodes represent the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t; for each spatial graph, based on the edges between nodes, the N×N dimensional adjacency matrix A of each spatial graph can be obtained t , where N represents the number of aircraft. Therefore, by stacking the adjacency matrix A at all times t The adjacency matrix sequence A of the space-time graph can be obtained. The management terminal is used to perform intelligent approach sorting, abnormal behavior detection, and / or AI-assisted decision-making based on the predicted trajectory of each aircraft output by the trajectory prediction model.

[0118] To further validate the predictive performance of the method / system proposed in this application, simulations were conducted based on two datasets. The two datasets used were generated based on real open-source ADS-B data from the airspace within a 50km radius centered on two large airports, and two dense airspaces with a radius of 100km above Europe and North America. The dataset contains 45,463 trajectory data, each of which includes five pieces of information: time, aircraft number, aircraft longitude, aircraft latitude, and aircraft altitude. The time interval between two trajectory data for each aircraft is 10 seconds.

[0119] This example uses the average displacement error (ADE) and final displacement error (FDE) to quantitatively evaluate the performance of the proposed method. The specific formulas are as follows:

[0120]

[0121] in and are respectively the actual horizontal or vertical trajectory of the aircraft and the horizontal or vertical trajectory predicted by the model, T p is the prediction timeframe, and N is the number of aircraft. The mean displacement error indicates the model's average prediction performance, while the final displacement error indicates the model's endpoint prediction performance. Smaller values ​​for these two metrics indicate better prediction performance.

[0122] In this example, the model was trained in batches of 128, using the ReLU activation function. Adam was selected as the training optimizer, with an initial learning rate of 0.001. After 200 training rounds, the learning rate was decayed to 0.0002, for a total of 400 rounds. In the specific implementation, the model predicted the trajectory distribution for the next 120 seconds based on the observed state of the past 80 seconds.

[0123] This embodiment was compared with existing LSTM, S-LSTM, ST-GAT, and S-STGCNN methods, using the aforementioned mean displacement error and final displacement error as comparison metrics. The final comparison results are shown in Table 1, where the values ​​on the left represent the mean displacement error and the values ​​on the right represent the final displacement error. The comparison results show that this embodiment achieved the best prediction metrics on both datasets, demonstrating superior performance.

[0124] This embodiment also visualizes the aircraft trajectory prediction results and compares them with the visualization results of the S-STGCNN method. Figure 5 As shown in Figure 5. (5a) and (5b) are the horizontal and vertical prediction visualization results of this embodiment, respectively, and (5c) and (5d) are the horizontal and vertical prediction visualization results of the S-STGCNN method, respectively. It can be seen from the visualization results that the prediction of the future trajectory of the aircraft by this embodiment is very close to the actual future trajectory, but the S-STGCNN method has a large deviation when predicting aircraft No. 1 and aircraft No. 2, which shows that the prediction performance of this embodiment is better.

[0125] Although the above describes the illustrative specific embodiments of the present invention to facilitate understanding of the present invention by those skilled in the art, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations using the concepts of the present invention are protected.

[0126] Table 1 Comparison of trajectory prediction performance of this embodiment with that of other methods

[0127]

[0128] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0129] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features being referred to. Thus, a feature defined as "first," "second," etc. may explicitly or implicitly include at least one of the features.

[0130] Any process or method description described in a flowchart or otherwise in this specification may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, as should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0131] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0132] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. Aircraft 4D trajectory prediction method based on dual-attention spatiotemporal graph convolution, characterized by: The following steps are involved: Step 1: Construct a spatiotemporal graph based on the aircraft trajectory data. The spatiotemporal graph is a sequence of spatial graphs at multiple moments. Each node in the spatial graph represents an aircraft, and the node feature of each node is used to standardize the flight status feature information of the corresponding aircraft. The edges between nodes represent the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t. For each spatial graph, based on the edges between nodes, the dimensional adjacency matrix , where N represents the number of aircraft; Step 2: Reconstruct the adjacency matrix of each spatial graph based on the self-attention network to obtain the reconstructed adjacency matrix ; Based on a trainable query weight matrix , key weight matrix , value weight matrix The normalized adjacency matrix The corresponding rows are linearly transformed to generate nodes The query vector ,node The key vector Sum value vector ,in and The adjacency matrix No. Hedi Row, where ; According to the formula Calculating attention score , and normalize it to get the attention weight ; The attention weight With value vector Perform weighted summation to get the node Reconstruction features ; Reconstruction features for all nodes Splicing to obtain the reconstructed adjacency matrix sequence ; Step 3: Extract features from the spatiotemporal graph based on the spatiotemporal graph convolutional network and graph attention network; Get spatial graph features based on the node features of all nodes in each spatial graph , splicing the spatial graph features of the same spatial graph and reconstruct the adjacency matrix Obtaining a first fused initial feature of each spatial graph; stacking the first fused initial features of all spatial graphs and using them as input data of a spatiotemporal graph convolutional network, and obtaining a first extracted feature based on the output features of the spatiotemporal graph convolutional network; Splicing spatial graph features of the same spatial graph And the normalized adjacency matrix Obtain the second fused initial features of each spatial graph, stack the second fused initial features of all spatial graphs and use them as input data of the graph attention network, and obtain the second extracted features based on the output features of the graph attention network; Then, the first extracted feature and the second extracted feature are added together to obtain the extracted feature of the spatiotemporal graph; Step 4: Input the extracted features of the spatiotemporal graph into the trajectory prediction network to obtain the predicted trajectory of each aircraft.

2. The method according to claim 1, wherein In step 1, the flight status characteristic information of the flying communication device includes: the position, speed and heading of the aircraft.

3. The method according to claim 1, wherein In step 1, a kernel function is used to characterize the degree of mutual influence between the two aircraft.

4. The method according to claim 1, wherein Normalized adjacency matrix , where the auxiliary matrix , is the identity matrix, is the normalized adjacency matrix The diagonal node degree matrix of .

5. The method according to claim 1, wherein In step 3, a multi-head graph attention network is used to extract the second fused initial features to obtain the second extracted features.

6. The method according to claim 1, wherein In step 4, the trajectory prediction network is used to output the trajectory distribution mean and trajectory distribution covariance matrix of the aircraft.

7. The method according to claim 1, wherein In step 4, the trajectory prediction network uses a time-extrapolated convolutional network.

8. The method according to claim 6 or 7, wherein: When jointly training the self-attention network, spatiotemporal graph convolutional network, graph attention network, and trajectory prediction network to reconstruct the adjacency matrix of each spatial graph, the loss function used is: ； in, are all learnable parameters of the trajectory prediction model based on the self-attention network, spatiotemporal graph convolutional network, graph attention network and trajectory prediction network. To predict the length of the time series, Indicates the Aircraft in The trajectory of time, for Moment The mean value of the trajectory distribution of the aircraft, for Moment The covariance matrix of the trajectory distribution of the aircraft.

9. Aircraft 4D trajectory prediction system based on dual-attention spatiotemporal graph convolution, characterized by: Includes sensors, data transmission channels, data preprocessing modules and trajectory prediction models; Among them, the sensor is used to collect the aircraft trajectory data in real time and transmit it to the data preprocessing module through the data transmission channel; The trajectory prediction model includes an adjacency matrix reconstruction module based on the self-attention mechanism, a spatiotemporal graph convolutional network, a graph attention network, and a trajectory prediction network; The data preprocessing module is used to construct a space-time graph based on the aircraft trajectory data and extract the adjacency matrix sequence of the space-time graph , and normalize it to obtain the normalized adjacency matrix sequence , and The adjacency matrix reconstruction module based on the self-attention mechanism is fed into the trajectory prediction model; and the reconstructed adjacency matrix sequence output by splicing the spatiotemporal graph features and the adjacency matrix reconstruction module based on the self-attention mechanism is Get the first fused initial features and feed them into the spatiotemporal graph convolutional network of the trajectory prediction model; concatenate the spatiotemporal graph features and the adjacency matrix sequence Get the second fused initial features and send them into the graph attention network of the trajectory prediction model; The trajectory prediction model's self-attention-based adjacency matrix reconstruction module is used to reconstruct the adjacency matrix. The spatiotemporal graph convolutional network is used to extract the first extracted features. The graph attention network is used to extract the second extracted features. The first extracted features and the second extracted features are added together based on the concatenation layer to obtain the extracted features of the spatiotemporal graph and fed into the trajectory prediction network to output the predicted trajectory of each aircraft. The spatiotemporal graph constructed by the data preprocessing module is a sequence of spatial graphs at multiple moments, wherein each node of the spatial graph represents an aircraft, and the node feature of each node is used to standardize the flight status feature information of the corresponding aircraft; the edge between the nodes represents the degree of mutual influence between the aircraft corresponding to the two nodes at the current moment t; for each spatial graph, based on the edge between the nodes, the dimensional adjacency matrix , where N represents the number of aircraft; Among them, the reconstructed adjacency matrix sequence Specifically: Based on a trainable query weight matrix , key weight matrix , value weight matrix The normalized adjacency matrix The corresponding rows are linearly transformed to generate nodes The query vector ,node The key vector Sum value vector ,in and The adjacency matrix No. Hedi Row, where ; According to the formula Calculating attention score , and normalize it to get the attention weight ; The attention weight With value vector Perform weighted summation to get the node Reconstruction features ;Reconstruction features of all nodes Splicing to obtain the reconstructed adjacency matrix sequence .

Images