Track intelligent prediction method based on transformer-lstm

Abstract

The embodiment of the application relates to the field of aerospace technology, and particularly relates to a track intelligent prediction method based on a Transformer-LSTM, which comprises the following steps: obtaining information of a multi-sensor track intelligent processing system and performing preprocessing, association and fusion to obtain original track data; wherein the original track data comprises original longitude and latitude data and original height data; converting the original track data to a unified Cartesian coordinate system, and performing normalization processing on the track data after spatial coordinate conversion and the original height data by using a maximum-minimum normalization function; inputting the normalized track data and the normalized height data into a Transformer network and an LSTM network of a pre-constructed track intelligent prediction model respectively, performing extraction and learning of spatial position features and time sequence features, and finally outputting a track prediction result by an output network of the track intelligent prediction model based on the spatial position features and the time sequence features, so that more accurate track prediction is realized.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of aerospace technology, in particular to a flight path intelligent prediction method based on Transformer-LSTM. BACKGROUND

[0002] With the rapid development of the aviation industry, air traffic flow is increasing, and the complexity and safety risk of flight operation are also increasing. In order to ensure the safe and efficient operation of flights, air traffic management needs to accurately predict the flight path of the aircraft. Flight path prediction can help air traffic management personnel to master the flight state of the aircraft in real time, and timely discover and handle potential flight conflicts, so as to ensure the safe operation of flights. The progress of sensor technology, optimization of intelligent algorithm and progress of aviation technology make the aircraft trajectory completion algorithm become a hot spot in the field of aviation research.

[0003] Currently, flight path prediction methods can be roughly divided into three categories: flight path prediction methods based on aerodynamic models, flight path prediction methods based on hybrid estimation theory models, and flight path prediction methods based on deep learning models.

[0004] The flight path prediction method based on aerodynamic model regards the air target as a particle, usually uses the physical characteristics of the target aircraft and the environmental conditions to construct the motion model during its flight, and relies on the dynamics formula to predict the future short-term flight path of the target. Wang Chao et al. established a 4D flight path prediction method based on the basic flight model, successfully modeled the horizontal flight path, height profile and speed profile of the aircraft, thereby realizing the overall fitting of the 4D flight path. Zhang Junfeng et al. used aircraft performance data, navigation adaptability and other trends to construct the target motion model, improving the accuracy of four-dimensional flight path prediction. Lee et al. derived a nonlinear dynamics model of the air target in multiple flight states, and used the continuous state transition probability to model the discrete transition between flight modes to predict and track the air target. Takeichi et al. analyzed the characteristics of the air target flight path data in depth, and combined with the time error and meteorological data, proposed an adaptive target flight path prediction model.

[0005] The track prediction method based on hybrid estimation theory model regards the track prediction problem as a stochastic linear mixed estimation problem, mainly using Kalman filter algorithm, improved Kalman filter algorithm, and adaptive filter algorithm to estimate the future trend of the track. Han Yunxiang et al. divided the aircraft operating state into three states: climbing, descending and flying, and established the aircraft dynamics model under different states based on hybrid system theory. Zhang Junfeng et al. proposed an improved interactive multi-mode algorithm, redefined the likelihood function to update the flight mode probability of the aerial target, eliminated the problem of flight mode difficult to convert caused by the incomplete mode set in the IMM algorithm (Interacting Multiple Model), and improved the track prediction accuracy. Lv Bo et al. introduced the improved six-degree-of-freedom model based on the motion of the aerial target and the wind field disturbance function into the Kalman filter algorithm, improved the track prediction accuracy and optimized the prediction interval.

[0006] With the development of artificial intelligence and deep learning technology, more and more research teams apply deep learning models to track prediction. Deep learning models can learn the motion rules and features of targets from massive data, and have strong adaptive and generalization capabilities. Hu Yukexie et al. established a track prediction model based on RNN (Recurrent Neural Network), and used the symmetric segmented path distance model to preprocess the original track data, which greatly eliminated the abnormal data and noise in the original track data, thereby reducing the influence of abnormal track data on the prediction accuracy. Shi Qingyan et al. proposed an online updating LSTM (Long Short Term Memory), which learned the potential association between a large number of historical track data, and realized real-time prediction of the flight target. Liu Shanshan et al. comprehensively utilized the advantages of convolutional neural network and bidirectional LSTM, and proposed a hybrid deep learning network to predict the future track of the target. Yu Chen et al. combined the difference autoregressive moving average model with the bidirectional gated recurrent neural network, and proposed an ARIMA-BIGRU track hybrid prediction model, which used the ARIMA model to preliminarily predict the track, and then corrected the residual sequence generated in the prediction process through the BIGRU (Bidirectional Gated Recurrent Unit) neural network, thereby improving the accuracy of track prediction.

[0007] However, the above-mentioned track prediction method still faces some difficulties. First, it is difficult to establish an accurate dynamic model because the prior information of the aerial target is usually unknown. Second, the deep learning models such as RNN and LSTM do not fully process the spatial position and time sequence features when processing track data, and can only capture part of the features, which leads to poor prediction accuracy. Third, although the models proposed so far can process time series data, they are mostly sequential calculation methods, which leads to serialization of calculation and reduction of calculation efficiency, making it difficult to effectively capture long-distance dependencies in long sequence data and limiting the prediction effect. SUMMARY

[0008] To solve the above technical problems, the embodiments of the present application propose a track intelligent prediction method based on Transformer-LSTM, which adopts a track intelligent prediction model based on Transformer-LSTM, effectively captures the long-term dependencies and time sequence information in the track data, more deeply understands and fully utilizes the space-time relationship in the track data, and thus realizes more accurate track prediction, providing reliable support for real-time decision-making in the field of aerospace technology.

[0009] To achieve the above-mentioned purpose, the embodiments of the present application propose a track intelligent prediction method based on Transformer-LSTM, which comprises the following steps: obtaining information of a multi-sensor track intelligent processing system and pre-processing, associating and fusing to obtain original track data; wherein the original track data includes original latitude and longitude data and original height data; converting the original track data to a unified Cartesian coordinate system, and using a maximum and minimum normalization function to normalize the track data after spatial coordinate conversion and the original height data respectively; inputting the normalized track data and the normalized original height data into the Transformer network and the LSTM network of the track intelligent prediction model respectively, extracting and learning the spatial position features and the time sequence features, and finally outputting the track prediction result based on the spatial position features and the time sequence features by the output network of the track intelligent prediction model.

[0010] In order to achieve the above object, the embodiment of the present application also proposes a track intelligent prediction system based on Transformer-LSTM, which comprises an original processing module, a normalization processing module and a track intelligent prediction model, and the track intelligent prediction model is specifically composed of a parallel Transformer network and an LSTM network, and an output network; the original processing module is used for obtaining information of a multi-sensor track intelligent processing system and performing preprocessing, association and fusion to obtain original track data, wherein the original track data comprises original latitude and longitude data and original height data; the normalization processing module is used for converting the original track data to a unified Cartesian coordinate system and performing normalization processing on the track data after spatial coordinate conversion and the original height data by using a maximum and minimum normalization function; the Transformer network is used for receiving the normalized track data, performing extraction and learning of spatial position features based on the normalized track data, and obtaining spatial position features; the LSTM network is used for receiving the normalized original height data, performing extraction and learning of time sequence features based on the normalized original height data, and obtaining time sequence features; and the output network is used for outputting a track prediction result based on the spatial position features and the time sequence features.

[0011] The track intelligent prediction method based on Transformer-LSTM proposed by the embodiment of the present application, compared with the traditional method, adopts a track intelligent prediction model constructed by a Transformer network and an LSTM network as cores to perform a track intelligent prediction task, the Transformer network is used for extraction and learning of spatial position features, thereby effectively capturing long-term dependence in track data, and the LSTM network is used for extraction and learning of time sequence features, thereby effectively capturing time sequence information in track data. The outputs of the Transformer network and the LSTM network are fused to perform track intelligent prediction, which can more deeply understand and fully utilize the space-time relationship in track data, and provides reliable support for real-time decision-making in the field of aerospace technology.

[0012] In some optional embodiments, the coordinates of the pth track point in the original track data in the geodetic coordinate system are (lat p ,lon p ,H p ), lat p represents original dimension data of the track point p, lon p represents original longitude data of the track point p, and H p represents original height data of the track point p, and converting the original track data to a unified Cartesian coordinate system comprises:

[0013] First, based on the geometric model of the earth, the original flight data is converted from the geodetic coordinate system to the meridian plane coordinate system Oxy, and the coordinates of the flight point p in the meridian plane coordinate system Oxy are represented as:

[0014] x p =(N+H)cos(lat p );

[0015] y p =[N(1-e 2 )+H]sin(lat p );

[0016] e 2 =(a 2 -b 2 ) / a 2 ;

[0017]

[0018] Wherein, e 2 is the eccentricity of the earth's ellipsoid, N is the radius of curvature of the earth, a is the length of the long axis of the earth, b is the length of the short axis of the earth, (x p , y p ) represents the coordinates of the flight point p in the meridian plane coordinate system Oxy;

[0019] Then, the original flight data is converted from the meridian plane coordinate system Oxy to the Cartesian coordinate system, and the coordinates of the flight point p in the Cartesian coordinate system are represented as:

[0020] X p =x p cos(lon p )=(N+H)cos(lat p )cos(lon p );

[0021] Y p =x p sin(lon p )=(N+H)cos(lat p )cos(lon p );

[0022] Z p =y p =[N(1-e 2 )+H]sin(lat p );

[0023] Wherein, (X p , Y p , Z p) represents the coordinates of the track point p in the Cartesian coordinate system.

[0024] In some optional embodiments, the track data after spatial coordinate conversion and the original height data are normalized respectively using a max-min normalization function, which is expressed by the formula:

[0025]

[0026] wherein min(·) and max(·) represent the minimum value function and the maximum value function respectively, data represents the track data after spatial coordinate conversion, data' represents the normalized track data, H p represents the original height data of the track point p, H' p represents the normalized original height data of the track point p.

[0027] In some optional embodiments, the Transformer network is composed of an input embedding layer, a Transformer encoder, an output embedding layer, a Transformer decoder, a linear layer and a Softmax layer, the Transformer encoder is composed of six layers of Transformer encoder layers connected in sequence, and the Transformer decoder is composed of six layers of Transformer decoder layers connected in sequence;

[0028] The normalized track data is input into the input embedding layer, is converted by the linear layer and is added with position encoding, is then input into the Transformer encoder, the delayed normalized track data is input into the output embedding layer, is converted by the linear layer and is added with position encoding, and is then input into the Transformer decoder;

[0029] The output of the input embedding layer is the input of the first layer of the Transformer encoder layer, the output of the previous layer of the Transformer encoder layer is the input of the next layer of the Transformer encoder layer, and the output of the last layer of the Transformer encoder layer is input into each layer of the Transformer decoder layer;

[0030] The output of the output embedding layer is the input of the first layer of the Transformer decoder layer, the output of the previous layer of the Transformer decoder layer is the input of the next layer of the Transformer decoder layer, and the output of the last layer of the Transformer decoder layer is processed by the linear layer and the Softmax layer to obtain the spatial position feature.

[0031] In some optional embodiments, each layer of the Transformer encoder layer is composed of a multi-head attention mechanism module, two layer normalization modules and a feedforward neural network layer.

[0032] The calculation process of the single-layer attention mechanism is represented by the formula:

[0033]

[0034] wherein Q, K, and V are query vector, key vector, and value vector respectively, d k represents the dimension of the key vector, softmax(·) represents the Softmax function, and Attention(Q, K, V) represents the single-layer attention mechanism.

[0035] After the single-layer attention mechanism with h attention heads is processed, the similar functions of different modes are obtained, the output vectors of multiple attention heads are combined, and are projected back to the original dimension to obtain the output of the multi-head attention mechanism.

[0036] The output of the multi-head attention mechanism is represented by the formula:

[0037] MultiHeadAttention(Q, K, V)=Concat(head1, head2, …, head h )W O ;

[0038]

[0039] wherein W O is a projection matrix, , and are attention score matrices corresponding to Q, K, and V of the hth attention head respectively, Concat(·) represents the concatenation operation, and MultiHeadAttention(Q, K, V) represents the output of the multi-head attention mechanism.

[0040] The multi-head attention mechanism module crosses the original input through the multi-head attention mechanism module to the first layer normalization module through a residual connection, and continues to cross the added output through the feedforward neural network layer to the second layer normalization module through a residual connection.

[0041] The calculation process of the layer normalization module is represented by the formula:

[0042] Add&Norm(x in )=ReLU[(x in W1+b1)W2+b2];

[0043] wherein x in is the input track point feature, W1 and W2 are weight matrices of two linear transformation layers respectively, b1 and b2 are bias vectors, and ReLU(·) represents the ReLU activation function.

[0044] In some alternative embodiments, each layer of the Transformer decoder is composed of a masked multi-head attention mechanism module, three layer normalization modules and two feed-forward neural network layers;

[0045] The masked multi-head attention mechanism module needs to mask the input with the predicted part;

[0046] The output of the output embedding layer is converted by a linear layer and added with position encoding, and then serves as the V of the multi-head attention mechanism, and the Q and K of the multi-head attention mechanism come from the linear projection of the input of the Transformer encoder.

[0047] In some alternative embodiments, the LSTM network is composed of three sequentially connected LSTM layers, the output of the previous LSTM layer is the input of the next LSTM layer, and the output of the last LSTM layer is the time sequence feature;

[0048] Each LSTM layer includes a forget gate, an input gate and an output gate, which are constructed by the following formulas:

[0049] f t =σ(W f ·[h t-1 ,x t ]+b f );

[0050] i t =σ(W i ·[h t-1 ,x t ]+b i );

[0051] o t =σ(W o ·[h t-1 ,x t ]+b o );

[0052] Wherein, x t represents the input of the t-th time step, h t-1 represents the hidden state of the t-1-th time step, which contains the understanding and memory of the input, W f , W i , W o are the weight matrices of the forget gate, the input gate and the output gate respectively, b f , b i , b o are the bias vectors of the forget gate, the input gate and the output gate respectively, and σ(·) is the Sigmoid function;

[0053] The output and the update state are selected by the following formulas:

[0054]

[0055] wherein W c is the weight matrix of the long-term memory unit, b c is the bias vector of the long-term memory unit;

[0056] The long-term memory unit and the state output are updated by the following formula:

[0057] c t = i t *c t +f t *c t-1 ;

[0058] h t =o t *tanh(c t );

[0059] The mapping output of the LSTM layer is constructed by the following formula:

[0060] y l =w y *h t +b y ;

[0061] wherein y l is the mapping output of the LSTM layer, w y is the output matrix of the fully connected layer, and b y is the bias vector of the fully connected layer.

[0062] In some optional embodiments, when training the intelligent track prediction model, a joint loss function is used, which is represented by the formula as follows:

[0063]

[0064] wherein B is the total number of data, subscript i is the i-th component in the data, X i is the i-th component in the output of the Transformer network, is the i-th component in the real label of the Transformer, H i is the i-th component in the output of the LSTM network, is the i-th component in the real label of the LSTM, and LOSS is the joint loss function, which uses mean square error to evaluate the difference between the two, and uses Adam optimizer to update the network parameters of the intelligent track prediction model. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0066] Figure 1 is a flowchart of a track intelligent prediction method based on Transformer-LSTM provided in an embodiment of the present application;

[0067] Figure 2 is a structural schematic diagram of a track intelligent prediction model provided in an embodiment of the present application;

[0068] Figure 3 is a structural schematic diagram of a Transformer network provided in an embodiment of the present application;

[0069] Figure 4 is a connection relationship schematic diagram of a Transformer encoder layer and a Transformer decoder layer in a Transformer network provided in an embodiment of the present application;

[0070] Figure 5 is a structural schematic diagram of an LSTM network provided in an embodiment of the present application;

[0071] Figure 6 is a structural schematic diagram of an LSTM layer provided in an embodiment of the present application;

[0072] Figure 7 is a structural schematic diagram of a track intelligent prediction system based on Transformer-LSTM provided in an embodiment of the present application;

[0073] Figure 8 is a structural schematic diagram of an electronic device deployed in another embodiment of the present application. DETAILED DESCRIPTION

[0074] To make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with the accompanying drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are presented in order to make the readers better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and based on various changes and modifications of the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation modes of the present application. The embodiments can be combined and referenced with each other without contradiction.

[0075] One embodiment of the present application proposes a track intelligent prediction method based on Transformer-LSTM, applied to an electronic device, wherein the electronic device can be a terminal or a server. In the embodiments of the present application and the following embodiments, the server is taken as an example for description. The implementation details of the track intelligent prediction method based on Transformer-LSTM proposed in the present embodiment will be described in detail below. The following content only provides related implementation details for the convenience of understanding, and is not necessary for implementing the present solution.

[0076] The specific process of the track intelligent prediction method based on Transformer-LSTM proposed in the present embodiment can be as shown in Figure 1 , which includes:

[0077] In step 11, information of the multi-sensor track intelligent processing system is acquired and preprocessed, associated, and fused to obtain original track data, wherein the original track data includes original latitude and longitude data and original height data.

[0078] In the specific implementation, the server first needs to acquire information from the multi-sensor track intelligent processing system, pre-process, associate, and fuse the acquired information to obtain original track data, wherein the original track data includes original latitude and longitude data and original height data.

[0079] In one example, each piece of original track data contains P track points, and each track point corresponds to time data, latitude data, longitude data, and height data.

[0080] The kth original track data can be represented by a formula as:

[0081] S k ={(t1,lat1,lon1,H1),(t2,lat2,lon2,H2),...,(t P ,lat P ,lon P ,H P )};

[0082] wherein t P , lat P , lon P and H P represent time data, latitude data, longitude data and height data of the track point P respectively, S k represents the kth original track data.

[0083] In one example, the preprocessing operation specifically comprises parsing the obtained information of the multi-sensor track intelligent processing system with a radar message format, and extracting valid information and storing the same into a database.

[0084] Step 12, converting the original track data into a unified Cartesian coordinate system, and using a maximum-minimum normalization function to normalize the track data after spatial coordinate conversion and the original height data respectively.

[0085] In a specific implementation, after obtaining the original track data, the server needs to convert the original track data into a unified Cartesian coordinate system, and use a maximum-minimum normalization function to normalize the track data after spatial coordinate conversion and the original height data respectively.

[0086] In one example, it is assumed that the coordinate of the pth track point in the original track data in the geodetic coordinate system (latitude and longitude coordinate system) is (lat p , lon p , H p ), lat p represents the original dimension data of the track point p, lon p represents the original longitude data of the track point p, and H p represents the original height data of the track point p.

[0087] When the server converts the original track data into a unified Cartesian coordinate system, it first needs to convert the original track data from the geodetic coordinate system to the meridian plane coordinate system Oxy based on the geometric model of the earth.

[0088] The coordinate of the track point p in the original track data in the meridian plane coordinate system Oxy is represented as:

[0089] x p = (N + H)cos(lat p );

[0090] y p = [N(1-e 2 ) + H]sin(lat p );

[0091] e 2 = (a 2 -b2 ) / a 2 ;

[0092]

[0093] Among them, e 2 Let be the eccentricity of the Earth's ellipsoid, N be the radius of curvature of the Earth, a be the length of the Earth's semi-major axis (6378137 m), and b be the length of the Earth's semi-minor axis (6356752.3142451793 m). p ,y p ) represents the coordinates of track point p in the meridian plane coordinate system Oxy.

[0094] Subsequently, the server transforms the original track data from the meridian plane coordinate system Oxy to the Cartesian coordinate system (geocentric rectangular coordinate system). The coordinates of track point p in the original track data in the Cartesian coordinate system are represented as follows:

[0095] X p =x p cos(lon p )=(N+H)cos(lat p )cos(lon p );

[0096] Y p =x p sin(lon p )=(Z+H)cos(lat p )cos(lon p );

[0097] Z p =y p =[N(1-e 2 )+H]sin(lat p );

[0098] Among them, (X) p ,Y p Z p ) represents the coordinates of track point p in the Cartesian coordinate system.

[0099] In one example, the server uses the max-min normalization function to normalize the track data after spatial coordinate transformation and the original altitude data, respectively, as expressed by the formula:

[0100]

[0101] wherein min(·) and max(·) represent minimum value function and maximum value function respectively, data represents the spatial coordinate converted track data, data' represents the normalized track data, H p represents the original height data of the track point p, H' p represents the normalized original height data of the track point p.

[0102] Step 13, input the normalized track data and the normalized original height data into the Transformer network and the LSTM network of the track intelligent prediction model respectively, to extract and learn the spatial position features and the time sequence features, and finally output the track prediction result based on the spatial position features and the time sequence features by the output network of the track intelligent prediction model.

[0103] In a specific implementation, the server pre-constructs and trains the track intelligent prediction model to perform the task of track intelligent prediction, the track intelligent prediction model is composed of a Transformer network, an LSTM network and an output network, and the Transformer network and the LSTM network are parallel. The server inputs the normalized track data and the normalized original height data into the Transformer network and the LSTM network of the track intelligent prediction model respectively, to extract and learn the spatial position features and the time sequence features, and finally output the track prediction result based on the spatial position features and the time sequence features by the output network of the track intelligent prediction model.

[0104] In one example, the specific structure of the track intelligent prediction model can be as shown in Figure 2 .

[0105] In one example, the specific structure of the Transformer network can be as shown in Figure 3 . The Transformer network is composed of an input embedding layer, a Transformer encoder, an output embedding layer, a Transformer decoder, a linear layer and a Softmax layer, the Transformer encoder is sequentially connected by six layers of Transformer encoder layers, the Transformer decoder is sequentially connected by six layers of Transformer decoder layers. The connection relationship between the Transformer encoder layer and the Transformer decoder layer can be as shown in Figure 4 .

[0106] In one example, the normalized trajectory data is input into an input embedding layer, transformed by a linear layer and added with position encoding, and then input into a Transformer encoder. The delayed normalized trajectory data is input into an output embedding layer, transformed by a linear layer and added with position encoding, and then input into a Transformer decoder.

[0107] In one example, the position encoding can be represented by the following formula:

[0108]

[0109] where PositionalEncoding(pos, i) represents the result of position encoding.

[0110] The output of the input embedding layer is the input of the first layer of the Transformer encoder. The output of the previous layer of the Transformer encoder is the input of the next layer of the Transformer encoder. The output of the last layer of the Transformer encoder is input into each layer of the Transformer decoder.

[0111] The output of the output embedding layer is the input of the first layer of the Transformer decoder. The output of the previous layer of the Transformer decoder is the input of the next layer of the Transformer decoder. The output of the last layer of the Transformer decoder is processed by a linear layer and a Softmax layer to obtain the spatial position feature.

[0112] In one example, each layer of the Transformer encoder is composed of a multi-head attention mechanism module, two layer normalization modules, and a feedforward neural network layer.

[0113] The calculation process of a single-layer attention mechanism is represented by the following formula:

[0114]

[0115] where Q, K, and V are query vector, key vector, and value vector, respectively, d k represents the dimension of the key vector, softamx(·) represents the Softmax function, and Attention(Q, K, V) represents a single-layer attention mechanism.

[0116] After processing by the single-layer attention mechanism with h attention heads, the similarity function of different modes is obtained. The output vectors of multiple attention heads are combined and projected back to the original dimension to obtain the output of the multi-head attention mechanism.

[0117] The output of the multi-head attention mechanism is represented by the following formula:

[0118] MultiHeadAttention(Q, L, V) = Concat(head1, head2, …, head h )W O ;

[0119]

[0120] where W O is a projection matrix, are the attention score matrices corresponding to Q, K, V of the h-th attention head respectively, Concat(·) denotes the concatenation operation, and MultiHeadAttention(Q, K, V) denotes the output of the multi-head attention mechanism.

[0121] The multi-head attention mechanism module passes the original input across the multi-head attention mechanism module to the first layer normalization module through a residual connection, and continues to pass the added output across the feedforward neural network layer to the second layer normalization module through a residual connection.

[0122] The calculation process of the layer normalization module is represented by the formula:

[0123] Add&Norm(x in ) = ReLU[(x in W1 + b1)W2 + b2];

[0124] where x in is the input track point feature, W1 and W2 are the weight matrices of the two linear transformation layers, b1 and b2 are the bias vectors, and ReLU(·) represents the ReLU activation function.

[0125] In one example, each Transformer decoder layer is composed of a masked multi-head attention mechanism module, three layer normalization modules, and two feedforward neural network layers. The masked multi-head attention mechanism module needs to mask the input with the predicted part. The output of the output embedding layer is converted by a linear layer and added with position encoding, and then used as the V of the multi-head attention mechanism. The Q and K of the multi-head attention mechanism come from the linear projection of the input of the Transformer encoder. The output of the Transformer decoder is converted by a linear layer to calculate the Softmax probability distribution result.

[0126] In one example, the specific structure of the LSTM network can be as shown in Figure 5 The LSTM network is composed of three sequentially connected LSTM layers, the output of the previous LSTM layer is the input of the next LSTM layer, and the output of the last LSTM layer is the time sequence feature.

[0127] In one example, the structure of the LSTM layer can be as shown in FIG. 1, each LSTM layer includes a forget gate, an input gate and an output gate, which are constructed by the following formulas: Figure 6

[0128] f t =σ(W f ·[h t-1 ,x t ]+b f );

[0129] i t =σ(W i ·[h t-1 ,x t ]+b i );

[0130] o t =σ(W o ·[h t-1 ,x t ]+b o );

[0131] where x t represents the input of the t-th time step, h t-1 represents the hidden state of the t-1-th time step, which contains the understanding and memory of the input, W f , W i , W o are the weight matrices of the forget gate, the input gate and the output gate, respectively, b f , b i , b o are the bias vectors of the forget gate, the input gate and the output gate, respectively, and σ(·) is a Sigmoid function.

[0132] The LSTM layer selects the output and updates the state by the following formulas:

[0133]

[0134] where W c is the weight matrix of the long-term memory unit, and b c is the bias vector of the long-term memory unit.

[0135] The LSTM layer updates the long-term memory unit and the state output by the following formulas:

[0136] c t =i t *c t +f t *c t-1 ;

[0137] h t =o​t • tanh(c t ).

[0138] The mapping output of the LSTM layer is constructed by the following formula:

[0139] y l = w y · h t + b y ;

[0140] where y l is the mapping output of the LSTM layer, w y is the output matrix of the fully connected layer, and b y is the bias vector of the fully connected layer.

[0141] In one example, the joint loss function used by the server when training the intelligent track prediction model can be represented by the following formula:

[0142]

[0143] where B is the total number of data, subscript i is the i-th component in the data, X i is the i-th component in the output of the Transformer network, is the i-th component in the true label of the Transformer , H i is the i-th component in the output of the LSTM network, is the i-th component in the true label of the LSTM , and LOSS is the joint loss function. The joint loss function uses mean square error to evaluate the difference between the two, and uses the Adam optimizer to update the network parameters of the intelligent track prediction model.

[0144] In one example, the formula for updating the network parameters is:

[0145]

[0146] where θ1 represents the network parameters before updating, and θ2 represents the network parameters after updating.

[0147] ​Compared with the traditional method, the intelligent track prediction method based on the Transformer-LSTM proposed in this embodiment adopts an intelligent track prediction model constructed by a Transformer network and an LSTM network as the core to perform an intelligent track prediction task. The Transformer network is used to extract and learn spatial position features, thereby effectively capturing long-term dependencies in track data. The LSTM network is used to extract and learn time sequence features, thereby effectively capturing time sequence information in track data. The outputs of the Transformer network and the LSTM network are fused to perform intelligent track prediction, which can more deeply understand and fully utilize the space-time relationship in track data, thereby providing reliable support for real-time decision-making in the field of aerospace technology.

[0148] The step division of the above various methods is only for the purpose of clear description. In implementation, one step can be combined or some steps can be split and decomposed into multiple steps, as long as the same logical relationship is included, which is within the protection scope of the present application. Adding insignificant modifications or introducing insignificant designs in the algorithm or process, but not changing the core design of the algorithm and process, are within the protection scope of the present application.

[0149] Another embodiment of the present application proposes an intelligent track prediction system based on Transformer-LSTM. The details of the intelligent track prediction system based on Transformer-LSTM proposed in this embodiment are specifically described below. The following implementation details are provided for the convenience of understanding, and are not essential for implementing this embodiment.

[0150] Figure 7 FIG. 1 is a structural schematic diagram of an intelligent track prediction system based on Transformer-LSTM proposed in this embodiment, which includes an original processing module 21, a normalization processing module 22, and an intelligent track prediction model 30. The intelligent track prediction model 30 is specifically composed of a parallel Transformer network 31 and an LSTM network 32, and an output network 33.

[0151] The original processing module 21 is used to acquire information of a multi-sensor intelligent track processing system and perform preprocessing, association, and fusion to obtain original track data. The original track data includes original longitude and latitude data and original height data.

[0152] The normalization processing module 22 is used to convert the original track data to a unified Cartesian coordinate system and perform normalization processing on the track data after spatial coordinate conversion and the original height data using a maximum and minimum normalization function.

[0153] The transformer network 31 is configured to receive the normalized track data, extract and learn spatial position features based on the normalized track data, and obtain the spatial position features.

[0154] The LSTM network 32 is configured to receive the normalized original height data, extract and learn time sequence features based on the normalized original height data, and obtain the time sequence features.

[0155] The output network 33 is configured to output a track prediction result based on the spatial position features and the time sequence features.

[0156] It can be found that the embodiment is a system embodiment corresponding to the method embodiment, and the embodiment can be implemented in cooperation with the method embodiment. The related technical details and technical effects mentioned in each of the above embodiments are still valid in the embodiment. In order to reduce repetition, they will not be described here. Correspondingly, the related technical details mentioned in the embodiment can also be applied to the above embodiments.

[0157] It is worth mentioning that each module involved in the embodiment is a logical module. In actual application, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, units not closely related to solving the technical problems proposed in the present application are not introduced in the embodiment. However, this does not mean that there are no other units in the embodiment.

[0158] Another embodiment of the present application deploys an electronic device, which can have a specific structure as shown in Figure 8 The electronic device includes at least one processor 41 and a memory 42 connected with the at least one processor 41. The memory 42 stores instructions executable by the at least one processor 41. The instructions are executed by the at least one processor 41 to enable the at least one processor 41 to perform the track intelligent prediction method based on the Transformer-LSTM described in each of the above method embodiments.

[0159] The bus can include any number of interconnecting buses and bridges, allowing for a variety of configurations of peripheral devices, memory, and processors. The bus also can include various other circuits and devices that are well known in the art, such as power management, clock, and memory management circuits. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single device or a plurality of devices, such as a plurality of receivers and transmitters, that provide for communication with various other devices over a transmission medium. Processors are responsible for managing the bus and general processing, including the execution of software stored on the storage medium. The storage medium can be used for storing data that is placed into or retrieved from the processors.

[0160] The processor is responsible for managing the bus and general processing, including the execution of software stored on the storage medium. The storage medium can be used for storing data that is placed into or retrieved from the processors.

[0161] Those skilled in the art will understand that the steps of the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the programs are stored in a storage medium, including a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0162] Those skilled in the art will understand that the above-mentioned embodiments are specific embodiments for implementing the present application, and in actual applications, various changes can be made in form and details without departing from the spirit and scope of the present application.

Claims

1. A trajectory intelligent prediction method based on Transformer-LSTM, characterized in that, include: Information from a multi-sensor intelligent trajectory processing system is acquired, preprocessed, correlated, and fused to obtain raw trajectory data; the raw trajectory data includes raw latitude and longitude data and raw altitude data. The original track data is transformed into a unified Cartesian coordinate system, and the maximum-minimum normalization function is used to normalize the track data after spatial coordinate transformation and the original altitude data respectively. The normalized track data and the normalized original altitude data are respectively input into the Transformer network and LSTM network of the track intelligent prediction model to extract and learn spatial location features and temporal features. Finally, the output network of the track intelligent prediction model outputs the track prediction result based on the spatial location features and temporal features. The Transformer network consists of an input embedding layer, a Transformer encoder, an output embedding layer, a Transformer decoder, a linear layer, and a Softmax layer. The Transformer encoder is composed of six Transformer encoder layers connected in sequence, and the Transformer decoder is composed of six Transformer decoder layers connected in sequence. Normalized track data is input to the input embedding layer, transformed by the linear layer and with position encoding added, and then input to the Transformer encoder. Delayed normalized track data is input to the output embedding layer, transformed by the linear layer and with position encoding added, and then input to the Transformer decoder. The output of the input embedding layer is the input of the first Transformer encoder layer, the output of the previous Transformer encoder layer is the input of the next Transformer encoder layer, and the output of the last Transformer encoder layer will be input to each Transformer decoder layer. The output of the output embedding layer is the input of the first Transformer decoder layer, the output of the previous Transformer decoder layer is the input of the next Transformer decoder layer, and the output of the last Transformer decoder layer is processed by a linear layer and a softmax layer to obtain the spatial location features. Each Transformer decoder layer consists of a masked multi-head attention mechanism module, three layers of normalization modules, and two feedforward neural network layers; The masked multi-head attention mechanism module needs to mask the part of the input to be predicted; The output of the output embedding layer is transformed by a linear layer and then positional encoding is added, serving as the multi-head attention mechanism. Multi-head attention mechanism and The input is linearly projected from the Transformer encoder, and the output of the Transformer encoder is processed through a linear layer to calculate its Softmax probability distribution. in, , , These are the query vector, key vector, and value vector, respectively.

2. The intelligent trajectory prediction method based on Transformer-LSTM according to claim 1, characterized in that, Let the first one in the original track data be... The coordinates of each track point in the geodetic coordinate system are: , Indicates waypoints The original dimensional data, Indicates waypoints The original longitude data, Indicates waypoints The original altitude data was used to transform the original track data into a unified Cartesian coordinate system, including: First, based on the geometric model of the Earth, the original track data is transformed from the geodetic coordinate system to the meridian plane coordinate system Oxy. The track points in the original track data... The coordinates in the meridian plane coordinate system Oxy are represented as follows: ； ； ； ； in, The eccentricity of the Earth's ellipsoid. Let be the radius of curvature of the Earth. This is the length of the Earth's semi-major axis. This is the length of the Earth's minor axis. Indicates waypoints Coordinates in the meridian plane coordinate system Oxy; Then, the original track data is transformed from the meridian plane coordinate system Oxy to the Cartesian coordinate system, and the track points in the original track data are... The coordinates in the Cartesian coordinate system are represented as follows: ； ； ； in, Indicates waypoints Coordinates in the Cartesian coordinate system.

3. The intelligent trajectory prediction method based on Transformer-LSTM according to claim 2, characterized in that, The maximum-minimum normalization function is used to normalize both the track data after spatial coordinate transformation and the original altitude data, as expressed by the formula: ； ； in, and Let these represent the minimum and maximum value functions, respectively. This represents the track data after spatial coordinate transformation. This represents the normalized track data. Indicates waypoints The original height data, Indicates waypoints The normalized original height data.

4. The intelligent trajectory prediction method based on Transformer-LSTM according to claim 1, characterized in that, Each Transformer encoder layer consists of a multi-head attention mechanism module, two layer normalization modules, and a feedforward neural network layer; The computational process of a single-layer attention mechanism can be expressed by the following formula: ； in, , , These are the query vector, key vector, and value vector, respectively. This represents the dimension of the key vector. This represents the Softmax function. This represents a single-layer attention mechanism; through After processing by a single-layer attention mechanism with multiple attention heads, similarity functions of different modes are obtained. The output vectors of multiple attention heads are merged and projected back to the original dimension to obtain the output of the multi-head attention mechanism. The output of the multi-head attention mechanism can be expressed by the following formula: ； ； in, Let be the projection matrix. , , The respective A person's attention , , The corresponding attention score matrix, This indicates a splicing operation. This represents the output of the multi-head attention mechanism; The multi-head attention mechanism module passes the original input through the multi-head attention mechanism module to the first layer normalization module through residual connections, and then passes the summed output through the feedforward neural network layer to the second layer normalization module through residual connections. The calculation process of the layer normalization module is expressed by the formula: ； in, The input is the feature of the waypoints. and These are the weight matrices of the two linear transformation layers. and It is a bias vector. This represents the ReLU activation function.

5. The intelligent trajectory prediction method based on Transformer-LSTM according to claim 1, characterized in that, An LSTM network consists of three sequentially connected LSTM layers. The output of the previous LSTM layer is the input of the next LSTM layer, and the output of the last LSTM layer is the temporal feature. Each LSTM layer includes a forget gate, an input gate, and an output gate, which are constructed using the following formula: ； ； ； in, Indicates the first Input at each time step, Indicates the first The hidden states at each time step contain an understanding and memory of the input. 、 、 These are the weight matrices for the forget gate, input gate, and output gate, respectively. 、 、 These are the bias vectors for the forget gate, input gate, and output gate, respectively. For the Sigmoid function; Select the output and update status using the following formula: ； in, It is the weight matrix of long-term memory units. It is the bias vector of the long-term memory unit; Update the long-term memory unit and state output using the following formula: ； ； The mapping output of the LSTM layer is constructed using the following formula: ； in, The output of the LSTM layer is the mapping. This is the output matrix of the fully connected layer. This is the bias vector for the fully connected layer.

6. A trajectory intelligent prediction method based on Transformer-LSTM according to any one of claims 1 to 5, characterized in that, The joint loss function used when training the intelligent trajectory prediction model is expressed by the following formula: ； in, The total number of data points, index For the data One portion, The first in the output of the Transformer network One portion, Real labels for Transformer The first in One portion, The first in the output of the LSTM network One portion, True label for LSTM The first in One portion, For the joint loss function, The mean squared error was used to evaluate the difference between the two, and the Adam optimizer was used to update the network parameters of the trajectory intelligent prediction model.

7. A trajectory intelligent prediction system based on Transformer-LSTM, characterized in that, include: The system consists of a raw processing module, a normalization processing module, and a trajectory intelligent prediction model. The trajectory intelligent prediction model is specifically composed of a parallel Transformer network and an LSTM network, as well as an output network. The raw processing module is used to acquire information from the multi-sensor intelligent trajectory processing system and perform preprocessing, correlation and fusion to obtain raw trajectory data, which includes raw latitude and longitude data and raw altitude data. The normalization module is used to transform the original track data into a unified Cartesian coordinate system, and uses the maximum and minimum normalization functions to normalize the track data after spatial coordinate transformation and the original altitude data respectively. The Transformer network is used to receive normalized track data, extract and learn spatial location features based on the normalized track data, and obtain spatial location features. An LSTM network is used to receive normalized raw height data, and to extract and learn temporal features based on the normalized raw height data to obtain temporal features. The output network is used to output trajectory prediction results based on spatial location features and temporal features; The Transformer network consists of an input embedding layer, a Transformer encoder, an output embedding layer, a Transformer decoder, a linear layer, and a Softmax layer. The Transformer encoder is composed of six Transformer encoder layers connected in sequence, and the Transformer decoder is composed of six Transformer decoder layers connected in sequence. Normalized track data is input to the input embedding layer, transformed by the linear layer and with position encoding added, and then input to the Transformer encoder. Delayed normalized track data is input to the output embedding layer, transformed by the linear layer and with position encoding added, and then input to the Transformer decoder. The output of the input embedding layer is the input of the first Transformer encoder layer, the output of the previous Transformer encoder layer is the input of the next Transformer encoder layer, and the output of the last Transformer encoder layer will be input to each Transformer decoder layer. The output of the output embedding layer is the input of the first Transformer decoder layer, the output of the previous Transformer decoder layer is the input of the next Transformer decoder layer, and the output of the last Transformer decoder layer is processed by a linear layer and a softmax layer to obtain the spatial location features. Each Transformer decoder layer consists of a masked multi-head attention mechanism module, three layers of normalization modules, and two feedforward neural network layers; The masked multi-head attention mechanism module needs to mask the predictive part of the input; The output of the output embedding layer is transformed by a linear layer and then positional encoding is added, serving as the multi-head attention mechanism. Multi-head attention mechanism and The input is linearly projected from the Transformer encoder, and the output of the Transformer encoder is processed through a linear layer to calculate its Softmax probability distribution. in, , , These are the query vector, key vector, and value vector, respectively.

Images