Intelligent track prediction method based on Transform-LSTM

Abstract

The embodiment of the invention relates to the technical field of aerospace, in particular to an intelligent track prediction method based on Transform-LSTM (Long Short Term Memory), and the method comprises the steps: obtaining the information of a multi-sensor track intelligent processing system, and carrying out the preprocessing, association and fusion, and obtaining the original track data; wherein the original track data comprises original latitude and longitude data and original height data; converting the original track data into a unified Cartesian coordinate system, and performing normalization processing on the track data after space coordinate conversion and the original height data by using maximum and minimum normalization functions; inputting the normalized track data and the normalized height data into a Transform network and an LSTM (Long Short Term Memory) network of a pre-constructed track intelligent prediction model respectively, and extracting and learning spatial position features and time sequence features; and finally, an output network of the intelligent track prediction model outputs a track prediction result based on the spatial position features and the time sequence features, so that more accurate track prediction is realized.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of aerospace technology, and in particular to a track intelligent prediction method based on Transformer-LSTM. Background Art

[0002] With the rapid development of the aviation industry, air traffic volume is increasing, and the complexity and safety risks of flight operations are also increasing. In order to ensure the safety and efficient operation of flights, air traffic management needs to accurately predict the trajectory of aircraft. Trajectory prediction can help air traffic management personnel understand the flight status of aircraft in real time, detect and deal with potential flight conflicts in a timely manner, and ensure the safe operation of flights. The advancement of sensor technology, the optimization of intelligent algorithms, and the advancement of aviation technology have made aircraft trajectory completion algorithms a hot topic in the aviation field.

[0003] Currently, track prediction methods can be roughly divided into three categories, namely, track prediction methods based on aerodynamic models, track prediction methods based on hybrid estimation theory models, and track prediction methods based on deep learning models.

[0004] The trajectory prediction method based on aerodynamic models regards the aerial target as a particle. It usually uses the physical characteristics and environmental conditions of the target aircraft to construct its motion model during flight, and relies on dynamic formulas to predict the target's future short-term trajectory. Wang Chao et al. established a 4D trajectory prediction method based on the basic flight model, successfully modeled the horizontal trajectory, altitude profile and velocity profile of the aircraft, thereby achieving a comprehensive fit of the 4D trajectory. Zhang Junfeng et al. used trends such as aircraft performance data and navigation adaptability to construct a target motion model, improving the accuracy of four-dimensional trajectory prediction. Lee et al. derived a nonlinear dynamic model of aerial targets under various flight states, and used continuous state transition probabilities to model discrete transitions between flight modes to predict and track aerial targets. Takeichi et al. conducted an in-depth analysis of the characteristics of aerial target trajectory data, and combined time errors and meteorological data to propose an adaptive target trajectory prediction model.

[0005] The track prediction method based on the hybrid estimation theory model regards the track prediction problem as a random linear hybrid estimation problem, and mainly estimates the future trend of the track based on the Kalman filter algorithm, the improved Kalman filter algorithm, and the adaptive filter algorithm. Han Yunxiang et al. divided the operating state of the aircraft into three states: climbing, descending, and level flight, and established a dynamic model based on the aircraft under different states based on the hybrid system theory. Zhang Junfeng et al. proposed an improved interactive multi-mode algorithm, redefined the likelihood function to complete the update of the probability of the flight mode of the air target, eliminated the problem of the difficulty of timely conversion of the flight mode caused by the incomplete mode set in the IMM algorithm (Interacting Multiple Model), and improved the accuracy of track prediction. Lv Bo et al. introduced the six-degree-of-freedom model based on the improved air target motion and the wind field disturbance function into the Kalman filter algorithm, which improved the accuracy of track prediction and optimized the prediction interval.

[0006] With the development of artificial intelligence and deep learning technology, more and more research teams are applying deep learning models to track prediction. Deep learning models can learn the movement laws and characteristics of targets from massive data, and have strong adaptability and generalization capabilities. Hu Yuke et al. established a track prediction model based on RNN (Recurrent Neural Network), and used a symmetric segmented path distance model in the track data to preprocess the original track data, eliminating the abnormal data and noise in the original track data to a large extent, thereby reducing the impact of abnormal track data on prediction accuracy. Shi Qingyan et al. proposed an online update LSTM (Long Short Term Memory) to learn the potential associations between data in a large amount of historical track data, and realized real-time prediction of flight targets. Liu Shanshan et al. combined the advantages of convolutional neural networks and bidirectional LSTMs and proposed a hybrid deep learning network to predict the future navigation trajectory of the target. Yu Chen et al. combined the differential autoregressive moving average model with the bidirectional gated recurrent neural network and proposed an ARIMA-BIGRU track hybrid prediction model. They used the ARIMA model to make a preliminary prediction of the track, and then used the BIGRU (Bidirectional Gated Recurrent Unit) neural network to correct the residual sequence generated in the prediction process, thereby improving the accuracy of track prediction.

[0007] However, the above track prediction methods still face some difficulties. First, since the prior information of aerial targets is usually unknown, it is difficult to establish an accurate dynamic model. Second, deep learning models such as RNN and LSTM are not comprehensive enough in processing spatial position and time series features when processing track data, and can only capture some features, which leads to poor prediction accuracy. Third, although the currently proposed models can process time series data, most of them are sequential calculation methods, which leads to serialization of calculations and reduced calculation efficiency. It is difficult to effectively capture long-distance dependencies in long sequence data, which limits the prediction effect. Summary of the invention

[0008] In order to solve the above technical problems, the embodiments of the present application propose a Transformer-LSTM-based intelligent track prediction method, which adopts a Transformer-LSTM-based intelligent track prediction model to effectively capture the long-term dependencies and timing information in the track data, and more deeply understand and fully utilize the spatiotemporal relationships in the track data, thereby achieving more accurate track prediction and providing reliable support for real-time decision-making in the field of aerospace technology.

[0009] In order to achieve the above-mentioned purpose, an embodiment of the present application proposes a track intelligent prediction method based on Transformer-LSTM, which includes the following steps: obtaining information from a multi-sensor track intelligent processing system and performing preprocessing, association and fusion to obtain original track data; wherein the original track data includes original latitude and longitude data and original altitude data; converting the original track data into a unified Cartesian coordinate system, and using the maximum and minimum normalization functions to normalize the track data and the original altitude data after the spatial coordinate conversion respectively; inputting the normalized track data and the normalized original altitude data into the Transformer network and LSTM network of the track intelligent prediction model respectively, extracting and learning spatial position features and time series features, and finally outputting the track prediction result based on the spatial position features and time series features by the output network of the track intelligent prediction model.

[0010] In order to achieve the above-mentioned purpose, the embodiment of the present application also proposes a track intelligent prediction system based on Transformer-LSTM, the system comprising: an original processing module, a normalization processing module and a track intelligent prediction model, the track intelligent prediction model is specifically composed of a parallel Transformer network and an LSTM network, and an output network; the original processing module is used to obtain the information of the multi-sensor track intelligent processing system and perform preprocessing, association and fusion to obtain the original track data, wherein the original track data includes original latitude and longitude data and original altitude data; the normalization processing module is used to convert The original track data is converted to a unified Cartesian coordinate system, and the maximum and minimum normalization functions are used to normalize the track data and the original altitude data after spatial coordinate conversion. The Transformer network is used to receive the normalized track data, extract and learn the spatial position features based on the normalized track data, and obtain the spatial position features. The LSTM network is used to receive the normalized original altitude data, extract and learn the time series features based on the normalized original altitude data, and obtain the time series features. The output network is used to output the track prediction results based on the spatial position features and the time series features.

[0011] The embodiment of the present application proposes a track intelligent prediction method based on Transformer-LSTM. Compared with the traditional method, it adopts a track intelligent prediction model built with Transformer network and LSTM network as the core to perform track intelligent prediction tasks. The Transformer network is used to extract and learn spatial position features, so as to effectively capture the long-term dependencies in the track data, and the LSTM network is used to extract and learn time series features, so as to effectively capture the time series information in the track data. Fusion of the outputs of the Transformer network and the LSTM network for track intelligent prediction can more deeply understand and fully utilize the spatiotemporal relationship in the track data, providing reliable support for real-time decision-making in the field of aerospace technology.

[0012] In some optional embodiments, the coordinates of the pth track point in the original track data in the geodetic coordinate system are (lat p ,lon p ,H p ), lat p Represents the original dimension data of track point p, lon p Represents the original longitude data of track point p, H p Represents the original height data of the track point p, converting the original track data into a unified Cartesian coordinate system, including:

[0013] First, based on the geometric model of the earth, the original track data is converted from the geodetic coordinate system to the meridian plane coordinate system Oxy. The coordinates of the track point p in the original track data in the meridian plane coordinate system Oxy are expressed as:

[0014] x p =(N+H)cos(lat p );

[0015] y p =[N(1-e 2 )+H]sin(lat p );

[0016] e 2 =(a 2 -b 2 ) / a 2 ;

[0017]

[0018] Among them, e 2 is the eccentricity of the earth's ellipsoid, N is the radius of curvature of the earth, a is the length of the earth's major axis, b is the length of the earth's minor axis, (x p ,y p ) represents the coordinates of the track point p in the meridian plane coordinate system Oxy;

[0019] Then the original track data is converted from the meridian plane coordinate system Oxy to the Cartesian coordinate system. The coordinates of the track point p in the original track data in the Cartesian coordinate system are expressed as:

[0020] X p =x p cos(lon p )=(N+H)cos(lat p )cos(lon p );

[0021] Y p =x p sin(lon p )=(N+H)cos(lat p )cos(lon p );

[0022] Z p =y p =[N(1-e 2 )+H]sin(lat p );

[0023] Among them, (X p ,Y p ,Z p) represents the coordinates of the track point p in the Cartesian coordinate system.

[0024] In some optional embodiments, the maximum and minimum normalization functions are used to normalize the track data and the original altitude data after the spatial coordinate conversion, respectively, which can be expressed by the formula:

[0025]

[0026] Where min(·) and max(·) represent the minimum function and the maximum function respectively, data represents the track data after spatial coordinate conversion, data′ represents the normalized track data, and H p Represents the original height data of track point p, H′ p Represents the normalized raw altitude data of track point p.

[0027] In some optional embodiments, the Transformer network is composed of an input embedding layer, a Transformer encoder, an output embedding layer, a Transformer decoder, a linear layer and a Softmax layer, the Transformer encoder is composed of six layers of Transformer encoder layers connected sequentially, and the Transformer decoder is composed of six layers of Transformer decoder layers connected sequentially;

[0028] The normalized track data is input into the input embedding layer, transformed by the linear layer and added with position coding, and then input into the Transformer encoder. The delayed normalized track data is input into the output embedding layer, transformed by the linear layer and added with position coding, and then input into the Transformer decoder.

[0029] The output of the input embedding layer is the input of the first Transformer encoder layer, the output of the previous Transformer encoder layer is the input of the next Transformer encoder layer, and the output of the last Transformer encoder layer will be input into each Transformer decoder layer;

[0030] The output of the output embedding layer is the input of the first Transformer decoder layer, the output of the previous Transformer decoder layer is the input of the next Transformer decoder layer, and the output of the last Transformer decoder layer is processed by the linear layer and the Softmax layer to obtain the spatial position feature.

[0031] In some optional embodiments, each Transformer encoder layer is composed of a multi-head attention mechanism module, two layer normalization modules and a feed-forward neural network layer;

[0032] The calculation process of a single-layer attention mechanism is expressed by the formula:

[0033]

[0034] Among them, Q, K, and V are query vectors, key vectors, and value vectors respectively. k represents the dimension of the key vector, softmax(·) represents the Softmax function, and Attention(Q,K,V) represents a single-layer attention mechanism;

[0035] After being processed by a single-layer attention mechanism with h attention heads, similar functions of different modes are obtained. The output vectors of multiple attention heads are merged and projected back to the original dimension to obtain the output of the multi-head attention mechanism.

[0036] The output of the multi-head attention mechanism is expressed as:

[0037] MultiHeadAttention(Q,K,V)=Concat(head1,head2,…,head h )W O ;

[0038]

[0039] Among them, W O is the projection matrix, The attention score matrices corresponding to Q, K, and V of the h-th attention head, respectively. Concat(·) represents the concatenation operation, and MultiHeadAttention(Q,K,V) represents the output of the multi-head attention mechanism.

[0040] The multi-head attention mechanism module uses residual connections to transfer the original input across the multi-head attention mechanism module to the first layer normalization module, and then uses residual connections to transfer the added output across the feedforward neural network layer to the second layer normalization module;

[0041] The calculation process of the layer normalization module is expressed by the formula:

[0042] Add&Norm(x in )=ReLU[(x in W1+b1)W2+b2];

[0043] Among them, x in is the input track point feature, W1 and W2 are the weight matrices of the two linear transformation layers, b1 and b2 are bias vectors, and ReLU(·) represents the ReLU activation function.

[0044] In some optional embodiments, each Transformer decoder layer is composed of a masked multi-head attention mechanism module, three layer normalization modules and two feed-forward neural network layers;

[0045] The masked multi-head attention mechanism module needs to mask the predicted part of the input;

[0046] The output of the output embedding layer is transformed by the linear layer and the position code is added to serve as V of the multi-head attention mechanism. The Q and K of the multi-head attention mechanism come from the input linear projection of the Transformer encoder.

[0047] In some optional embodiments, the LSTM network is composed of three LSTM layers connected sequentially, the output of the previous LSTM layer is the input of the next LSTM layer, and the output of the last LSTM layer is the time series feature;

[0048] Each LSTM layer includes a forget gate, an input gate, and an output gate. The forget gate, input gate, and output gate are constructed using the following formula:

[0049] f t =σ(W f ·[h t-1 ,x t ]+b f );

[0050] i t =σ(W i ·[h t-1 ,x t ]+b i );

[0051] o t =σ(W o ·[h t-1 ,x t ]+b o );

[0052] Among them, x t represents the input of the tth time step, h t-1 represents the hidden state at the t-1th time step, which includes the understanding and memory of the input, W f , W i , W o are the weight matrices of the forget gate, input gate, and output gate, respectively, and b f , b i , b o are the bias vectors of the forget gate, input gate, and output gate respectively, and σ(·) is the Sigmoid function;

[0053] The output and update status are selected by the following formula:

[0054]

[0055] Among them, W c is the weight matrix of the long-term memory unit, b c is the bias vector of the long-term memory unit;

[0056] Update the long-term memory unit and state output by the following formula:

[0057] c t =i t *c t +f t *c t-1 ;

[0058] h t =o t ·tanh(c t );

[0059] The mapping output of the LSTM layer is constructed by the following formula:

[0060] y l =w y ·h t +b y ;

[0061] Among them, y l is the mapping output of the LSTM layer, w y is the output matrix of the fully connected layer, b y is the bias vector of the fully connected layer.

[0062] In some optional embodiments, when training the track intelligent prediction model, the joint loss function used is expressed by the formula:

[0063]

[0064] Among them, B is the total number of data, subscript i is the i-th component in the data, and X i is the i-th component in the output of the Transformer network, is the true label of Transformer The i-th component in H i is the i-th component in the output of the LSTM network, is the true label of LSTM LOSS is the joint loss function. LOSS uses the mean square error to evaluate the difference between the two, and uses the Adam optimizer to update the network parameters of the track intelligent prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 is a flowchart of a track intelligent prediction method based on Transformer-LSTM provided in one embodiment of the present application;

[0067] Figure 2 is a schematic diagram of the structure of a track intelligent prediction model provided in one embodiment of the present application;

[0068] Figure 3 is a schematic diagram of the structure of a Transformer network provided in one embodiment of the present application;

[0069] Figure 4 is a schematic diagram of the connection relationship between the Transformer encoder layer and the Transformer decoder layer in the Transformer network provided in one embodiment of the present application;

[0070] Figure 5 It is a schematic diagram of the structure of an LSTM network provided by an embodiment of the present application;

[0071] Figure 6 It is a schematic diagram of the structure of the LSTM layer provided by an embodiment of the present application;

[0072] Figure 7 It is a structural diagram of a track intelligent prediction system based on Transformer-LSTM provided in one embodiment of the present application;

[0073] Figure 8 It is a structural diagram of an electronic device deployed in another embodiment of the present application. DETAILED DESCRIPTION

[0074] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0075] An embodiment of the present application proposes a Transformer-LSTM-based intelligent track prediction method, which is applied to an electronic device, wherein the electronic device may be a terminal or a server. This embodiment and the following embodiments are all described using the server as an example. The implementation details of the Transformer-LSTM-based intelligent track prediction method proposed in this embodiment are specifically described below. The following content is only the relevant implementation details provided for ease of understanding and is not necessary for the implementation of this solution.

[0076] The specific process of the intelligent track prediction method based on Transformer-LSTM proposed in this embodiment can be as follows: Figure 1 As shown, including:

[0077] Step 11, obtaining information from the multi-sensor track intelligent processing system and performing preprocessing, association and fusion to obtain original track data, wherein the original track data includes original latitude and longitude data and original altitude data.

[0078] In the specific implementation, the server first needs to obtain information from the multi-sensor track intelligent processing system, pre-process, associate and fuse the obtained information to obtain the original track data, where the original track data includes original latitude and longitude data and original altitude data.

[0079] In an example, each piece of original track data includes P track points, and each track point corresponds to time data, latitude data, longitude data, and altitude data.

[0080] The kth original track data can be expressed by the formula:

[0081] S k ={(t1,lat1,lon1,H1),(t2,lat2,lon2,H2),...,(t P ,lat P ,lon P ,H P )};

[0082] Among them, t P ,lat P ,lon P and H P Respectively represent the time data, latitude data, longitude data and altitude data of the track point P, S k Represents the kth original track data.

[0083] In one example, the preprocessing operation is specifically to parse the information obtained from the multi-sensor track intelligent processing system and the radar message format, and extract valid information and store it in a database.

[0084] Step 12, convert the original track data into a unified Cartesian coordinate system, and use the maximum and minimum normalization functions to normalize the track data after the spatial coordinate conversion and the original height data respectively.

[0085] In a specific implementation, after obtaining the original track data, the server needs to convert the original track data into a unified Cartesian coordinate system, and use the maximum and minimum normalization functions to normalize the track data after the spatial coordinate conversion and the original height data respectively.

[0086] In an example, suppose the coordinates of the pth track point in the original track data in the geodetic coordinate system (latitude and longitude coordinate system) are (lat p ,lon p ,H p ), lat p Represents the original dimension data of track point p, lon p Represents the original longitude data of track point p, H p Represents the raw altitude data of track point p.

[0087] When the server converts the original track data into a unified Cartesian coordinate system, it first needs to convert the original track data from the geodetic coordinate system to the meridian plane coordinate system Oxy based on the geometric model of the earth.

[0088] The coordinates of the track point p in the original track data in the meridian plane coordinate system Oxy are expressed as:

[0089] x p =(N+H)cos(lat p );

[0090] y p =[N(1-e 2 )+H]sin(lat p );

[0091] e 2 =(a 2 -b2 ) / a 2 ;

[0092]

[0093] Among them, e 2 is the eccentricity of the earth's ellipsoid, N is the radius of curvature of the earth, a is the length of the earth's major axis, which is 6378137m, b is the length of the earth's minor axis, which is 6356752.3142451793m, (x p ,y p ) represents the coordinates of the track point p in the meridian plane coordinate system Oxy.

[0094] Subsequently, the server converts the original track data from the meridian plane coordinate system Oxy to the Cartesian coordinate system (geodetic rectangular coordinate system). The coordinates of the track point p in the original track data in the Cartesian coordinate system are expressed as:

[0095] X p =x p cos(lon p )=(N+H)cos(lat p )cos(lon p );

[0096] Y p =x p sin(lon p )=(Z+H)cos(lat p )cos(lon p );

[0097] Z p =y p =[N(1-e 2 )+H]sin(lat p );

[0098] Among them, (X p ,Y p ,Z p ) represents the coordinates of the track point p in the Cartesian coordinate system.

[0099] In one example, the server uses the maximum and minimum normalization functions to normalize the track data and the original altitude data after the spatial coordinate conversion, respectively, which is expressed by the formula:

[0100]

[0101] Where min(·) and max(·) represent the minimum function and the maximum function respectively, data represents the track data after spatial coordinate conversion, data′ represents the normalized track data, and H p Represents the original height data of track point p, H′ p Represents the normalized raw altitude data of track point p.

[0102] Step 13, the normalized track data and the normalized original altitude data are respectively input into the Transformer network and LSTM network of the track intelligent prediction model to extract and learn the spatial position features and time series features, and finally the output network of the track intelligent prediction model outputs the track prediction result based on the spatial position features and time series features.

[0103] In the specific implementation, the server pre-builds and trains the intelligent track prediction model to perform the task of intelligent track prediction. The intelligent track prediction model consists of a Transformer network, an LSTM network, and an output network. The Transformer network and the LSTM network are parallel. The server inputs the normalized track data and the normalized original altitude data into the Transformer network and the LSTM network of the intelligent track prediction model respectively to extract and learn the spatial position features and time series features. Finally, the output network of the intelligent track prediction model outputs the track prediction results based on the spatial position features and time series features.

[0104] In an example, the specific structure of the track intelligent prediction model can be as follows: Figure 2 shown.

[0105] In an example, the specific structure of the Transformer network can be as follows Figure 3 As shown in Figure 1. The Transformer network consists of an input embedding layer, a Transformer encoder, an output embedding layer, a Transformer decoder, a linear layer, and a Softmax layer. The Transformer encoder consists of six layers of Transformer encoder layers connected in sequence, and the Transformer decoder consists of six layers of Transformer decoder layers connected in sequence. The connection relationship between the Transformer encoder layer and the Transformer decoder layer can be shown as follows: Figure 4 shown.

[0106] In one example, the normalized track data is input into the input embedding layer, transformed by the linear layer and adding the position code, and then input into the Transformer encoder. The delayed normalized track data is input into the output embedding layer, transformed by the linear layer and adding the position code, and then input into the Transformer decoder.

[0107] In an example, the position encoding can be expressed by the formula:

[0108]

[0109] Among them, PositionalEncoding(pos,i) represents the result after position encoding.

[0110] The output of the input embedding layer is the input of the first Transformer encoder layer, the output of the previous Transformer encoder layer is the input of the next Transformer encoder layer, and the output of the last Transformer encoder layer will be input into each Transformer decoder layer.

[0111] The output of the output embedding layer is the input of the first Transformer decoder layer, the output of the previous Transformer decoder layer is the input of the next Transformer decoder layer, and the output of the last Transformer decoder layer is processed by the linear layer and the Softmax layer to obtain the spatial position feature.

[0112] In one example, each Transformer encoder layer consists of a multi-head attention module, two layer normalization modules, and a feed-forward neural network layer.

[0113] The calculation process of a single-layer attention mechanism is expressed by the formula:

[0114]

[0115] Among them, Q, K, and V are query vectors, key vectors, and value vectors respectively. k represents the dimension of the key vector, softamx(·) represents the Softmax function, and Attention(Q,K,V) represents a single-layer attention mechanism;

[0116] After being processed by a single-layer attention mechanism with h attention heads, similar functions of different modes are obtained. The output vectors of multiple attention heads are merged and projected back to the original dimension to obtain the output of the multi-head attention mechanism.

[0117] The output of the multi-head attention mechanism is expressed as:

[0118] MultiHeadAttention(Q,L,V)=Concat(head1,head2,…,head h )W O ;

[0119]

[0120] Among them, W O is the projection matrix, The attention score matrices corresponding to Q, K, and V of the h-th attention head, respectively. Concat(·) represents the concatenation operation, and MultiHeadAttention(Q,K,V) represents the output of the multi-head attention mechanism.

[0121] The multi-head attention mechanism module uses residual connections to transfer the original input across the multi-head attention mechanism module to the first layer normalization module, and then uses residual connections to transfer the added output across the feedforward neural network layer to the second layer normalization module;

[0122] The calculation process of the layer normalization module is expressed by the formula:

[0123] Add&Norm(x in )=ReLU[(x in W1+b1)W2+b2];

[0124] Among them, x in is the input track point feature, W1 and W2 are the weight matrices of the two linear transformation layers, b1 and b2 are bias vectors, and ReLU(·) represents the ReLU activation function.

[0125] In one example, each Transformer decoder layer consists of a masked multi-head attention module, three layer normalization modules, and two feedforward neural network layers. The masked multi-head attention module requires masking the predicted part of the input. The output of the output embedding layer is transformed by a linear layer and the position encoding is added to serve as the V of the multi-head attention mechanism. The Q and K of the multi-head attention mechanism come from the linear projection of the input of the Transformer encoder. The output of the Transformer decoder will be calculated after the linear layer to calculate its Softmax probability distribution result.

[0126] In an example, the specific structure of the LSTM network can be as follows Figure 5 As shown in the figure, the LSTM network consists of three LSTM layers connected sequentially. The output of the previous LSTM layer is the input of the next LSTM layer, and the output of the last LSTM layer is the time series feature.

[0127] In an example, the structure of the LSTM layer can be as follows Figure 6 As shown in the figure, each LSTM layer includes a forget gate, an input gate, and an output gate. The forget gate, input gate, and output gate are constructed by the following formula:

[0128] f t =σ(W f ·[h t-1 ,x t ]+b f );

[0129] i t =σ(W i ·[h t-1 ,x t ]+b i );

[0130] o t =σ(W o ·[h t-1 ,x t ]+b o );

[0131] Among them, x t represents the input of the tth time step, h t-1 represents the hidden state at the t-1th time step, which includes the understanding and memory of the input, W f , W i , W o are the weight matrices of the forget gate, input gate, and output gate, respectively, and b f , b i , b o are the bias vectors of the forget gate, input gate, and output gate respectively, and σ(·) is the Sigmoid function.

[0132] The LSTM layer selects output and updates state using the following formula:

[0133]

[0134] Among them, W c is the weight matrix of the long-term memory unit, b c is the bias vector of the long-term memory unit.

[0135] The LSTM layer updates the long-term memory unit and state output using the following formula:

[0136] c t =i t *c t +f t *c t-1 ;

[0137] h t =ot ·tanh(c t ).

[0138] The mapping output of the LSTM layer is constructed by the following formula:

[0139] y l =w y ·h t +b y ;

[0140] Among them, y l is the mapping output of the LSTM layer, w y is the output matrix of the fully connected layer, b y is the bias vector of the fully connected layer.

[0141] In an example, the joint loss function used by the server when training the track intelligent prediction model can be expressed by the formula:

[0142]

[0143] Among them, B is the total number of data, subscript i is the i-th component in the data, and X i is the i-th component in the output of the Transformer network, is the true label of Transformer The i-th component in H i is the i-th component in the output of the LSTM network, is the true label of LSTM LOSS is the joint loss function. LOSS uses the mean square error to evaluate the difference between the two, and uses the Adam optimizer to update the network parameters of the track intelligent prediction model.

[0144] In one example, the update formula of the network parameters is:

[0145]

[0146] Among them, θ1 represents the network parameters before update, Indicates the updated network parameters.

[0147] This embodiment proposes a track intelligent prediction method based on Transformer-LSTM. Compared with the traditional method, it adopts a track intelligent prediction model built with Transformer network and LSTM network as the core to perform track intelligent prediction tasks. Transformer network is used to extract and learn spatial position features, so as to effectively capture the long-term dependencies in track data, and LSTM network is used to extract and learn time series features, so as to effectively capture the time series information in track data. Fusion of the outputs of Transformer network and LSTM network for track intelligent prediction can more deeply understand and fully utilize the spatiotemporal relationship in track data, and provide reliable support for real-time decision-making in the field of aerospace technology.

[0148] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0149] Another embodiment of the present application proposes a track intelligent prediction system based on Transformer-LSTM. The details of the track intelligent prediction system based on Transformer-LSTM proposed in this embodiment are described in detail below. The following content is only the implementation details provided for easy understanding and is not necessary for the implementation of this example.

[0150] Figure 7 It is a structural diagram of a track intelligent prediction system based on Transformer-LSTM proposed in this embodiment, including: an original processing module 21, a normalization processing module 22 and a track intelligent prediction model 30. The track intelligent prediction model 30 is specifically composed of a parallel Transformer network 31 and an LSTM network 32, and an output network 33.

[0151] The original processing module 21 is used to obtain the information of the multi-sensor track intelligent processing system and perform preprocessing, association and fusion to obtain the original track data, wherein the original track data includes original latitude and longitude data and original altitude data.

[0152] The normalization processing module 22 is used to convert the original track data into a unified Cartesian coordinate system, and use the maximum and minimum normalization functions to perform normalization processing on the track data after the spatial coordinate conversion and the original altitude data respectively.

[0153] The Transformer network 31 is used to receive the normalized track data, extract and learn the spatial position features based on the normalized track data, and obtain the spatial position features.

[0154] The LSTM network 32 is used to receive the normalized original height data, extract and learn the time series features based on the normalized original height data, and obtain the time series features.

[0155] The output network 33 is used to output the track prediction result based on the spatial position characteristics and the time series characteristics.

[0156] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiments.

[0157] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0158] Another embodiment of the present application deploys an electronic device, whose specific structure can be as follows: Figure 8 As shown, it includes: at least one processor 41; and a memory 42 that is communicatively connected to the at least one processor 41; wherein the memory 42 stores instructions that can be executed by the at least one processor 41, and the instructions are executed by the at least one processor 41 so that the at least one processor 41 can execute a Transformer-LSTM-based track intelligent prediction method described in the above-mentioned method embodiments.

[0159] Among them, the memory and the processor can be connected in a bus manner, and the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators and power management circuits together, which are all well known in the art, so this article will not describe them further. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0160] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0161] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk and other media that can store program codes.

[0162] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A track intelligent prediction method based on Transformer-LSTM, characterized in that: include: Acquire the information of the multi-sensor track intelligent processing system and perform pre-processing, correlation and fusion to obtain the original track data; wherein the original track data includes the original latitude and longitude data and the original altitude data; The original track data is converted into a unified Cartesian coordinate system, and the maximum and minimum normalization functions are used to normalize the track data after the spatial coordinate conversion and the original height data respectively; The normalized track data and the normalized original altitude data are respectively input into the Transformer network and LSTM network of the track intelligent prediction model to extract and learn the spatial position features and time series features. Finally, the output network of the track intelligent prediction model outputs the track prediction results based on the spatial position features and time series features.

2. According to the Transformer-LSTM-based intelligent track prediction method of claim 1, it is characterized in that The coordinates of the pth track point in the original track data in the geodetic coordinate system are (lat p ,lon p ,H p ), lat p Represents the original dimension data of track point p, lon p Represents the original longitude data of track point p, H p Represents the original height data of the track point p, converting the original track data into a unified Cartesian coordinate system, including: First, based on the geometric model of the earth, the original track data is converted from the geodetic coordinate system to the meridian plane coordinate system Oxy. The coordinates of the track point p in the original track data in the meridian plane coordinate system Oxy are expressed as: x p =(N+H)cos(lat p ); and p =[N(1-e 2 )+H]sin(lat p ); e 2 =(a 2 -b 2 ) / a 2 ; Among them, e 2 is the eccentricity of the earth's ellipsoid, N is the radius of curvature of the earth, a is the length of the earth's major axis, b is the length of the earth's minor axis, (x p ,y p ) represents the coordinates of the track point p in the meridian plane coordinate system Oxy; Then the original track data is converted from the meridian plane coordinate system Oxy to the Cartesian coordinate system. The coordinates of the track point p in the original track data in the Cartesian coordinate system are expressed as: X p =x p cos(lon p )=(N+H)cos(lat p )cos(lon p ); Y p =x p sin(lon p )=(N+H)cos(lat p )cos(lon p ); WITH p =y p =[N(1-e 2 )+H]sin(lat p ); Among them, (X p ,Y p ,Z p ) represents the coordinates of the track point p in the Cartesian coordinate system.

3. According to claim 2, a track intelligent prediction method based on Transformer-LSTM is characterized in that: The maximum and minimum normalization functions are used to normalize the track data and the original height data after spatial coordinate conversion, respectively, which can be expressed by the formula: Among them, min(·) and max(·) represent the minimum function and the maximum function respectively, data represents the track data after spatial coordinate conversion, and data ′ represents the normalized track data, H p Represents the original height data of track point p, H′ p Represents the normalized raw altitude data of track point p.

4. The intelligent track prediction method based on Transformer-LSTM according to claim 1, characterized in that: The Transformer network consists of an input embedding layer, a Transformer encoder, an output embedding layer, a Transformer decoder, a linear layer, and a Softmax layer. The Transformer encoder consists of six layers of Transformer encoder layers connected sequentially, and the Transformer decoder consists of six layers of Transformer decoder layers connected sequentially. The normalized track data is input into the input embedding layer, transformed by the linear layer and added with position coding, and then input into the Transformer encoder. The delayed normalized track data is input into the output embedding layer, transformed by the linear layer and added with position coding, and then input into the Transformer decoder. The output of the input embedding layer is the input of the first Transformer encoder layer, the output of the previous Transformer encoder layer is the input of the next Transformer encoder layer, and the output of the last Transformer encoder layer will be input into each Transformer decoder layer; The output of the output embedding layer is the input of the first Transformer decoder layer, the output of the previous Transformer decoder layer is the input of the next Transformer decoder layer, and the output of the last Transformer decoder layer is processed by the linear layer and the Softmax layer to obtain the spatial position feature.

5. The intelligent track prediction method based on Transformer-LSTM according to claim 4 is characterized in that: Each Transformer encoder layer consists of a multi-head attention mechanism module, two layer normalization modules and a feed-forward neural network layer; The calculation process of a single-layer attention mechanism is expressed by the formula: Among them, Q, K, and V are query vectors, key vectors, and value vectors respectively. k represents the dimension of the key vector, softmax(·) represents the Softmax function, and Attention(Q,K,V) represents a single-layer attention mechanism; After being processed by a single-layer attention mechanism with h attention heads, similar functions of different modes are obtained. The output vectors of multiple attention heads are merged and projected back to the original dimension to obtain the output of the multi-head attention mechanism. The output of the multi-head attention mechanism is expressed as: MultiHeadAttention(Q,K,V)=Concat(head1,head2,…,head h )W O ; Among them, W O is the projection matrix, The attention score matrices corresponding to Q, K, and V of the h-th attention head, respectively. Concat(·) represents the concatenation operation, and MultiHeadAttention(Q,K,V) represents the output of the multi-head attention mechanism. The multi-head attention mechanism module uses residual connections to transfer the original input across the multi-head attention mechanism module to the first layer normalization module, and then uses residual connections to transfer the added output across the feedforward neural network layer to the second layer normalization module; The calculation process of the layer normalization module is expressed by the formula: Add&Norm(x in )=ReLU[(x in W1+b1)W2+b2]; Among them, x in is the input track point feature, W1 and W2 are the weight matrices of the two linear transformation layers, b1 and b2 are bias vectors, and ReLU(·) represents the ReLU activation function.

6. The intelligent track prediction method based on Transformer-LSTM according to claim 5, characterized in that: Each Transformer decoder layer consists of a masked multi-head attention mechanism module, three layer normalization modules, and two feed-forward neural network layers; The masked multi-head attention mechanism module needs to mask the predicted part of the input; The output of the output embedding layer is transformed by the linear layer and the position code is added to serve as V of the multi-head attention mechanism. The Q and K of the multi-head attention mechanism come from the input linear projection of the Transformer encoder.

7. The intelligent track prediction method based on Transformer-LSTM according to claim 1, characterized in that: The LSTM network consists of three LSTM layers connected sequentially. The output of the previous LSTM layer is the input of the next LSTM layer, and the output of the last LSTM layer is the time series feature. Each LSTM layer includes a forget gate, an input gate, and an output gate. The forget gate, input gate, and output gate are constructed using the following formula: f t =σ(W f ·[h t-1 ,x t ]+b f ); i t =σ(W i ·[h t-1 ,x t ]+b i ); the t =σ(W o ·[h t-1 ,x t ]+b o ); Among them, x t represents the input of the tth time step, h t-1 represents the hidden state at the t-1th time step, which includes the understanding and memory of the input, W f , W i , W o are the weight matrices of the forget gate, input gate, and output gate, respectively, and b f , b i , b o are the bias vectors of the forget gate, input gate, and output gate respectively, and σ(·) is the Sigmoid function; The output and update status are selected by the following formula: Among them, W c is the weight matrix of the long-term memory unit, b c is the bias vector of the long-term memory unit; Update the long-term memory unit and state output by the following formula: c t =i t *c t +f t *c t-1 ; h t =o t ·tanh(c t ); The mapping output of the LSTM layer is constructed by the following formula: y l =w y ·h t +b y ; Among them, y l is the mapping output of the LSTM layer, w y is the output matrix of the fully connected layer, b y is the bias vector of the fully connected layer.

8. A track intelligent prediction method based on Transformer-LSTM according to any one of claims 1 to 7, characterized in that: When training the track intelligent prediction model, the joint loss function used is expressed by the formula: Among them, B is the total number of data, subscript i is the i-th component in the data, and X i is the i-th component in the output of the Transformer network, is the true label of Transformer The i-th component in H i is the i-th component in the output of the LSTM network, is the true label of LSTM LOSS is the joint loss function. LOSS uses the mean square error to evaluate the difference between the two, and uses the Adam optimizer to update the network parameters of the track intelligent prediction model.

9. A track intelligent prediction system based on Transformer-LSTM, characterized in that: include: The original processing module, the normalization processing module and the track intelligent prediction model. The track intelligent prediction model is specifically composed of a parallel Transformer network and an LSTM network, as well as an output network; The original processing module is used to obtain the information of the multi-sensor track intelligent processing system and perform pre-processing, association and fusion to obtain the original track data, wherein the original track data includes the original latitude and longitude data and the original altitude data; A normalization processing module is used to convert the original track data into a unified Cartesian coordinate system, and use the maximum and minimum normalization functions to normalize the track data after the spatial coordinate conversion and the original height data respectively; The Transformer network is used to receive the normalized track data, extract and learn the spatial position features based on the normalized track data, and obtain the spatial position features; The LSTM network is used to receive the normalized original height data, extract and learn the time series features based on the normalized original height data, and obtain the time series features; The output network is used to output the track prediction results based on spatial position features and temporal features.

Images