Multi-feature fusion target trajectory prediction method based on Transform model

Abstract

The invention provides a multi-feature fusion target trajectory prediction method based on a Transform model. The method comprises the following steps: step 1, inputting position information of a target trajectory into a feature generation module to generate multi-feature information such as speed, acceleration and angular speed; respectively embedding the generated multi-feature information and the original position information, then carrying out vector representation, and carrying out convolution fusion to form a multi-feature vector; step 2, transforming a multi-head attention mechanism of a Transform model, combining the extracted Fourier adaptive filtering and multi-head attention to propose a new Fourier adaptive filtering attention algorithm, modeling a dependency relationship of track sequence features by using the new algorithm, and adding convolution and pooling operations at the end of Encoder so as to further extract features; and step 3, the features processed by the Encoder for multiple times are sent to Decoder of the Transform model to be decoded, and prediction is carried out. The feature information of the target trajectory data can be efficiently mined and extracted, and high-precision trajectory prediction of the target is realized.

Description

Technical Field

[0001] The present invention belongs to the field of trajectory prediction, and in particular relates to a target trajectory prediction method based on multi-feature fusion of a Transformer model. Background Art

[0002] Trajectory prediction predicts the future path and behavior of a target (such as a vehicle, pedestrian, drone, or any other moving entity) by observing its motion patterns or based on its historical trajectory. This is a typical multidisciplinary cross-research problem involving transportation engineering and intelligent science and technology, and has important theoretical research and practical application value in the fields of intelligent traffic supervision, abnormal behavior detection, and autonomous navigation of unmanned aerial vehicles.

[0003] At present, the mainstream target trajectory prediction methods can be roughly divided into two categories: physical modeling based on motion characteristics and historical trajectory data driven. Trajectory prediction based on motion characteristics is to establish kinematic equations based on the physical laws of the target motion system, which means that many factors often need to be considered. However, some factors are difficult to avoid in trajectory prediction, such as adverse environmental factors such as wind speed and magnetic field, which is also an important reason for the large errors of these methods. With the widespread application of GPS, radar and mobile devices, a large amount of data has been generated, and emerging technologies such as data mining and cloud computing have developed rapidly. Data-driven methods, especially deep learning methods, have become increasingly popular. Compared with motion modeling methods, data-driven methods are more universal. They only need to study the historical data of their target trajectories, and no longer need to conduct separate motion analysis on specific targets and establish different kinematic equations.

[0004] The key to trajectory prediction based on deep learning methods lies in how to enable the model to learn useful information from the historical data sequence of the target trajectory and how to efficiently extract effective features. Summary of the invention

[0005] The purpose of this invention is to provide a target trajectory prediction method based on multi-feature fusion of Transformer model, which can efficiently extract effective features of historical trajectory sequence, thereby reducing trajectory prediction error and improving the prediction trajectory accuracy. To achieve the above purpose, the following scheme is adopted:

[0006] Compared with the prior art, the advantages of the present invention are:

[0007] 1. The feature generation network proposed in the present invention can fully explore the potential features of the target trajectory and better extract effective features. Especially when some trajectory data sets have single features, the feature generation network can obtain more features.

[0008] 2. The Fourier adaptive filtering network proposed in the present invention converts features from the time domain to the frequency domain. Since it covers all frequencies, it captures the long-term and short-term interactions of sequence features and better obtains global feature information.

[0009] 3. The present invention combines Fourier adaptive filtering with multi-head attention to propose a new multi-head attention mechanism, which performs Fourier adaptive filtering network transformation on Q, K, and V of the multi-head attention respectively, thereby enhancing the model's ability to extract long-distance trend features and short-distance detail features of trajectories.

[0010] 4. The present invention adds convolution and pooling operations at the end of each layer of the Encoder to further fuse the features modeled by the new multi-head attention model and learn more complex feature representations. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is the structure diagram of Fourier adaptive filtering network;

[0012] Figure 2 This is the structure diagram of the new Fourier adaptive filter attention algorithm;

[0013] Figure 3 This is the structure diagram of the Decoder part of the Transformer model;

[0014] Figure 4 This is a structural diagram of the network of a target trajectory prediction method based on multi-feature fusion of the Transformer model; DETAILED DESCRIPTION

[0015] The following is a more detailed description of a target trajectory prediction method based on multi-feature fusion of a Transformer model of the present invention in conjunction with a schematic diagram, which shows a preferred embodiment of the present invention. It should be understood that those skilled in the art can modify the present invention described herein and still achieve the advantageous effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art and not as a limitation of the present invention.

[0016] like Figures 1 to 4 , a target trajectory prediction method based on multi-feature fusion of Transformer model, based on the classic Transformer framework. Compared with CNNs and RNNs model frameworks, the Transformer model framework has a global vision and better parallel computing capabilities in time series tasks that the former do not have. At the same time, the attention mechanism also brings better explainability to the operation of the entire model.

[0017] The present invention first uses the original position information of the historical trajectory to generate its speed, acceleration and angular velocity information with the help of a feature generation network, and embeds them into feature vectors of uniform dimension respectively, so as to further concatenate these feature vectors and fuse them into a multi-feature vector through convolution operation, which is convenient for input into the subsequent encoder structure.

[0018] Secondly, in the Encoder part of each layer, the Fourier adaptive filtering network is combined with multi-head attention to transform the multi-head attention mechanism, and a multi-head Fourier adaptive filtering attention algorithm is proposed to enhance the global feature representation of Q, K, and V in the multi-head attention, improve the model's ability to extract long-term and short-term dependencies of historical trajectory sequence data, and add convolution and maximum pooling operations at the end of the Encoder to further extract features.

[0019] Finally, the feature vector obtained by the Encoder is decoded by the Transformer's Decoder and outputs the prediction result through a fully connected feedforward network.

[0020] A target trajectory prediction method based on multi-feature fusion of Transformer model includes:

[0021] Step 1: The original position information of the historical trajectory sequence data is input through the feature generation network to generate its velocity, acceleration and angular velocity information, and are embedded into feature vectors with a dimension size of 512 respectively, and then concatenated and fused into a multi-feature vector.

[0022] Step 11: Generate velocity, acceleration and angular velocity information using a feature generation network.

[0023] Different trajectory data sets often have different information, but most trajectory data sets have position information. Therefore, generally speaking, assuming that the horizontal coordinate of the trajectory position information of a two-dimensional target is represented by x and the vertical coordinate is represented by y, the speed calculation formula is as follows.

[0024]

[0025] Where T is the observation time of the historical trajectory sequence and t is the sampling time of the data.

[0026] The acceleration and angular velocity information can be calculated based on the obtained velocity information. The calculation formula is as follows.

[0027] a=diff(v)

[0028]

[0029] Where diff is the difference operation and r is the target radius.

[0030] Step 12: Embed the generated feature encoding position features and then concatenate and fuse them.

[0031] Embedding is a technique that maps discrete data into continuous vectors, which can capture the potential relationship of data. The new features generated by the feature generation network and the original position features are embedded into vectors with a uniform dimension of 512. The embedded position vectors are positionally encoded, and the formula is as follows.

[0032] PE (pos，2i) =sin(pos / 10000 2i / d )

[0033] PE (pos,2i+1) =cos(pos / 10000 2i / d )

[0034] Where pos is the position information and i is the dimension.

[0035] The embedded features are consistent with the feature dimensions of the position encoding, and they are concatenated and fused through convolution operations.

[0036] Step 2: Based on the Transformer Encoder, we propose a new multi-head Fourier adaptive filter attention algorithm by modifying the multi-head attention algorithm, and add a convolution pooling operation at the end of the Encoder to further extract features.

[0037] Step 21: Transform the multi-head attention and propose a new multi-head Fourier adaptive filtering attention algorithm.

[0038] Multi-Head Attention is a method that calculates scaled dot-product attention (Scaled Dot-Product Attention) from h different sets of Query, Key, and Value matrices, and then obtains the output through matrix concatenation and linear transformation. In this way, the sequence input features that each head pays attention to are different, and multiple multi-head attentions can better extract input features. The calculation formula for scaled dot-product attention is as follows.

[0039]

[0040] Where Q represents the query matrix; K represents the key matrix; V represents the value matrix; d k Represents the dimensions of matrices Q and K, used to control the range of dot product values.

[0041] The proposed multi-head Fourier adaptive filter attention algorithm combines the Fourier adaptive filter network with the multi-head attention. The Fourier adaptive filter converts the feature signal from the time domain to the frequency domain, which makes it easier to process the feature signal and is more conducive to feature extraction. The network diagram is shown in Figure 1 As shown in the figure, it consists of fast Fourier transform, learnable filter, 1D convolution, GLU activation function and inverse fast Fourier transform. The fast Fourier transform converts the fused multi-feature signal from the time domain to the frequency domain, and captures the long-term and short-term interactions of sequence features by covering all frequencies; the multi-feature frequency signal is multiplied by a learnable filter to assign weights to the frequency features to find effective features; the 1D convolution and GLU activation function fuse features and adjust the output feature dimensions; the inverse fast Fourier transform transforms the obtained frequency features back to the time domain. The process is described by the formula as follows.

[0042] x f =FFT(x)

[0043] x′ f =x f ⊙w f

[0044] x c =GLU(CONV(x′ f ))

[0045] x t =IFFT(x c )

[0046] Where FFT is the fast Fourier transform, ⊙ is the Hadamard product, w f is a learnable weight matrix, GLU is the activation function, CONV is the convolution operation, and IFFT is the inverse fast Fourier transform.

[0047] After Fourier adaptive filtering of Q, K, and V of the multi-head attention mechanism, new Q′, K′, and V′ with richer feature information are formed, and then the scaled dot product attention is calculated. The structure diagram of the proposed multi-head Fourier adaptive filtering attention new algorithm is shown in the figure Figure 2 shown.

[0048] Step 22: The fully connected feedforward network enhances the model's ability to handle complex nonlinearities.

[0049] The main task of the fully connected feedforward network is to further extract and deepen features so that the model can better understand and process the input data. It consists of two linear transformations with a ReLU activation in the middle. The first linear mapping can be regarded as a preliminary processing of the input data. Subsequently, since the data in the real world is nonlinear, the ReLU activation function performs a nonlinear transformation on the processed data to capture more complex features. Finally, the feedforward network performs a second linear mapping to produce the output. Its calculation formula is shown below.

[0050] FFN(x)=max(0,xW1+b1)W2+b2

[0051] Where W1 and W2 are weight matrices, and b1 and b2 are bias terms.

[0052] Step 23: The convolution pooling operation at the end of the encoder further extracts the features processed by multi-head attention.

[0053] Convolution and pooling are performed at the end of each layer of Encoder to further extract features, reduce the risk of overfitting of the model, and improve the computational efficiency of the model. The calculation formula is as follows.

[0054] x′=MaxPooling(Conv(x))

[0055] Among them, MaxPooling is the maximum pooling.

[0056] Step 3: Use the Transformer's Decoder to decode the feature vector obtained by the Encoder and output the prediction result by connecting the fully connected feedforward network.

[0057] Step 31, the Decoder part of Transformer is as follows Figure 2 As shown in the figure. The multi-head masked attention mechanism uses a masking method to prevent the current position input from being associated with the future position input, ensuring that the current position input can only rely on the known input in the past. The starting token is concatenated with a placeholder with a value of 0 and a predicted length as the input of the multi-head masked attention, and the output q vector is cross-attentioned with the k and v vectors output by the Encoder to obtain the output result, which is the final decoding result.

[0058] Step 32: The features obtained after decoding are connected through a fully connected feed-forward network to adjust the output dimension size required and output the prediction result.

[0059] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. A target trajectory prediction method based on multi-feature fusion of Transformer model, characterized in that: The following steps are involved: Step 1: The position information of the trajectory target is input into the multi-feature generation network to generate multi-feature information such as speed, acceleration, angular velocity, etc.; the generated speed, acceleration, angular velocity information is embedded with the original position information, and these features are further fused into a feature vector through convolution operation; Step 2: In the Encoder of the classic Transformer model architecture Encoder-Decoder, the multi-head attention mechanism of the Transformer model is modified, and Fourier adaptive filtering is combined with multi-head attention to propose a new Fourier adaptive filtering attention algorithm, and the dependency of trajectory sequence features is modeled using the new attention mechanism; and convolution and pooling operations are added at the end of the Encoder to further extract the features modeled by the new multi-head attention; Step 3: The trajectory features encoded by multiple encoders are sent to the Decoder of the Transformer model for decoding and prediction.

2. According to claim 1, a target trajectory prediction method based on multi-feature fusion of Transformer model is characterized in that: Step 1 comprises the following steps: Step 11, generating the speed information v according to the original position information x, y of the target trajectory; Where t is the observation time. Step 12: Generate acceleration information a based on the velocity information v generated by the original position information x, y of the target trajectory; a=diff(v) Where diff is a difference operation. Step 13, generating the angular velocity information w according to the velocity information v generated by the original position information x, y of the target trajectory; Where r is the target radius. Step 14: The generated velocity, acceleration, angular velocity information and the original position information are respectively processed by embedding technology to be embedded into a higher-dimensional feature vector; x=Embed(p),p=v,a,w,pos Step 15: Concatenate the embedded feature vectors and use convolution operation to fuse the generated multiple features to form a multi-feature vector, so as to be sent to the Transformer model for further prediction; x′=Concat(x v ,x a ,x w ,x pos ) Concat is a concatenation operation, which concatenates two or more tensors in a certain dimension; Conv is a convolution operation.

3. According to claim 1, a target trajectory prediction method based on multi-feature fusion of Transformer model is characterized in that: Step 2 comprises the following steps: Step 21: Propose a Fourier adaptive filtering network to enrich feature information. x f =FFT(x) x′ f =x f ⊙w f x c =GLU(CONV(x′ f )) x t =IFFT(x c ) Where FFT is the fast Fourier transform, ⊙ is the Hadamard product, w f is a learnable weight matrix, GLU is the activation function, CONV is the convolution operation, and IFFT is the inverse fast Fourier transform. Step 22: Transform the multi-head attention of the Transformer model and propose a new Fourier adaptive filter attention algorithm. x=Q、K、V Where FAN is the Fourier adaptive filtering network Step 23. Finally, add convolution and pooling operations at the end of the Transformer model Encoder to further extract features.

4. According to claim 1, a target trajectory prediction method based on multi-feature fusion of Transformer model is characterized in that: Step 3 comprises the following steps: Step 31, Decoder decodes the features obtained by Encoder through masked attention and cross-attention. Step 32: The features obtained after decoding are connected through a fully connected feed-forward network to output the prediction results.

Images