A vehicle trajectory prediction method based on intention perception spatio-temporal attention network

Abstract

The application discloses a vehicle trajectory prediction method based on an intention perception space-time attention network and relates to the technical field of intelligent vehicles. Step 1: offline training of a vehicle trajectory prediction model: the step is divided into two parts of constructing a training database through collection and processing of large-scale real driving scenes and training a vehicle trajectory prediction model; step 2: online real-time prediction of the vehicle trajectory prediction model: after real-time collection of vehicle data and data preprocessing, the trained vehicle trajectory prediction model is used for online real-time prediction, so that the trajectory of a target vehicle can be predicted. Considering the coupling of driving intention and neighbor vehicle interaction, an intention attention mechanism is proposed to allocate attention weight in the time dimension to extract the historical information of the vehicle; an interaction relationship capturing module is introduced to obtain the influence of different neighbor vehicles on the target vehicle based on a multi-head attention mechanism, so that the interaction information between the vehicles in the space dimension is captured, the accuracy of long-term trajectory prediction is improved, and the lane-changing intention of surrounding vehicles can be effectively predicted.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle technology, and in particular to a vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network. Background Technology

[0002] In recent years, autonomous driving has received increasing attention from research both domestically and internationally due to its enormous potential in addressing many long-standing transportation challenges related to safety, congestion, and energy conservation. The key technologies for autonomous vehicles can be divided into three parts: perception, decision-making, and control. Autonomous vehicles use sensors to perceive the surrounding road environment, identify the state of surrounding obstacles, and make decisions, plan, and control the vehicle's future movement to ensure safe operation. In complex and dynamic traffic environments, the movements of surrounding vehicles are highly uncertain, and accurate planning cannot be made based solely on perception results. Traffic participant prediction is a crucial aspect of ensuring the safe operation of autonomous vehicles.

[0003] With the tremendous success of deep learning and other artificial intelligence technologies in the field of autonomous driving, vehicle trajectory prediction methods are gradually transitioning to deep learning. The literature (Tang L, Wang HY, Zhang WH, et al. Driver lanechange intention recognition of intelligent vehicle based on long short-term memory network[J].IEEE Access,2020,8:136898-136905.) considers the vehicle's own driving state and the influence of surrounding vehicles, proposing a lane change intention prediction method based on Multi-LSTM (Long Short-Term Memory). However, intention recognition cannot obtain explicit trajectory information. The paper (Zyner A, Worrall S, Nebot E. Naturalistic driver intention and path prediction using recurrentneural networks[J].IEEE Transactions on Intelligent Transportation Systems,2020,21(4):1584-1594.) proposes a method for predicting driver intention at urban intersections based on uncertain multimodal trajectory prediction. This method can predict the driving intention and trajectory of the target vehicle, but it only considers the movement of the target vehicle and ignores the interaction between the target vehicle and neighboring vehicles. This method cannot obtain satisfactory results in complex situations.

[0004] Therefore, this invention provides a vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network. By considering the coupling between driving intent and interactions with neighboring vehicles, the accuracy of long-term trajectory prediction is improved, which greatly contributes to enhancing the safety performance of autonomous vehicles. Summary of the Invention

[0005] The purpose of this invention is to solve the above-mentioned problems in the prior art and to propose a vehicle trajectory prediction method based on an intention-aware spatiotemporal attention network. Considering the coupling between driving intention and the interaction with neighboring vehicles, an intention attention mechanism is proposed to achieve a real-time and accurate vehicle trajectory prediction method.

[0006] The vehicle trajectory prediction method of this invention consists of two parts: offline training of the vehicle trajectory prediction model and online real-time prediction, including the following steps:

[0007] Step 1: Offline training of the vehicle trajectory prediction model, which consists of two parts: building the training database and training the vehicle trajectory prediction model. The specific implementation steps are as follows:

[0008] Step 1.1: Construct a training database through the collection and processing of large-scale real-world driving scenarios;

[0009] Step 1.1.1: Data Acquisition: The vehicle driving data collected by the vehicle's onboard sensors, such as cameras and millimeter-wave radar, mainly includes: the longitudinal distance and lateral distance between the target vehicle and neighboring vehicles, the longitudinal relative speed, the lateral relative speed, the longitudinal acceleration, the lateral relative acceleration, and the lateral distance between the vehicle and the lane lines.

[0010] Step 1.1.2: Data preprocessing: Impute missing values ​​in the data. Use the nearest neighbor imputation method to impute single missing values, use interpolation to impute multiple consecutive missing values, use the 3-σ rule to remove outliers, and finally use the Savitzky-Golay filter to filter the data to obtain a smoother data curve.

[0011] Step 1.1.3: Dataset preparation: Define a coordinate system to facilitate the description of vehicle positions, calculate the position coordinates of the target vehicle and neighboring vehicles at various times; define vehicle driving intentions as changing lanes to the left, changing lanes to the right, and going straight, and set intention labels for each segment of driving sequence data;

[0012] Step 1.2: The training steps for the vehicle trajectory prediction model are as follows:

[0013] Step 1.2.1: Intent Recognition Module: Identifies the vehicle's driving intent by analyzing the temporal evolution of the vehicle's trajectory and its spatial relationship with neighboring vehicles, thereby obtaining a driving intent feature vector;

[0014] Step 1.2.1.1: Input to the intent recognition module: The input features of the intent recognition module include the motion state features of the target vehicle and the interaction state features of the target vehicle;

[0015] Step 1.2.1.2: The vehicle driving intention recognition model of the present invention is based on a BiLSTM network, which is composed of a forward LSTM and a backward LSTM. The forward and backward hidden layer vectors are concatenated together, and its final output is the combined output of the forward layer and the backward layer at each time step.

[0016] Step 1.2.1.3: Introduce quick connections to achieve efficient information flow across different layers and improve prediction performance;

[0017] Step 1.2.1.4: The output vector of the last moment of the input historical sequence is processed by an FC layer and a SoftMax function to obtain an intention probability vector;

[0018] Step 1.2.2: Trajectory Feature Encoder: Combining the target vehicle intent feature vector obtained by the intent recognition module, an intent attention mechanism is proposed to obtain the historical trajectory feature encoding vectors of the target vehicle and its neighboring vehicles. Then, using the feature encoding vectors of the neighboring vehicles and the target vehicle, an interaction relationship capture module is established based on the multi-head attention mechanism to capture the importance of the neighboring vehicles, and finally an intermediate semantic vector representing the influence of "context" is formed.

[0019] Step 1.2.2.1: Trajectory Prediction Encoder Input: The input to the trajectory prediction model includes historical state information of the target vehicle and surrounding neighboring vehicles over a period of time;

[0020] Step 1.2.2.2: The trajectory prediction model is based on an encoder-decoder framework using LSTM; the historical state information of the target vehicle and its surrounding neighboring vehicles is encoded from a past period; first, a fully connected layer is used as an embedding layer to embed the input state vector of each vehicle to form an embedding vector; then, LSTM is used to encode the embedding vectors of different vehicles respectively.

[0021] Step 1.2.2.3: Intent Attention Mechanism: The intent vector of the target vehicle and the hidden state vector of the predicted trajectory decoder at the previous time step are concatenated by a concatenation operation. The vector obtained by the fully connected layer is used as the "query" in the key-value pair attention mechanism. The hidden state of the vehicle is processed by different fully connected layers to form "keys" and "values". The attention features are calculated as a weighted sum of "values".

[0022] Step 1.2.2.4: Interaction Relationship Capture Module: Employs a multi-head attention mechanism, using different learned linear projections to calculate multiple attention features; then, these attention features are concatenated and processed using a fully connected layer; the historical trajectory feature vector of the target vehicle processed by the fully connected layer is used as the "query," and the feature vectors of neighboring vehicles are processed by different fully connected layers to form "keys" and "values."

[0023] Step 1.2.2.5: Generate intermediate semantic vector: Use multi-head attention to aggregate information from different subspaces to generate an intermediate semantic vector that combines all vehicle interaction information;

[0024] Step 1.2.3: Trajectory Prediction Decoder: The decoder receives an intermediate semantic vector containing the trajectory features of the target vehicle and the interaction features of neighboring vehicles, and generates the predicted trajectory point location of the target vehicle within a certain future time period.

[0025] Step 1.2.3.1: The trajectory prediction decoder uses a fully connected layer to embed the coordinates of the previous time step of the predicted trajectory and forms an embedding vector; at this time, the intermediate semantic vector contains the selected vehicle interaction information and the motion state encoding of the target vehicle, and is fed into the LSTM decoder together with the embedding vector.

[0026] Step 1.2.3.2: Add shortcut connections to the LSTM decoder to predict and generate the trajectory point positions of the target vehicle at future time steps;

[0027] Step 2: Online real-time prediction of vehicle trajectory prediction model: After real-time data collection and preprocessing of the vehicle, the trajectory of the target vehicle can be predicted online in real time using the trained vehicle trajectory prediction model.

[0028] Compared with the prior art, the advantages of the present invention are as follows:

[0029] The vehicle trajectory prediction method of this invention considers the coupling between driving intention and interaction with neighboring vehicles, proposes an intention attention mechanism, allocates attention weights in the time dimension to extract historical information of vehicles, and introduces an interaction relationship capture module, which obtains the influence of different neighboring vehicles on the target vehicle based on a multi-head attention mechanism, captures the interaction information between vehicles in the spatial dimension, improves the accuracy of long-term trajectory prediction, and effectively realizes the prediction of lane-changing intentions of surrounding vehicles. Attached Figure Description

[0030] Figure 1 This is a flowchart of the data preprocessing process of the present invention;

[0031] Figure 2 This is a schematic diagram of the coordinate system of the present invention;

[0032] Figure 3This is a structural diagram of the vehicle trajectory prediction model of the present invention;

[0033] Figure 4 This is a structural diagram of the intent recognition module of the present invention;

[0034] Figure 5 This is a diagram of the Bi-LSTM network structure of the present invention;

[0035] Figure 6 This is a structural diagram of the attention mechanism of the present invention;

[0036] Figure 7 This is a structural diagram of the interaction relationship capture module of the present invention. Detailed Implementation

[0037] The following embodiments are in conjunction with the appendix. Figure 1-7 The method of the present invention will be described in further detail below.

[0038] The vehicle trajectory prediction method of this invention consists of two parts: offline training of the vehicle trajectory prediction model and online real-time prediction, including the following steps:

[0039] Step 1: The offline training process of the vehicle trajectory prediction model consists of two parts: building the training database and training the vehicle trajectory prediction model. The implementation steps are as follows:

[0040] Step 1.1: The construction of the training database in this invention is achieved through the collection and processing of large-scale real driving scenarios.

[0041] Step 1.1.1: Data Acquisition: The vehicle driving data collected by the vehicle's onboard sensors, such as cameras and millimeter-wave radar, mainly includes: the longitudinal distance and lateral distance of the target vehicle and its neighboring vehicles, the longitudinal relative speed, the lateral relative speed, the longitudinal acceleration, the lateral relative acceleration, and the lateral distance between the vehicle and the lane lines.

[0042] Step 1.1.2: Data Preprocessing: Figure 1 The data preprocessing workflow is as follows: For a single missing value, nearest neighbor imputation is used to fill it with data from the previous or next time step; for multiple consecutive missing values, interpolation is used to calculate the average of the preceding and following values ​​to fill the missing value; for outliers, the 3-σ criterion is used to remove outliers, calculating the standard deviation σ and mean μ of each feature, and removing data whose values ​​fall outside the interval (μ-3σ, μ+3σ); finally, the Savitzky-Golay filter is used to filter the data. M sampling points are taken near the original data x(i), and a window of 2M+1 sampling points is constructed around x to fit the p-order polynomial y(i), the expression of which is:

[0043]

[0044]

[0045]

[0046] In the formula, y(i) represents the value at time t. i The processed data at time t, x(t) i ) represents in t i The raw data at time i is unprocessed, i = -M, ..., 0, ..., M; E is the total sum of squared errors, p ≤ 2M; k = 0, 1, 2, ..., p.

[0047] Step 1.1.3: Dataset Preparation: Figure 2 This is a schematic diagram of a coordinate system. A fixed reference frame is defined, with the origin fixed at t. p The system is constantly preparing to predict the target vehicle's position. The y-axis points to the direction of movement along the road, and the x-axis is perpendicular to it. The system describes the vehicle's position and calculates the position coordinates of the target vehicle and neighboring vehicles at various times. Simultaneously, the driving intentions of each vehicle are defined as changing lanes to the left, changing lanes to the right, and going straight, and intention labels are assigned to each segment of driving sequence data.

[0048] Step 1.2: Train the vehicle trajectory prediction model: as follows Figure 3 The diagram shown is a network structure diagram of the vehicle trajectory prediction model. The model consists of three main parts: an intent recognition module, a trajectory feature encoder, and a trajectory prediction decoder. The training steps are as follows:

[0049] Step 1.2.1: Intent Recognition Module: This module identifies the vehicle's driving intent by analyzing the temporal evolution of the vehicle's trajectory and its spatial relationship with neighboring vehicles, obtaining a driving intent feature vector m. t The specific steps are as follows:

[0050] Step 1.2.1.1: Input to the Intent Recognition Module: The input features of the intent recognition module include the motion state features of the target vehicle and the interaction state features of the target vehicle, as shown below:

[0051] I t =[s t ,Δs t ]

[0052]

[0053]

[0054]

[0055] In the formula, s t Δs represents the motion state characteristics of the target vehicle. tThe interaction state characteristics of the target vehicle, v t a represents the speed of the target vehicle at the current time t. t v represents the acceleration of the vehicle. xt v represents the lateral velocity of the vehicle. e Represents the driver's expected speed. This represents the relative lateral displacement between the target vehicle and its neighboring vehicles. This represents the relative longitudinal displacement between the target vehicle and its neighboring vehicles, where i represents the neighboring vehicle, and i = 1, 2, ..., 6.

[0056] Step 1.2.1.2: Reference Figure 4 This is a schematic diagram of the vehicle driving intention recognition model of the present invention; the model is based on a BiLSTM network, and the BiLSTM structure is referenced. Figure 5 It is composed of a forward LSTM and a backward LSTM, concatenating the forward and backward hidden layer vectors. Its final output is the combined output of the forward and backward layers at each time step. To fully utilize the input information, the input vector X... t First, a fixed-length embedding vector e is obtained by passing a fully connected layer (FC). t , will e t The hidden state h of the BiLSTM cell at time t is used as the input. t The embedding vector e at the current time t And the hidden state of the previous moment h t-1 Joint update. As follows:

[0057]

[0058] h t =BiLSTM(h t-1 ,e t W l )

[0059] In the formula, It's an FC layer, W emb It is the weight of the FC layer, W l These are the weights of the BiLSTM layer.

[0060] Step 1.2.1.3: Introduce a shortcut connection, using an FC layer to embed the input vector I. t This yields a fixed-length embedding vector r. t Its length is the same as the output vector of the MBi-LSTM. t The same. The resulting embedding vector r t The transformation is performed using the ReLU activation function, and then combined with the output vector of the MBi-LSTM. t Add them together to get a new output vector. Construct a shortcut link as follows:

[0061]

[0062]

[0063] In the formula, It's an FC layer, W r It is the weight of the FC layer.

[0064] Step 1.2.1.4: Input the output vector of the last moment of the historical sequence, i.e., the current moment t. An intent probability vector is obtained by processing through an FC layer and a SoftMax function, as shown in the following formula:

[0065]

[0066] In the formula, Represents the intent category vector, where These represent the probabilities of three different driving intentions; Represents a fully connected layer, W f This represents the weight matrix of the fully connected layer;

[0067] Step 1.2.2: Trajectory Feature Encoder: Combining the target vehicle's intent feature vector obtained by the intent recognition module, an intent attention mechanism is proposed to obtain the historical trajectory feature encoding vectors of the target vehicle and its neighboring vehicles. Then, using the feature encoding vectors of the neighboring vehicles and the target vehicle, an interaction relationship capture module is established based on a multi-head attention mechanism to capture the importance of neighboring vehicles, ultimately forming an intermediate semantic vector representing the influence of "context". The specific steps are as follows:

[0068] Step 1.2.2.1: Trajectory Prediction Encoder Input: The input to the trajectory prediction model includes the target vehicle and surrounding neighboring vehicles over a past time period t. p -t h to t p The historical status information is shown below:

[0069]

[0070]

[0071]

[0072] In the formula, This represents the state of vehicle i at time t, including the vehicle's lateral coordinate. Vertical axis speed and acceleration th The input history sequence represents the length of the input sequence, and N represents the number of neighboring vehicles.

[0073] Step 1.2.2.2: The trajectory prediction model is based on an LSTM encoder-decoder framework; first, the target vehicle and its surrounding neighboring vehicles are analyzed from the past time period t. p -t h to t p The historical state information of each vehicle is encoded. A fully connected layer is used as the embedding layer to encode the input state vector of each vehicle. Embedding is performed to form an embedding vector. Then, LSTM is used to extract different vehicles from t. p -t h To t p The embedding vector for this time period is encoded as follows:

[0074]

[0075]

[0076] In the formula, ψ represents a fully connected layer with a LeakyReLU nonlinear activation function, and W emb Represents the embedding layer weights used for learning. W represents the hidden state of the LSTM cell representing vehicle i at time step t. enc The weight matrix represents the LSTM.

[0077] Step 1.2.2.3: Intent Attention Mechanism: Figure 6 This is a schematic diagram of the intent attention mechanism. The hidden state of the LSTM-encoded vehicle is denoted as... The intent vector of the target vehicle is denoted as The hidden state vector of the trajectory decoder at the previous time step is denoted as p. t-1 . and p t-1 The pieces are joined together using a splicing operation, and then connected by a fully connected layer θ. l The obtained vectors are processed and used as the "query" Q in the key-value pair attention mechanism. l This is used to calculate the correlation between the vehicle's hidden state H and its future trajectory at each historical moment under the influence of intent features. i Composed of fully connected layer φ l and ρ l Processing to form a "bond" K l And "value" V l Then, the attention feature head... l Calculated as "value" v lj The weighted sum formula is as follows:

[0078]

[0079]

[0080]

[0081]

[0082] In the formula, and Represents the weight matrix to be learned in each attention head l, Concat represents the concatenation operation, α lj represents the attention weight, calculated using the dot product attention method, and d represents the scaling factor.

[0083] Step 1.2.2.4: Interaction Relationship Capture Module: Figure 7 This is a schematic diagram of the interaction relationship capture module. It employs a multi-head attention mechanism, using different learned linear projections Q. l K l and V l Calculated n h Attention features head l l = 1, 2, ..., n h Then, these attention features are headed. l They are concatenated together and processed using a fully connected layer to extract higher-order interactions. This is achieved through a fully connected layer θ. s The historical trajectory feature vector of the target vehicle being processed Used as a "query", the feature vector of neighboring vehicles Composed of fully connected layer φ s and ρ s The process forms "keys" and "values", and the cuboid Z... t It consists of six small cuboids, representing the feature vectors of six neighboring vehicles. Blank, transparent cuboids indicate that no neighboring vehicles exist at that location. Similar to the intent attention mechanism, the "query" Q... s "Key" K s And "value" V s Calculation using the scaled dot product attention mechanism:

[0084]

[0085]

[0086] K s =φ s (Z t W φs )

[0087] V s =ρ s (Zt W ρs )

[0088]

[0089] In the formula, z t Let represent the vehicle historical trajectory feature encoding vector obtained by the decoder during time step t prediction. Here, the feature vectors of the target vehicle and surrounding vehicles extracted by the intent attention mechanism are represented as follows: Represents a fully connected layer, W iq This represents the weight matrix of the fully connected layer. and α represents the learnable weight matrix corresponding to the transformation. sj This represents the degree of relevance between neighboring vehicles and the target vehicle.

[0090] Step 1.2.2.5: Generate intermediate semantic vectors: the obtained α sj It can be used to compute attention features head that have interactions with surrounding neighboring vehicles. s Additionally, multi-head attention is used to aggregate information from different subspaces, thereby enhancing the model's representational power.

[0091]

[0092]

[0093] In the formula, c t This represents the intermediate semantic vector that the decoder combines all vehicle interaction information when making predictions at time step t. Represents a fully connected layer, W si This represents the weight matrix of the fully connected layer, n. h This represents the number of attention features in parallel computing.

[0094] Step 1.2.3: Trajectory Prediction Decoder: The decoder receives an intermediate semantic vector containing the trajectory features of the target vehicle and the interaction features of neighboring vehicles, and generates the predicted trajectory point location of the target vehicle within a certain future time period.

[0095] Step 1.2.3.1: When making predictions at time step t, the trajectory prediction decoder also uses a fully connected layer with a LeakyReLU nonlinear activation function, embedding the coordinates Y of the predicted trajectory's position at the previous time step. t-1 And form the embedding vector e t The intermediate semantic vector C at this point t Includes selected vehicle interaction information c t and motion state coding of the target vehicle With embedding vector e t They were fed together to the LSTM decoder

[0096] e t =ψ(Y t-1 W ed )

[0097]

[0098] U t =Concat(C t ,e t )

[0099] In the formula, ψ represents a fully connected layer with a LeakyReLU nonlinear activation function, and W ed This represents the embedding layer weights used for learning. U t This represents the input vector of the LSTM decoder when performing prediction at time step t, and Concat represents the concatenation operation.

[0100] Step 1.2.3.2: The LSTM decoder predicts and generates the future time step t = t_0 of the target vehicle. p +1,t p +2,...,t p +t f The location of the trajectory points. The decoder also includes the same shortcut connections as in the intent recognition model in Section 3.3.3. The formula is as follows:

[0101] p t =LSTM(p t-1 U t W dec )

[0102]

[0103]

[0104] In the formula, p t This represents the hidden state vector of the LSTM decoder. W represents the new state vector obtained by introducing a shortcut connection. dec W d and W p This is the weight matrix.

[0105] Step 2: Online real-time prediction of lane change intention: After real-time data collection and preprocessing, the trained vehicle trajectory prediction model is used to predict the trajectory of the target vehicle online in real time.

[0106] The above description, in conjunction with preferred technical solutions, provides a further detailed explanation of the present invention and should not be construed as limiting the specific implementation of the invention to these descriptions. For those skilled in the art, simple deductions and substitutions can be made without departing from the concept of the present invention, and all such deductions and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network, characterized in that... It consists of two parts: offline training of the vehicle trajectory prediction model and online real-time prediction, including the following steps: Step 1: Offline training of the vehicle trajectory prediction model, which consists of two parts: building the training database and training the vehicle trajectory prediction model. The specific implementation steps are as follows: Step 1.1: Construct a training database through the collection and processing of large-scale real-world driving scenarios; Step 1.1.1: Data Acquisition: The vehicle driving data collected by the vehicle's onboard sensors, such as cameras and millimeter-wave radar, mainly includes: the longitudinal distance and lateral distance between the target vehicle and neighboring vehicles, the longitudinal relative speed, the lateral relative speed, the longitudinal acceleration, the lateral relative acceleration, and the lateral distance between the vehicle and the lane lines. Step 1.1.2: Data preprocessing: Impute missing values ​​in the data. Use the nearest neighbor imputation method to impute single missing values, use interpolation to impute multiple consecutive missing values, use the 3-σ rule to remove outliers, and finally use the Savitzky-Golay filter to filter the data to obtain a smoother data curve. Step 1.1.3: Dataset preparation: Define a coordinate system to facilitate the description of vehicle positions, calculate the position coordinates of the target vehicle and neighboring vehicles at various times; define vehicle driving intentions as changing lanes to the left, changing lanes to the right, and going straight, and set intention labels for each segment of driving sequence data; Step 1.2: The training steps for the vehicle trajectory prediction model are as follows: Step 1.2.1: Intent Recognition Module: Identifies the vehicle's driving intent by analyzing the temporal evolution of the vehicle's trajectory and its spatial relationship with neighboring vehicles, thereby obtaining a driving intent feature vector; Step 1.2.2: Trajectory Feature Encoder: Combining the target vehicle intent feature vector obtained by the intent recognition module, an intent attention mechanism is proposed to obtain the historical trajectory feature encoding vectors of the target vehicle and its neighboring vehicles. Then, using the feature encoding vectors of the neighboring vehicles and the target vehicle, an interaction relationship capture module is established based on a multi-head attention mechanism to capture the importance of the neighboring vehicles, and finally an intermediate semantic vector representing the influence of "context" is formed. Step 1.2.3: Trajectory Prediction Decoder: The decoder receives an intermediate semantic vector containing the trajectory features of the target vehicle and the interaction features of neighboring vehicles, and generates the predicted trajectory point location of the target vehicle within a certain future time period. Step 2: Online real-time prediction of vehicle trajectory prediction model: After real-time data collection and preprocessing of the vehicle, the trajectory of the target vehicle can be predicted online in real time using the trained vehicle trajectory prediction model.

2. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 1, characterized in that... In step 1.1.2, the specific steps of data preprocessing include: for a single missing value in the data, the nearest neighbor imputation method is used to fill it with data from the previous or next time step; for multiple consecutive missing values, interpolation is used to calculate the average of the previous and next missing values ​​to fill them; for outliers in the data, the 3-σ criterion is used to remove outliers, the standard deviation σ and mean μ of each feature are calculated, and data with values ​​outside the interval (μ-3σ, μ+3σ) are removed; finally, the Savitzky-Golay filter is used to filter the data, taking M sampling points near the original data x(i), and constructing a window of 2M+1 sampling points around x to fit the p-order polynomial y(i), the expression of which is: In the formula, y(i) represents the value at time t. i The processed data at time t, x(t) i ) represents in t i The raw data at time i is unprocessed, i = -M, ..., 0, ..., M; E is the total sum of squared errors, p ≤ 2M; k = 0, 1, 2, ..., p.

3. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 1, characterized in that... In step 1.2.1, the specific steps for obtaining the driving intention feature vector include: Step 1.2.1.1: Input to the intent recognition module: The input features of the intent recognition module include the motion state features of the target vehicle and the interaction state features of the target vehicle; Step 1.2.1.2: The vehicle driving intention recognition model is based on a BiLSTM network, which is composed of a forward LSTM and a backward LSTM. The forward and backward hidden layer vectors are concatenated together, and its final output is the combined output of the forward and backward layers at each time step. Step 1.2.1.3: Introduce a shortcut connection to achieve efficient information flow across different layers and improve prediction performance; Step 1.2.1.4: The output vector of the last moment of the input history sequence is processed by an FC layer and a SoftMax function to obtain an intention probability vector.

4. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 3, characterized in that... In step 1.2.1.1, the input features of the intent recognition module include the motion state features of the target vehicle and the interaction state features of the target vehicle, as shown below: I t =[s t ,Δs t ] In the formula, s t Δs represents the motion state characteristics of the target vehicle. t v represents the interactive state characteristics of the target vehicle. t a represents the speed of the target vehicle at the current time t. t v represents the acceleration of the vehicle. xt v represents the lateral velocity of the vehicle. e Represents the driver's expected speed. This represents the relative lateral displacement between the target vehicle and its neighboring vehicles. The relative longitudinal displacement between the target vehicle and its neighboring vehicles is represented by , where i represents the neighboring vehicle, and i = 1, 2, ..., 6; In step 1.2.1.2, the vehicle driving intention recognition model, in order to make full use of the input information, inputs vector X. t First, a fixed-length embedding vector e is obtained by passing a fully connected layer (FC). t , will e t As the input to the BiLSTM cell at time t; The hidden state h of the BiLSTM cell at time t t The embedding vector e at the current time t And the hidden state of the previous moment h t-1 Joint update; as follows: h t =BiLSTM(h t-1 ,e t ; W l ) In the formula, It's an FC layer, W emb It is the weight of the FC layer, W l These are the weights of the BiLSTM layer; In step 1.2.1.3, a shortcut connection is introduced, using an FC layer to embed the input vector I. t This yields a fixed-length embedding vector r. t Its length is the same as the output vector of the MBi-LSTM. t Same; the resulting embedding vector r t The transformation is performed using the ReLU activation function, and then combined with the output vector of the MBi-LSTM. t Add them together to get a new output vector. Construct the shortcut as follows: In the formula, It's an FC layer, W r It is the weight of the FC layer; In step 1.2.1.4, an intention probability vector is obtained, which is the output vector of the last moment of the input historical sequence, i.e., the current moment t. An intent probability vector is obtained by processing through an FC layer and a SoftMax function, as shown in the following formula: In the formula, Represents the intent category vector, where These represent the probabilities of three different driving intentions; W represents a fully connected layer. f This represents the weight matrix of the fully connected layer.

5. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 1, characterized in that... In step 1.2.2, the specific steps for forming an intermediate semantic vector representing the influence of "context" include: Step 1.2.2.1: Trajectory Prediction Encoder Input: The input to the trajectory prediction model includes historical state information of the target vehicle and surrounding neighboring vehicles over a period of time; Step 1.2.2.2: The trajectory prediction model is based on an encoder-decoder framework using LSTM; the historical state information of the target vehicle and its surrounding neighboring vehicles is encoded from a past period; firstly, a fully connected layer is used as the embedding layer to embed the input state vector of each vehicle to form an embedding vector; then, LSTM is used to encode the embedding vectors of different vehicles respectively. Step 1.2.2.3: Intent Attention Mechanism: The intent vector of the target vehicle and the hidden state vector of the predicted trajectory decoder at the previous time step are concatenated by a concatenation operation. The vector obtained by the fully connected layer is then used as the "query" in the key-value pair attention mechanism. The hidden state of the vehicle is processed by different fully connected layers to form "keys" and "values". Then the attention features are calculated as a weighted sum of the "values". Step 1.2.2.4: Interaction Relationship Capture Module: Employs a multi-head attention mechanism, using different learned linear projections to calculate multiple attention features; then, these attention features are concatenated and processed using a fully connected layer; the historical trajectory feature vector of the target vehicle processed by the fully connected layer is used as the "query," and the feature vectors of neighboring vehicles are processed by different fully connected layers to form "keys" and "values." Step 1.2.2.5: Generate intermediate semantic vector: Use multi-head attention to aggregate information from different subspaces to generate an intermediate semantic vector that combines all vehicle interaction information.

6. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 5, characterized in that... In step 1.2.2.1, the trajectory prediction encoder input is as follows: In the formula, This represents the state of vehicle i at time t, including the vehicle's lateral coordinate. Vertical axis speed and acceleration t h The input history sequence represents the length of the input sequence, and N represents the number of neighboring vehicles. In step 1.2.2.2, the encoding of the historical state information of the target vehicle and its surrounding neighboring vehicles over a past period of time is performed. p -t h to t p The historical state information of each vehicle is encoded; a fully connected layer is used as the embedding layer to encode the input state vector of each vehicle. Embedding is performed to form an embedding vector. Then, LSTM is used to extract different vehicles from t. p -t h To t p The embedding vector for this time period is encoded as follows: In the formula, ψ represents a fully connected layer with a LeakyReLU nonlinear activation function, and W emb Represents the embedding layer weights used for learning. W represents the hidden state of the LSTM cell representing vehicle i at time step t. enc The weight matrix represents the LSTM.

7. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 5, characterized in that... In step 1.2.2.3, the intent attention mechanism records the hidden state of the LSTM-encoded vehicle as... The intent vector of the target vehicle is denoted as The hidden state vector of the trajectory decoder at the previous time step is denoted as p. t-1 ; and p t-1 The pieces are joined together using a splicing operation, and then connected by a fully connected layer θ. l The obtained vectors are processed and used as the "query" Q in the key-value pair attention mechanism. l To calculate the correlation between the state information at each historical moment under the intervention of intentional features and the future trajectory; Vehicle's hidden state H i Composed of fully connected layer φ l and ρ l Processing to form a "bond" K l And "value" V l Then focus on the head feature. l Calculated as a "value" The weighted sum formula is as follows: In the formula, and Represents the weight matrix to be learned in each attention head l, Concat represents the concatenation operation, α lj represents the attention weight, calculated using the dot product attention method, and d represents the scaling factor.

8. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 5, characterized in that... In step 1.2.2.4, the interaction relationship capture module employs a multi-head attention mechanism, using different learned linear projections Q. l K l and V l Calculate n h Attention features head l l = 1, 2, ..., n h ; these attention features head l They are stitched together and processed using a fully connected layer to extract higher-order interactions; this is achieved by the fully connected layer θ. s The historical trajectory feature vector of the target vehicle being processed Used as a "query", the feature vector of neighboring vehicles Composed of fully connected layer φ s and ρ s The process forms "keys" and "values", and the cuboid Z... t It consists of six small cuboids, representing the feature vectors of six neighboring vehicles. A blank, transparent cuboid indicates that there are no neighboring vehicles at that location. Similar to the intent attention mechanism, "query" Q s "key" K s And "value" V s Calculation using the scaled dot product attention mechanism: K s =φ s (WITH t ;IN φs ) V s =ρ s (WITH t ;IN ρs ) In the formula, z t Let represent the vehicle historical trajectory feature encoding vector obtained by the decoder during time step t prediction. Here, the feature vectors of the target vehicle and surrounding vehicles extracted by the intent attention mechanism are represented as follows: W represents a fully connected layer. iq This represents the weight matrix of the fully connected layer; and α represents the learnable weight matrix corresponding to the transformation. sj This represents the degree of relevance between neighboring vehicles and the target vehicle. In step 1.2.2.5, the intermediate semantic vector is generated: the obtained α sj It can be used to compute attention features head that have interactions with surrounding neighboring vehicles. s Additionally, multi-head attention is used to aggregate information from different subspaces, thereby enhancing the model's representational power. In the formula, c t This represents the intermediate semantic vector that the decoder combines all vehicle interaction information when making predictions at time step t. W represents a fully connected layer. si This represents the weight matrix of the fully connected layer, n. h This represents the number of attention features in parallel computing.

9. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 1, characterized in that... In step 1.2.3, the specific steps for generating the predicted trajectory point location of the target vehicle within a future time period include: Step 1.2.3.1: The trajectory prediction decoder uses a fully connected layer to embed the coordinates of the position of the predicted trajectory at the previous time step and forms an embedding vector; at this time, the intermediate semantic vector contains the selected vehicle interaction information and the motion state encoding of the target vehicle, which is fed into the LSTM decoder together with the embedding vector. Step 1.2.3.2: Add shortcut connections to the LSTM decoder to predict and generate the trajectory point positions of the target vehicle at future time steps.

10. The vehicle trajectory prediction method based on an intent-aware spatiotemporal attention network as described in claim 9, characterized in that... In step 1.2.3.1, the trajectory prediction decoder, when performing prediction at time step t, also employs a fully connected layer with a LeakyReLU nonlinear activation function to embed the coordinates Y of the position of the predicted trajectory at the previous time step. t-1 And form the embedding vector e t The intermediate semantic vector C at this point t Includes selected vehicle interaction information c t and motion state coding of the target vehicle With embedding vector e t They are fed together to the LSTM decoder: e t =ψ(Y t-1 ;W ed ) U t =Concat(C t ,e t ) In the formula, ψ represents a fully connected layer with a LeakyReLU nonlinear activation function, and W ed Represents the embedding layer weights used for learning; U t This represents the input vector of the LSTM decoder when performing prediction at time step t, and Concat represents the concatenation operation. In step 1.2.3.2, the LSTM decoder predicts and generates the future time step t = t_future for the target vehicle. p +1,t p +2,...,t p +t f The trajectory point locations; the decoder also adds the same shortcut connections as in the intent recognition model; the formula is as follows: p t =LSTM(p t-1 ,U t ;W dec ) In the formula, p t This represents the hidden state vector of the LSTM decoder. W represents the new state vector obtained by introducing a shortcut connection. dec W d and W p This is the weight matrix.

Images