A speed-aware spatio-temporal interactive vehicle trajectory prediction method
By employing a speed-aware spatiotemporal interactive vehicle trajectory prediction method, which utilizes a multi-head attention mechanism and raster modeling to dynamically adjust the perception range, this approach solves the problems of relying on high-precision maps and heavy computational burden in existing technologies, achieving more efficient and flexible trajectory prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2025-06-12
- Publication Date
- 2026-06-09
AI Technical Summary
Existing autonomous driving trajectory prediction methods rely on high-precision maps, which are difficult to apply in unstructured scenarios. They also have a heavy computational burden and cannot simulate the driver's perception characteristics or flexibly adjust the perception range.
A speed-aware spatiotemporal interactive vehicle trajectory prediction method is adopted. Through encoder and decoder modules, multi-head attention mechanism and raster modeling are used to dynamically adjust the perception range, capture the spatial dependency relationship between the vehicle and the scene, and simulate the driver's perception characteristics.
It reduces computational resource overhead, improves the accuracy and efficiency of trajectory prediction, and enhances the model's adaptability and application flexibility in unstructured scenarios.
Smart Images
Figure CN120611190B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, and in particular to a spatiotemporal interactive vehicle trajectory prediction method based on speed perception. Background Technology
[0002] Autonomous driving technology has gradually become a focus of attention due to its enormous potential in alleviating traffic congestion, improving passenger experience, and significantly reducing traffic accidents caused by human error. Thanks to the rapid development of technologies such as high-performance computing, vehicle-to-everything (V2X) communication, and artificial intelligence, autonomous driving systems are rapidly advancing from basic L1 / L2 automation functions to higher levels of intelligent driving such as L3 and L4. This transformation is profoundly reshaping people's travel habits and propelling transportation systems into a new era of intelligence and automation.
[0003] With the rapid development of autonomous driving technology, accurately predicting the future trajectories of surrounding vehicles in complex and dynamic traffic environments has become one of the key challenges for autonomous driving systems to achieve safe and efficient decision-making. Traditional trajectory prediction methods mostly rely on global attention mechanisms, modeling the pairwise interactions of all traffic participants in the scene, ignoring the perception limitations during driving. This not only increases the computational burden but also makes it difficult to reflect the rationality of driving behavior. To address these issues, there is an urgent need for a prediction method that can flexibly adjust the perception range according to the actual situation, just like a human driver. On the other hand, existing methods often rely on high-precision maps to provide environmental constraints, limiting their application in unstructured or map-unavailable scenarios. Therefore, there is an urgent need for a trajectory prediction technology that does not require high-precision maps, can dynamically model the spatiotemporal relationships between traffic participants, and has adjustable perception range characteristics, in order to improve the system's generalization ability and operational efficiency in diverse traffic situations. Summary of the Invention
[0004] The purpose of this invention is to provide a speed-aware spatiotemporal interactive vehicle trajectory prediction method that can accurately capture the spatial dependency between the vehicle and the scene, improving the accuracy of trajectory prediction. Furthermore, it can adaptively adjust the interaction range based on the target vehicle's current speed, thereby simulating the driver's perception characteristics at different speeds and improving the efficiency of trajectory prediction. The technical solution adopted by this invention is as follows.
[0005] On one hand, the present invention provides a spatiotemporal interactive vehicle trajectory prediction method based on speed perception, comprising:
[0006] Acquire the driving trajectory data of the target vehicle and surrounding vehicles within its perception range;
[0007] The driving trajectory data is input into a pre-trained trajectory prediction model to obtain the target vehicle trajectory data for the prediction period; wherein, the trajectory prediction model includes an encoder module, a speed perception interaction modeling module, and a decoder module;
[0008] The encoder module includes a spatiotemporal feature encoder and a scene perception encoder. The spatiotemporal feature encoder is used to encode the driving trajectories of the target vehicle and other vehicles respectively, and fill the encoded result data into the perception range grid according to the relative position of the surrounding vehicles and the target vehicle to obtain the spatiotemporal feature encoding. The scene perception encoder is used to extract the global spatial dependency relationship between vehicles based on the driving trajectory data to obtain the scene perception spatial encoding.
[0009] The speed-sensing interaction modeling module is used to: extract the spatiotemporal interaction features and scene interaction features of the target vehicle and surrounding vehicles at each historical moment based on the spatiotemporal feature encoding and scene-sensing spatial encoding, and obtain global interaction features based on the spatiotemporal interaction features and scene interaction features.
[0010] The decoder module is used to obtain the driving trajectory of the target vehicle during the prediction period based on the spatiotemporal interaction features and global interaction features.
[0011] Optionally, the method for constructing the sensing range grid is as follows:
[0012] Get the current speed of the target vehicle. The number of lanes on the road where the target vehicle is located And preset perception range parameters, the perception range parameters including the basic target vehicle line-of-sight range. Expected headway And the vertical length of a single grid cell (e.g., along the lane direction) ;
[0013] The perception range of the target vehicle is calculated based on the perception range parameters: ;
[0014] The perception range of the target vehicle is divided into: Grid size .
[0015] Optionally, the driving trajectory data includes the vehicle's position coordinates, speed, angular velocity, and lane number data at each moment within a set time period, wherein the position coordinates are relative position coordinates determined with the position of the target vehicle at the same moment as the coordinate origin.
[0016] The driving trajectory data is represented as follows: ,in Indicates the current moment. Indicates the duration covered by the driving trajectory. express target vehicles and The trajectory information of the vehicles surrounding the vehicle. Corresponding target vehicle, Indicates the first The surrounding vehicles at all times The trajectory information is represented as: , They represent the first The surrounding vehicles at all times The relative longitudinal position, relative lateral position, speed, acceleration, and lane number.
[0017] Optionally, the spatiotemporal feature encoder encodes the driving trajectories of the target vehicle and other vehicles separately, and fills the encoded data into the sensing range grid to obtain spatiotemporal features, including:
[0018] Two Long Short-Term Memory (LSTM) networks are used to encode the trajectory features of the target vehicle and surrounding vehicles, respectively, to obtain the trajectory codes of the target vehicle and surrounding vehicles. , , respectively represented as:
[0019]
[0020]
[0021] in, This represents the hidden state dimension of the LSTM;
[0022] Encode the trajectories of surrounding vehicles According to the current time The corresponding grid position of the vehicle within the perception range is filled into the corresponding grid to obtain the spatiotemporal feature code. The spatiotemporal characteristic value corresponding to the current moment is .
[0023] Optionally, the scene-aware encoder extracts the global spatial dependencies between vehicles based on the driving trajectory data to obtain scene-aware spatial coding, including:
[0024] Based on the speed and acceleration data in the driving trajectory data, calculate the speed difference matrix between vehicles within the sensing range grid. And acceleration difference matrix :
[0025]
[0026]
[0027] velocity difference matrix And acceleration difference matrix By splicing :
[0028]
[0029] Will The data is fed into a convolutional layer for feature extraction, resulting in... The convolutional layer is selected as a set of convolutional layers with a kernel of 1;
[0030] Two multilayer perceptron (MLP) sublayers are alternately applied to the MLPMixer network to process the extracted data along the feature dimension and the grid node dimension, respectively. The feature sequences are mixed to obtain : ;
[0031] Will As node feature inputs to the Graph Attention Network (GATv2), it captures global spatial dependencies between vehicles, leading to scene-aware spatial encoding. : .
[0032] Optionally, the speed perception interaction modeling module includes a vehicle-to-vehicle interaction unit, a vehicle-to-scene interaction unit, and a scene-to-vehicle interaction unit;
[0033] The vehicle-to-vehicle interaction unit is used to extract the spatiotemporal interaction features between the target vehicle and surrounding vehicles at historical moments based on the spatiotemporal feature encoding using a multi-head attention mechanism.
[0034] The vehicle-scene interaction unit is used to extract the scene interaction features between the target vehicle and surrounding vehicles at historical moments based on the scene perception space and the spatiotemporal interaction features using a multi-head attention mechanism.
[0035] The scene-vehicle interaction unit is used to obtain global interaction features based on the spatiotemporal interaction features and scene interaction features.
[0036] Optionally, the method by which the vehicle-to-vehicle interaction unit obtains spatiotemporal interaction features includes:
[0037] Encode the target vehicle's trajectory As a query Q, the spatiotemporal feature encoding As key K and value V, As a mask for cross-attention, a spatial attention network based on a multi-head attention mechanism and a gated linear unit (GLU) are used to extract the interaction information between the target vehicle and surrounding vehicles at various historical moments. The formula is expressed as:
[0038]
[0039]
[0040]
[0041] in, Here, is the dimension of the key vector. , This represents the maximum perception range of the target vehicle. For each node position in the perception range grid, if it is within the perception range... If it is inside, the mask value is 0; if it is located inside... Beyond but still in Inside, the mask value is ;
[0042] Will and target vehicle trajectory coding By inputting residual connections and layer normalization networks, we obtain the encoding of the target vehicle at each historical timestamp, which incorporates spatial interaction features. :
[0043]
[0044]
[0045] Will As the query Q, key K, and value V, the input is a temporal attention network based on a multi-head attention mechanism. Then, it sequentially passes through residual connections and a layer normalization network, a feed-forward network, and another residual connection and layer normalization network to obtain the vehicle-to-vehicle interaction code. This refers to the spatiotemporal interaction features, specifically the features of the last time step. Used for the scene-vehicle interaction unit; the process is represented as:
[0046]
[0047]
[0048] .
[0049] Optionally, the method by which the vehicle-scene interaction unit obtains the scene interaction features includes:
[0050] Encode the scene-aware space As a query Q, the spatiotemporal feature encoding As key K and value V, As a mask for cross-attention, the input is a Masked Multi-Head Attention network, which then sequentially passes through residual connections and a layer normalization network, a Feed-Forward Network, and another residual connection and layer normalization network to obtain the vehicle-scene interaction feature encoding. This refers to the scene interaction features; the process is represented as:
[0051]
[0052]
[0053] .
[0054] Optionally, the scene-vehicle interaction unit is used to obtain global interaction features based on the spatiotemporal interaction features and scene interaction features, including:
[0055] The features of the last time step in the spatiotemporal interaction features As query Q, vehicle-scenario interaction feature encoding As key K and value V, As a mask for cross-attention, the input is a Masked Multi-Head Attention network, which then sequentially passes through residual connections and a layer normalization network, a Feed-Forward Network, and another residual connection and layer normalization network to obtain a global interactive feature encoding containing vehicle dynamics and scene information. This refers to the global interaction feature, which is represented as:
[0056] .
[0057]
[0058] .
[0059] Optionally, the decoder module obtains the target vehicle's driving trajectory during the prediction period based on the spatiotemporal interaction features and global interaction features, including:
[0060] Encoding vehicle-to-vehicle interaction features and global interactive feature encoding The data is stitched together and fed into an LSTM decoder to obtain the future trajectory of the target vehicle. , is represented as:
[0061] .
[0062] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the speed-aware spatiotemporal interactive vehicle trajectory prediction method described in the first aspect.
[0063] Beneficial effects
[0064] Compared with the prior art, the present invention has the following advantages and advancements:
[0065] The perceptual range gridded modeling of this invention realizes a speed-adaptive interactive modeling method, which can dynamically adjust the attention range according to the motion state of the target vehicle during trajectory prediction, effectively simulating the perceptual characteristics of a human driver. This mechanism significantly reduces the resource overhead caused by redundant computation while ensuring modeling accuracy, and improves the interpretability of the attention mechanism and overall computational efficiency.
[0066] This invention introduces a scene-aware encoder based on rasterization modeling, which can effectively model the spatiotemporal relationships between local traffic participants without relying on high-precision map information. While possessing structured expression capabilities, it also improves the adaptability and application flexibility of the model in unstructured or map-deficient scenarios, and has broader engineering application prospects. Attached Figure Description
[0067] Figure 1 The diagram shown is a schematic representation of the implementation process of an embodiment of the method of the present invention.
[0068] Figure 2 The diagram shown is a schematic of the trajectory prediction model architecture in an embodiment of the present invention.
[0069] Figure 3 The figure shown is a schematic diagram of the trajectory prediction effect in an embodiment of the present invention. Detailed Implementation
[0070] The following description, in conjunction with the accompanying drawings and specific embodiments, provides further details.
[0071] Example 1
[0072] This embodiment introduces a spatiotemporal interactive vehicle trajectory prediction method based on speed perception, including:
[0073] Acquire the driving trajectory data of the target vehicle and surrounding vehicles within its perception range;
[0074] The driving trajectory data is input into a pre-trained trajectory prediction model to obtain the target vehicle trajectory data for the prediction period; wherein, the trajectory prediction model includes an encoder module, a speed perception interaction modeling module, and a decoder module;
[0075] The encoder module includes a spatiotemporal feature encoder and a scene perception encoder. The spatiotemporal feature encoder is used to encode the driving trajectories of the target vehicle and other vehicles respectively, and fill the encoded result data into the perception range grid according to the relative position of the surrounding vehicles and the target vehicle to obtain the spatiotemporal feature encoding. The scene perception encoder is used to extract the global spatial dependency relationship between vehicles based on the driving trajectory data to obtain the scene perception spatial encoding.
[0076] The speed-sensing interaction modeling module is used to: extract the spatiotemporal interaction features and scene interaction features of the target vehicle and surrounding vehicles at each historical moment based on the spatiotemporal feature encoding and scene-sensing spatial encoding, and obtain global interaction features based on the spatiotemporal interaction features and scene interaction features.
[0077] The decoder module is used to obtain the driving trajectory of the target vehicle during the prediction period based on the spatiotemporal interaction features and global interaction features.
[0078] Combination Figure 1 and Figure 2 The specific implementation of the method in this embodiment involves the following content.
[0079] I. Construction and Training of Trajectory Prediction Model
[0080] refer to Figure 2 The trajectory prediction model network architecture shown includes an encoder module, a velocity-aware interactive modeling module, and a decoder module; wherein:
[0081] The encoder module includes a spatiotemporal feature encoder and a scene perception encoder. The spatiotemporal feature encoder is used to encode the driving trajectories of the target vehicle and other vehicles respectively, and fill the encoded result data into the perception range grid according to the relative position of the surrounding vehicles and the target vehicle to obtain the spatiotemporal feature encoding. The scene perception encoder is used to extract the global spatial dependency relationship between vehicles based on the driving trajectory data to obtain the scene perception spatial encoding.
[0082] The speed-sensing interaction modeling module is used to: extract the spatiotemporal interaction features and scene interaction features of the target vehicle and surrounding vehicles at each historical moment based on the spatiotemporal feature encoding and scene-sensing spatial encoding, and obtain global interaction features based on the spatiotemporal interaction features and scene interaction features.
[0083] The decoder module is used to obtain the driving trajectory of the target vehicle during the prediction period based on the spatiotemporal interaction features and global interaction features.
[0084] refer to Figure 1The implementation process shown involves first acquiring driving trajectory data of multiple vehicles in various typical road scenarios during model training, and then preprocessing this data to obtain model training samples. This preprocessing includes, but is not limited to:
[0085] (1) Exclude vehicles whose total trajectory duration is less than 8 seconds;
[0086] (2) Number the collected road sections and lanes in order from top to bottom, such as 1, 2, 3...
[0087] The historical trajectory sequences of the target vehicle and its surrounding vehicles, as well as the actual target vehicle trajectory sequences used for loss function calculation, are extracted from the selected multi-vehicle driving trajectory data and used as training sample data.
[0088] Vehicle trajectory information includes time, location coordinates (X and Y coordinates), vehicle speed, acceleration, lane, and vehicle type. Historical trajectories can be represented as:
[0089]
[0090] In the formula, Represents historical trajectory, including the length of historical time period. Vehicle trajectory information, express target vehicles and The trajectory information of the vehicles surrounding the vehicle. Indicates the target vehicle; Indicates the first The car at any time The trajectory information can be represented as:
[0091]
[0092] In the formula, They represent the first The vehicle is the target vehicle at the current moment. The position is taken as the origin of the coordinate system at time 10:00. The longitudinal position, lateral position, speed, acceleration, and lane number of the vehicle.
[0093] During model training, historical trajectories are selected as the model input, and the model output is the future trajectory coordinates of the target vehicle (which are known data in the training samples), represented as:
[0094]
[0095] In the formula, It is the output vector. It is the future time step of the predicted trajectory. express The position coordinates of the target vehicle at any given time.
[0096] Specifically, after inputting the preprocessed historical trajectory sequences of the target vehicle and its surrounding vehicles into the trajectory prediction model, the trajectory prediction model is based on... Figure 2 The model network architecture shown performs the following data processing.
[0097] 1) Encoder module based on speed-aware line-of-sight analysis:
[0098] In this section, this embodiment constructs a sensing range grid capable of adapting to the speed of a target vehicle, using the following method:
[0099] Get the current speed of the target vehicle. The number of lanes on the road where the target vehicle is located And preset perception range parameters, the perception range parameters including the basic target vehicle line-of-sight range. Expected headway And the vertical length of a single grid cell (e.g., along the lane direction) ;
[0100] The perception range of the target vehicle is calculated based on the perception range parameters: ;
[0101] The perception range of the target vehicle is divided into: Grid size .
[0102] To support batch training in situations where the perception range is variable, this embodiment introduces a visibility mask. ,in This is the maximum sensing range of the target vehicle (e.g., corresponding to 120 km / h). For each spatial location within the full sensing range grid, if a neighboring vehicle is within the effective sensing range... If the vehicle is inside the effective range, the mask value is 0; if the vehicle is outside the effective range but still within the maximum range, the mask value is 0. Inside, the mask value is This mechanism ensures that the trajectory prediction model focuses its attention on the effective area, filtering out distant and irrelevant regions.
[0103] 2) The encoder module encodes the driving trajectories of the target vehicle and other vehicles separately using a spatiotemporal feature encoder, filling the perceptual range grid to obtain spatiotemporal feature codes. Specifically:
[0104] The spatiotemporal feature encoder uses two Long Short-Term Memory (LSTM) networks to encode the trajectory features of the target vehicle and surrounding vehicles, respectively, to obtain the trajectory codes of the target vehicle and surrounding vehicles. , , respectively represented as:
[0105]
[0106]
[0107] in, This represents the hidden state dimension of the LSTM;
[0108] Encode the trajectories of surrounding vehicles According to the current time The corresponding grid position of the vehicle within the perception range is filled into the corresponding grid to obtain the spatiotemporal feature code. The spatiotemporal characteristic value corresponding to the current moment is .
[0109] 3) The encoder module extracts the global spatial dependencies between vehicles based on driving trajectory data using the scene-aware encoder, obtaining scene-aware spatial coding. Specifically:
[0110] The scene perception encoder calculates the speed difference matrix between vehicles within the perception range grid based on the speed and acceleration data in the driving trajectory data. And acceleration difference matrix :
[0111]
[0112]
[0113] velocity difference matrix And acceleration difference matrix By splicing :
[0114]
[0115] Will The data is fed into a convolutional layer for feature extraction, resulting in... The convolutional layer is selected as a set of convolutional layers with a kernel of 1;
[0116] Two multilayer perceptron (MLP) sublayers are alternately applied to the MLPMixer network to process the extracted data along the feature dimension and the grid node dimension, respectively. The feature sequences are mixed to achieve information exchange between different grids and between different vehicles, resulting in... : .
[0117] Will As node feature inputs to the Graph Attention Network (GATv2), it captures global spatial dependencies between vehicles, leading to scene-aware spatial encoding. : .
[0118] 4) The vehicle-to-vehicle interaction unit of the speed perception interaction modeling module uses a multi-head attention mechanism based on spatiotemporal feature encoding to extract the spatiotemporal interaction features between the target vehicle and surrounding vehicles at historical moments.
[0119] The vehicle-to-vehicle interaction unit encodes the trajectory of the target vehicle. As a query Q, the spatiotemporal feature encoding As key K and value V, As a mask for cross-attention, a spatial attention network based on a multi-head attention mechanism and a gated linear unit (GLU) are used to extract the interaction information between the target vehicle and surrounding vehicles at various historical moments. The formula is expressed as:
[0120]
[0121]
[0122]
[0123] in, Here, is the dimension of the key vector. , This represents the maximum perception range of the target vehicle. For each node position in the perception range grid, if it is within the perception range... If it is inside, the mask value is 0; if it is located inside... Beyond but still in Inside, the mask value is ;
[0124] Will and target vehicle trajectory coding By inputting residual connections and layer normalization networks, we obtain the encoding of the target vehicle at each historical timestamp, which incorporates spatial interaction features. :
[0125]
[0126]
[0127] Will As the query Q, key K, and value V, the input is a temporal attention network based on a multi-head attention mechanism. Then, it sequentially passes through residual connections and a layer normalization network, a feed-forward network, and another residual connection and layer normalization network to obtain the vehicle-to-vehicle interaction code. This refers to the spatiotemporal interaction features, specifically the features of the last time step. Used for the scene-vehicle interaction unit; the process is represented as:
[0128]
[0129]
[0130] .
[0131] 5) The vehicle-scene interaction unit of the speed perception interaction modeling module extracts the scene interaction features between the target vehicle and surrounding vehicles at historical moments based on the scene perception space and the spatiotemporal interaction features using a multi-head attention mechanism.
[0132] The vehicle-scene interaction unit encodes the scene perception space. As a query Q, the spatiotemporal feature encoding As key K and value V, As a mask for cross-attention, the input is a Masked Multi-HeadAttention network, which then sequentially passes through residual connections and a layer normalization network, a Feed-Forward Network, and another residual connection and layer normalization network to obtain the vehicle-scene interaction feature encoding. This refers to the scene interaction features; the process is represented as:
[0133]
[0134]
[0135] .
[0136] 6) The scene-vehicle interaction unit of the speed perception interaction modeling module obtains global interaction features based on the spatiotemporal interaction features and scene interaction features.
[0137] The scene-vehicle interaction unit will incorporate the features of the last time step in the spatiotemporal interaction characteristics. As query Q, vehicle-scenario interaction feature encoding As key K and value V, As a mask for cross-attention, the input is a Masked Multi-Head Attention network, which then sequentially passes through residual connections and a layer normalization network, a Feed-Forward Network, and another residual connection and layer normalization network to obtain a global interactive feature encoding containing vehicle dynamics and scene information. This refers to the global interaction feature, which is represented as:
[0138] .
[0139]
[0140] .
[0141] Finally, using the actual target vehicle trajectory sequence corresponding to the training samples, the trajectory prediction model is trained based on the mean squared error (MSE) loss function to obtain the trained trajectory prediction model, which can then be used to predict vehicle trajectories.
[0142] 7) Based on the aforementioned spatiotemporal interaction features and global interaction features, the decoder module obtains the target vehicle's driving trajectory during the prediction period, specifically:
[0143] The decoder module encodes vehicle-to-vehicle interaction features. and global interactive feature encoding The data is stitched together and fed into an LSTM decoder to obtain the future trajectory of the target vehicle. , is represented as:
[0144] .
[0145] 8) Iterate the trajectory prediction model based on the mean squared error (MSE) loss function, specifically:
[0146] The formula for calculating the mean squared error loss function (MSE) is as follows:
[0147]
[0148] In the formula, Indicates the target vehicle is in The true value of the longitudinal displacement at time t. Indicates the target vehicle is in The true value of the lateral displacement at time t. Indicates the target vehicle is in Predicted longitudinal displacement at time t. Indicates the target vehicle is in The predicted value of the lateral displacement at time t; This indicates the predicted time step.
[0149] The root mean square error (RMSE) can also be used as a performance metric to evaluate the model, with the following formula:
[0150]
[0151] When using a trajectory prediction model that has been trained or whose performance meets the requirements for predicting the future trajectory of a real vehicle, it is only necessary to obtain the trajectory of the target vehicle for a historical period up to the current moment, as well as the trajectory data of surrounding vehicles within the target vehicle's perception range, as input to the trajectory prediction model. This will yield the predicted trajectory of the target vehicle within a set future timeframe. The data processing approach performed by each functional module of the model is the same as that during the model training phase.
[0152] Figure 3 The trajectory prediction effect diagram of the trajectory prediction model obtained in this embodiment is shown. The results show that after adopting the trajectory prediction model, the trajectory prediction error of the vehicle within 5 seconds is effectively controlled. That is, the trajectory prediction model implemented by the method in this embodiment can significantly improve the trajectory prediction accuracy, provide more accurate information support for the vehicle decision planning module, and thus effectively improve the driving safety, comfort and traffic efficiency of autonomous vehicles.
[0153] Example 2
[0154] Based on the same inventive concept as Embodiment 1, the implementation process of the speed-sensing-based spatiotemporal interactive vehicle trajectory prediction method in this embodiment is described in detail below.
[0155] Step 1: During the model training phase, collect and process data: Collect vehicle trajectory information for typical road scenarios and process the data.
[0156] (1) Taking the highly interactive scenario of highways as an example, the NGSIM dataset for highways is selected:
[0157] The NGSIM dataset uses US highway driving data collected by the US FHWA, which includes the driving conditions of all vehicles on roads such as US101 and I-80 over a period of time, with a data sampling interval of 0.1 seconds.
[0158] The dataset contains the following data information: Vehicle_ID (vehicle ID in the dataset), Frame_ID (frame time), Global_Time (global time, unit: ms), Local_X (relative X coordinate, unit: ft), Local_Y (relative Y coordinate, unit: ft), v_leng (vehicle length, unit: ft), v_class (vehicle type), etc.
[0159] (2) Data preprocessing of the dataset on typical roads is as follows:
[0160] The duration of historical trajectories is set to 3 seconds, and the predicted duration is 5 seconds. Therefore, vehicles with a total trajectory duration of less than 8 seconds are excluded. The road segments and lanes in the NGSIM dataset are numbered from top to bottom, starting with 1, for a total of 3 numbers. ;
[0161] Step 2: Construct and train the trajectory prediction model, as follows:
[0162] (1) Historical trajectory modeling: The historical trajectory can be represented as:
[0163] (1)
[0164] In the formula, Represents historical trajectory, including historical periods. Vehicle information, express target vehicles and The trajectory information of the surrounding vehicles. Indicates the target vehicle; Indicates the first The car at any time The trajectory information can be represented as:
[0165] (2)
[0166] In the formula, They represent the first The vehicle is the target vehicle at the current time. The position is taken as the origin of the coordinate system at time 10:00. The longitudinal position, lateral position, velocity, acceleration, and lane number of the vehicle;
[0167] Using historical trajectories as input to the model, the model outputs the future trajectory coordinates of the target vehicle, represented as:
[0168] (3)
[0169] In the formula, It is the output vector. It is the future time step of the predicted trajectory. express The location coordinates of the target vehicle at any given time.
[0170] (2) Calculate the line-of-sight range of the target vehicle, as follows:
[0171] Assume the target vehicle's current speed is km / h, basic target vehicle line of sight range m, expected headway s, m, calculate the target vehicle's line-of-sight range m, the maximum line-of-sight range of the target vehicle m is divided into a total of A grid, a visibility mask This means the mask value is 0 within a range of 82m, and the mask value is [value missing] within a range of 127m (outside 82m). .
[0172] (3) Spatiotemporal feature encoding and scene-aware encoding are performed on the target vehicle and surrounding vehicles respectively, as follows:
[0173] Two Long Short-Term Memory (LSTM) networks are used to encode the historical trajectory features of the target vehicle and surrounding vehicles, respectively. Assuming the hidden state dimension of the LSTM is 64, the historical trajectory codes of the target vehicle and surrounding vehicles are obtained. , Assume that at the current moment, there are two vehicles within 82m of the target vehicle's line of sight, and no vehicles within 127m beyond 82m. Encode the historical trajectories of the two surrounding vehicles. According to the current time The location of the given grid cell is filled into the corresponding grid cell to obtain the spatiotemporal feature. The current value is .
[0174] Calculate the current time Speed difference matrix between vehicles within a grid Acceleration difference matrix The two are concatenated and fed into a convolutional layer Conv with a kernel of 1 for feature extraction.
[0175] Subsequently, by alternately applying two multilayer perceptron (MLP) sublayers through MLPMixer, the input sequence is mixed along the feature dimension (channel mixing) and the node dimension (spatial mixing) respectively, to achieve information interaction between different grids and between different vehicles, resulting in... As the node feature input for subsequent graph attention network (GAT) graph construction, it further captures the global spatial dependencies between vehicles, ultimately obtaining the scene-aware spatial encoding. .
[0176] It should be noted here that the extracted feature shape after passing through the ConvNet network is as follows: ,in, For node dimensions, The feature dimension is defined as follows. According to the definition of the MLPMixer network, it internally implements two functions: channel mixing and spatial mixing. In this embodiment, MLPMixer is implemented along the node dimension. The operation performed is called channel blending, along the feature dimension. The operation performed is called spatial blending. Here, a node corresponds to a different cell within the grid.
[0177] (4) The speed perception interaction modeling module extracts the spatiotemporal interaction information between the vehicle and the scene, as follows:
[0178] The vehicle-to-vehicle interaction modeling unit utilizes a multi-head attention mechanism to extract interaction information between the target vehicle and surrounding vehicles at various historical moments, and encodes the historical trajectory of the target vehicle. As a query (Q), spatiotemporal features As key (K) and value (V) As a mask for cross-attention, The target vehicle's encoding, which incorporates spatial interaction features at each historical timestamp, is obtained. .
[0179] Next, it passes through a multi-head attention and feed-forward network (FFN), with each module followed by residual connections and layer normalization. As the query (Q), key (K), and value (V), the vehicle-to-vehicle interaction code is obtained. Features of the last time step Used for vehicle-scenario interaction modules.
[0180] The vehicle-scene interaction unit utilizes a multi-head attention mechanism and a feed-forward network (FFN). Each module is followed by residual connections and layer normalization to capture scene interaction information between the target vehicle and surrounding vehicles at the current moment, thus spatially encoding the environment perception. As a query (Q), As key (K) and value (V) As a mask for cross-attention, the vehicle-scene interaction feature encoding is obtained. .
[0181] The scene-vehicle interaction modeling unit will As a query (Q), vehicle-scenario interaction feature encoding As the key (K) and value (V), a global interactive feature encoding containing vehicle dynamics information and scene information is obtained. .
[0182] (5) The decoder encodes the vehicle-to-vehicle interactions. and global interactive feature encoding The data is stitched together and fed into an LSTM decoder to obtain the future trajectory of the target vehicle. .
[0183] Step 4: Using the training sample dataset, iteratively train the trajectory prediction model based on the mean squared error (MSE) loss function to obtain the trained trajectory prediction model. Assume that the number of iterations is 16 and the learning rate is set to 0.0005.
[0184] The gradient of the loss function with respect to each trainable parameter in the network is calculated based on the loss function value, and then the network parameters are updated according to the gradient descent rule. This is based on existing technology and will not be elaborated here.
[0185] Step 5: Evaluate the trajectory prediction model based on the root mean square error (RMSE), and apply the trajectory prediction model to predict vehicle trajectories in a real-world application scenario. Specifically, the RMSE is used to evaluate the model's performance; the smaller the RMSE value, the better the trajectory prediction effect of the vehicle trajectory prediction model.
[0186] Figure 3 The image shows the trajectory prediction effect of the method described in this embodiment. The results show that the trajectory prediction error of the vehicle within 5 seconds is effectively controlled after adopting this method. In summary, this method can significantly improve the trajectory prediction accuracy, provide more accurate information support for the vehicle decision planning module, and thus effectively improve the driving safety, comfort and traffic efficiency of autonomous vehicles.
[0187] Example 3
[0188] Based on the same inventive concept as Embodiments 1 to 3, this embodiment introduces a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the speed-aware spatiotemporal interactive vehicle trajectory prediction method as described in Embodiment 1 or 2.
[0189] In summary, the present invention possesses the ability to adaptively adjust attention at speed and to model spatial interactions without the need for high-precision maps. It can accurately capture the spatial dependencies between the vehicle and the scene, improve the accuracy of trajectory prediction, and adaptively adjust the interaction range according to the current speed of the target vehicle, thereby simulating the driver's perception characteristics at different speeds. This effectively improves the model's perception accuracy and computational efficiency for key interactive objects.
[0190] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0191] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0192] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0193] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0194] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. A spatiotemporal interactive vehicle trajectory prediction method based on speed perception, characterized in that, include: Acquire the driving trajectory data of the target vehicle and surrounding vehicles within its perception range; The driving trajectory data is input into a pre-trained trajectory prediction model to obtain the target vehicle trajectory data for the prediction period; wherein, the trajectory prediction model includes an encoder module, a speed perception interaction modeling module, and a decoder module; The encoder module includes a spatiotemporal feature encoder and a scene perception encoder. The spatiotemporal feature encoder is used to encode the driving trajectories of the target vehicle and other vehicles respectively, and fill the encoded result data into the perception range grid according to the relative position of the surrounding vehicles and the target vehicle to obtain the spatiotemporal feature encoding. The scene perception encoder is used to extract the global spatial dependency relationship between vehicles based on the driving trajectory data to obtain the scene perception spatial encoding. The speed-sensing interaction modeling module is used to: extract the spatiotemporal interaction features and scene interaction features of the target vehicle and surrounding vehicles at each historical moment based on the spatiotemporal feature encoding and scene-sensing spatial encoding, and obtain global interaction features based on the spatiotemporal interaction features and scene interaction features. The decoder module is used to obtain the driving trajectory of the target vehicle during the prediction period based on the spatiotemporal interaction features and global interaction features. The method for constructing the perception range grid is as follows: obtaining the current speed of the target vehicle. The number of lanes on the road where the target vehicle is located And preset perception range parameters, the perception range parameters including the basic target vehicle line-of-sight range. Expected headway And the vertical length of a single cell in the grid ; Calculate the perception range of the target vehicle based on the perception range parameters: The perception range of the target vehicle is divided into... Grid size ; The driving trajectory data includes the vehicle's position coordinates, speed, angular velocity, and lane number at each moment within a set time period, wherein the position coordinates are relative position coordinates determined with the target vehicle's position at the same moment as the coordinate origin; The driving trajectory data is represented as follows: ,in Indicates the current moment. express target vehicles and The trajectory information of the vehicles surrounding the vehicle. Corresponding target vehicle, Indicates the first The surrounding vehicles at all times The trajectory information is represented as: , They represent the first The surrounding vehicles at all times The relative longitudinal position, relative lateral position, speed, acceleration, and lane number.
2. The method according to claim 1, characterized in that, The spatiotemporal feature encoder encodes the driving trajectories of the target vehicle and other vehicles separately, and fills the encoded data into the sensing range grid to obtain spatiotemporal features, including: Two Long Short-Term Memory (LSTM) networks are used to encode the trajectory features of the target vehicle and surrounding vehicles, respectively, to obtain the trajectory codes of the target vehicle and surrounding vehicles. , , respectively represented as: ， ， in, This represents the hidden state dimension of the LSTM; Encode the trajectories of surrounding vehicles According to the current time The corresponding grid position of the vehicle within the perception range is filled into the corresponding grid to obtain the spatiotemporal feature code. The spatiotemporal characteristic value corresponding to the current moment is .
3. The method according to claim 2, characterized in that, The scene-aware encoder extracts the global spatial dependencies between vehicles based on the driving trajectory data to obtain scene-aware spatial coding, including: Based on the speed and acceleration data in the driving trajectory data, calculate the speed difference matrix between vehicles within the sensing range grid. And acceleration difference matrix : ， ， velocity difference matrix And acceleration difference matrix By splicing : ， Will The data is fed into a convolutional layer for feature extraction, resulting in... The convolutional layer is selected as a set of convolutional layers with a kernel of 1; Two multilayer perceptron (MLP) sublayers are alternately applied to the MLPMixer network to process the extracted data along the feature dimension and the grid node dimension, respectively. The feature sequences are mixed to obtain : ; Will As node feature inputs to the Graph Attention Network (GATv2), it captures global spatial dependencies between vehicles, leading to scene-aware spatial encoding. : .
4. The method according to claim 3, characterized in that, The speed perception interaction modeling module includes a vehicle-to-vehicle interaction unit, a vehicle-to-scene interaction unit, and a scene-to-vehicle interaction unit. The vehicle-to-vehicle interaction unit is used to extract the spatiotemporal interaction features between the target vehicle and surrounding vehicles at historical moments based on the spatiotemporal feature encoding using a multi-head attention mechanism. The vehicle-scene interaction unit is used to extract the scene interaction features between the target vehicle and surrounding vehicles at historical moments based on the scene perception space and the spatiotemporal interaction features using a multi-head attention mechanism. The scene-vehicle interaction unit is used to obtain global interaction features based on the spatiotemporal interaction features and scene interaction features.
5. The method according to claim 4, characterized in that, The method for the vehicle-to-vehicle interaction unit to obtain spatiotemporal interaction features includes: Encode the target vehicle's trajectory As a query Q, the spatiotemporal feature encoding As key K and value V, As a mask for cross-attention, a spatial attention network based on a multi-head attention mechanism and a gated linear unit (GLU) are used to extract the interaction information between the target vehicle and surrounding vehicles at various historical moments. The formula is expressed as: ， ， ， in, Let be the dimension of the key vector. This represents the maximum perception range of the target vehicle. For each node position in the perception range grid, if it is within the perception range... If it is inside, the mask value is 0; if it is located inside... Beyond but still in Inside, the mask value is ; Will and target vehicle trajectory coding By inputting residual connections and layer normalization networks, we obtain the encoding of the target vehicle at each historical timestamp, which incorporates spatial interaction features. : ， ， Will As the query Q, key K, and value V, the input is a temporal attention network based on a multi-head attention mechanism. Then, it passes through a residual connection and layer normalization network, a feed-forward network, and another residual connection and layer normalization network to obtain the vehicle-to-vehicle interaction code. This refers to the spatiotemporal interaction feature; the process is represented as: ， ， 。 6. The method according to claim 5, characterized in that, The method by which the vehicle-scene interaction unit obtains the scene interaction features includes: Encode the scene perception space As a query Q, the spatiotemporal feature encoding As key K and value V, As a mask for cross-attention, the input mask is passed to a Masked Multi-Head Attention network, and then sequentially through residual connections and a layer normalization network, a Feed-Forward Network, and another residual connection and layer normalization network to obtain the vehicle-scene interaction feature encoding. This refers to the scene interaction features; the process is represented as: ， ， ； The scene-vehicle interaction unit is used to obtain global interaction features based on the spatiotemporal interaction features and scene interaction features, including: The features of the last time step in the spatiotemporal interaction features As query Q, vehicle-scenario interaction feature encoding As key K and value V, As a mask for cross-attention, the input is a MaskedMulti-Head Attention network, which then sequentially passes through residual connections and a layer normalization network, a Feed-Forward Network, and another residual connection and layer normalization network to obtain a global interactive feature encoding containing vehicle dynamics and scene information. This refers to the global interaction feature, which is represented as: ， ， 。 7. The method according to claim 6, characterized in that, The decoder module obtains the target vehicle's driving trajectory during the prediction period based on the spatiotemporal interaction features and global interaction features, including: Encoding vehicle-to-vehicle interaction features and global interactive feature encoding The data is stitched together and fed into an LSTM decoder to obtain the future trajectory of the target vehicle. , represented as: 。 8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.