Automatic driving vehicle multi-modal trajectory prediction method based on transformer and end-point induction

Abstract

The application provides an automatic driving vehicle multi-modal trajectory prediction method based on a Transformer and end point induction, which firstly extracts surrounding vehicle historical trajectory features, target vehicle historical trajectory features, relative position information and scene map features; then utilizes the Transformer to locally and globally fuse the extracted features, efficiently and comprehensively fusing the features of various elements in the scene; then predicts multi-modal trajectory endpoints through a dynamic weight multi-layer perception (MLP), simplifies the model architecture and reduces the model parameter quantity; finally uses an attention mechanism to interact endpoint information for trajectory refinement, outputs multi-modal trajectory prediction, so that the predicted vehicle future trajectory is reasonable and accurate. The method can not only improve the multi-modal trajectory prediction accuracy of the automatic driving vehicle, but also improve the model prediction efficiency.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a multimodal trajectory prediction method for autonomous vehicles based on Transformer and endpoint guidance. Background Technology

[0002] Autonomous driving has broad application prospects in solving problems such as traffic accidents, traffic congestion, and emissions. In recent years, the perception, decision-making, and control modules of autonomous driving have developed rapidly; however, the full deployment of autonomous driving requires verification of its safety first. In complex traffic scenarios, even if the driver's final intention is certain, the real-time changes in the dynamic driving paths of other vehicles during the decision-making process will cause the vehicle's path to exhibit multimodal attributes, meaning that the vehicle's future trajectory has multiple possibilities. Trajectory prediction plays a crucial role in autonomous driving, serving not only as an extension of environmental perception and a basis for decision-making and planning, but also as the foundation for risk assessment and a bridge for human-machine interaction. Through accurate trajectory prediction, autonomous driving systems can better adapt to complex and ever-changing traffic environments, improving driving safety and efficiency.

[0003] Existing vehicle trajectory prediction methods can be mainly divided into three categories: prediction methods based on physical models, prediction methods based on machine learning, and prediction methods based on deep learning. The core of physical model-based vehicle trajectory prediction methods lies in simplifying the prediction process of the target vehicle into a vehicle dynamics or kinematic model. This method accurately predicts the future state of the vehicle through iterative calculations based on the model's input parameters, such as acceleration and steering angle, and external conditions, such as the road friction coefficient. While this method has low computational complexity and good real-time performance, it ignores the influence of prior knowledge such as road structure and environmental information, as well as the subjective intentions and driving styles of other drivers in the scene, making it only suitable for trajectory prediction within a short time domain of 1 second. Machine learning-based methods are mostly based on driving behavior prediction. They first determine driving behavior and then combine this behavior to predict the future trajectory. However, in the prediction process, they assume that vehicles are independent of each other and do not fully consider the influence of the driving behavior of other vehicles in the driving scene. Ignoring this interaction may generate unreasonable trajectories. Deep learning-based trajectory prediction methods can not only extract historical trajectory features using temporal networks, but also process trajectory features through different network structures to extract interaction information of traffic participants and road information. Compared with physics-based methods and classic machine learning-based methods, predictions over long time domains are more accurate. The application of deep learning in multimodal trajectory prediction tasks mainly falls into three mainstream directions: methods based on sequence networks, methods based on graph neural networks, and methods based on generative models. However, compared to classic sequence networks such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), the Transformer is a non-recurrent network structure based entirely on a self-attention mechanism. It breaks the limitation of sequential input through encoder-decoder and attention mechanisms, thus solving the problem of non-parallel training inherent in Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models. Simultaneously, the Transformer model has better memory capacity, can solve long-distance dependency problems, and has better interpretability when handling interaction problems. Therefore, there is an urgent need for a Transformer method that can quickly adapt to autonomous driving scenarios and ensure the completion of multimodal trajectory prediction tasks under conditions of strong safety, comfort, timeliness, and generalization. Therefore, it is of great significance to study multimodal trajectory prediction methods for autonomous vehicles based on Transformer and endpoint guidance. Summary of the Invention

[0004] To address the current issues related to multimodal trajectory prediction in the field of autonomous driving, this invention provides a method for multimodal trajectory prediction of autonomous vehicles based on Transformer and endpoint guidance. This method utilizes Transformer to perform local and global fusion of extracted features, and then uses a dynamic weighted multilayer perceptron (MLP) to predict the endpoint of the multimodal trajectory. At the same time, an attention mechanism is used to interact with endpoint information to refine the trajectory. This method improves the accuracy and efficiency of multimodal trajectory prediction for autonomous vehicles.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] A multimodal trajectory prediction method for autonomous vehicles based on Transformer and endpoint guidance, the method comprising the following steps:

[0007] Step 1: Use an encoder to encode the road scene information, surrounding vehicle information including the current state and historical trajectory of the vehicles, the current state, historical trajectory and relative position information of the target vehicle, and generate a high-dimensional feature tensor.

[0008] Step 2: Perform hierarchical feature fusion on the encoded high-dimensional feature tensor. First, use the attention mechanism to perform local feature fusion on vehicle features and map features respectively. Then, use Transformer to perform global feature fusion on vehicle features, map features and relative position information.

[0009] Step 3: Input the fused feature tensor into the dynamic weight MLP to output the multimodal trajectory endpoint prediction in different driving scenarios.

[0010] Step 4: Use the attention mechanism to interactively output the trajectory endpoint offset, realize feature interaction over a longer spatiotemporal range to obtain more accurate endpoint information, and finally perform trajectory completion to output a complete multimodal predicted trajectory.

[0011] Step 5: Design a suitable loss function based on the multimodal trajectory prediction model. The loss function includes three components: regression loss of all trajectories, classification loss, and endpoint regression loss.

[0012] The feature extraction module in step 1 includes a vehicle trajectory encoder, a map information encoder, and a relative position information encoder. The specific steps are as follows:

[0013] S1: The trajectory information is a time sequence containing the vehicle's position. The vehicle trajectory encoder input is represented by displacement. In map coding, the position and direction of each lane segment are represented by the position and direction of the midpoint of the lane centerline.

[0014] S2: Represent the historical trajectory information of the i-th vehicle at time step H as a displacement sequence ΔX i ={ΔX i,-H+1 ,...,ΔX i,0}, where ΔX i,t Let V represent the displacement of the i-th vehicle from time t-1 to time t. To improve the effectiveness of multi-scale feature extraction and the efficiency of parallel computation, a one-dimensional residual convolution and feature pyramid network (FPN) are used to process the historical trajectory input. The encoder outputs the vehicle feature tensor V. a Size is [N a

[128] , where N a This indicates the number of vehicles within the historical step.

[0015] S3: Considering that lane segments are static during the observation period, a simple two-layer MLP is used to encode map information to improve operational efficiency. The map feature tensor V output by the encoder is... m Size is [N l

[128] , where N l This indicates the number of lane center lines on the map during the observation period.

[0016] S4: In relative position encoding, the heading angle difference α is used. i→j Relative azimuth angle β i→j and distance ||d i→j These three quantities describe the relative position and pose between element i and element j, and the relative position information r between scene elements i and j. i→j It can be represented by a five-dimensional vector:

[0017]

[0018] For a scene with N elements within the observation time, the relative position information is encoded using MLP to obtain a relative position tensor rpe of size [N,N,128].

[0019] In step 2, the multi-feature fusion method based on hierarchical feature fusion includes: local feature fusion of vehicles and lanes based on a single-head self-attention mechanism, and global feature fusion of map features, vehicle trajectory features, and relative position information based on Symmetrical Transformer (SFT). Because complex traffic scenarios involve numerous features requiring comprehensive and efficient fusion, this invention designs a hierarchical feature fusion module based on multiple considerations.

[0020] 1) Local feature fusion

[0021] In complex traffic scenarios, the behaviors of traffic participants are often complex. To allow the model to focus on more important features and improve the efficiency and accuracy of trajectory prediction, local feature fusion is performed first. In local feature fusion, a single-head attention mechanism is used to realize trajectory feature interaction and map feature interaction separately, which can both realize interactive functions and improve the efficiency of interaction.

[0022] The trajectory feature tensor and map feature tensor output from the feature extraction module are input separately. A single-head self-attention mechanism is then used to assign different weights to the scene element features, where F represents the input feature tensor and F' represents the feature tensor after the attention mechanism. The specific formula is as follows:

[0023] Q = W q F,K=W k F,V=W V F

[0024]

[0025] Among them W q W k W V This is the weight matrix. Finally, after passing through a layer normalization network and a feedforward network, the interactive feature tensor is obtained, where Q, K, and V are the input vectors of the attention mechanism.

[0026] 2) Global Feature Fusion

[0027] Local feature fusion does not consider the relative positional relationships of elements in the scene and the interaction between vehicles and lane segments. Transformer models are widely used in computer vision and feature fusion, as they can easily handle and fuse features at different scales, reducing the number of model parameters. Therefore, a Symmetric Fusion Transformer (SFT) is used for global feature fusion. This feature fusion method performs directional information transmission symmetrically, enabling the network to predict the future movements of all road users in a single feedforward pass. This not only facilitates effective interaction but also improves model efficiency. The specific steps are as follows:

[0028] S1: Tensor V is obtained by concatenating the vehicle trajectory feature tensor and the map feature tensor after local fusion. n The tensor size is [N, 128], where N is the sum of vehicles and lane segments in the scene.

[0029] S2: For V n Repeat along two different dimensions and stack them to obtain the target tensor and the source tensor, both of which are [N,N,128].

[0030] S3: The target tensor, source tensor, and relative position feature tensor are concatenated and then passed through an MLP to obtain the inputs K and V for multi-head attention. The Q of the multi-head attention is the tensor V.n .

[0031] S4: Output the updated vehicle trajectory feature tensor V after passing through the multi-head attention mechanism. a 'and relative position feature tensor rpe', where the number of attention heads is 8. Tensor V a Size is [N] a The tensor rpe' has a size of [N,N,128], where N... a This represents the number of vehicles in the scene. The global feature fusion module effectively integrates scene traffic participant information and road structure information.

[0032] The trajectory endpoint prediction method based on dynamic weighted MLP in step 3 includes:

[0033] Traditional trajectory endpoint prediction methods rely on probability distributions to describe the uncertainty of vehicle trajectories. However, these methods exhibit poor generalization performance and are prone to overfitting when dealing with data from different regions and data collection methods. This invention utilizes two dynamically weighted MLPs to generate different vehicle features, thereby adapting to various traffic scenarios. The specific steps are as follows:

[0034] S1: Vehicle trajectory feature tensor V output by the multi-feature fusion module a 'Use an MLP layer to generate tensor V w ' a .

[0035] S2: Introduce trainable weight parameters W from the two MLP layers. w1 and W w2 , making V w ' a Two dynamic weight parameters, W1 and W2, are generated after passing through these two MLP layers, as shown in the formula:

[0036]

[0037] S3: The trajectory endpoint prediction process is transformed into first passing through an MLP layer with weight W1, then through a normalization layer and an activation layer, and finally through an MLP layer with weight W2 to generate the multimodal endpoint y. pred .

[0038] The endpoint refinement method based on trajectory endpoint information interaction in step 4 includes:

[0039] Traditional trajectory endpoint prediction methods lack effective endpoint refinement, focusing only on interaction with historical observation information while neglecting the interaction with endpoint information. This invention utilizes an attention mechanism to efficiently interact with feature tensors containing trajectory endpoint information, extracting effective features over a longer spatiotemporal range to achieve accurate endpoint refinement, thereby enhancing the rationality and accuracy of predicted trajectories. The specific steps are as follows:

[0040] S1: Encode the trajectory endpoint output by the endpoint prediction module to obtain the trajectory endpoint feature tensor E. pred :

[0041] E pred =MLP(y pred )

[0042] S2: The spliced ​​vehicle trajectory feature tensor V a 'and trajectory endpoint feature tensor E pred In the input attention mechanism, the output trajectory endpoint offset b pred :

[0043] b pred =MHA(concat(V a ',E pred ))

[0044] S3: Use a simple MLP to complete the remaining trajectory points on the vehicle trajectory feature tensor and predict the probability of each mode.

[0045] The loss function in step 5 includes three components: regression loss for the entire trajectory, classification loss, and endpoint regression loss. Regression loss for the entire trajectory optimizes the overall shape and continuity of the trajectory, making the predicted trajectory closer to the actual trajectory; classification loss improves the model's accuracy in recognizing the vehicle's motion intentions, thus providing a more reliable basis for trajectory prediction; endpoint regression loss optimizes the endpoint position of the trajectory, making the predicted endpoint closer to the actual endpoint, and improving the accuracy of trajectory prediction.

[0046] Specifically:

[0047] 1) Regression loss of all trajectories

[0048] The regression loss for the entire trajectory includes the vehicle position coordinate regression loss. and heading angle loss Introducing heading angle loss improves the smoothness and feasibility of the predicted trajectory, as shown in the equation:

[0049]

[0050] In the formula, T is the prediction time step; Let be the position coordinates of the trajectory of the optimal mode at time t; represents the position coordinates of the true trajectory at time t; CosSim is the cosine similarity measure, which takes a value of 1 when the two vectors are in the same direction and -1 when they are in opposite directions. The heading angle of the vehicle in the optimal mode; This represents the vehicle's actual heading angle.

[0051] 2) Classification loss

[0052] The classification loss function measures the difference between the class predicted by the model and the true class. Cross-entropy loss is used to calculate the classification loss, and the model's performance is evaluated by calculating the difference between the predicted probability distribution and the true label.

[0053] 3) Terminal regression loss

[0054] Terminal regression loss is a loss function used to measure the difference between the model's predicted terminal location and the actual terminal location. Its core purpose is to optimize the model's predictive performance by minimizing this difference, as shown in the equation:

[0055]

[0056] In the formula The coordinates of the endpoint of the trajectory in the optimal mode; These are the coordinates of the actual endpoint of the trajectory.

[0057] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: when the processor executes the program, it implements the aforementioned method for predicting multimodal trajectories of autonomous vehicles based on Transformer and endpoint guidance.

[0058] A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the aforementioned method for multimodal trajectory prediction of autonomous vehicles based on Transformer and endpoint guidance.

[0059] Based on the above technical solution, the present invention has the following beneficial technical effects:

[0060] 1) The road scene information, surrounding vehicle information including the current state and historical trajectory of the vehicles, the current state and historical trajectory of the target vehicle and the relative position information are encoded by the encoder to generate a high-dimensional feature tensor.

[0061] 2) Hierarchical feature fusion is performed on the encoded high-dimensional feature tensor. First, attention mechanism is used to perform local feature fusion on vehicle features and map features respectively. Then, Transformer is used to perform global feature fusion on vehicle features, map features and relative position information, which enhances the interaction of scene element features.

[0062] 3) Utilizing dynamic weighted MLP to output multimodal trajectory endpoint predictions in different driving scenarios not only simplifies the model structure but also improves the accuracy of endpoint prediction.

[0063] 4) By using the attention mechanism to interactively output the trajectory endpoint offset, feature interaction within a longer spatiotemporal range is achieved to obtain more accurate endpoint information. Finally, trajectory completion is performed to output a complete multimodal predicted trajectory, which improves the rationality and comfort of the output trajectory.

[0064] 5) Design a suitable loss function based on the multimodal trajectory prediction model. The loss function includes three terms: the weighted sum of the regression loss of all trajectories, the classification loss, and the endpoint regression loss, so that the predicted multimodal trajectory is closer to the real trajectory. Attached Figure Description

[0065] Figure 1 This is a structural diagram of the multimodal trajectory prediction method for autonomous vehicles based on Transformer and endpoint guidance according to the present invention.

[0066] Figure 2 This is a diagram of the local feature fusion structure based on the attention mechanism of the present invention;

[0067] Figure 3 This is a diagram of the global feature fusion structure based on Transformer in this invention;

[0068] Figure 4 This is a diagram of the endpoint prediction structure based on dynamic weighted MLP in this invention.

[0069] Figure 5 The results of the multimodal trajectory prediction of the present invention are shown in the embodiments. Detailed Implementation

[0070] The following will take an unlit intersection in an urban traffic scenario as an example, and, in conjunction with the accompanying drawings of the embodiments of the present invention, clearly and completely describe the technical solutions of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Example 1: As Figure 1 As shown, a multimodal trajectory prediction method for autonomous vehicles based on Transformer and endpoint guidance includes the following steps:

[0072] Step 1: Use an encoder to encode the road scene information, surrounding vehicle information including the current state and historical trajectory of the vehicles, the current state, historical trajectory and relative position information of the target vehicle, and generate a high-dimensional feature tensor.

[0073] Step 2: Perform hierarchical feature fusion on the encoded high-dimensional feature tensor. First, use the attention mechanism to perform local feature fusion on vehicle features and map features respectively. Then, use Transformer to perform global feature fusion on vehicle features, map features and relative position information.

[0074] Step 3: Input the fused feature tensor into the dynamic weight MLP to output the multimodal trajectory endpoint prediction in different driving scenarios.

[0075] Step 4: Use the attention mechanism to interactively output the trajectory endpoint offset, realize feature interaction over a longer spatiotemporal range to obtain more accurate endpoint information, and finally perform trajectory completion to output a complete multimodal predicted trajectory.

[0076] Step 5: Design a suitable loss function based on the multimodal trajectory prediction model. The loss function includes three components: regression loss of all trajectories, classification loss, and endpoint regression loss.

[0077] In step 2, the local features are fused into an attention-based structure, such as... Figure 2 As shown, specifically:

[0078] In local feature fusion, a single-head attention mechanism is used to realize trajectory feature interaction and map feature interaction respectively, which can both realize interactive functions and improve the efficiency of interaction.

[0079] The trajectory feature tensor and map feature tensor output from the feature extraction module are input separately. A single-head self-attention mechanism is then used to assign different weights to the scene element features, where F represents the input feature tensor and F' represents the feature tensor after the attention mechanism. The specific formula is as follows:

[0080] Q = W q F,K=W k F,V=W V F

[0081]

[0082] Among them W q W k W V This is the weight matrix. Finally, after passing through a layer normalization network and a feedforward network, the interactive feature tensor is obtained, where Q, K, and V are the input vectors of the attention mechanism.

[0083] In step 2, the global feature fusion is performed into a structure based on Symmetric Fusion Transformer (SFT), as follows: Figure 3 As shown, the specific steps are as follows:

[0084] S1: Tensor V is obtained by concatenating the vehicle trajectory feature tensor and the map feature tensor after local fusion. n The tensor size is [N, 128], where N is the sum of vehicles and lane segments in the scene.

[0085] S2: For V n Repeat along two different dimensions and stack them to obtain the target tensor and the source tensor, both of which are [N,N,128].

[0086] S3: The target tensor, source tensor, and relative position feature tensor are concatenated and then passed through an MLP to obtain the inputs K and V for multi-head attention. The Q of the multi-head attention is the tensor V. n .

[0087] S4: Output the updated vehicle trajectory feature tensor V after passing through the multi-head attention mechanism. a 'and relative position feature tensor rpe', where the number of attention heads is 8. Tensor V a Size is [N] a The tensor rpe' has a size of [N,N,128], where N... a This represents the number of vehicles in the scene. The global feature fusion module effectively integrates scene traffic participant information and road structure information.

[0088] The structure of the endpoint prediction based on dynamic weighted MLP in step 3 is as follows: Figure 4 As shown, the specific steps are as follows:

[0089] S1: Vehicle trajectory feature tensor V output by the multi-feature fusion module a 'Use an MLP layer to generate tensor V w ' a .

[0090] S2: Introduce trainable weight parameters W from the two MLP layers. w1 and W w2 , making V w ' a Two dynamic weight parameters, W1 and W2, are generated after passing through these two MLP layers, as shown in the formula:

[0091]

[0092] S3: The trajectory endpoint prediction process is transformed into first passing through an MLP layer with weight W1, then through a normalization layer and an activation layer, and finally through an MLP layer with weight W2 to generate the multimodal endpoint y.pred .

[0093] In step 4, the endpoint information is refined through interaction based on an attention mechanism. The specific steps are as follows:

[0094] S1: Encode the trajectory endpoint output by the endpoint prediction module to obtain the trajectory endpoint feature tensor E. pred :

[0095] E pred =MLP(y pred )

[0096] S2: The spliced ​​vehicle trajectory feature tensor V a 'and trajectory endpoint feature tensor E pred In the input attention mechanism, the output trajectory endpoint offset b pred :

[0097] b pred =MHA(concat(V a ',E pred ))

[0098] S3: Use a simple MLP to complete the remaining trajectory points on the vehicle trajectory feature tensor and predict the probability of each mode.

[0099] The loss function designed in step 5 includes three parts: regression loss for all trajectories, classification loss, and endpoint regression loss, specifically:

[0100] 1) Regression loss of all trajectories

[0101] The regression loss for the entire trajectory includes the vehicle position coordinate regression loss. and heading angle loss Introducing heading angle loss improves the smoothness and feasibility of the predicted trajectory, as shown in the equation:

[0102]

[0103] In the formula, T is the prediction time step; Let be the position coordinates of the trajectory of the optimal mode at time t; represents the position coordinates of the true trajectory at time t; CosSim is the cosine similarity measure, which takes a value of 1 when the two vectors are in the same direction and -1 when they are in opposite directions. The heading angle of the vehicle in the optimal mode; This represents the vehicle's actual heading angle.

[0104] 2) Classification loss

[0105] The classification loss function measures the difference between the class predicted by the model and the true class. Cross-entropy loss is used to calculate the classification loss, and the model's performance is evaluated by calculating the difference between the predicted probability distribution and the true label.

[0106] 3) Terminal regression loss

[0107] Terminal regression loss is a loss function used to measure the difference between the model's predicted terminal location and the actual terminal location. Its core purpose is to optimize the model's predictive performance by minimizing this difference, as shown in the equation:

[0108]

[0109] In the formula The coordinates of the endpoint of the trajectory in the optimal mode; These are the coordinates of the actual endpoint of the trajectory.

[0110] like Figure 5 The figure illustrates the process of multimodal trajectory prediction for autonomous vehicles based on Transformer and endpoint guidance in an urban scenario, as described in this invention. In this invention, red represents the vehicle itself, blue represents surrounding vehicles, the actual trajectory endpoint is represented by a star, the historical trajectory by a solid line, the predicted multimodal trajectory by a dashed line, and the vehicle's final attitude by an arrow. As can be seen from the figure, the algorithm in this invention can simultaneously predict the future trajectories of all vehicles, and in complex traffic environments with dense traffic, the model accurately predicts and avoids collisions.

[0111] In summary, this invention proposes a multimodal trajectory prediction method for autonomous vehicles based on Transformer and endpoint guidance. During the autonomous driving trajectory prediction process, it can focus on important vehicles in the scene and ignore vehicles that are irrelevant to itself, thereby improving the accuracy of trajectory prediction and reducing model complexity.

[0112] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a multimodal trajectory prediction model for autonomous vehicles based on Transformer and destination guidance, characterized in that, The method includes the following steps: Step 1: Encode the road scene information, surrounding vehicle information (including the current state and historical trajectory of vehicles), and the target vehicle's current state, historical trajectory, and relative position information using an encoder to generate a high-dimensional feature tensor. Step 2: Perform hierarchical feature fusion on the encoded high-dimensional feature tensor. First, use an attention mechanism to perform local feature fusion on vehicle features and map features respectively. Then, use a Transformer to perform global feature fusion on vehicle features, map features, and relative position information. Step 3: Input the fused feature tensor into the dynamic weighted MLP to output multimodal trajectory endpoint predictions for different driving scenarios. Step 4: Utilize an attention mechanism to interactively output the trajectory endpoint offset, achieving feature interaction over a longer spatiotemporal range to obtain more accurate endpoint information. Finally, perform trajectory completion to output a complete multimodal predicted trajectory. Step 5: Design a suitable loss function based on the multimodal trajectory prediction model. The loss function includes three terms: the regression loss of all trajectories, the classification loss, and the weighted sum of the endpoint regression loss. In step 1, the feature extraction module includes a vehicle trajectory encoder, a map information encoder, and a relative position information encoder. The specific steps are as follows: S1: Trajectory information is a time-series sequence containing vehicle positions; the vehicle trajectory encoder input is represented by displacement. In map coding, the position and direction of each lane segment are represented by the position and direction of the midpoint of the lane centerline. S2: The first One vehicle Historical trajectory information over a time step is represented as a displacement sequence. ,in Indicates the first One vehicle from Time's up To improve the effectiveness of multi-scale feature extraction and the efficiency of parallel computation, a one-dimensional residual convolution and feature pyramid network (FPN) are used to process the historical trajectory input, and the encoder outputs the vehicle feature tensor. Size is ,in This indicates the number of vehicles within the historical step size. S3: Considering that lane segments are static during the observation period, a simple two-layer MLP is used to encode map information to improve operational efficiency. The encoder outputs a map feature tensor. Size is ,in This indicates the number of lane center lines on the map during the observation period. S4: In relative position encoding, the heading angle difference is used. relative azimuth and distance These three quantities describe the element and elements The relative positions and postures of the elements in the scene and elements Relative position information between It can be represented by a five-dimensional vector: For those with during the observation time In a scenario with elements, MLP is used to encode the relative position information to obtain a value of [size missing]. relative position tensor ; In step 2, the multi-feature fusion method based on hierarchical feature fusion includes: 1) Local feature fusion, In complex traffic scenarios, the behaviors of traffic participants are often complex. In local feature fusion, a single-head attention mechanism is used to realize trajectory feature interaction and map feature interaction separately, which can not only realize the interaction function but also improve the interaction efficiency. The trajectory feature tensor and map feature tensor output from the feature extraction module are input separately, and different weights are assigned to the scene element features through a single-head self-attention mechanism. Represents the input feature tensor. The feature tensor after the attention mechanism is represented by the following formula: in It is a weight matrix, which is then processed through a layer normalization network and a feedforward network to obtain the interactive feature tensor. This is the input vector for the attention mechanism. 2) Global feature fusion, Local feature fusion does not consider the relative positional relationships of elements in the scene and the interaction between vehicles and lane segments. The Transformer model is widely used in computer vision and feature fusion. Symmetrical Fusion Transformer (SFT) is used for global feature fusion. This feature fusion method performs directional information transmission in a symmetrical manner, enabling the network to predict the future motion of all road users in a single feedforward pass. This not only enables effective interaction but also improves model efficiency. The specific steps are as follows: S1: The tensor is obtained by concatenating the vehicle trajectory feature tensor and the map feature tensor after local fusion. The size of the tensor is ,in It is the sum of vehicles and lane segments in the scene. S2: Yes Repeating and stacking along two different dimensions yields a target tensor and a source tensor, both of which are of the same size. , S3: The target tensor, source tensor, and relative position feature tensor are concatenated and then passed through an MLP to obtain the input for multi-head attention. Multi-headed attention For tensor , S4: Output the updated vehicle trajectory feature tensor after passing through the multi-head attention mechanism. and relative position feature tensor The number of attention heads is 8, and the tensor is... Size is tensor Size is ,in To determine the number of vehicles in the scene, the global feature fusion module effectively integrates information on traffic participants and road structure. Step 3, the trajectory endpoint prediction method based on dynamic weighted MLP, includes: S1: Vehicle trajectory feature tensor output by the multi-feature fusion module Tensors are generated using an MLP layer. , S2: Introduce trainable weight parameters from two MLP layers. and ,make Two dynamic weight parameters are generated by passing the two MLP layers respectively. and As shown in the formula: S3: Transform the trajectory endpoint prediction process into first passing through a weighted... The MLP layer then passes through a normalization layer and an activation layer, and finally through a weighted array. MLP layer generates multimodal endpoints ; Step 4, the endpoint refinement method based on trajectory endpoint information interaction, includes: S1: Encode the trajectory endpoint output by the endpoint prediction module to obtain the trajectory endpoint feature tensor. : S2: The stitched vehicle trajectory feature tensor and trajectory endpoint feature tensor In the input attention mechanism, the output trajectory endpoint offset : S3: Use a simple MLP to complete the remaining trajectory points on the vehicle trajectory feature tensor and predict the probability of each mode.

2. The method for constructing a multimodal trajectory prediction model for autonomous vehicles based on Transformer and endpoint guidance according to claim 1, characterized in that, In step 5, the loss function includes three terms: regression loss of all trajectories, classification loss, and endpoint regression loss, specifically: 1) Regression loss for all trajectories, The regression loss for the entire trajectory includes the vehicle position coordinate regression loss. and heading angle loss Introducing heading angle loss makes the predicted trajectory smoother and more feasible, as shown in the following equation: In the formula To predict the time step; The trajectory for the optimal mode is in Position coordinates at that moment; For the true trajectory in Position coordinates at that moment; Cosine similarity is a metric where 1 is the value for two vectors moving in the same direction and 0 is the value for two vectors moving in opposite directions. ; The heading angle of the vehicle in the optimal mode; This represents the vehicle's actual heading angle. 2) Classification loss, The classification loss function measures the difference between the class predicted by the model and the true class. Cross-entropy loss is used to calculate the classification loss, and the model's performance is evaluated by calculating the difference between the predicted probability distribution and the true label. 3) Endpoint regression loss, Terminal regression loss is a loss function used to measure the difference between the predicted terminal position and the actual terminal position. Its core purpose is to optimize the model's predictive performance by minimizing this difference, as shown in the equation: In the formula The coordinates of the endpoint of the trajectory in the optimal mode; These are the coordinates of the actual endpoint of the trajectory.

3. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the method for constructing a multimodal trajectory prediction model for autonomous vehicles based on Transformer and endpoint guidance as described in any one of claims 1 to 2.

4. A computer-readable storage medium storing computer instructions thereon, characterized in that: When executed by a processor, the computer instructions implement the method for constructing a multimodal trajectory prediction model for autonomous vehicles based on Transformer and endpoint guidance as described in any one of claims 1-2.

Images