A vehicle trajectory prediction method and model based on an improved Transformer model and target point guidance, and an electronic device
Patent Information
- Application Number
- CN202410801976.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-20
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2044-06-20
AI Technical Summary
Existing methods for predicting the trajectory of autonomous vehicles are computationally expensive and have long latency when dealing with complex scenarios. Furthermore, they are not accurate enough in capturing multimodal distributions and cannot effectively capture fine-grained information, resulting in high prediction uncertainty.
We construct a scene context attention encoder and a multimodal motion decoder based on an improved Transformer model. We process scene elements through a local spatiotemporal coordinate system and adopt a target point-guided method to achieve efficient multimodal trajectory prediction.
It improves the accuracy and real-time performance of trajectory prediction, reduces computational complexity, enables efficient multi-agent prediction in dynamic and complex scenarios, and generates highly reliable and interpretable multimodal trajectory outputs.
Smart Images
Figure CN118823731B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of autonomous driving, specifically a method, model, and electronic device for predicting the trajectory of vehicles around an autonomous vehicle based on an improved Transformer model and target point guidance. Background Technology
[0002] In the field of autonomous driving, accurately predicting the future trajectories of surrounding vehicles is crucial, but this task is fraught with challenges. Vehicle motion is influenced by a variety of factors, including road geometry, traffic rules, and the behavior of other vehicles. Furthermore, the intentions and goals of surrounding vehicles are often unknown, leading to a high degree of uncertainty in future state predictions.
[0003] Motion prediction tasks have traditionally relied on road map information based on physical kinematic constraints and hand-crafted rules. However, these methods fail to capture the rich behavioral policies and interactions within complex scenes. With advancements in deep learning (DL), we are able to use data-driven approaches to understand and capture the complexity of driving scenarios, achieving promising state-of-the-art results by learning these intrinsic rules. Early research rasterized scenes into bird's-eye views and applied CNNs for prediction. While these methods are relatively easy to implement using off-the-shelf image models, they are computationally expensive and have limited receptive fields. Given these limitations, vectorization methods have gained increasing attention due to their efficient sparse encoding and ability to capture complex structural information. Furthermore, existing research methods model all relationships in both spatial and temporal dimensions to capture fine-grained interactions between vectorized entities. Notably, Transformer-based models have been widely used for trajectory prediction to model spatial and temporal contexts and vehicle-to-vehicle interactions. While these models improve prediction accuracy, their scalability remains limited by the high computational complexity of attention. When dealing with dense scenes and extreme conditions, this can lead to exacerbated inference latency, hindering timely predictions.
[0004] The output of trajectory prediction is subject to significant uncertainty. To avoid missing any potential behaviors, the model must be able to capture potential multimodal distributions. Currently, many studies utilize hand-crafted anchor points as guidance for multimodal prediction. However, when the number of anchor points cannot accurately cover the actual ground conditions, it leads to a failure to capture fine-grained information, thus affecting the accuracy of trajectory prediction. Summary of the Invention
[0005] To overcome the limitations of previous solutions, this invention provides a method, model, and electronic device for predicting the trajectory of vehicles surrounding an autonomous vehicle based on an improved Transformer model and target point guidance. The constructed model mainly consists of two modules: a scene context attention encoder and a multimodal motion decoder. The model processes multi-scene element inputs in the local spatiotemporal coordinate system to generate a scene code. This scene code is used as the context for the decoder to generate k possible multimodal trajectories covering the output space.
[0006] First, a vectorized scene representation is employed. For each semantic element, such as vehicles and lane segments, a local reference frame is constructed to decouple the inherent features and relative information between elements, learning a traffic scene representation independent of the global spatiotemporal coordinate system to generate a unique and invariant representation. Second, a scene context encoding module based on factorized attention Transformer is constructed. A feature fusion method based on cross-attention query along the target vehicle axis is proposed. By injecting multiple key-value pairs to guide and enhance the acquisition of target vehicle information, the goal of global feature fusion of scene context is achieved, improving computational efficiency and real-time prediction accuracy, and helping the model achieve efficient multi-agent prediction in dynamic and complex scenes. Then, a multimodal prediction and decoding module based on target points is constructed. Dense target candidates and their probabilities are generated from the scene context. A target point predictor is designed to select only candidate targets on highly potential lane segments, and a multimodal target decoder is introduced to further capture a set of high-quality target states, thereby effectively filtering and suppressing redundant target points and reducing the representation requirements for subsequent target trajectory generation. Finally, a trajectory generation module is used to obtain a final set of compact and diverse predictions, achieving highly reliable and interpretable multimodal trajectory output.
[0007] Based on the above prediction model, the proposed method for predicting the trajectory of vehicles surrounding an autonomous vehicle based on an improved Transformer model and target point guidance is implemented using the following technical solution, including the following steps:
[0008] Step 1: Trajectory Prediction Problem Description: In a certain driving environment, assume there are A surrounding targets in the scene where the target vehicle i is located. It is known that the target vehicle i has a trajectory prediction time of t... h The historical trajectory information with time step is represented as follows: Predict the future t of the target vehicle f Trajectory information of time step segment Each trajectory This refers to the location coordinates at a future time. Specifically, it is represented as follows:
[0009] Step 1.1: Vectorize all scene information: For target vehicle i, the motion state features at time step t include its spatial position. Heading angle acceleration and speed Equal motion parameters, where (t∈t) h Each trajectory motion vector is defined as follows: in Let represent the start and end positions of target vehicle i within time step t, respectively. Simultaneously, obtain the M-shaped polygon from the high-resolution map; for a lane segment C... Each lane vector Where N represents the total vector length. These are the starting and ending coordinates of the lane segment, respectively.
[0010] Step 1.2: Determine the spatiotemporal location of the target vehicle. and reference direction Establish a local spatiotemporal coordinate system. For lanes and pedestrian crossings, select the location and direction of the centerline entrance point as a reference.
[0011] Step 1.3: Embedding of Multiple Scene Elements: For each spatiotemporal scene element, the polar coordinates are converted into Fourier features, and then embedded using a multilayer perceptron (MLP) to obtain the embedding of the target vehicle i. Attention-based pooling is applied to the sampling point embedding m within each map polygon. i .
[0012] Step 1.4: To preserve all spatiotemporal position information of scene element pairs, the relative spatiotemporal positions are embedded. Specifically: taking target vehicle i and surrounding target j as examples, their absolute spatiotemporal positions are represented as follows: and Including their relative distance relative direction relative direction And the time interval ut. Then, through processing by a multilayer perceptron (MLP), a relative position embedding representation is generated as follows. If either of the two scene elements is static (such as a lane segment), the embedding can be simplified to τ. j:i .
[0013] Step 2: Encode scene elements into their local coordinate system and, with the help of relative spatiotemporal position embedding, propose an efficient feature fusion method for querying cross-attention along the target vehicle's axis. By injecting multiple key-value pairs to guide and enhance the acquisition of target vehicle scene context information features, and iteratively updating the target vehicle feature representation, the final target information embedding is obtained. The specific implementation process is as follows:
[0014] Step 2.1: First, use self-attention to capture the relationships between map elements, starting from the map embedding m. i Extract a query value and make it focus on its adjacent lanes and crosswalks. Where N i It is a map polygon m i The steps for settling the neighbors are as follows:
[0015]
[0016] Where MHA(a, b, c) represents attention, a is the query, b is the key, and c is the value. The attention function is calculated as follows:
[0017]
[0018]
[0019] This involves combining spatial awareness with map coding, and embedding cascaded operations from adjacent polygon codes and relative positions. Brief Notes To calculate the j-th key and value. These are the query matrix, key matrix, and value matrix, respectively, W. Q W K W V It is a learnable linear projection matrix, d k M represents the dimensions of the Q, K, and V vectors. i This is represented as a map encoding using self-attention output.
[0020] Step 2.2: Concatenate the key and value elements in the attention layer with the spatiotemporal position embeddings relative to the query element. This includes embedding from the target vehicle state. Derived query vector Query value
[0021] By mapping target vehicle information from past time steps to the current state, the model learns the encoded motion information from previous moments. First, the historical attention module is used to learn the temporal dependencies within the region. The formula is:
[0022]
[0023] This involves considering the target vehicle i from time step tu(0) <u<t h The embeddings up to time step t-1 and the corresponding relative position embedding combinations are denoted as follows: Calculate the key and value to perform time attention operations.
[0024] Step 2.3: Next, to reflect the impact of map environment information on vehicle prediction, the self-attention-updated map information is fused into the intelligent agent. The formula is:
[0025]
[0026] This utilizes the output map encoding. Exporting Key and Value values completes the objective - map note.
[0027] Step 2.4: Spatial information is then aggregated through an interactive attention module to learn environmental information and surrounding vehicle information that significantly impacts the target vehicle's driving. The formula is:
[0028]
[0029] This is composed of the state of surrounding vehicles and their relative spatiotemporal positions. Exporting key and value pairs completes the social attention between vehicles. Neighbor set N i It is determined by a distance threshold of 50 meters.
[0030] Step 2.5: Repeat steps 2.2 to 2.4, stacking historical time attention, target-map attention, and vehicle interaction attention sequentially as a fusion block, and iteratively updating L. enc This is used to realize the update information flow between different scene elements and output the final tensor.
[0031] Step 3: Generate dense target candidates and their probabilities from the scene context. Design a target point predictor to select only highly potential candidate targets on lane segments. Introduce multimodal target decoding to further effectively capture a set of high-quality target states, thereby effectively filtering and suppressing redundant target points and reducing the representation requirements for subsequent target trajectory generation. The specific implementation process is as follows:
[0032] Step 3.1: First, lane scoring selection is used to predict the lane the target vehicle will appear in, thereby reducing the number of target candidates. The first k lane segments {C1, C2, ... C...} are selected. k}, then we have:
[0033] D(C,C gt )=min(||C1-C gt || 2 ,||C2-C gt || 2 ,…||C k -C gt || 2 )
[0034] L lane =L cls (C,C gt )
[0035] Where D(C,C) gt () represents lane C and the actual ground lane segment C. gt The distance between them, L lane Estimate the loss for the lane.
[0036] Step 3.2: Next, target probability estimation is used to obtain the probability associated with future target points on the candidate lanes. An attention mechanism is then used to obtain local information between the target vehicle and the lane. The specific calculation formula is as follows:
[0037] q g =W q Z,k c =W k L,v c =W v L
[0038]
[0039] Z and L are the feature matrices obtained using MLP to gather dense target candidates and all map elements.
[0040] Then, the maximum entropy model is used to score all candidate target points, where the predicted score of the i-th target can be written as:
[0041]
[0042] The trainable function g(·) is also implemented using a 2-layer MLP.
[0043]
[0044] Where L goal This is the target point loss. The predicted target score δ and the ground reality target score are calculated using the cross-entropy loss function. The differences between them.
[0045] Step 3.3: Then, a multimodal target generator is used, employing k=6 heads with the same network structure to simultaneously predict multiple target points, achieving a balance between decoding efficiency and performance. The specific formula is:
[0046]
[0047] Where ψ k,x This means the prediction head passes through a linear layer, and then the softmax function is used to obtain the assigned scores. The decoder output g k The candidate coordinates Si Obtain the corresponding prediction target.
[0048] Step 4: Finally, a two-layer MLP network structure is used to complete the multimodal trajectory generation.
[0049]
[0050] Where L traj It is the trajectory completion loss. Y t and These are the future trajectory and the predicted trajectory of the target vehicle, respectively.
[0051] L = L lane +L goal +L traj
[0052] Where L is the loss function of the entire model.
[0053] Step 5: Validate and compare this model on the test set.
[0054] The beneficial effects of this invention are:
[0055] 1. For each semantic element, such as a vehicle and a lane segment, this invention constructs a local reference frame to decouple the vectorized features between elements and their relative poses, in order to learn a traffic scene representation independent of the global spatiotemporal coordinate system, so that each scene element generates a unique and invariant representation, thereby enabling faster online inference.
[0056] 2. This invention constructs a scene context encoding module based on factorized attention Transformer. Under local spatiotemporal coordinate system encoding, a feature fusion method is proposed that queries cross-attention along the target vehicle axis. By injecting multiple key-value pairs to guide and enhance the acquisition of target vehicle information, the goal of global feature fusion of scene context is achieved, improving computational efficiency and real-time prediction accuracy. Simultaneously, when performing attention-based scene context fusion, historical time attention, target-map attention, and inter-vehicle interaction attention are stacked sequentially as a fusion block. Through iterative learning of the fusion block, information updates between scene elements are achieved, helping the model achieve accurate predictions in dynamic and complex scenes.
[0057] 3. This invention constructs a multimodal motion prediction and decoding module based on target points. Dense target candidate points and their probabilities are generated from the scene context. A target point predictor is designed to select candidate targets on highly potential lane segments, and a multimodal target decoder is introduced to further capture a set of high-quality target states, thereby effectively filtering and suppressing redundant target points and reducing the representation requirements for subsequent target trajectory generation. Finally, a trajectory generation module is used to obtain a final set of compact and diverse predictions, achieving highly reliable and interpretable multimodal trajectory output. Attached Figure Description
[0058] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention.
[0059] Figure 2 A diagram illustrating contextual awareness encoding.
[0060] Figure 3 This is a schematic diagram of multimodal motion prediction decoding. Detailed Implementation
[0061] The invention will now be further described with reference to the accompanying drawings.
[0062] This invention provides an accurate and efficient method for predicting the trajectories of vehicles around an autonomous vehicle, such as... Figure 1 As shown. The model processes multimodal input to produce scene encoding. This scene encoding serves as the context for the decoder to generate highly reliable and interpretable multimodal trajectory outputs covering the output space. Specifically, it includes the following steps:
[0063] (1) First, the trajectory prediction problem is described: In a certain driving environment, assume there are A surrounding targets in the scene where the target vehicle is located. It is known that the target vehicle i has a trajectory prediction time of t. h The historical trajectory information with time step is represented as follows: Predict the future t of the target vehicle f Trajectory information of time step segment Each trajectory This refers to the location coordinates at future times, where the superscript t = 1, 2, ... t f Specifically, it is expressed as follows:
[0064] 1) Vectorize all scene information: For target vehicle i, the motion state features at time step t include its spatial position. Heading angle acceleration and speed Equal motion parameters, where t∈t h , These are the vehicle's x-coordinate and y-coordinate in the coordinate system, respectively. Each trajectory motion vector is defined as... in Let represent the start and end positions of target vehicle i within time step t, respectively. Simultaneously, obtain the M-shaped polygon from the high-resolution map; for a lane segment C... Each lane vector Where N represents the total vector length. These are the starting and ending coordinates of the lane segment, respectively.
[0065] 2) A local spatiotemporal coordinate system was established for each scene element (including lanes, crosswalks, vehicles, pedestrians, etc.). The establishment of the local spatiotemporal coordinate system was determined by the spatiotemporal position of the target vehicle and the reference direction. For lanes and crosswalks, the position and direction of the centerline entrance point were selected as the reference.
[0066] 3) Embedding of Multiple Scene Elements: Polar coordinates of all geometric attributes (such as the velocity and motion vector of the target vehicle state, and the positions of all sampling points on the lane) are calculated based on the spatiotemporal position and orientation referenced in the local coordinate system. Each polar coordinate is converted into a Fourier feature and embedded using a multilayer perceptron (MLP) to obtain the embedding A of the target vehicle i. i t Attention-based pooling is applied to the sampling point embedding m within each map polygon. i .
[0067] 4) To preserve all spatiotemporal location information of scene element pairs, relative spatiotemporal locations are embedded. Specifically: taking target vehicle i at time step t and surrounding target j at time step u as an example (0 <u<t h Their absolute spatiotemporal positions are respectively represented as and Including their relative distance relative direction relative direction And the time interval ut. Then, through processing by a multilayer perceptron (MLP), a relative position embedding representation is generated as follows. If either of the two scene elements is static (such as a lane segment), the embedding can be simplified to τ. j:i .
[0068] (2) This invention constructs a Transformer scene attention context encoding module based on factorized attention. Scene elements are encoded into their local coordinate systems, and by leveraging relative spatiotemporal position embedding, an efficient feature fusion method for querying cross-attention along the target vehicle axis is proposed, such as... Figure 2As shown, by injecting multiple key-value pairs to enhance the ability to acquire target vehicle scene context information features, and iteratively updating the target vehicle feature representation, the goal of obtaining the final target information embedding is achieved. The specific implementation process is as follows:
[0069] 1) First, self-attention is used to capture the relationships between map elements, starting from the map embedding m. i Extract a query value and make it focus on its adjacent lanes and crosswalks. Where N i It is a map polygon m i The steps for settling the neighbors are as follows:
[0070]
[0071] Where MHA(a, b, c) represents attention, a is the query, b is the key, and c is the value. The attention function is calculated as follows:
[0072]
[0073] The spatial awareness combined with map coding, and the concatenation operation of embedding adjacent polygon codes with relative positions, is denoted as... Brief Notes To calculate the j-th key and value. These are the query matrix, key matrix, and value matrix, respectively, W. Q W K W V It is a learnable linear projection matrix, d k It represents the dimensions of the Q, K, and V vectors.
[0074]
[0075] Where M i It uses self-attention output for map encoding, which can be shared across all vehicles and all time steps.
[0076] 2) Before performing attention, concatenate the key and value elements in the attention layer with their spatiotemporal positions relative to the query element.
[0077] First, target vehicle information from past time steps is mapped to the current state, enabling the model to learn the encoded motion information from previous moments. Historical time attention is then used to learn temporal dependencies within the region. The formula is the same as described above:
[0078]
[0079] Among them, embedding from the target vehicle state Derived query vector Query value By considering the target vehicle i from time step tx(0) <x<t h The embeddings up to time step t-1 and the corresponding relative position embedding combinations are denoted as follows: The key and value are calculated to perform historical time attention operations, resulting in updated target features.
[0080] 3) Next, to reflect the impact of map environment information on vehicle prediction, the self-attention-updated map information is integrated into the intelligent agent. The formula is:
[0081]
[0082] Among them, the output map encoding is used. Exporting the key and value completes the objective - map attention, obtaining the target features of interest in the map information.
[0083] 4) Spatial information is then aggregated through an interactive attention module to learn environmental information and surrounding vehicle information that significantly impacts the target vehicle's driving. The formula is:
[0084]
[0085] This is composed of the state of surrounding intelligent agents and their relative spatiotemporal positions. Exporting key and value data completes the interaction and attention between vehicles, resulting in the final fused target features. Neighbor set N i It is determined by a distance threshold of 50 meters.
[0086] 5) Repeat steps 1)-3), continuously updating the initial, unchanging query using the same keys and values. Stack historical time attention, target-map attention, and vehicle interaction attention sequentially as a fusion block, and iterate and update it multiple times, denoted as L. enc This is used to realize the update information flow between different scene elements and output the final tensor.
[0087] (3) This invention constructs a multimodal motion prediction and decoding module based on the target point, such as Figure 3 As shown, a dense set of target candidates and their probabilities are generated from the scene context. A target point predictor is designed to select only highly potential candidate targets on lane segments. A multimodal target decoder is introduced to further effectively capture a set of high-quality target states, thereby effectively filtering and suppressing redundant target points and reducing the representation requirements for subsequent target trajectory generation. The specific implementation process is shown below:
[0088] 1) The number of target candidates depends on the lanes surrounding the target vehicle. Since multiple hypothetical targets are freely output based on map elements, a lane scoring selection strategy is used to predict the lane the target vehicle will appear in. This effectively avoids some unreasonable target points and obtains a reasonable driving lane for the target vehicle. To reduce the number of target candidates, the first k lane segments {C1, C2, ... C...} are selected. k}, then we have:
[0089] D(C,C gt )=min(||C1-C gt || 2 ,||C2-C gt || 2 ,…||C k -C gt || 2 )
[0090]
[0091] in Estimate the loss for the lane. To select the binary cross-entropy loss function, D(C,C) gt () represents lane C and the actual ground lane segment C. gt The distance between them.
[0092] 2) Since the sampling of dense target points is random, no probability value is provided. Next, target probability estimation is used to obtain the probability associated with future target points on the candidate lanes. An attention mechanism is then used to obtain local information between the target vehicle and the lane. The specific calculation formula is as follows:
[0093] q g =W q Z,k c =W k L,v c =W v L
[0094]
[0095] Among them, W q W k W v It is a linear projection matrix, d k q represents the dimensions of the Q, K, and V vectors. g It is the query vector, k c It is a key vector, v c It is a value vector, where Z and L represent the feature matrices obtained by using MLP to obtain dense target candidates and all map elements, and Attn(·) represents the attention mechanism operation.
[0096] The maximum entropy model is then used to score all candidate target points, and the scores are normalized using the softmax function. By comprehensively considering various environmental factors and trajectory features, the probability of each target point can be assessed more accurately. The predicted score for the i-th target can be written as:
[0097]
[0098] The trainable function g(·) is implemented using a two-layer MLP, Z i The information obtained on the lane is based on the attention mechanism for target point i.
[0099]
[0100] in, It is the target point loss. Let δ be the cross-entropy loss function, and δ be the predicted target score. The target score is the ground-based objective score. The predicted target score δ and the ground-based objective score are calculated using the cross-entropy loss function. The differences between them.
[0101] 3) After passing through the target point predictor above, a heatmap describing the probability distribution of potential targets is obtained. The top X candidate targets with the highest scores are selected, and their corresponding features, predicted scores, and coordinates are obtained. First, these vectors are concatenated, and an MLP is used to obtain new embeddings for the selected nodes. At the same time, a self-attention mechanism is used to fuse global heatmap information.
[0102] Then, a multimodal target decoder is used, employing k=6 heads with the same network structure to simultaneously predict X=100 target points, thereby achieving a balance between decoding efficiency and performance.
[0103]
[0104] Where, ψ k,x This means the prediction head passes through a linear layer, and then the softmax function is used to obtain the assigned scores. The decoder output g k The candidate coordinates S i Obtain the corresponding prediction targets, including the 6 most likely target points.
[0105] (4) Finally, each trajectory is completed based on the predicted target set. The features of each target point are passed to the decoder, and a two-layer MLP network structure is used to complete the multimodal trajectory generation.
[0106]
[0107] in It is the trajectory completion loss. Let Y be the Huber loss function. t and These are the future trajectory and the predicted trajectory of the target vehicle, respectively.
[0108]
[0109] in This is the loss function for the entire model. The loss is estimated from the lane estimation. target point loss Trajectory Completion Loss composition.
[0110] (5) Validate and compare the model on the test set.
[0111] The present invention also proposes an electronic device for autonomous vehicles, or a control device, for executing the program of the above prediction method, or a storage device for storing the program of the above prediction method.
[0112] In summary, this invention provides a method, model, and electronic device for predicting the trajectory of vehicles surrounding an autonomous vehicle based on an improved Transformer model and target point guidance. The model architecture mainly consists of a scene context attention encoding module and a multimodal target decoding module. The encoder learns the vector representation of the dynamic scene and its components, enabling the target vehicle to efficiently and accurately capture long-term behavior in complex scenes and its interactions with map structures and other surrounding traffic entities. The decoder makes predictions based on target points and uses a trajectory generation module to generate the vehicle's multimodal future trajectory. Using the autonomous vehicle surrounding vehicle trajectory prediction model provided by this invention to predict multimodal trajectories can yield reliable and accurate prediction results.
[0113] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent methods or modifications that do not depart from the technology of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for predicting the trajectory of an autonomous vehicle based on an improved Transformer model and target point guidance, characterized in that, Includes the following steps: Step 1: Trajectory Prediction Problem Description: In a certain driving environment, assume there are A surrounding targets in the scene where the target vehicle is located, and the target vehicle is known. in the past The historical trajectory information with time step is represented as follows: Predicting the future of the target vehicle Trajectory information of time step segment Each trajectory This refers to the location coordinates at a future time. Step 2: Encode scene elements into their local coordinate system and embed them using relative spatiotemporal positions. Employ an efficient feature fusion method that queries cross-attention along the target vehicle's axis. By injecting multiple key-value pairs, enhance the target vehicle's ability to acquire scene contextual information features and iteratively update the target vehicle's feature representation to achieve the final target information embedding goal. The specific implementation is as follows: Step 2.1: First, use self-attention to capture the relationships between map elements from the map embedding. Extract a query value and make it focus on its adjacent lanes and crosswalks. ,in It is a map polygon The steps for settling the neighbors are as follows: Where MHA(a, b, c) represents attention, a is the query, b is the key, and c is the value; the attention function is calculated as follows: ， ， This involves combining spatial awareness with map coding, and embedding cascaded operations from adjacent polygon codes and relative positions. abbreviated To calculate the first Each key and value, , and These are the query matrix, key matrix, and value matrix, respectively. , , It is a learnable linear projection matrix. yes , and The dimension of a vector This is represented as a map encoding using self-attention output; Step 2.2: Concatenate the key and value elements in the attention layer with the spatiotemporal position embeddings relative to the query element, where the embeddings are from the target vehicle state. Derived query vector Query value ; The target vehicle information from past time steps is mapped to the current state to learn the previous motion information encoding. First, historical time attention is used to learn the temporal dependencies within the region, as shown in the formula: Among them, by considering the target vehicle From time step Time to step The embedding and the corresponding relative position embedding combination are denoted as The key and value are calculated to perform time attention operations; Step 2.3: Next, to reflect the impact of map environment information on vehicle prediction, the self-attention updated map information is fused into the intelligent agent, using the following formula: Among them, the output map encoding is used. Exporting Key and Value data completes the objective - map notes; Step 2.4: Then, through interactive attention, aggregate spatial information to learn environmental information and surrounding vehicle information that significantly impacts the target vehicle's driving. The formula is: This is achieved by embedding and combining the status of surrounding vehicles and their relative spatiotemporal positions. Exporting key and value pairs completes the social attention between vehicles and the neighbor set. Determined by a distance threshold of 50 meters; Step 2.5: Repeat steps 2.2 to 2.4, stacking historical time attention, target-map attention, and vehicle interaction attention sequentially as a fusion block, and iteratively updating L. enc This is used to realize the update information flow between elements in different scenes and output the final tensor. Step 3: Generate dense target candidates and their probabilities from the scene context, design a target point predictor to select only highly potential candidate targets on lane segments, and introduce multimodal target decoding to further capture a set of high-quality target states, effectively filtering and suppressing redundant target points, and reducing the representation requirements for subsequent target trajectory generation; Step 4: Finally, a two-layer MLP network structure is used to complete the multimodal trajectory generation.
2. The method for predicting the trajectory of an autonomous vehicle based on an improved Transformer model and target point guidance according to claim 1, characterized in that, The specific implementation of step 1 includes the following: Step 1.1: Vectorize all scene information: For the target vehicle time step The characteristics of a state of motion include its spatial position. heading angle acceleration and speed Motion parameters, among which Each trajectory motion vector is defined as follows: ,in , They represent the target vehicles respectively. time step The starting and ending points within the area; simultaneously acquire the high-resolution map. Polygonal shape, for a lane segment In other words, Each lane vector ,in Indicates the total vector length. , These are the starting and ending coordinates of the lane segment, respectively. Step 1.2: Determine the spatiotemporal location of the target vehicle. and reference direction Establish a local spatiotemporal coordinate system, and for lanes and pedestrian crossings, select the location and direction of the centerline entrance point as a reference. Step 1.3: Embedding of Multiple Scene Elements: The polar coordinates of each spatiotemporal scene element are converted into Fourier features, and then embedded using a multilayer perceptron (MLP) to obtain the target vehicle. Embedded Embedding sampling points within each map polygon Attention-based pooling is employed; Step 1.4: In order to preserve all spatiotemporal position information of scene element pairs, the relative spatiotemporal position is embedded. Specifically: for the target vehicle... and surrounding targets Their absolute spatiotemporal positions are respectively represented as and Specifically, this includes relative distance. relative direction Relative orientation and time interval Then, through processing by a multilayer perceptron (MLP), a relative position embedding is generated, represented as... If either of the two scene elements is static, the embedding can be simplified as follows: .
3. The method for predicting the trajectory of an autonomous vehicle based on an improved Transformer model and target point guidance according to claim 1, characterized in that, The specific implementation of step 3 includes the following: Step 3.1: First, a lane scoring selection strategy is used to predict the lane in which the target vehicle will appear, in order to reduce the number of target candidates and select the lane that will be selected. Each lane section Then we have: in Indicated as lane With ground real-time lane section C gt The distance between them Estimate the loss for the lane; Step 3.2: Next, target probability estimation is used to obtain the probability associated with future target points on the candidate lanes. An attention mechanism is then used to obtain local information between the target vehicle and the lane. The specific calculation formula is as follows: ， ， in , The purpose is to use MLP to obtain the feature matrix of dense target candidates and all map elements; Then, the maximum entropy model is used to score all candidate target points, where the th... The predicted score for each target can be expressed as: Where the trainable function Implemented using a two-layer MLP; in It is the target point loss, which uses the cross-entropy loss function to calculate the predicted target score. Score of ground-based target The differences between them; Step 3.3: After obtaining a heatmap describing the probability distribution of potential targets using the target point predictor described above, select the top X candidate targets with the highest scores and obtain their corresponding features, predicted scores, and coordinates. Then, concatenate these vectors and use an MLP to obtain new embeddings for the selected nodes. Simultaneously, a self-attention mechanism is employed to fuse global heatmap information. Next, a multimodal target decoder is used, employing k=6 heads with the same network structure to simultaneously predict multiple target points, achieving a balance between decoding efficiency and performance. The specific formula is as follows: in This is represented as the prediction head passing through a linear layer, then using the softmax function to obtain the assigned scores, and the decoder output. Candidate coordinates Obtain the corresponding prediction target.
4. The method for predicting the trajectory of an autonomous vehicle based on an improved Transformer model and target point guidance according to claim 1, characterized in that, In step 4, the loss function of the two-layer MLP network is designed as follows: in It is trajectory completion loss. and These are the future trajectory and the predicted trajectory of the target vehicle, respectively.
5. A trajectory prediction model for autonomous vehicles based on an improved Transformer model and target point guidance, characterized in that, include: Scene context attention encoder and multimodal motion decoder; The scene context attention encoder is used to process multi-scene element input in the local spatiotemporal coordinate system to generate a scene code, which is used as the context of the multimodal motion decoder to capture a set of high-quality target states, and the trajectory generation module generates k possible multimodal trajectories covering the output space. The scene context attention encoder implements the content of step 2 as described in any one of claims 1-4; The multimodal motion decoder implements the content of step 3 as described in any one of claims 1-4.
6. The prediction model according to claim 5, characterized in that, The prediction model further includes a vectorized scene representation module, which is capable of implementing the content of step 1 as described in any one of claims 1-4.
7. The prediction model according to claim 5, characterized in that, The trajectory generation module employs the two-layer MLP network described in any one of claims 1-4.
8. The prediction model according to claim 5, characterized in that, The total loss function of the prediction model Lane estimation loss as described in claim 3 Target point loss And the trajectory completion loss of claim 4 composition: 。 9. An electronic device for an autonomous vehicle, characterized in that, The electronic device may be a control device capable of executing the program of any one of the prediction methods of claims 1-4, or the electronic device may be a storage device storing the program of the prediction method of claims 1-4.