A vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion

Abstract

This invention relates to the field of autonomous driving technology and provides a vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion. The method includes: acquiring historical vehicle trajectories and map lane line information and constructing a local coordinate system; constructing a masked autoencoder for occlusion reconstruction pre-training of trajectory and lane line tokens; designing a time-stamped dynamic interaction module to capture dynamic time-varying relationships; jointly predicting longitudinal, lateral, and interaction intents to guide multimodal trajectory decoding; constructing a risk assessment module that fuses dynamic traffic risk and map deviation risk to reweight the confidence of candidate trajectories based on risk perception; and constructing a multi-task joint loss function for end-to-end optimization. This invention improves prediction robustness, multimodal trajectory discriminability, and trajectory safety in occluded scenarios and is applicable to autonomous driving decision-making and planning systems.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving technology, and in particular relates to a vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion. Background Technology

[0002] With the rapid development of autonomous driving technology, the ability of autonomous driving systems to accurately predict the future movement trends of surrounding traffic participants is a key challenge currently facing the field. Vehicle trajectory prediction, as a crucial intermediate link connecting the perception and decision-making / planning modules in an autonomous driving system, directly impacts the overall driving safety and reliability of the vehicle due to its accuracy and stability. Autonomous vehicles need to predict various possible movement trends within the next few seconds based on the historical states of target vehicles, surrounding traffic participants, and road structure, thereby providing a basis for driving decisions such as obstacle avoidance, following, lane changing, yielding, overtaking, and intersection maneuvering.

[0003] Real-world road traffic scenarios are characterized by multiple stakeholders, strong interaction, uncertainty, and high risk. The future trajectory of a target vehicle depends not only on its own historical position, speed, acceleration, and heading changes, but also on the time-varying interactions of surrounding vehicles, lane geometry, road topology constraints, and changes in driving intent. Especially in scenarios with perceptual obstruction, dense traffic flow, intersection merging, lane changes, or overtaking, directly regressing the future trajectory point sequence based solely on short-term historical trajectories can easily lead to problems such as trajectory deviation, insufficient modality, uncontrollable risks, and weak scenario adaptability.

[0004] In recent years, deep learning-based multimodal vehicle trajectory prediction methods typically employ Transformer, graph neural networks, recurrent neural networks, or attention mechanisms to encode the agent's historical state and map features, and output several candidate trajectories through a multimodal decoder. However, existing methods still have the following shortcomings:

[0005] (1) Insufficient scene understanding: Traditional methods mostly input historical trajectories and maps, and directly train the model to predict future trajectories. This results in the model only learning the mapping from "input to future trajectory labels", and the agent fails to truly understand the traffic scene patterns. When faced with occluded perception, missing historical trajectories, or insufficient map information, the model is prone to only learning the local patterns of visible segments in the early stages of training.

[0006] (2) Insufficient time-varying interaction capability: Traditional interaction modeling is difficult to depict the dynamic interaction process of agent-agent and agent-map on a time-stamp basis. The interaction relationship between the target vehicle and surrounding vehicles is often statically aggregated, which makes it impossible to update in time sequence, and thus it is difficult to depict the dynamic changes of the interaction impact at each historical moment.

[0007] (3) Lack of explicit modeling of driving intentions: Many trajectory prediction methods only predict by regressing future trajectory points, lacking modeling of the agent's horizontal and vertical driving intentions and interaction intentions. This results in driving intentions not being tightly coupled with trajectory decoding, leading to insignificant differences between different modalities.

[0008] (4) Insufficient risk assessment capability: Most trajectory prediction models typically optimize multimodal candidate trajectories using geometric error as the main optimization objective, lacking explicit constraints on dynamic collision risk and map deviation risk. Summary of the Invention

[0009] The purpose of this invention is to provide a vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion, which aims to solve the problems mentioned in the background art.

[0010] The present invention is implemented as follows: a vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion includes the following steps:

[0011] Step 1: Fuse the multi-sensor information obtained by the sensors to obtain the agent's historical trajectory information and HD map information. Construct a local coordinate system for the target vehicle based on the target vehicle's current position and heading angle, and unify the vehicle position, lane line points, and speed direction into the local coordinate system.

[0012] Step 2: The MAE encoder encodes the vehicle trajectory token and the map lane line token respectively. A portion of the vehicle trajectory token and map lane line token are randomly occluded proportionally, so that the encoder only receives the unoccluded tokens. The MAE decoder uses the learnable mask token to recover the complete token set and reconstructs the coordinates of the occluded trajectory and lane line points. The pre-trained encoder weights are obtained based on the occlusion reconstruction loss. The shared scene encoder is initialized using the pre-trained encoder weights. The scene encoder consists of a vehicle temporal encoder, a lane line polyline encoder, and a scene fusion encoder, which encode the vehicle's historical trajectory and map lane lines respectively and fuse them to obtain scene features.

[0013] Step 3: Perform agent-agent self-attention on all valid vehicles at each historical timestamp, and perform agent-map cross-attention on the vehicle token at each timestamp. Then, perform temporal Transformer updates along the historical time dimension of each vehicle to obtain dynamic interaction features. After fusing the scene features and dynamic interaction features, output the probabilities of vertical intent, horizontal intent, and interaction intent respectively. Multiply the probabilities of the three types of intents with the learnable intent embedding matrix to obtain the intent embedding. Generate a dynamic traffic risk potential field based on the assisted prediction trajectory of surrounding vehicles, and generate a map-constrained risk potential field based on the minimum distance from the candidate trajectory to the lane line point set. Combine these to obtain a comprehensive risk score, and use the comprehensive risk score to reweight the original modality confidence.

[0014] Step 4: Use K learnable modal queries. Each modal query is fused with the scene features, dynamic interaction features and intent embedding of the target vehicle to form the pattern features of the corresponding candidate trajectory. The decoder predicts the displacement increment of each future time step and accumulates it starting from the last valid historical position of the target vehicle to obtain continuous future predicted trajectories. At the same time, the original modal confidence of each candidate trajectory and the auxiliary predicted trajectory of surrounding vehicles are output.

[0015] Step 5: Train the trajectory prediction model using multi-task joint loss, which includes trajectory regression loss, pattern classification loss, surrounding vehicle auxiliary prediction loss, intent prediction loss, and risk assessment loss.

[0016] A further technical solution involves, in step 1, using the current position of the target vehicle... and heading angle Based on this, a local coordinate system for the target vehicle is constructed, where, Let x be the x-coordinate of the target vehicle in the global coordinate system at the current moment. Let be the ordinate of the target vehicle in the global coordinate system at the current moment; for any vehicle or lane line point, its position in the global coordinate system is... Its local coordinates Calculate as follows:

[0017]

[0018] in, This represents the global x-coordinate of a point on a vehicle or lane line. This represents the global vertical coordinate of a point on a vehicle or lane line. Let be a rotation matrix, and its expression is:

[0019]

[0020] For velocity vector The rotation is performed in the same way as follows:

[0021]

[0022] in, Vehicle speed in global coordinate system Directional components, Vehicle speed in global coordinate system Directional components, This is the velocity vector after rotation to the target vehicle's local coordinate system.

[0023] A further technical solution, in step 2, is to reduce occlusion reconstruction loss. and pre-trained encoder weights The calculation is as follows:

[0024]

[0025]

[0026] in, Loss weights for reconstructing future trajectories. Weighting for reconstructing historical trajectories Weights for lane line reconstruction loss on the map. Losses due to reconstruction of future trajectories Losses due to the reconstruction of historical trajectory For lane line reconstruction losses, and These are the sets of indices for the obscured vehicle trajectory tokens and the map lane line tokens, respectively. To reconstruct weights for the map, The index set of the obscured vehicle trajectory tokens The true two-dimensional coordinate vector of each trajectory token. for The reconstructed two-dimensional coordinate prediction results The index set of the lane line tokens on the obscured map The true two-dimensional coordinate vector of each lane line token for The predicted two-dimensional coordinates obtained from the reconstruction.

[0027] A further technical solution, in step 2, is the vehicle timing encoder. Transformer encoding is performed on the historical state sequence of each vehicle to obtain the time token at each historical moment and the agent token features of the entire trajectory.

[0028]

[0029] in, Features of the agent token This refers to the historical trajectory characteristics of vehicles. This represents the number of vehicles participating in the modeling process in the current scene. For historical observation time steps;

[0030] Lane line polyline encoder Transformer encoding is performed on the sampling point sequence of each lane line to obtain the map lane token features as follows:

[0031]

[0032] in, For map lane token features, For map lane line features, This represents the number of lane lines involved in modeling in the current scene. This refers to the number of sampling points on each lane.

[0033] agent token features and map lane token features The stitched data is then input into a Transformer-based scene fusion encoder. By utilizing the self-attention mechanism, agent and map information are fused globally to obtain fused scene features. :

[0034] .

[0035] A further technical solution involves, in step 3, the agent-agent interaction characteristics. for:

[0036]

[0037] in, Indicates the first Vehicle characteristics at each historical time step This indicates a bullish attention-based strategy;

[0038] Interaction features of agent-map for:

[0039]

[0040] Finally, a time-series Transformer update is performed along the historical timeline of each vehicle to create a dynamic interactive state. Dynamic interaction features are obtained through mask averaging. :

[0041]

[0042] in, It is a Transformer encoding module in the time dimension, used to perform temporal updates and fusion of agent-map interaction features of vehicles in continuous historical time steps.

[0043] A further technical solution, in step 3, is the longitudinal intent probability. Horizontal Intent Probability and interaction intent probability The calculation is as follows:

[0044]

[0045]

[0046]

[0047] in, For predicting longitudinal intent, a multilayer perceptron classification head, For predicting lateral intent, a multilayer perceptron classification head, For predicting interaction intent, a multilayer perceptron classification head, The function is softmax; longitudinal intents include four categories: keep, accelerate, decelerate, and stop; lateral intents include six categories: keep lane, dodge left / change lanes left, dodge right / change lanes right, go straight, turn, invalid, or others; interactive intents include four categories: yield, cut in front, follow, and overtake.

[0048] A further technical solution, in step 3, is dynamic traffic risk. Represented as:

[0049]

[0050] in, For the first The candidate trajectories of the target vehicles are in Location at any given moment For the first Vehicle trajectory prediction with assistance Location at any given moment To appropriately increase the width of the risk field over the prediction period, For all predicted time steps The risk potential energy value is averaged.

[0051] Map constraint risks Represented as:

[0052]

[0053] in, Represents the set of candidate trajectory points to valid lane line points. The minimum distance, The safe lane distance threshold, For the map potential field scale parameters. For smoothing nonlinear activation functions;

[0054] Comprehensive Risk Score for:

[0055]

[0056] in, Map risk weights;

[0057] Then, the original confidence levels are reweighted using the comprehensive risk score:

[0058]

[0059] in, The first output of the decoder The original modal confidence of each candidate trajectory. This is the risk reweighting coefficient. For the first The final modal confidence of each candidate trajectory after risk reweighting.

[0060] A further technical solution involves, in step 4, predicting the displacement increment for each future time step. And starting from the last valid historical position of the target vehicle, the data is accumulated to obtain continuous future predicted trajectories:

[0061]

[0062] in, For the first The candidate trajectory in the 1st The position of a future moment The predicted starting point position of the target vehicle. Predicting time steps for the future.

[0063] A further technical solution is provided in step 5.

[0064] Total loss for:

[0065]

[0066] in, For trajectory regression loss weights, Weights for pattern classification loss. To assist in predicting loss weights for surrounding vehicles, To predict the loss weights, For risk assessment loss weighting, For trajectory regression loss, For pattern classification loss, To assist in predicting losses for surrounding vehicles, In order to predict losses, To assess the risk of loss.

[0067] The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion provided in this invention has the following beneficial effects:

[0068] (1) By pre-training through occlusion reconstruction, the model can learn the completion rules of occluded vehicle trajectories and map lane lines, thereby improving its robustness to occlusion, missing information and complex traffic scenarios.

[0069] (2) Establish a local coordinate system with the target vehicle as a reference, and uniformly represent the vehicle trajectory and map information so that the model can reduce the impact of changes in different road positions and directions and improve its adaptability in different scenarios.

[0070] (3) By using time-stamped agent-agent self-attention and agent-map cross attention, dynamic interaction changes in the historical process can be captured, rather than just static interaction aggregation.

[0071] (4) By jointly predicting multiple intentions, interpretable driving semantic intermediate variables are formed before the output trajectory, thereby enhancing the separation degree and behavioral rationality of multimodal trajectories.

[0072] (5) By predicting the future trajectories of surrounding vehicles and constructing a dynamic risk potential field, the model can explicitly evaluate the potential collision risk of each candidate trajectory of the target vehicle; by constraining the risk potential field through the map, it can suppress candidate trajectories that deviate from the lane lines or do not conform to the road geometry, thereby improving the road fit of the trajectory. At the same time, risk perception is added for reweighting, and the final output probability not only reflects the trajectory fitting confidence, but also reflects the safety risk, thus making it more suitable for subsequent tasks such as autonomous driving planning and decision-making. Attached Figure Description

[0073] Figure 1 A schematic diagram illustrating a vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion provided in an embodiment of the present invention;

[0074] Figure 2This is an architecture diagram of a vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion provided in an embodiment of the present invention;

[0075] Figure 3 This is a flowchart of step 2 in a vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion provided in an embodiment of the present invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0077] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0078] like Figure 1 As shown, an embodiment of the present invention provides a vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion, comprising the following steps:

[0079] Step 1: Constructing the local coordinate system;

[0080] Information obtained from sensors such as cameras, LiDAR, and millimeter-wave radar is fused to obtain historical trajectory information and HD map information of the intelligent agent, which are then used as input to the prediction module. The obtained information is further processed to establish a local coordinate system centered on the target vehicle. Specifically, this involves using the target vehicle's current position... and heading angle Based on this, a local coordinate system for the target vehicle is constructed, where, Let x be the x-coordinate of the target vehicle in the global coordinate system at the current moment. This represents the current ordinate of the target vehicle in the global coordinate system. For any vehicle or lane line point, the position in the global coordinate system... Its local coordinates Calculate as follows:

[0081]

[0082] in, This represents the global x-coordinate of a point on a vehicle or lane line. This represents the global vertical coordinate of a point on a vehicle or lane line. Let be a rotation matrix, and its expression is:

[0083]

[0084] For velocity vector The rotation is performed in the same way as follows:

[0085]

[0086] in, Vehicle speed in global coordinate system Directional components, Vehicle speed in global coordinate system Directional components, This is the velocity vector after rotation to the target vehicle's local coordinate system.

[0087] Through the coordinate transformation described above, the vehicle position, lane line points, and velocity direction are all unified into a local coordinate system based on the target vehicle's current position and heading angle. This local coordinate system allows similar traffic scenarios to have similar representations under different road orientations and global locations, reducing the difficulty of model learning and improving the predictive model's adaptability to scene rotation and translation changes.

[0088] Step 2: Occlusion Reconstruction Pre-training and Scene Encoding;

[0089] The occlusion reconstruction pre-training module learns general scene representations of vehicle trajectories and map lane lines before the downstream prediction task. It uses a MAE encoder (Masked Autoencoder) to encode vehicle trajectory tokens (specifically, historical and future trajectory tokens) and map lane line tokens. During training, a portion of the vehicle trajectory tokens and map lane line tokens are randomly occluded according to a preset ratio, ensuring the encoder only receives unoccluded tokens. The MAE decoder uses the learnable mask tokens to recover the complete token set and reconstructs the coordinates of the occluded trajectory and lane line points. Simultaneously, it calculates the occlusion reconstruction loss... Obtain pre-trained encoder weights :

[0090]

[0091]

[0092] in, Loss weights for reconstructing future trajectories. Weighting for reconstructing historical trajectories Weights for lane line reconstruction loss on the map. Losses due to reconstruction of future trajectories Losses due to the reconstruction of historical trajectory For lane line reconstruction losses, and These are the sets of indices for the obscured vehicle trajectory tokens and the map lane line tokens, respectively. To reconstruct weights for the map, The index set of the obscured vehicle trajectory tokens The true two-dimensional coordinate vector of each trajectory token. for The reconstructed two-dimensional coordinate prediction results The index set of the lane line tokens on the obscured map The true two-dimensional coordinate vector of each lane line token for The predicted two-dimensional coordinates obtained from the reconstruction.

[0093] The vehicle trajectory reconstruction targets include historical trajectory coordinates and future trajectory coordinates, while the map lane line reconstruction targets are the lane line sampling point coordinates. During the pre-training phase, the model is forced to infer the missing mask content from surrounding vehicles, road structures, and unoccluded trajectories, thereby gaining stronger occlusion completion capabilities and scene context understanding capabilities.

[0094] The shared scene encoder is initialized using pre-trained encoder weights. The scene encoder consists of a vehicle time sequence encoder, a lane line polyline encoder, and a scene fusion encoder, which encode the vehicle's historical trajectory and the map lane lines respectively and fuse them to obtain scene features.

[0095] Among them, vehicle timing encoder Transformer encoding is performed on the historical state sequence of each vehicle to obtain the time token at each historical moment and the agent token features of the entire trajectory.

[0096]

[0097] in, Features of the agent token It is the vehicle's historical trajectory characteristics. This represents the number of vehicles participating in the modeling process in the current scene. For historical observation time steps;

[0098] Lane line polyline encoder Transformer encoding is performed on the sampling point sequence of each lane line to obtain the map lane token features as follows:

[0099]

[0100] in, For map lane token features, It is a feature of the lane lines on the map. This represents the number of lane lines involved in modeling in the current scene. This refers to the number of sampling points on each lane.

[0101] agent token features and map lane token features The stitched data is then input into a Transformer-based scene fusion encoder. By utilizing the self-attention mechanism, agent and map information are fused globally to obtain fused scene features. :

[0102]

[0103] This design enables the target vehicle, surrounding vehicles, and map elements to interact within the same feature space, providing a foundational representation for subsequent dynamic interactions, intent inference, and risk assessment.

[0104] Step 3: Combine dynamic interaction, driving intention, risk assessment and risk perception with weighted joint training;

[0105] The dynamic interaction module addresses the problem of static aggregation of interaction relationships in existing technologies. This module first rearranges the vehicle's historical time tokens along the time dimension, and then performs agent-agent self-attention on all valid vehicles at each historical timestamp, enabling the model to determine the influence of different vehicles on the target vehicle's motion trend at that moment. (Agent-agent interaction features are then discussed.) for:

[0106]

[0107] in, Indicates the first Vehicle characteristics at each historical time step This indicates multi-head attention operation.

[0108] Then, using the map-encoded lane token as memory, agent-map cross-attention is performed on the vehicle token for each timestamp, enabling the vehicle state to perceive lane geometry, road boundaries, and local map constraints, thus obtaining the agent-map interaction features. :

[0109]

[0110] Finally, the module performs a time-series Transformer update along the historical time dimension of each vehicle to form a dynamic interactive state. Dynamic interaction features are obtained through mask averaging. :

[0111]

[0112] in, It is a Transformer encoding module in the time dimension, used to perform temporal updates and fusion of agent-map interaction features of vehicles in continuous historical time steps.

[0113] Through three levels of processing—timestamp-level agent-agent interaction, timestamp-level agent-map interaction, and vehicle-level time update—it is possible to express in greater detail the evolution of behaviors such as following other vehicles, preparing to change lanes, cutting in front, yielding, and overtaking in the historical process.

[0114] The driving intent module will incorporate scene features With dynamic interaction features The data is concatenated and fused, and then output as vertical intent logits, horizontal intent logits, and interaction intent logits respectively through three classification heads. The vertical intent probability is obtained after applying a softmax function. Horizontal Intent Probability and interaction intent probability :

[0115]

[0116]

[0117]

[0118] in, For predicting longitudinal intent, a multilayer perceptron classification head, For predicting lateral intent, a multilayer perceptron classification head, The multilayer perceptron classifier head is used to predict interaction intentions. Vertical intentions include four categories: keep, accelerate, decelerate, and stop. Lateral intentions include six categories: keep lane, dodge left / change lanes left, dodge right / change lanes right, go straight, turn, invalid, or other. Interaction intentions include four categories: yield, cut in front, follow, and overtake. The model multiplies the probabilities of the three types of intentions by the learnable intention embedding matrix to obtain the intention embedding, and then concatenates and projects it with the fused features to obtain the Intent Embedding (intention embedding features).

[0119] This intention embedding serves as input to the multimodal trajectory decoder, ensuring that each candidate trajectory is not only dependent on the geometric history trajectory but also guided by driving semantics, thereby enhancing the discriminability and interpretability between different prediction modalities.

[0120] The risk assessment module is used to perform explicit safety evaluation on each candidate trajectory. This module includes a dynamic traffic risk potential field and a map-constrained risk potential field. The dynamic traffic risk potential field is generated by the trajectory prediction assisted by surrounding vehicles. The closer the candidate trajectory is to the predicted trajectory of surrounding vehicles, the higher the risk potential. The map-constrained risk potential field is generated by the minimum distance from the candidate trajectory to the lane line point set. The farther the candidate trajectory deviates from the drivable lane, the higher the map risk.

[0121] Dynamic traffic risks It can be represented as:

[0122]

[0123] in, For the first The candidate trajectories of the target vehicles are in Location at any given moment For the first Vehicle trajectory prediction with assistance Location at any given moment To appropriately increase the width of the risk field over the prediction period, For all predicted time steps The risk potential energy value is averaged.

[0124] Map constraint risks It can be represented as:

[0125]

[0126] in, Represents the set of candidate trajectory points to valid lane line points. The minimum distance, The safe lane distance threshold, For the map potential field scale parameters. This is a smooth nonlinear activation function used to smoothly convert the "degree of deviation from the lane line" into a map risk score; the greater the deviation, the higher the risk.

[0127] Comprehensive Risk Score for:

[0128]

[0129] in, Map risk weights;

[0130] Then, the original confidence levels are reweighted using the comprehensive risk score:

[0131]

[0132] in, The first output of the decoder The original modal confidence of each candidate trajectory. This is the risk reweighting coefficient. For the first The final modal confidence score of each candidate trajectory after risk reweighting. Through this mechanism, low-risk trajectories that conform to map constraints are more likely to obtain higher final probabilities, while trajectories with high collision risk or deviation from the road structure are automatically suppressed.

[0133] Step 4: Multimodal trajectory decoding;

[0134] The multimodal trajectory decoding module employs K learnable modal queries. Each modal query is fused with the target vehicle's scene features, dynamic interaction features, and intent embedding to form the pattern features of the corresponding candidate trajectory. The decoder does not directly regress future absolute coordinates; instead, it predicts the displacement increment at each future time step. And starting from the last valid historical position of the target vehicle, the data is accumulated to obtain continuous future predicted trajectories:

[0135]

[0136] in, For the first The candidate trajectory in the 1st The position of a future moment The predicted starting point position of the target vehicle. Predicting time steps for the future;

[0137] This decoding method can solve the problem of trajectory discontinuity caused by directly predicting absolute coordinates, making the predicted trajectory smoother and more stable in time. The decoder also outputs the original modal confidence of each candidate trajectory. To support subsequent risk assessment, auxiliary predictions were also made for the future trajectories of all surrounding vehicles, resulting in a set of auxiliary predicted trajectories for surrounding vehicles. The weekly prediction function serves both as auxiliary supervision during the training process and as input to dynamic risk sources in the risk assessment module.

[0138] Step 5: Design the joint training loss function;

[0139] A multi-task joint loss training model is employed for the trajectory prediction model. For the target vehicle's multimodal trajectory, the mean displacement error (ADE) between each candidate trajectory and the true future trajectory is first calculated, and the candidate mode with the smallest ADE is selected as the optimal candidate mode. Then, a smooth L1 trajectory regression loss is calculated for the trajectory corresponding to the optimal candidate mode. The best candidate pattern is used as the classification label to calculate the pattern classification loss. Calculate the surrounding vehicle prediction loss based on the surrounding vehicle's predicted trajectory. The cross-entropy loss is calculated for vertical, horizontal, and interactive intentions respectively, and then summed to obtain the intention prediction loss. The risk assessment loss is obtained by averaging the risk scores of the trajectories corresponding to the best candidate patterns. .

[0140] Total loss for:

[0141]

[0142] in, For trajectory regression loss weights, Weights for pattern classification loss. To assist in predicting loss weights for surrounding vehicles, To predict the loss weights, For risk assessment loss weighting.

[0143] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion, characterized in that, Includes the following steps: Step 1: Fuse the multi-sensor information obtained by the sensors to obtain the agent's historical trajectory information and HD map information. Construct a local coordinate system for the target vehicle based on the target vehicle's current position and heading angle, and unify the vehicle position, lane line points, and speed direction into the local coordinate system. Step 2: The MAE encoder encodes the vehicle trajectory token and the map lane line token respectively. A portion of the vehicle trajectory token and map lane line token are randomly occluded proportionally, so that the encoder only receives the unoccluded tokens. The MAE decoder uses the learnable mask token to recover the complete token set and reconstructs the coordinates of the occluded trajectory and lane line points. The pre-trained encoder weights are obtained based on the occlusion reconstruction loss. The shared scene encoder is initialized using the pre-trained encoder weights. The scene encoder consists of a vehicle temporal encoder, a lane line polyline encoder, and a scene fusion encoder, which encode the vehicle's historical trajectory and map lane lines respectively and fuse them to obtain scene features. Step 3: Perform agent-agent self-attention on all valid vehicles at each historical timestamp, and perform agent-map cross-attention on the vehicle token at each timestamp. Then, perform temporal Transformer updates along the historical time dimension of each vehicle to obtain dynamic interaction features. After fusing the scene features and dynamic interaction features, output the probabilities of vertical intent, horizontal intent, and interaction intent respectively. Multiply the probabilities of the three types of intents with the learnable intent embedding matrix to obtain the intent embedding. Generate a dynamic traffic risk potential field based on the assisted prediction trajectory of surrounding vehicles, and generate a map-constrained risk potential field based on the minimum distance from the candidate trajectory to the lane line point set. Combine these to obtain a comprehensive risk score, and use the comprehensive risk score to reweight the original modality confidence. Step 4: Use K learnable modal queries. Each modal query is fused with the scene features, dynamic interaction features and intent embedding of the target vehicle to form the pattern features of the corresponding candidate trajectory. The decoder predicts the displacement increment of each future time step and accumulates it starting from the last valid historical position of the target vehicle to obtain continuous future predicted trajectories. At the same time, the original modal confidence of each candidate trajectory and the auxiliary predicted trajectory of surrounding vehicles are output. Step 5: Train the trajectory prediction model using multi-task joint loss, which includes trajectory regression loss, pattern classification loss, surrounding vehicle auxiliary prediction loss, intent prediction loss, and risk assessment loss.

2. The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion according to claim 1, characterized in that, In step 1, the current position of the target vehicle is used. and heading angle Based on this, a local coordinate system for the target vehicle is constructed, where, Let x be the x-coordinate of the target vehicle in the global coordinate system at the current moment. Let be the ordinate of the target vehicle in the global coordinate system at the current moment; for any vehicle or lane line point, its position in the global coordinate system is... its local coordinates Calculate as follows: in, This represents the global x-coordinate of a point on a vehicle or lane line. This represents the global vertical coordinate of a point on a vehicle or lane line. Let be a rotation matrix, and its expression is: For velocity vector The rotation is performed in the same way: in, Vehicle speed in global coordinate system Directional components, Vehicle speed in global coordinate system Directional components, This is the velocity vector after rotation to the target vehicle's local coordinate system.

3. The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion according to claim 2, characterized in that, In step 2, occlusion reconstruction loss and pre-trained encoder weights The calculation is as follows: in, Loss weights are used to reconstruct future trajectories. Weighting for reconstructing historical trajectories Weights for lane line reconstruction loss on the map. Losses due to reconstruction of future trajectories Losses due to the reconstruction of historical trajectory Losses due to lane line reconstruction and These are the sets of indices for the obscured vehicle trajectory tokens and the map lane line tokens, respectively. To reconstruct weights for the map, The index set of the obscured vehicle trajectory tokens The true two-dimensional coordinate vector of each trajectory token. for The reconstructed two-dimensional coordinate prediction results The index set of the lane line tokens on the obscured map The true two-dimensional coordinate vector of each lane line token for The predicted two-dimensional coordinates obtained from the reconstruction.

4. The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion according to claim 3, characterized in that, In step 2, the vehicle timing encoder Transformer encoding is performed on the historical state sequence of each vehicle to obtain the time token at each historical moment and the agent token features of the entire trajectory. in, The characteristics of the agent token This refers to the historical trajectory characteristics of vehicles. This represents the number of vehicles participating in the modeling process in the current scene. For historical observation time steps; Lane line polygon encoder Transformer encoding is performed on the sampling point sequence of each lane line to obtain the map lane token features as follows: in, For map lane token features, For map lane line features, This represents the number of lane lines involved in modeling in the current scene. The number of sampling points on each lane line; agent token features and map lane token features The stitched data is then input into a Transformer-based scene fusion encoder. By utilizing the self-attention mechanism, agent and map information are fused globally to obtain fused scene features. : 。 5. The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion according to claim 4, characterized in that, In step 3, the agent-agent interaction characteristics for: in, Indicates the first Vehicle characteristics at each historical time step This indicates a bullish focus operation; Interaction features of agent-map for: Finally, a time-series Transformer update is performed along the historical timeline of each vehicle to create a dynamic interactive state. Dynamic interaction features are obtained through mask averaging. : in, It is a Transformer encoding module in the time dimension, used to perform temporal updates and fusion of agent-map interaction features of vehicles in continuous historical time steps.

6. The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion according to claim 5, characterized in that, In step 3, the longitudinal intent probability Horizontal Intent Probability and interaction intent probability The calculation is as follows: in, For predicting longitudinal intent, a multilayer perceptron classification head, For predicting lateral intent, a multilayer perceptron classification head, For predicting interaction intent, a multilayer perceptron classification head, The function is softmax; longitudinal intents include four categories: keep, accelerate, decelerate, and stop; lateral intents include six categories: keep lane, dodge left / change lanes left, dodge right / change lanes right, go straight, turn, invalid, or others; interactive intents include four categories: yield, cut in front, follow, and overtake.

7. The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion according to claim 6, characterized in that, In step 3, dynamic traffic risk Represented as: in, For the first The candidate trajectories of the target vehicles are in Location at any given moment For the first Vehicle trajectory prediction with assistance Location at any given moment To appropriately increase the width of the risk field over the prediction period, For all predicted time steps The risk potential energy value is averaged. Map constraint risk Represented as: in, Represents the set of candidate trajectory points to valid lane line points. The minimum distance, The safe lane distance threshold, For the map potential field scale parameters. For smoothing nonlinear activation functions; Comprehensive Risk Score for: in, Map risk weights; Then, the original confidence levels are reweighted using the comprehensive risk score: in, The first output of the decoder The original modal confidence of each candidate trajectory. This is the risk reweighting coefficient. For the first The final modal confidence of each candidate trajectory after risk reweighting.

8. The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion according to claim 7, characterized in that, In step 4, the displacement increment for each future time step is predicted. And starting from the last valid historical position of the target vehicle, the data is accumulated to obtain continuous future predicted trajectories: in, For the first The candidate trajectory in the 1st The position of a future moment The predicted starting point position of the target vehicle. Predicting time steps for the future.

9. The vehicle trajectory prediction method based on occlusion reconstruction and intent risk fusion according to claim 8, characterized in that, In step 5, Total loss for: in, For trajectory regression loss weights, Weights for pattern classification loss. To assist in predicting loss weights for surrounding vehicles, To predict the loss weights, For risk assessment loss weighting, For trajectory regression loss, For pattern classification loss, To assist in predicting losses for surrounding vehicles, In order to predict losses, To assess the risk of loss.

Images