A scene-level trajectory prediction method and system based on a conditional diffusion model, medium and device

Abstract

The application discloses a kind of intelligent driving technical field's scene level trajectory prediction method, system, medium and equipment based on condition diffusion model, comprising: obtaining the navigation constraint condition of the agent motion extracted based on the current time acquisition multi-agent trajectory feature and city high-precision map data of ego-vehicle position, input the condition diffusion model trained, output future multi-modal trajectory prediction result.The application improves the generalization ability of model in complex traffic scene by deeply fusing multi-source heterogeneous data, especially in rare scene that training data does not cover, can maintain high prediction reliability, solves the bottleneck that traditional method is difficult to deal with diversified scene;The application uses condition diffusion model, learns multi-modal trajectory distribution through iterative denoising process, combines semantic probability and direction matching probability, effectively constrains predicted trajectory to conform to road topology and traffic rules, reduces the error accumulation effect of long-term prediction.

Description

Technical Field

[0001] This invention relates to a scenario-level trajectory prediction method, system, medium, and device based on a conditional diffusion model, belonging to the field of intelligent driving technology. Background Technology

[0002] With the rapid development of autonomous driving technology, the problem of trajectory prediction for intelligent vehicles in complex traffic environments is becoming increasingly prominent. As a key component of autonomous driving decision-making systems, multi-agent trajectory prediction not only needs to consider the vehicle's own motion state but also accurately predict the behavioral intentions of surrounding vehicles, pedestrians, and other traffic participants. Currently, with the widespread adoption of high-precision positioning technology and environmental perception equipment, in-vehicle systems can acquire multi-dimensional data, including historical trajectories, high-precision maps, and real-time traffic conditions. This provides a data foundation for in-depth research into the movement patterns of traffic participants.

[0003] In the field of intelligent transportation, multi-agent trajectory prediction is essentially a probabilistic estimation problem with multimodal characteristics. Due to the complexity of traffic scenarios and the randomness of participant behavior, the prediction results often present multiple possibilities. Existing research mainly addresses this problem through two technical approaches: one is a deterministic prediction method based on physical motion models, and the other is a probabilistic generation method based on deep learning. While the former is computationally efficient, it struggles to handle complex interaction scenarios; while the latter can generate diverse trajectories, it still has shortcomings in terms of prediction accuracy and physical feasibility.

[0004] Current mainstream trajectory prediction methods face three main technical bottlenecks: First, the use of a single data modality limits the model's adaptability to complex scenarios; second, the cumulative error increases significantly over long periods, causing the trajectory to deviate from actual physical constraints; and most importantly, the predictive reliability of existing models drops drastically when encountering rare scenarios not covered by training data. The root cause of these problems lies in the difficulty of effectively integrating multi-source heterogeneous data and the lack of in-depth analysis of the essential characteristics of traffic scenarios. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a scene-level trajectory prediction method, system, medium and device based on a conditional diffusion model.

[0006] To solve the above-mentioned technical problems, the present invention is implemented using the following technical solution.

[0007] In a first aspect, this invention discloses a scene-level trajectory prediction method based on a conditional diffusion model, comprising:

[0008] The system acquires the trajectory features of multiple agents collected at the current moment and the navigation constraints of agent motion extracted based on the vehicle's location and high-precision urban map data. It inputs these into a trained conditional diffusion model and outputs the future multimodal trajectory prediction results. The multiple agents are all traffic participants within a preset spatial range.

[0009] The training of the conditional diffusion model includes:

[0010] Collect historical trajectory data of multiple agents in real traffic scenarios, and preprocess the historical trajectory data of the multiple agents to generate standardized historical trajectory features;

[0011] Extract the historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, with the vehicle as the center.

[0012] Based on the vehicle's location and high-precision urban map data, navigation constraints for the agent's movement are extracted;

[0013] A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's motion, and the vehicle's planned path information as inputs. The trajectory distribution pattern is learned through an iterative denoising training process. After the iteration terminates, the trained conditional diffusion model is obtained.

[0014] Furthermore, the step of collecting multi-agent historical trajectory data from real traffic scenarios and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features includes:

[0015] The historical trajectory data of multiple agents in real traffic scenarios is transformed from the world coordinate system to a local coordinate system with the vehicle's center of gravity as the origin.

[0016] The historical trajectory data of the multi-agents after the coordinate system transformation is synchronized in the time domain, and the sampling timestamps of the historical trajectory data of each agent are unified by linear interpolation.

[0017] A trajectory interpolation algorithm based on fifth-order polynomials is used to smooth and enhance the historical trajectory data of each agent after time-domain synchronization processing, construct a continuous and differentiable trajectory sample space, and obtain standardized historical trajectory features.

[0018] Furthermore, the step of extracting the historical movement trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, centered on the vehicle, includes:

[0019] Based on the type of traffic participant, the circular sensing area is dynamically adjusted with the vehicle as the center to filter out the standardized historical trajectory features of the traffic participants.

[0020] The method of dynamically adjusting the circular sensing area centered on the vehicle based on the type of traffic participant includes:

[0021] For motor vehicle traffic participants, a circular sensing area with a radius of R1 is set up;

[0022] For non-motorized traffic participants, a circular sensing area with a radius of R2 is set up, where R2 <R1;

[0023] For pedestrian traffic participants, a circular sensing area with a radius of R3 is set, where R3 <R2。

[0024] Furthermore, the process of extracting navigation constraints for the agent's movement based on the vehicle's location and high-precision urban map data includes:

[0025] With the current position of the vehicle as the center, a dynamic capture area with a radius of D is established. The rasterized map data and vector map data within this area are acquired simultaneously, and a transformation matrix between the raster coordinate system and the global coordinate system is established.

[0026] Semantic analysis is performed on the captured raster map data to extract structured environmental information, including the boundaries of drivable areas, the location of pedestrian crossings, zebra crossing areas, and various prohibited areas, which serves as environmental constraints for trajectory prediction.

[0027] For each agent's current location, a nearest neighbor search is performed in the vector map data to determine the lane centerline associated with each agent;

[0028] The environmental constraints and the lane centerline associated with each agent are used as navigation constraints for the agent's movement.

[0029] Furthermore, the construction of a conditional diffusion model, which takes as input the historical trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's movement, and the vehicle's planned path information, learns the trajectory distribution pattern through an iterative denoising training process. Upon reaching the iteration termination condition, a trained conditional diffusion model is obtained, including...

[0030] A training set is constructed based on multiple driving scenarios of the autonomous vehicle. Each driving scenario in the training set...

[0031] This includes the motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent agent's motion, and the vehicle's planned path information;

[0032] Set the number of iteration rounds and the learning rate, and train the conditional diffusion model using a mixed precision algorithm based on the training set to finally obtain the trained conditional diffusion model;

[0033] Each round of iterative training includes:

[0034] The state matrix of neighboring agents centered on the vehicle is analyzed using an MLP-Mixer network. S neighbor Temporal feature extraction is performed to obtain the agent's trajectory features, where... , T h Indicates the time step in history. Represents the set of real numbers;

[0035] The discrete point sequence composed of the left boundary, right boundary and center line S Road Inputting the data into a lightweight CNN network extracts lane geometry features, resulting in lane boundary features. , P This indicates the number of sampling points for each lane. D lane Includes coordinate and angle information;

[0036] Using a pre-trained EfficientNet-B0 as the backbone network, a three-channel semantic map containing drivable regions, obstacle regions, and pedestrian regions is generated. S map Multi-scale feature extraction is performed, and spatial information is fused using a feature pyramid network to obtain raster map features. , H , W These are the height and width of the feature map, respectively;

[0037] The agent trajectory features and lane boundary features are concatenated along the channel dimension, and relative position information is injected through a spatial location embedding layer to form a joint representation feature. The relative coordinates in physical space are mapped to the spatial location of the feature map using the following formula:

[0038] ;

[0039] in, Indicates the mapped position. x , y Indicates the original position. p This indicates the number of pixels per meter. d This represents the map downsampling ratio;

[0040] The semantic features extracted from the corresponding positions in the grid map feature map are used as the Key and Value in the multi-head cross attention mechanism. The fusion features of the agent and the lane boundary are used as the Query in the multi-head cross attention. Information alignment and cross-modal fusion are performed through multi-head cross attention, and a unified high-dimensional scene representation is output and input into the cross attention module of the DiT model.

[0041] The centerline trajectory associated with the agent S navigation The navigation semantic vector is generated after processing by a cross-height convolution module and compression by an adaptive pooling layer. ;

[0042] The navigation semantic vector is added to the noisy step features of the conditional diffusion model to obtain the navigation constraints of the agent's motion. The navigation constraints of the agent's motion are then fused with the trajectory features through the adaptive normalization module of the DiT model.

[0043] The construction of the noisy step features of the conditional diffusion model includes:

[0044] Gaussian noise is applied to the trajectory to be predicted based on a linear scheduling strategy to obtain a noisy trajectory. Y N , is represented as:

[0045] ;

[0046] in, For the first k Step-by-step cumulative noise attenuation coefficient, Y 0 represents the actual trajectory. Indicates Gaussian noise. Represents a normal distribution;

[0047] Predicting noise using a DiT decoder loss function L for:

[0048] ;

[0049] in, Expressing expectations, Indicates DiT network, Indicates contextual information, denoted as the square of the Euclidean norm.

[0050] Furthermore, the output of the future multimodal trajectory prediction result includes:

[0051] The trajectory features of the multi-agent agents at the current moment A t and navigation constraints C Input the trained conditional diffusion model and perform the following inference:

[0052] Multi-agent trajectory features A t and navigation constraints C Standardized to a 256-dimensional vector;

[0053] Based on a preset number of iterations, the DPM-Solver algorithm is used for multiple skip-step denoising to generate multi-mode denoising.

[0054] The multimodal trajectory distribution is used to output future multimodal trajectory prediction results. The denoising formula is as follows:

[0055] ;

[0056] In the formula, Y k-1 This represents the updated output or state variable, in the [number]th [year]. k The result after step -1, Y k Indicates the current step k Input or state variables, α k Represents a time step k The relevant parameters.

[0057] Secondly, this invention also discloses a scene-level trajectory prediction system based on a conditional diffusion model, comprising:

[0058] The acquisition module is used to acquire the trajectory features of the multi-agents collected at the current moment and the navigation constraints of the agent's motion extracted based on the vehicle's position and high-precision urban map data, and input them into the trained conditional diffusion model; the multi-agents are all traffic participants within a preset spatial range;

[0059] The processing module is used to output future multimodal trajectory prediction results based on the multi-agent trajectory features collected at the current moment and the navigation constraints.

[0060] The training of the conditional diffusion model includes:

[0061] Collect historical trajectory data of multiple agents in real traffic scenarios, and preprocess the historical trajectory data of the multiple agents to generate standardized historical trajectory features;

[0062] Extract the historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, with the vehicle as the center.

[0063] Based on the vehicle's location and high-precision urban map data, navigation constraints for the agent's movement are extracted;

[0064] A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's motion, and the vehicle's planned path information as inputs. The trajectory distribution pattern is learned through an iterative denoising training process. After the iteration terminates, the trained conditional diffusion model is obtained.

[0065] Thirdly, the present invention also discloses a computer-readable storage medium for storing one or more programs, characterized in that the one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the method of the first aspect.

[0066] Fourthly, the present invention also discloses a computer device, comprising,

[0067] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing the method of the first aspect.

[0068] The beneficial effects achieved by this invention are as follows:

[0069] To address the limitation of adaptability caused by a single data modality in multi-agent trajectory prediction, this invention deeply integrates multi-source heterogeneous data, including historical trajectories, high-precision maps, and planned path information, and uses a cross-attention mechanism to generate unified conditional features. This improves the model's generalization ability in complex traffic scenarios, especially in rare scenarios not covered by training data, and maintains high prediction reliability, thus solving the bottleneck of traditional methods in dealing with diverse scenarios.

[0070] To address the challenge of trajectory deviation from physical constraints caused by accumulated errors in long-term predictions, this invention employs a conditional diffusion model. Through an iterative denoising process, it learns the multimodal trajectory distribution and combines semantic probability and direction matching probability to effectively constrain the predicted trajectory to conform to road topology and traffic rules, thereby reducing the cumulative effect of errors in long-term predictions.

[0071] To address the high demands of autonomous driving systems on real-time performance and computational efficiency, this invention introduces the DPM-Solver fast denoising algorithm, which reduces computational complexity from low to high by skipping steps in denoising. Down to It significantly improves inference speed while maintaining high accuracy in multimodal prediction, providing an efficient and robust trajectory prediction solution for intelligent transportation and autonomous driving. Attached Figure Description

[0072] Figure 1 This is a flowchart of the method of the present invention;

[0073] Figure 2 This is the model prediction framework of the present invention;

[0074] Figure 3 This is a diagram illustrating the trajectory prediction results of an autonomous vehicle according to an embodiment of the present invention. Detailed Implementation

[0075] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0076] Example 1: This example introduces a scene-level trajectory prediction method based on a conditional diffusion model, including:

[0077] The system acquires the trajectory features of multiple agents collected at the current moment and the navigation constraints of agent motion extracted based on the vehicle's location and high-precision urban map data. It inputs these into a trained conditional diffusion model and outputs the future multimodal trajectory prediction results. The multiple agents are all traffic participants within a preset spatial range.

[0078] The training of the conditional diffusion model includes:

[0079] Collect historical trajectory data of multiple agents in real traffic scenarios, and preprocess the historical trajectory data of the multiple agents to generate standardized historical trajectory features;

[0080] Extract the historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, with the vehicle as the center.

[0081] Based on the vehicle's location and high-precision urban map data, navigation constraints for the agent's movement are extracted;

[0082] A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's motion, and the vehicle's planned path information as inputs. The trajectory distribution pattern is learned through an iterative denoising training process. After the iteration terminates, the trained conditional diffusion model is obtained.

[0083] The process of collecting multi-agent historical trajectory data from real traffic scenarios and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features includes:

[0084] The historical trajectory data of multiple agents in real traffic scenarios is transformed from the world coordinate system to a local coordinate system with the vehicle's center of gravity as the origin.

[0085] The historical trajectory data of the multi-agents after the coordinate system transformation is synchronized in the time domain, and the sampling timestamps of the historical trajectory data of each agent are unified by linear interpolation.

[0086] A trajectory interpolation algorithm based on fifth-order polynomials is used to smooth and enhance the historical trajectory data of each agent after time-domain synchronization processing, construct a continuous and differentiable trajectory sample space, and obtain standardized historical trajectory features.

[0087] The step of extracting the historical movement trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, centered on the vehicle, includes:

[0088] Based on the type of traffic participant, the circular sensing area is dynamically adjusted with the vehicle as the center to filter out the standardized historical trajectory features of the traffic participants.

[0089] The method of dynamically adjusting the circular sensing area centered on the vehicle based on the type of traffic participant includes:

[0090] For motor vehicle traffic participants, a circular sensing area with a radius of R1 is set up;

[0091] For non-motorized traffic participants, a circular sensing area with a radius of R2 is set up, where R2 <R1;

[0092] For pedestrian traffic participants, a circular sensing area with a radius of R3 is set, where R3 <R2。

[0093] The navigation constraints for the agent's movement, extracted based on the vehicle's location and high-precision urban map data, include:

[0094] With the current position of the vehicle as the center, a dynamic capture area with a radius of D is established. The rasterized map data and vector map data within this area are acquired simultaneously, and a transformation matrix between the raster coordinate system and the global coordinate system is established.

[0095] Semantic analysis is performed on the captured raster map data to extract structured environmental information, including the boundaries of drivable areas, the location of pedestrian crossings, zebra crossing areas, and various prohibited areas, which serves as environmental constraints for trajectory prediction.

[0096] For each agent's current location, a nearest neighbor search is performed in the vector map data to determine the lane centerline associated with each agent;

[0097] The environmental constraints and the lane centerline associated with each agent are used as navigation constraints for the agent's movement.

[0098] The construction of a conditional diffusion model, which takes the historical trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's movement, and the vehicle's planned path information as input, learns the trajectory distribution pattern through an iterative denoising training process. Upon reaching the iteration termination condition, the trained conditional diffusion model is obtained, including:

[0099] A training set is constructed based on multiple driving scenarios of the autonomous vehicle. Each driving scenario in the training set...

[0100] This includes the motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent agent's motion, and the vehicle's planned path information;

[0101] Set the number of iteration rounds and the learning rate, and train the conditional diffusion model using a mixed precision algorithm based on the training set to finally obtain the trained conditional diffusion model;

[0102] Each round of iterative training includes:

[0103] The state matrix of neighboring agents centered on the vehicle is analyzed using an MLP-Mixer network. S neighbor Temporal feature extraction is performed to obtain the agent's trajectory features, where... , T h Indicates the time step in history. Represents the set of real numbers;

[0104] The discrete point sequence composed of the left boundary, right boundary and center line S Road Inputting the data into a lightweight CNN network extracts lane geometry features, resulting in lane boundary features. , P This indicates the number of sampling points for each lane. D lane Includes coordinate and angle information;

[0105] Using a pre-trained EfficientNet-B0 as the backbone network, a three-channel semantic map containing drivable regions, obstacle regions, and pedestrian regions is generated. S map Multi-scale feature extraction is performed, and spatial information is fused using a feature pyramid network to obtain raster map features. , H , W These are the height and width of the feature map, respectively;

[0106] The agent trajectory features and lane boundary features are concatenated along the channel dimension, and relative position information is injected through a spatial location embedding layer to form a joint representation feature. The relative coordinates in physical space are mapped to the spatial location of the feature map using the following formula:

[0107] ;

[0108] in, Indicates the mapped position. x , y Indicates the original position. p This indicates the number of pixels per meter. d This represents the map downsampling ratio;

[0109] The semantic features extracted from the corresponding positions in the grid map feature map are used as the Key and Value in the multi-head cross attention mechanism. The fusion features of the agent and the lane boundary are used as the Query in the multi-head cross attention. Information alignment and cross-modal fusion are performed through multi-head cross attention, and a unified high-dimensional scene representation is output and input into the cross attention module of the DiT model.

[0110] The centerline trajectory associated with the agent S navigation The navigation semantic vector is generated after processing by a cross-height convolution module and compression by an adaptive pooling layer. ;

[0111] The navigation semantic vector is added to the noisy step features of the conditional diffusion model to obtain the navigation constraints of the agent's motion. The navigation constraints of the agent's motion are then fused with the trajectory features through the adaptive normalization module of the DiT model.

[0112] The construction of the noisy step features of the conditional diffusion model includes:

[0113] Gaussian noise is applied to the trajectory to be predicted based on a linear scheduling strategy to obtain a noisy trajectory. Y N , is represented as:

[0114] ;

[0115] in, For the first k Step-by-step cumulative noise attenuation coefficient, Y 0 represents the actual trajectory. Indicates Gaussian noise. Represents a normal distribution;

[0116] Predicting noise using a DiT decoder loss function L for:

[0117] ;

[0118] in, Expressing expectations, Indicates DiT network, Indicates contextual information, denoted as the square of the Euclidean norm.

[0119] Furthermore, the output of the future multimodal trajectory prediction result includes:

[0120] The trajectory features of the multi-agent agents at the current moment A t and navigation constraintsC Input the trained conditional diffusion model and perform the following inference:

[0121] Multi-agent trajectory features A t and navigation constraints C Standardized to a 256-dimensional vector;

[0122] Based on a preset number of iterations, the DPM-Solver algorithm is used for multiple skip-step denoising to generate multi-mode denoising.

[0123] The multimodal trajectory distribution is used to output future multimodal trajectory prediction results. The denoising formula is as follows:

[0124] ;

[0125] In the formula, Y k-1 This represents the updated output or state variable, in the [number]th [year]. k The result after step -1, Y k Indicates the current step k Input or state variables, α k Represents a time step k The relevant parameters.

[0126] Example 2: This example introduces a scene-level trajectory prediction method based on a conditional diffusion model, such as... Figure 1 As shown, it includes the following steps:

[0127] Step 1: Collect and preprocess multi-agent trajectory data;

[0128] Multi-agent trajectory data in real traffic scenarios is collected using vehicle-mounted sensors (including but not limited to cameras and millimeter-wave radar) and road monitoring equipment. Data fields include a unique agent ID, a sampling timestamp accurate to milliseconds, and planar coordinates. x , y ),speed v and orientation θ To ensure data quality, the preprocessing procedure is as follows:

[0129] Format conversion: The raw trajectory data is converted to a standard format. This represents a set of intelligent agents. Each intelligent agent... Historical state is defined as ,in, This represents the length of the historical time and the state at each time step. Includes planar coordinates ( x , y ),speedv Orientation θ Three-dimensional dimensions ( l , w , h One-hot encoding of acceleration and agent type. Each agent in the future... The trajectory at each time step is ,in, Maintaining the historical status quo Same coordinate system reference.

[0130] 1) Coordinate Standardization: The trajectory points in the world coordinate system are converted to a local coordinate system with the vehicle's center of mass as the origin using a transformation matrix. The transformation formula is as follows:

[0131] ;

[0132] in, This refers to the vehicle's position and orientation.

[0133] 2) Time-domain synchronization: The sampling time interval is unified through linear interpolation. The interpolation formula is:

[0134] .

[0135] 3) Trajectory Smoothing: A fifth-order polynomial interpolation algorithm is used to generate a continuously differentiable trajectory. The polynomial is defined as:

[0136] ;

[0137] Optimize coefficients using the least squares method a 0 to a 5. Ensure the trajectory is smooth and has minimal deviation from the sampling points.

[0138] In this embodiment, the sampling frequency is 10Hz, and trajectory data from 1000 scenarios are processed. Each agent contains trajectory points at 40 time steps, generating a standardized trajectory feature set.

[0139] Step 2: Extract trajectory features of traffic participants;

[0140] With the position of the car A dynamic sensing area is constructed centered on a specific traffic participant, with sensing radii set for different traffic participants: motor vehicles. R 1 = 30 meters, non-motorized vehicles R 2 = 10 meters, pedestrians R 3 = 10 meters. Calculation of the intelligent agent. i Euclidean distance from the vehicle:

[0141] ;

[0142] like, (k If the type of agent corresponds to the type of agent, then its trajectory data is retained.

[0143] In this embodiment, it is assumed that the perception area contains 20 intelligent agents (10 motor vehicles, 5 non-motor vehicles, and 5 pedestrians). Their trajectory features are extracted to form a set. .

[0144] Step 3: Extract local environmental constraint features;

[0145] Based on the vehicle's location, a radius is extracted from the high-precision map. D For a local area of ​​80 meters, raster and vector map data are acquired. The processing procedure is as follows:

[0146] 1) Map Spatial Indexing: Constructing the Graph Structure G < V , E >, among which V It is a road intersection. E For the road centerline, use R-tree technology to generate a spatial index.

[0147] 2) Coordinate Transformation: Define the transformation matrix between the raster coordinate system and the global coordinate system. M :

[0148] .

[0149] 3) Semantic parsing: Semantic segmentation of the grid map is performed using a convolutional neural network (CNN) to extract features such as drivable areas and restricted areas, and semantic probabilities. for:

[0150] .

[0151] 4) Topological constraints: Determine the road centerlines associated with the agents through nearest neighbor search, and calculate the direction angle. Define the direction matching probability :

[0152] .

[0153] 5) In this embodiment, a constraint feature set containing 8 road centerlines and 10 semantic regions (including 2 pedestrian crossings) was extracted. C .

[0154] Step 4: Construct a conditional diffusion model, such as Figure 2 As shown, the conditional diffusion model includes:

[0155] Encoding stage: Multi-source information processing;

[0156] The encoding stage processes the perceived data in parallel through a dual-branch structure, dividing it into dynamic objects and static environmental elements to form a multimodal scene representation, thereby capturing historical motion trajectories and environmental semantic information.

[0157] Dynamic objects, i.e., the historical state of neighboring agents, are collected in a self-centered coordinate system and defined as follows: These states are encoded using an MLP-Mixer network to capture temporal motion patterns.

[0158] The static environment includes lane boundaries and a semantic map. Lane boundaries (including left, right, and center lines) are represented as a sequence of discrete points. ,in, P This indicates the number of points on each line. D lane It contains coordinate and orientation information. These features are extracted using a lightweight convolutional neural network (CNN). The semantic map contains three layers of information—drivable area, obstacle area, and pedestrian walkway—represented as follows: ,in, H and W These represent the height and width of the map, respectively. Multi-scale semantic features are extracted using a pre-trained EfficientNet-B0 backbone network, and spatial information is fused using a Feature Pyramid Network (FPN).

[0159] Navigation information is derived from the nearest lane centerline, represented as This information is encoded through height-oriented convolutional modules, adaptive pooling, and fully connected layers to form a compact navigation vector. This vector provides a directional prior for trajectory prediction.

[0160] Cross-modal fusion is achieved through a spatially enhanced fusion mechanism. The encoded features of the agent and lane boundaries are concatenated and spatial location information is added. The interaction between the agent and the lane is modeled using a self-attention mechanism. Subsequently, based on relative coordinates (… x , y The fused features are projected onto the semantic map feature map, and cross-modal fusion is achieved through a multi-head cross-attention mechanism to generate a unified scene representation. This representation encapsulates spatiotemporal interaction and environmental context for trajectory prediction.

[0161] Decoding phase: Trajectory generation based on conditional diffusion model;

[0162] The decoding stage employs a diffusion-based transformer (DiT) module, which generates trajectories through a conditional diffusion process and utilizes features from the encoding stage to generate accurate future trajectories.

[0163] During the forward process, predefined variance scheduling is applied. Gaussian noise is gradually injected. Diffusion time step t The disturbance trajectory is represented as Through iterative noise injection, the standard Gaussian distribution is eventually approximated.

[0164] In the reverse process, starting from a standard Gaussian distribution, the future trajectory is generated by solving the inverse-time diffusion ordinary differential equation (ODE). The generation process is based on conditional information. The guidance integrates scene representation and navigation priors.

[0165] By employing a cross-attention mechanism, spatiotemporal interaction features and semantic environment context are embedded into the non-equilibrium diffusion process, ensuring that the generated trajectory is jointly influenced by motion patterns and environmental constraints.

[0166] Navigation vectors The diffusion process is dynamically injected through the Adaptive Layer Normalization (AdaLN) mechanism, which promotes deep coupling between navigation information and trajectory evolution, ensuring that the predicted trajectory is aligned with the lane centerline and has target consistency.

[0167] The specific steps are as follows:

[0168] Step 4-1: Multimodal scene feature encoding:

[0169] 1) Neighbor agent state encoding: a state matrix of neighbor agents centered on the vehicle. Temporal feature extraction is performed using an MLP-Mixer network, where... T h Indicates the historical time step;

[0170] 2) Lane boundary feature encoding: a discrete point sequence composed of the left boundary, right boundary, and centerline. Input a lightweight CNN network to extract lane geometry features, where... P This indicates the number of sampling points for each lane. D lane Includes coordinate and angle information;

[0171] 3) Raster map feature encoding: Using a pre-trained EfficientNet-B0 as the backbone network, the three-channel semantic map containing drivable areas, obstacle areas, and pedestrian areas is encoded. Multi-scale feature extraction is performed, and spatial information fusion is achieved through a Feature Pyramid Network (FPN). H , W These are the height and width of the feature map, respectively;

[0172] Step 4-2: Spatial Enhancement Fusion Mechanism

[0173] 1) Primary Feature Fusion: The agent's trajectory features and lane boundary features are concatenated along the channel dimension. Relative position information is injected through a spatial position embedding layer. The calculation formula is as follows:

[0174] ;

[0175] in, p This indicates the number of pixels per meter. d This represents the map downsampling ratio.

[0176] 2) Cross-modal attention fusion: Features sampled from the map feature map are used as keys and values, combined with fused features from the agent and lane boundaries (as queries), and information alignment and cross-modal fusion are achieved through multi-head cross-attention. The final output is a unified high-dimensional scene representation for downstream prediction tasks.

[0177] Step 4-3: Navigation condition injection:

[0178] 1) Navigation trajectory encoding: Associating the agent with the centerline trajectory The navigation semantic vector is generated by processing through a cross-height convolution module and then compressed by an adaptive pooling layer.

[0179] Conditional fusion: The navigation vector is concatenated with the time-step features of the diffusion model, and the features are adjusted by the adaptive normalization module (AdaLN).

[0180] Step 4-4: Training:

[0181] 1) Noise addition: Gaussian noise is added based on a linear scheduling strategy. The noise trajectory is as follows:

[0182] ;

[0183] in, For the first k Step-by-step cumulative noise attenuation coefficient, Y 0 represents the actual trajectory.

[0184] 2) Noise Reduction Training: Predicting noise using a DiT decoder The loss function is:

[0185] ;

[0186] The model was optimized using a mixed-precision algorithm, iterated for 50 epochs, and the learning rate was set to 2*10. -4 .

[0187] In this embodiment, the model input dimension is 256, the training dataset contains 700 driving scenarios, the validation set contains 150 validation trajectory scenarios, and the mean squared error of the model on the validation set converges to 0.05.

[0188] Step 5: Trajectory prediction inference;

[0189] Trajectory characteristics at the current moment A t and map constraint features C Input the trained model and execute inference:

[0190] 1) Input preparation: A t and C It is standardized to a 256-dimensional vector.

[0191] 2) Fast denoising: The DPM-Solver algorithm is used for multi-step denoising. The denoising formula is as follows:

[0192] ;

[0193] In the formula, Y k-1 This represents the updated output or state variable, in the [number]th [year]. k The result after step -1, Y k Indicates the current step k Input or state variables, α k Represents a time step k The relevant parameters, which are usually part of noise scheduling, control the noise level at each step.

[0194] Output sampling: Sample from the denoised result to generate a multimodal trajectory distribution, outputting a high-probability trajectory, such as... Figure 3 As shown.

[0195] Example 3, based on the same inventive concept as Example 1, introduces a scene-level trajectory prediction system based on a conditional diffusion model, including:

[0196] The acquisition module is used to acquire the trajectory features of the multi-agents collected at the current moment and the navigation constraints of the agent's motion extracted based on the vehicle's position and high-precision urban map data, and input them into the trained conditional diffusion model; the multi-agents are all traffic participants within a preset spatial range;

[0197] The processing module is used to output future multimodal trajectory prediction results based on the multi-agent trajectory features collected at the current moment and the navigation constraints.

[0198] The training of the conditional diffusion model includes:

[0199] Collect historical trajectory data of multiple agents in real traffic scenarios, and preprocess the historical trajectory data of the multiple agents to generate standardized historical trajectory features;

[0200] Extract the historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, with the vehicle as the center.

[0201] Based on the vehicle's location and high-precision urban map data, navigation constraints for the agent's movement are extracted;

[0202] A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's motion, and the vehicle's planned path information as inputs. The trajectory distribution pattern is learned through an iterative denoising training process. After the iteration terminates, the trained conditional diffusion model is obtained.

[0203] Example 4, based on the same inventive concept as Example 2, describes a computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method described in Example 1.

[0204] Example 5, based on the same inventive concept as Example 1, describes a computer device, including,

[0205] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing the method described in Embodiment 1.

[0206] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0207] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0208] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0209] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0210] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A scene-level trajectory prediction method based on a conditional diffusion model, characterized in that, include: The system acquires the trajectory features of multiple agents collected at the current moment and the navigation constraints of agent motion extracted based on the vehicle's location and high-precision urban map data. It inputs these into a trained conditional diffusion model and outputs the future multimodal trajectory prediction results. The multiple agents are all traffic participants within a preset spatial range. The training of the conditional diffusion model includes: Collect historical trajectory data of multiple agents in real traffic scenarios, and preprocess the historical trajectory data of the multiple agents to generate standardized historical trajectory features; Extract the historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, with the vehicle as the center. Based on the vehicle's location and high-precision urban map data, navigation constraints for the agent's movement are extracted; A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's motion, and the vehicle's planned path information as input conditions. The trajectory distribution pattern is learned through an iterative denoising training process. After the iteration termination condition is reached, the trained conditional diffusion model is obtained. The navigation constraints for the agent's movement, extracted based on the vehicle's location and high-precision urban map data, include: With the current position of the vehicle as the center, a dynamic capture area with a radius of D is established. The rasterized map data and vector map data within this area are acquired simultaneously, and a transformation matrix between the raster coordinate system and the global coordinate system is established. Semantic analysis is performed on the captured raster map data to extract structured environmental information, including the boundaries of drivable areas, the location of pedestrian crossings, zebra crossing areas, and various prohibited areas, which serves as environmental constraints for trajectory prediction. For each agent's current location, a nearest neighbor search is performed in the vector map data to determine the lane centerline associated with each agent; The environmental constraints and the lane centerline associated with each agent are used as navigation constraints for the agent's movement. The construction of a conditional diffusion model, which takes the historical trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's movement, and the vehicle's planned path information as input, learns the trajectory distribution pattern through an iterative denoising training process. Upon reaching the iteration termination condition, the trained conditional diffusion model is obtained, including: A training set is constructed based on multiple driving scenarios of the autonomous vehicle. Each driving scenario in the training set... This includes the motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent agent's motion, and the vehicle's planned path information; Set the number of iteration rounds and the learning rate, and train the conditional diffusion model using a mixed precision algorithm based on the training set to finally obtain the trained conditional diffusion model; Each round of iterative training includes: The state matrix of neighboring agents centered on the vehicle is analyzed using an MLP-Mixer network. S neighbor Temporal feature extraction is performed to obtain the agent's trajectory features, where... , T h Indicates the time step in history. Represents the set of real numbers; The discrete point sequence composed of the left boundary, right boundary and center line S Road Inputting the data into a lightweight CNN network extracts lane geometry features, resulting in lane boundary features. , P This indicates the number of sampling points for each lane. D lane Includes coordinate and angle information; Using a pre-trained EfficientNet-B0 as the backbone network, a three-channel semantic map containing drivable regions, obstacle regions, and pedestrian regions is generated. S map Multi-scale feature extraction is performed, and spatial information is fused using a feature pyramid network to obtain raster map features. , H , W These are the height and width of the feature map, respectively; The agent trajectory features and lane boundary features are concatenated along the channel dimension, and relative position information is injected through a spatial location embedding layer to form a joint representation feature. The relative coordinates in physical space are mapped to the spatial location of the feature map using the following formula: ； in, Indicates the mapped position. x , y Indicates the original position. p This indicates the number of pixels per meter. d This represents the map downsampling ratio; The semantic features extracted from the corresponding positions in the grid map feature map are used as the Key and Value in the multi-head cross attention mechanism. The fusion features of the agent and the lane boundary are used as the Query in the multi-head cross attention. Information alignment and cross-modal fusion are performed through multi-head cross attention, and a unified high-dimensional scene representation is output and input into the cross attention module of the DiT model. The centerline trajectory associated with the agent S navigation The navigation semantic vector is generated after processing by a cross-height convolution module and compression by an adaptive pooling layer. ; The navigation semantic vector is added to the noisy step features of the conditional diffusion model to obtain the navigation constraints of the agent's motion. The navigation constraints of the agent's motion are then fused with the trajectory features through the adaptive normalization module of the DiT model. The construction of the noisy step features of the conditional diffusion model includes: Gaussian noise is applied to the trajectory to be predicted based on a linear scheduling strategy to obtain a noisy trajectory. Y N , is represented as: ； in, For the first k Step-by-step cumulative noise attenuation coefficient, Y 0 represents the actual trajectory. Indicates Gaussian noise. Represents a normal distribution; Predicting noise using a DiT decoder loss function L for: ； in, Expressing expectations, Indicates DiT network, Indicates contextual information, denoted as the square of the Euclidean norm.

2. The scene-level trajectory prediction method based on the conditional diffusion model according to claim 1, characterized in that, The process of collecting multi-agent historical trajectory data from real traffic scenarios and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features includes: The historical trajectory data of multiple agents in real traffic scenarios is transformed from the world coordinate system to a local coordinate system with the vehicle's center of gravity as the origin. The historical trajectory data of the multi-agents after the coordinate system transformation is synchronized in the time domain, and the sampling timestamps of the historical trajectory data of each agent are unified by linear interpolation. A trajectory interpolation algorithm based on fifth-order polynomials is used to smooth and enhance the historical trajectory data of each agent after time-domain synchronization processing, construct a continuous and differentiable trajectory sample space, and obtain standardized historical trajectory features.

3. The scene-level trajectory prediction method based on the conditional diffusion model according to claim 1, characterized in that, The step of extracting the historical movement trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, centered on the vehicle, includes: Based on the type of traffic participant, the circular sensing area is dynamically adjusted with the vehicle as the center to filter out the standardized historical trajectory features of the traffic participants. The method of dynamically adjusting the circular sensing area centered on the vehicle based on the type of traffic participant includes: For motor vehicle traffic participants, a circular sensing area with a radius of R1 is set up; For non-motorized traffic participants, a circular sensing area with a radius of R2 is set up, where R2 <R1; For pedestrian traffic participants, a circular sensing area with a radius of R3 is set, where R3 <R2。 4. The scene-level trajectory prediction method based on the conditional diffusion model according to claim 1, characterized in that, The output of the future multimodal trajectory prediction results includes: The trajectory features of the multi-agent agents at the current moment A t and navigation constraints C Input the trained conditional diffusion model and perform the following inference: Multi-agent trajectory features A t and navigation constraints C Standardized to a 256-dimensional vector; Based on a preset number of iterations, the DPM-Solver algorithm is used for multiple skip-step denoising to generate multi-mode denoising. The multimodal trajectory distribution is used to output future multimodal trajectory prediction results. The denoising formula is as follows: ； In the formula, Y k-1 This represents the updated output or state variable, in the [number]th [year]. k The result after step -1, Y k Indicates the current step k Input or state variables, α k Represents a time step k The relevant parameters.

5. A scene-level trajectory prediction system based on a conditional diffusion model, characterized in that, include: The acquisition module is used to acquire the trajectory features of the multi-agents collected at the current moment and the navigation constraints of the agent's motion extracted based on the vehicle's position and high-precision urban map data, and input them into the trained conditional diffusion model; the multi-agents are all traffic participants within a preset spatial range; The processing module is used to output future multimodal trajectory prediction results based on the multi-agent trajectory features collected at the current moment and the navigation constraints. The training of the conditional diffusion model includes: Collect historical trajectory data of multiple agents in real traffic scenarios, and preprocess the historical trajectory data of the multiple agents to generate standardized historical trajectory features; Extract the historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features, with the vehicle as the center. Based on the vehicle's location and high-precision urban map data, navigation constraints for the agent's movement are extracted; A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's motion, and the vehicle's planned path information as input conditions. The trajectory distribution pattern is learned through an iterative denoising training process. After the iteration termination condition is reached, the trained conditional diffusion model is obtained. The navigation constraints for the agent's movement, extracted based on the vehicle's location and high-precision urban map data, include: With the current position of the vehicle as the center, a dynamic capture area with a radius of D is established. The rasterized map data and vector map data within this area are acquired simultaneously, and a transformation matrix between the raster coordinate system and the global coordinate system is established. Semantic analysis is performed on the captured raster map data to extract structured environmental information, including the boundaries of drivable areas, the location of pedestrian crossings, zebra crossing areas, and various prohibited areas, which serves as environmental constraints for trajectory prediction. For each agent's current location, a nearest neighbor search is performed in the vector map data to determine the lane centerline associated with each agent; The environmental constraints and the lane centerline associated with each agent are used as navigation constraints for the agent's movement. The construction of a conditional diffusion model, which takes the historical trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the agent's movement, and the vehicle's planned path information as input, learns the trajectory distribution pattern through an iterative denoising training process. Upon reaching the iteration termination condition, the trained conditional diffusion model is obtained, including: A training set is constructed based on multiple driving scenarios of the autonomous vehicle. Each driving scenario in the training set... This includes the motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent agent's motion, and the vehicle's planned path information; Set the number of iteration rounds and the learning rate, and train the conditional diffusion model using a mixed precision algorithm based on the training set to finally obtain the trained conditional diffusion model; Each round of iterative training includes: The state matrix of neighboring agents centered on the vehicle is analyzed using an MLP-Mixer network. S neighbor Temporal feature extraction is performed to obtain the agent's trajectory features, where... , T h Indicates the time step in history. Represents the set of real numbers; The discrete point sequence composed of the left boundary, right boundary and center line S Road Inputting the data into a lightweight CNN network extracts lane geometry features, resulting in lane boundary features. , P This indicates the number of sampling points for each lane. D lane Includes coordinate and angle information; Using a pre-trained EfficientNet-B0 as the backbone network, a three-channel semantic map containing drivable regions, obstacle regions, and pedestrian regions is generated. S map Multi-scale feature extraction is performed, and spatial information is fused using a feature pyramid network to obtain raster map features. , H , W These are the height and width of the feature map, respectively; The agent trajectory features and lane boundary features are concatenated along the channel dimension, and relative position information is injected through a spatial location embedding layer to form a joint representation feature. The relative coordinates in physical space are mapped to the spatial location of the feature map using the following formula: ； in, Indicates the mapped position. x , y Indicates the original position. p This indicates the number of pixels per meter. d This represents the map downsampling ratio; The semantic features extracted from the corresponding positions in the grid map feature map are used as the Key and Value in the multi-head cross attention mechanism. The fusion features of the agent and the lane boundary are used as the Query in the multi-head cross attention. Information alignment and cross-modal fusion are performed through multi-head cross attention, and a unified high-dimensional scene representation is output and input into the cross attention module of the DiT model. The centerline trajectory associated with the agent S navigation The navigation semantic vector is generated after processing by a cross-height convolution module and compression by an adaptive pooling layer. ; The navigation semantic vector is added to the noisy step features of the conditional diffusion model to obtain the navigation constraints of the agent's motion. The navigation constraints of the agent's motion are then fused with the trajectory features through the adaptive normalization module of the DiT model. The construction of the noisy step features of the conditional diffusion model includes: Gaussian noise is applied to the trajectory to be predicted based on a linear scheduling strategy to obtain a noisy trajectory. Y N , is represented as: ； in, For the first k Step-by-step cumulative noise attenuation coefficient, Y 0 represents the actual trajectory. Indicates Gaussian noise. Represents a normal distribution; Predicting noise using a DiT decoder loss function L for: ； in, Expressing expectations, Indicates DiT network, Indicates contextual information, denoted as the square of the Euclidean norm.

6. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods of claims 1 to 4.

7. A computer device, characterized in that, include, One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method of any of claims 1 to 4.

Images