Conditional diffusion model-based scene-level trajectory prediction method and system, medium and equipment

Abstract

The invention discloses a scene-level trajectory prediction method, system, medium and equipment based on a conditional diffusion model in the technical field of intelligent driving, and the method comprises the steps: obtaining a multi-agent trajectory feature collected at a current moment and a navigation constraint condition of agent motion extracted based on a vehicle position and urban high-precision map data, and inputting the trained conditional diffusion model, and outputting a future multi-modal trajectory prediction result. According to the method, multi-source heterogeneous data is deeply fused, so that the generalization ability of the model in a complex traffic scene is improved, particularly, high prediction reliability can be kept in a rare scene which is not covered by training data, and the bottleneck that a traditional method is difficult to cope with diversified scenes is solved; according to the method, the conditional diffusion model is adopted, the multi-modal trajectory distribution is learned through the iterative denoising process, the semantic probability and the direction matching probability are combined, the prediction trajectory is effectively restrained to conform to the road topology and traffic rules, and the error accumulation effect of long-term prediction is reduced.

Description

Technical Field

[0001] The present invention relates to a scene-level trajectory prediction method, system, medium and equipment based on a conditional diffusion model, and belongs to the field of intelligent driving technology. Background Art

[0002] With the rapid development of autonomous driving technology, the problem of trajectory prediction for intelligent vehicles in complex traffic environments has become increasingly prominent. As a key component of autonomous driving decision-making systems, multi-agent trajectory prediction not only needs to consider the vehicle's motion state but also accurately predict the behavioral intentions of surrounding traffic participants, such as vehicles and pedestrians. Currently, with the widespread use of high-precision positioning technology and environmental perception devices, on-board systems can acquire multi-dimensional data, including historical trajectories, high-precision maps, and real-time traffic status. This provides a data foundation for in-depth research on the movement patterns of traffic participants.

[0003] In the field of intelligent transportation, multi-agent trajectory prediction is essentially a probabilistic estimation problem with multimodal characteristics. Due to the complexity of traffic scenarios and the randomness of participant behavior, prediction results often present multiple possibilities. Existing research primarily addresses this problem through two technical approaches: deterministic prediction methods based on physical motion models, and probabilistic generation methods based on deep learning. While the former offers high computational efficiency, it struggles with complex interactive scenarios. While the latter can generate diverse trajectories, it still lacks prediction accuracy and physical feasibility.

[0004] Current mainstream trajectory prediction methods face three major technical bottlenecks: First, the use of a single data modality limits the model's adaptability to complex scenarios; second, the cumulative error increases significantly during long-term predictions, causing trajectories to deviate from actual physical constraints; and most importantly, the prediction reliability of existing models decreases significantly when encountering rare scenarios not covered by the training data. The root cause of these problems lies in the difficulty of existing methods in effectively integrating multi-source heterogeneous data and their lack of in-depth exploration of the essential characteristics of traffic scenarios. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a scene-level trajectory prediction method, system, medium and device based on the conditional diffusion model.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions.

[0007] In a first aspect, the present invention discloses a scene-level trajectory prediction method based on a conditional diffusion model, comprising: The multi-agent trajectory features collected at the current moment and the navigation constraints of the agent's movement extracted based on the vehicle's position and high-precision city map data are input into the trained conditional diffusion model to output the future multimodal trajectory prediction results. The multi-agents are all traffic participants within the preset spatial range. The training of the conditional diffusion model includes: Collecting multi-agent historical trajectory data in a real traffic scene, and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features; Extracting historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center; Extract navigation constraints for the agent's motion based on the vehicle's position and high-precision city map data; A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle as conditional input. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is met, the trained conditional diffusion model is obtained.

[0008] Furthermore, the collecting of multi-agent historical trajectory data in a real traffic scene and preprocessing of the multi-agent historical trajectory data to generate standardized historical trajectory features include: The collected multi-agent historical trajectory data in real traffic scenes are converted from the world coordinate system to the local coordinate system with the center of mass of the vehicle as the origin; Perform time domain synchronization processing on the historical trajectory data of multiple agents after the coordinate system conversion, and unify the sampling timestamps of the historical trajectory data of each agent through linear interpolation method; A trajectory interpolation algorithm based on quintic polynomial is used to smoothly enhance the historical trajectory data of each agent after time domain synchronization processing, construct a continuous and differentiable trajectory sample space, and obtain standardized historical trajectory features.

[0009] Furthermore, the process of extracting historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center includes: Dynamically adjust the circular perception area centered on the vehicle based on the type of traffic participant to filter out the standardized historical trajectory characteristics of traffic participants; The method of dynamically adjusting the circular perception area centered on the vehicle according to the type of traffic participant includes: For motor vehicle traffic participants, a circular sensing area with a radius of R1 is set; For non-motorized vehicle traffic participants, a circular sensing area with a radius of R2 is set, where R2 <R1; For pedestrian traffic participants, a circular sensing area with a radius of R3 is set, where R3 <R2。

[0010] Furthermore, the extraction of navigation constraints for the movement of the intelligent body based on the vehicle position and the high-precision city map data includes: With the current position of the vehicle as the center, a dynamic interception area with a radius of D is established, the raster map data and vector map data in the area are acquired synchronously, and the conversion relationship matrix between the raster coordinate system and the global coordinate system is established; Perform semantic analysis on the intercepted rasterized map data to extract structured environmental information, including the boundaries of the drivable area, the location of pedestrian crossings, zebra crossings, and various prohibited areas, as environmental constraints for trajectory prediction. For each agent’s current position, a nearest neighbor search is performed in the vector map data to determine the lane centerline associated with each agent; The environmental constraints and the lane centerline associated with each agent are used as navigation constraints for the agent's movement.

[0011] Furthermore, the conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body movement, and the vehicle planning path information as conditional inputs. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is reached, the trained conditional diffusion model is obtained, including A training set is constructed based on multiple driving scenarios of the vehicle. Each driving scenario in the training set This includes the motion trajectory characteristics of all traffic participants within the preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle; Set the number of iterations and learning rate, and use the mixed precision algorithm to train the conditional diffusion model based on the training set to finally obtain the trained conditional diffusion model; The process of each round of iterative training includes: The state matrix of the neighboring agents centered on the vehicle is calculated through the MLP-Mixer network. S neighbor Perform time series feature extraction to obtain the agent trajectory features, where: , T h represents the historical time step, represents the set of real numbers; The discrete point sequence consisting of the left boundary, right boundary and center line S Road Input the lightweight CNN network to extract lane geometry features and obtain lane boundary features, where , P Indicates the number of sampling points for each lane,D lane Contains coordinate and angle information; The pre-trained EfficientNet-B0 is used as the backbone network to construct a three-channel semantic map containing the drivable area, obstacle area and pedestrian area. S map Perform multi-scale feature extraction and perform spatial information fusion through feature pyramid network to obtain raster map features, where , H 、 W are the height and width of the feature map respectively; The agent trajectory features and lane boundary features are spliced ​​along the channel dimension, and the relative position information is injected through the spatial position embedding layer to form a joint representation feature. The relative coordinates in the physical space are mapped to the spatial position of the feature map using the following formula: ; in, Indicates the position after mapping, x 、 y Indicates the original position, p Indicates the number of pixels per meter. d is the map downsampling ratio; The semantic features extracted from the corresponding positions in the grid map feature map are used as the key and value in the multi-head cross-attention mechanism. The fused features of the agent and lane boundaries are used as the query in the multi-head cross-attention mechanism. Information alignment and cross-modal fusion are performed through multi-head cross-attention, and a unified high-dimensional scene representation is output, which is input into the cross-attention module of the DiT model. Associate the agent with the centerline trajectory S navigation The navigation semantic vector is generated after processing through the cross-height convolution module and compression by the adaptive pooling layer. ; The navigation constraints of the agent's motion are obtained by adding the navigation semantic vector to the noisy step features of the conditional diffusion model. The navigation constraints of the agent's motion are then fused with the trajectory features through the adaptive normalization module of the DiT model. The construction of the noise step feature of the conditional diffusion model includes: Apply Gaussian noise to the trajectory to be predicted based on the linear scheduling strategy to obtain the noisy trajectory Y N , expressed as: ; in, For the k Step-by-step cumulative noise attenuation coefficient, Y 0 is the true trajectory, represents Gaussian noise, represents a normal distribution; Noise prediction by DiT decoder , loss function L for: ; in, Express expectations, represents the DiT network, Represents context information, represents the square of the Euclidean norm.

[0012] Furthermore, the outputting of future multimodal trajectory prediction results includes: The multi-agent trajectory features at the current moment A t and navigation constraints C Input the trained conditional diffusion model and perform the following inference: Multi-agent trajectory features A t and navigation constraints C Normalized to a 256-dimensional vector; Based on the preset number of iterations, the DPM-Solver algorithm is used to perform multiple step-by-step denoising to generate multi-mode The multimodal trajectory distribution is used to output the future multimodal trajectory prediction result according to the multimodal trajectory distribution. The denoising formula is: ; Where, Y k-1 Represents the updated output or state variable in the k The result after −1 step, Y k Indicates the current step k The input or state variables, α k Represents a time step k Related parameters.

[0013] In a second aspect, the present invention further discloses a scene-level trajectory prediction system based on a conditional diffusion model, comprising: An acquisition module is used to obtain the multi-agent trajectory features collected at the current moment and the navigation constraints of the agent's movement extracted based on the vehicle's position and high-precision city map data, and input them into the trained conditional diffusion model; the multi-agents are all traffic participants within a preset spatial range; A processing module, configured to output a future multimodal trajectory prediction result based on the multi-agent trajectory features collected at the current moment and the navigation constraints; The training of the conditional diffusion model includes: Collecting multi-agent historical trajectory data in a real traffic scene, and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features; Extracting historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center; Extract navigation constraints for the agent's motion based on the vehicle's position and high-precision city map data; A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle as conditional input. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is met, the trained conditional diffusion model is obtained.

[0014] In a third aspect, the present invention further discloses a computer-readable storage medium storing one or more programs, characterized in that the one or more programs include instructions that, when executed by a computing device, enable the computing device to execute the method of the first aspect.

[0015] In a fourth aspect, the present invention further discloses a computer device, comprising: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method of the first aspect.

[0016] The beneficial effects achieved by the present invention are: To address the problem of single data modality limiting adaptability in multi-agent trajectory prediction, the present invention deeply integrates multi-source heterogeneous data, including historical trajectories, high-precision maps, and planned path information, and uses a cross-attention mechanism to generate unified conditional features, thereby improving the model's generalization ability in complex traffic scenarios. In particular, it can maintain high prediction reliability in rare scenarios not covered by training data, solving the bottleneck that traditional methods have difficulty in coping with diverse scenarios.

[0017] To address the challenge of trajectory deviation from physical constraints caused by cumulative errors in long-term predictions, this paper adopts a conditional diffusion model to learn multimodal trajectory distribution through an iterative denoising process. By combining semantic probability and direction matching probability, it effectively constrains the predicted trajectory to conform to road topology and traffic rules, reducing the error accumulation effect of long-term predictions.

[0018] In view of the high requirements of the autonomous driving system for real-time performance and computational efficiency, this paper introduces the DPM-Solver fast denoising algorithm, which reduces the computational complexity from down to , significantly improving the inference speed while maintaining high accuracy of multimodal prediction, providing an efficient and robust trajectory prediction solution for intelligent transportation and unmanned driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flow chart of the method of the present invention; Figure 2 is the model prediction framework of the present invention; Figure 3 This is a diagram showing the trajectory prediction results of an autonomous driving vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0021] Example 1: This example introduces a scene-level trajectory prediction method based on a conditional diffusion model, including: The multi-agent trajectory features collected at the current moment and the navigation constraints of the agent's movement extracted based on the vehicle's position and high-precision city map data are input into the trained conditional diffusion model to output the future multimodal trajectory prediction results. The multi-agents are all traffic participants within the preset spatial range. The training of the conditional diffusion model includes: Collecting multi-agent historical trajectory data in a real traffic scene, and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features; Extracting historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center; Extract navigation constraints for the agent's motion based on the vehicle's position and high-precision city map data; A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle as conditional input. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is met, the trained conditional diffusion model is obtained.

[0022] The collecting of multi-agent historical trajectory data in a real traffic scene and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features include: Convert the multi-agent historical trajectory data collected in real traffic scenes from the world coordinate system to the local coordinate system with the center of mass of the vehicle as the origin; Perform time domain synchronization processing on the historical trajectory data of multiple agents after the coordinate system conversion, and unify the sampling timestamps of the historical trajectory data of each agent through linear interpolation method; A trajectory interpolation algorithm based on quintic polynomial is used to smoothly enhance the historical trajectory data of each agent after time domain synchronization processing, construct a continuous and differentiable trajectory sample space, and obtain standardized historical trajectory features.

[0023] The process of extracting the historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center includes: Dynamically adjust the circular perception area centered on the vehicle based on the type of traffic participant to filter out the standardized historical trajectory characteristics of traffic participants; The method of dynamically adjusting the circular perception area centered on the vehicle according to the type of traffic participant includes: For motor vehicle traffic participants, a circular sensing area with a radius of R1 is set; For non-motorized vehicle traffic participants, a circular sensing area with a radius of R2 is set, where R2 <R1; For pedestrian traffic participants, a circular sensing area with a radius of R3 is set, where R3 <R2。

[0024] The extraction of navigation constraints for the intelligent body's movement based on the vehicle's position and high-precision city map data includes: With the current position of the vehicle as the center, a dynamic interception area with a radius of D is established, the raster map data and vector map data in the area are acquired synchronously, and the conversion relationship matrix between the raster coordinate system and the global coordinate system is established; Perform semantic analysis on the intercepted rasterized map data to extract structured environmental information, including the boundaries of the drivable area, the location of pedestrian crossings, zebra crossings, and various prohibited areas, as environmental constraints for trajectory prediction. For each agent’s current position, a nearest neighbor search is performed in the vector map data to determine the lane centerline associated with each agent; The environmental constraints and the lane centerline associated with each agent are used as navigation constraints for the agent's movement.

[0025] The conditional diffusion model is constructed based on the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is met, the trained conditional diffusion model is obtained, including: A training set is constructed based on multiple driving scenarios of the vehicle. Each driving scenario in the training set This includes the motion trajectory characteristics of all traffic participants within the preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle; Set the number of iterations and learning rate, and use the mixed precision algorithm to train the conditional diffusion model based on the training set to finally obtain the trained conditional diffusion model; The process of each round of iterative training includes: The state matrix of the neighboring agents centered on the vehicle is calculated through the MLP-Mixer network. S neighbor Perform time series feature extraction to obtain the agent trajectory features, where: , T h represents the historical time step, represents the set of real numbers; The discrete point sequence consisting of the left boundary, right boundary and center line S Road Input the lightweight CNN network to extract lane geometry features and obtain lane boundary features, where , P Indicates the number of sampling points for each lane, D lane Contains coordinate and angle information; The pre-trained EfficientNet-B0 is used as the backbone network to construct a three-channel semantic map containing the drivable area, obstacle area and pedestrian area. S map Perform multi-scale feature extraction and perform spatial information fusion through feature pyramid network to obtain raster map features, where , H 、 W are the height and width of the feature map respectively; The agent trajectory features and lane boundary features are spliced ​​along the channel dimension, and the relative position information is injected through the spatial position embedding layer to form a joint representation feature. The relative coordinates in the physical space are mapped to the spatial position of the feature map using the following formula: ; in, Indicates the position after mapping, x 、 y Indicates the original position, p Indicates the number of pixels per meter. d is the map downsampling ratio; The semantic features extracted from the corresponding positions in the grid map feature map are used as the key and value in the multi-head cross-attention mechanism. The fused features of the agent and lane boundaries are used as the query in the multi-head cross-attention mechanism. Information alignment and cross-modal fusion are performed through multi-head cross-attention, and a unified high-dimensional scene representation is output, which is input into the cross-attention module of the DiT model. Associate the agent with the centerline trajectory S navigation The navigation semantic vector is generated after processing through the cross-height convolution module and compression by the adaptive pooling layer. ; The navigation constraints of the agent's motion are obtained by adding the navigation semantic vector to the noisy step features of the conditional diffusion model. The navigation constraints of the agent's motion are then fused with the trajectory features through the adaptive normalization module of the DiT model. The construction of the noise step feature of the conditional diffusion model includes: Apply Gaussian noise to the trajectory to be predicted based on the linear scheduling strategy to obtain the noisy trajectory Y N , expressed as: ; in, For the k Step-by-step cumulative noise attenuation coefficient, Y 0 is the true trajectory, represents Gaussian noise, represents a normal distribution; Noise prediction by DiT decoder , loss function L for: ; in, Express expectations, represents the DiT network, Represents context information, represents the square of the Euclidean norm.

[0026] Furthermore, the outputting of future multimodal trajectory prediction results includes: The multi-agent trajectory features at the current moment A t and navigation constraints C Input the trained conditional diffusion model and perform the following inference: Multi-agent trajectory features A t and navigation constraints C Normalized to a 256-dimensional vector; Based on the preset number of iterations, the DPM-Solver algorithm is used to perform multiple step-by-step denoising to generate multi-mode The multimodal trajectory distribution is used to output the future multimodal trajectory prediction result according to the multimodal trajectory distribution. The denoising formula is: ; Where, Y k-1 Represents the updated output or state variable in the k The result after −1 step, Y k Indicates the current step k The input or state variables, α k Represents a time step k Related parameters.

[0027] Example 2: This example introduces a scene-level trajectory prediction method based on a conditional diffusion model. Figure 1 As shown, the following steps are included: Step 1: Collect multi-agent trajectory data and preprocess it; The multi-agent trajectory data in real traffic scenes is collected through vehicle-mounted sensors (including but not limited to cameras, millimeter-wave radars) and road monitoring equipment. The data fields include the unique number of the agent, the sampling timestamp accurate to milliseconds, the plane coordinates ( x , y ),speed v and orientation θ To ensure data quality, the preprocessing process is as follows: Format conversion: Unify the original trajectory data into a standard format. Represents a collection of agents. Each agent The historical state is defined as ,in, Indicates the length of historical time, the state of each time step Contains plane coordinates ( x , y ),speed v , towards θ , three-dimensional dimensions ( l , w , h ), acceleration and agent type unique hot encoding. Each agent in the future The trajectory of the time step is ,in, Keeping state with history Same coordinate system datum.

[0028] 1) Coordinate standardization: The trajectory points in the world coordinate system are converted to the local coordinate system with the center of mass of the vehicle as the origin through the transformation matrix. The conversion formula is: ; in, is the vehicle's position and orientation.

[0029] 2) Time domain synchronization: The sampling time interval is unified through linear interpolation. The interpolation formula is: .

[0030] 3) Trajectory smoothing: A quintic polynomial interpolation algorithm is used to generate a continuous and differentiable trajectory. The polynomial is defined as: ; Optimize coefficients by least squares method a 0 to a 5. Ensure that the trajectory is smooth and the deviation from the sampling points is minimal.

[0031] In this embodiment, the sampling frequency is 10 Hz, and the trajectory data of 1000 scenes are processed. Each agent contains 40 time-step trajectory points to generate a standardized trajectory feature set.

[0032] Step 2: Extract the trajectory features of traffic participants; Based on the vehicle's position As the center, a dynamic perception area is constructed, and the perception radius is set for different traffic participants: motor vehicles R 1=30 meters, non-motorized vehicle R 2=10 meters, pedestrian R 3 = 10 meters. Computing Agent i Euclidean distance to the vehicle: ; like, ( k corresponding agent type), its trajectory data is retained.

[0033] In this embodiment, it is assumed that there are 20 agents (10 motor vehicles, 5 non-motor vehicles, and 5 pedestrians) in the sensing area, and their trajectory features are extracted to form a set .

[0034] Step 3: Extract local environmental constraint features; Based on the vehicle's position, the radius is intercepted from the high-precision map D = 80 meters, and obtain raster map and vector map data. The processing process is as follows: 1) Map spatial index: building a graph structure G < V , E>, where V For road intersections, E For the road centerline, a spatial index is generated using R-tree technology.

[0035] 2) Coordinate transformation: Define the transformation matrix between the grid coordinate system and the global coordinate system M : .

[0036] 3) Semantic parsing: Use convolutional neural networks (CNNs) to perform semantic segmentation on the grid map, extract features such as drivable areas and prohibited areas, and semantic probability for: .

[0037] 4) Topological constraints: Determine the centerline of the road associated with the agent through nearest neighbor search and calculate the direction angle , define the direction matching probability : .

[0038] 5) In this example, a constraint feature set consisting of 8 road centerlines and 10 semantic regions (including 2 crosswalks) was extracted. C .

[0039] Step 4: Construct a conditional diffusion model, such as Figure 2 As shown, the conditional diffusion model includes: Encoding stage: multi-source information processing; The encoding stage processes the perception data in parallel through a dual-branch structure, dividing it into dynamic objects and static environmental elements to form a multimodal scene representation to capture historical motion trajectories and environmental semantic information.

[0040] The dynamic objects, i.e., the historical states of neighboring agents, are collected in the ego-centered coordinate system and defined as These states are encoded through an MLP-Mixer network to capture temporal motion patterns.

[0041] The static environment includes lane boundaries and semantic maps. Lane boundaries (including left boundary, right boundary and center line) are represented as a sequence of discrete points. ,in, P Indicates the number of points per line, D lane Contains coordinate and direction information. These features are extracted by a lightweight convolutional neural network (CNN). The semantic map contains three layers of information - drivable area, obstacle area and sidewalk, represented as ,in, H and WRepresent the height and width of the map, respectively. A pre-trained EfficientNet-B0 backbone network is used to extract multi-scale semantic features, and spatial information is fused through a feature pyramid network (FPN).

[0042] The navigation information comes from the agent's nearest lane centerline, represented as This information is encoded through height-wise convolutional modules, adaptive pooling, and fully connected layers to form a compact navigation vector , which provides a directional prior for trajectory prediction.

[0043] Cross-modal fusion is achieved through a spatial enhancement fusion mechanism. The encoded features of the agent and lane boundaries are concatenated and the spatial position information is added. The interaction between the agent and the lane is modeled through the self-attention mechanism. Subsequently, based on the relative coordinates ( x , y ), projecting the fused features onto the semantic map feature map, and achieving cross-modal fusion through a multi-head cross-attention mechanism to generate a unified scene representation ,This representation encapsulates spatiotemporal interactions and environmental context for trajectory prediction.

[0044] Decoding stage: trajectory generation based on the conditional diffusion model; The decoding stage adopts a diffusion-based transformer (DiT) module to generate trajectories through a conditional diffusion process, and uses the features of the encoding stage to generate accurate future trajectories.

[0045] In the forward process, a predefined variance schedule is applied , gradually injecting Gaussian noise. Diffusion time step t The perturbation trajectory is expressed as , through iterative noise injection, it eventually approaches the standard Gaussian distribution.

[0046] In the reverse process, starting from the standard Gaussian distribution, the future trajectory is generated by solving the inverse time diffusion ordinary differential equation (ODE). The generation process is determined by the conditional information Guidance,integrates scene representation and navigation priors.

[0047] Through the cross-attention mechanism, spatiotemporal interaction features and semantic environment context are embedded into the unbalanced diffusion process, ensuring that the generated trajectories are jointly influenced by motion patterns and environmental constraints.

[0048] Navigation vector The diffusion process is dynamically injected through the adaptive layer normalization (AdaLN) mechanism to promote the deep coupling of navigation information and trajectory evolution, ensuring that the predicted trajectory is aligned with the lane centerline and has target consistency.

[0049] The specific steps are as follows: Step 4-1: Multimodal scene feature encoding: 1) Neighbor agent state encoding: Neighbor agent state matrix centered on the vehicle Temporal feature extraction is performed through the MLP-Mixer network, where T h represents the historical time step; 2) Lane boundary feature encoding: a discrete point sequence consisting of the left boundary, right boundary and center line Input lightweight CNN network to extract lane geometry features, where P Indicates the number of sampling points for each lane, D lane Contains coordinate and angle information; 3) Grid map feature encoding: Using the pre-trained EfficientNet-B0 as the backbone network, a three-channel semantic map containing the drivable area, obstacle area, and pedestrian area is generated. Multi-scale feature extraction is performed and spatial information fusion is achieved through the Feature Pyramid Network (FPN), where H 、 W are the height and width of the feature map respectively; Step 4-2: Spatial Enhancement Fusion Mechanism: 1) Primary feature fusion: The agent trajectory features and lane boundary features are concatenated along the channel dimension, and relative position information is injected through the spatial position embedding layer. The calculation formula is: ; in, p Indicates the number of pixels per meter. d The map downsampling ratio.

[0050] 2) Cross-modal Attention Fusion: Features sampled from locations in the map feature map are used as keys and values, combined with fused features from the agent and lane boundaries (as queries). Multi-head cross-attention is used to achieve information alignment and cross-modal fusion. The final output is a unified high-dimensional scene representation for downstream prediction tasks.

[0051] Step 4-3: Navigation condition injection: 1) Navigation trajectory encoding: the centerline trajectory associated with the agent The navigation semantic vector is generated after processing through the cross-height convolution module and compression by the adaptive pooling layer; Conditional fusion: The navigation vector is concatenated with the time step features of the diffusion model and the features are adjusted through the adaptive normalization module (AdaLN).

[0052] Step 4-4: Training: 1) Noise addition: Gaussian noise is applied based on the linear scheduling strategy. The noise trajectory is: ; in, For the k Step-by-step cumulative noise attenuation coefficient, Y 0 is the true trajectory.

[0053] 2) Denoising training: predicting noise through DiT decoder , the loss function is: ; Use mixed precision algorithm to optimize the model, iterate 50 times, and set the learning rate to 2*10 -4 .

[0054] In this embodiment, the model input dimension is 256, the training data set contains 700 driving scenarios, the validation set contains 150 validation trajectory scenarios, and the mean square error of the model on the validation set converges to 0.05.

[0055] Step 5: Trajectory prediction reasoning; The trajectory features at the current moment A t and map constraint features C Input the trained model and perform inference: 1) Input preparation: A t and C Normalized to a 256-dimensional vector.

[0056] 2) Fast denoising: The DPM-Solver algorithm is used for sub-step denoising. The denoising formula is: ; Where, Y k-1 Represents the updated output or state variable in the k The result after −1 step, Y k Indicates the current step k The input or state variables, α k Represents a time step k Related parameters, usually part of the noise schedule, control the noise level at each step.

[0057] Output sampling: Sampling from the denoising results, generating multimodal trajectory distribution, and outputting high-probability trajectories, such as Figure 3 shown.

[0058] Example 3, based on the same inventive concept as Example 1, introduces a scene-level trajectory prediction system based on a conditional diffusion model, including: An acquisition module is used to obtain the multi-agent trajectory features collected at the current moment and the navigation constraints of the agent's movement extracted based on the vehicle's position and high-precision city map data, and input them into the trained conditional diffusion model; the multi-agents are all traffic participants within a preset spatial range; A processing module, configured to output a future multimodal trajectory prediction result based on the multi-agent trajectory features collected at the current moment and the navigation constraints; The training of the conditional diffusion model includes: Collecting multi-agent historical trajectory data in a real traffic scene, and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features; Extracting historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center; Extract navigation constraints for the agent's motion based on the vehicle's position and high-precision city map data; A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle as conditional input. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is met, the trained conditional diffusion model is obtained.

[0059] Example 4 is based on the same inventive concept as Example 2. This example introduces a computer-readable storage medium that stores one or more programs, wherein the one or more programs include instructions that, when executed by a computing device, enable the computing device to execute the method described in Example 1.

[0060] Example 5, based on the same inventive concept as Example 1, this example introduces a computer device, including: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method described in Example 1.

[0061] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0063] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0065] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A scene-level trajectory prediction method based on a conditional diffusion model, characterized in that: include: The multi-agent trajectory features collected at the current moment and the navigation constraints of the agent's movement extracted based on the vehicle's position and high-precision city map data are input into the trained conditional diffusion model to output the future multimodal trajectory prediction results. The multi-agents are all traffic participants within the preset spatial range. The training of the conditional diffusion model includes: Collecting multi-agent historical trajectory data in a real traffic scene, and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features; Extracting historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center; Extract navigation constraints for the agent's motion based on the vehicle's position and high-precision city map data; A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle as conditional input. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is met, the trained conditional diffusion model is obtained.

2. The scene-level trajectory prediction method based on the conditional diffusion model according to claim 1 is characterized in that: The collecting of multi-agent historical trajectory data in a real traffic scene and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features include: The collected multi-agent historical trajectory data in real traffic scenes are converted from the world coordinate system to the local coordinate system with the center of mass of the vehicle as the origin; Perform time domain synchronization processing on the historical trajectory data of multiple agents after the coordinate system conversion, and unify the sampling timestamps of the historical trajectory data of each agent through linear interpolation method; A trajectory interpolation algorithm based on quintic polynomial is used to smoothly enhance the historical trajectory data of each agent after time domain synchronization processing, construct a continuous and differentiable trajectory sample space, and obtain standardized historical trajectory features.

3. The scene-level trajectory prediction method based on the conditional diffusion model according to claim 1 is characterized in that: The process of extracting the historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center includes: Dynamically adjust the circular perception area centered on the vehicle based on the type of traffic participant to filter out the standardized historical trajectory characteristics of traffic participants; The method of dynamically adjusting the circular perception area centered on the vehicle according to the type of traffic participant includes: For motor vehicle traffic participants, a circular sensing area with a radius of R1 is set; For non-motorized vehicle traffic participants, a circular sensing area with a radius of R2 is set, where R2 <R1; For pedestrian traffic participants, a circular sensing area with a radius of R3 is set, where R3 <R2。 4. The scene-level trajectory prediction method based on the conditional diffusion model according to claim 1 is characterized in that: The extraction of navigation constraints for the intelligent body's movement based on the vehicle's position and high-precision city map data includes: With the current position of the vehicle as the center, a dynamic interception area with a radius of D is established, the raster map data and vector map data in the area are acquired synchronously, and the conversion relationship matrix between the raster coordinate system and the global coordinate system is established; Perform semantic analysis on the intercepted rasterized map data to extract structured environmental information, including the boundaries of the drivable area, the location of pedestrian crossings, zebra crossings, and various prohibited areas, as environmental constraints for trajectory prediction. For each agent’s current position, a nearest neighbor search is performed in the vector map data to determine the lane centerline associated with each agent; The environmental constraints and the lane centerline associated with each agent are used as navigation constraints for the agent's movement.

5. The scene-level trajectory prediction method based on the conditional diffusion model according to claim 4, characterized in that: The conditional diffusion model is constructed based on the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is met, the trained conditional diffusion model is obtained, including: A training set is constructed based on multiple driving scenarios of the vehicle. Each driving scenario in the training set This includes the motion trajectory characteristics of all traffic participants within the preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle; Set the number of iterations and learning rate, and use the mixed precision algorithm to train the conditional diffusion model based on the training set to finally obtain the trained conditional diffusion model; The process of each round of iterative training includes: The state matrix of the neighboring agents centered on the vehicle is calculated through the MLP-Mixer network. S neighbor Perform time series feature extraction to obtain the agent trajectory features, where: , T h represents the historical time step, represents the set of real numbers; The discrete point sequence consisting of the left boundary, right boundary and center line S Road Input the lightweight CNN network to extract lane geometry features and obtain lane boundary features, where , P Indicates the number of sampling points for each lane, D lane Contains coordinate and angle information; The pre-trained EfficientNet-B0 is used as the backbone network to construct a three-channel semantic map containing the drivable area, obstacle area and pedestrian area. S map Perform multi-scale feature extraction and perform spatial information fusion through feature pyramid network to obtain raster map features, where , H 、 W are the height and width of the feature map respectively; The agent trajectory features and lane boundary features are spliced ​​along the channel dimension, and the relative position information is injected through the spatial position embedding layer to form a joint representation feature. The relative coordinates in the physical space are mapped to the spatial position of the feature map using the following formula: ； in, Indicates the position after mapping, x 、 y Indicates the original position, p Indicates the number of pixels per meter. d is the map downsampling ratio; The semantic features extracted from the corresponding positions in the grid map feature map are used as the key and value in the multi-head cross-attention mechanism. The fused features of the agent and lane boundaries are used as the query in the multi-head cross-attention mechanism. Information alignment and cross-modal fusion are performed through multi-head cross-attention, and a unified high-dimensional scene representation is output, which is input into the cross-attention module of the DiT model. Associate the agent with the centerline trajectory S navigation The navigation semantic vector is generated after processing through the cross-height convolution module and compression by the adaptive pooling layer. ; The navigation constraints of the agent's motion are obtained by adding the navigation semantic vector to the noisy step features of the conditional diffusion model. The navigation constraints of the agent's motion are then fused with the trajectory features through the adaptive normalization module of the DiT model. The construction of the noise step feature of the conditional diffusion model includes: Apply Gaussian noise to the trajectory to be predicted based on the linear scheduling strategy to obtain the noisy trajectory Y N , expressed as: ； in, For the k Step-by-step cumulative noise attenuation coefficient, Y 0 is the true trajectory, represents Gaussian noise, represents a normal distribution; Noise prediction by DiT decoder , loss function L for: ； in, Express expectations, represents the DiT network, Represents context information, represents the square of the Euclidean norm.

6. The scene-level trajectory prediction method based on the conditional diffusion model according to claim 5 is characterized in that: The output of the future multimodal trajectory prediction result includes: The multi-agent trajectory features at the current moment A t and navigation constraints C Input the trained conditional diffusion model and perform the following inference: Multi-agent trajectory features A t and navigation constraints C Normalized to a 256-dimensional vector; Based on the preset number of iterations, the DPM-Solver algorithm is used to perform multiple step-by-step denoising to generate multi-mode The multimodal trajectory distribution is used to output the future multimodal trajectory prediction result according to the multimodal trajectory distribution. The denoising formula is: ； Where, Y k-1 Represents the updated output or state variable in the k The result after −1 step, Y k Indicates the current step k The input or state variables, α k Represents a time step k Related parameters.

7. A scene-level trajectory prediction system based on a conditional diffusion model, characterized in that: include: An acquisition module is used to obtain the multi-agent trajectory features collected at the current moment and the navigation constraints of the agent's movement extracted based on the vehicle's position and high-precision city map data, and input them into the trained conditional diffusion model; the multi-agents are all traffic participants within a preset spatial range; A processing module, configured to output a future multimodal trajectory prediction result based on the multi-agent trajectory features collected at the current moment and the navigation constraints; The training of the conditional diffusion model includes: Collecting multi-agent historical trajectory data in a real traffic scene, and preprocessing the multi-agent historical trajectory data to generate standardized historical trajectory features; Extracting historical motion trajectory features of all traffic participants within a preset range from the standardized historical trajectory features with the vehicle as the center; Extract navigation constraints for the agent's motion based on the vehicle's position and high-precision city map data; A conditional diffusion model is constructed with the historical motion trajectory characteristics of all traffic participants within a preset range, the navigation constraints of the intelligent body's motion, and the planned path information of the vehicle as conditional input. The trajectory distribution law is learned through an iterative denoising training process. After the iterative termination condition is met, the trained conditional diffusion model is obtained.

8. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 6.

9. A computer device, characterized in that: include, One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method of any one of claims 1 to 6.

Images