Dynamic scene simulation generation method for automatic driving

Abstract

The invention relates to the technical field of automatic driving, in particular to a dynamic scene simulation generation method for automatic driving, which comprises the following steps: step 1, scene initialization and static background reconstruction: utilizing an improved 3D Gaussian splash technology, fusing LiDAR point cloud prior information, reconstructing a large-scale static road environment, and performing semantic decoupling on a scene; step 2, dynamic object and behavior injection: endowing each dynamic object with an initial motion track represented by a learnable Bezier curve, outputting an adjustment instruction for behaviors of other traffic participants by using a generative AI model according to the semantic information of the current scene and a self-vehicle planning track, and finely adjusting a control point of the Bezier curve; the interactive authenticity is improved, human driving behaviors are simulated through generative AI, physical reasonability is ensured through explicit tracks, the test efficiency is improved, and the long-tail scene generation efficiency is improved by 50% or above through conditional generation.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method for generating dynamic scene simulations for autonomous driving. Background Technology

[0002] Before large-scale application, autonomous driving technology needs to undergo rigorous and comprehensive system testing to ensure its safety. Scenario-based simulation testing is an important component of autonomous driving technology testing. Scenario-based simulation testing can test specific functions, performance, or behaviors of the autonomous driving system using a simulation model based on a pre-set simulation scenario.

[0003] With the development of autonomous driving technology, simulation testing has become a key means of verifying algorithm safety. However, existing simulation methods have the following limitations:

[0004] Insufficient realism in scene interaction: Traditional rule-based methods have fixed behavior patterns, making it difficult to simulate the complex interactions of human driving. Low efficiency in generating long-tail scenes: Rare scenes rely on manual design, which is costly and incomplete.

[0005] The simulation confidence level is low: the sensor simulation has poor fidelity in complex environments. Summary of the Invention

[0006] The purpose of this invention is to provide a dynamic scene simulation generation method for autonomous driving, which combines the world model capabilities of generative AI with explicit trajectory control to form a self-optimizing dynamic scene simulation closed loop, thereby solving the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a dynamic scene simulation generation method for autonomous driving, comprising the following steps:

[0008] Step 1: Scene Initialization and Static Background Reconstruction: Utilizing improved 3D Gaussian splashing technology and fusing LiDAR point cloud prior information, a large-scale static road environment is reconstructed, and the scene is semantically decoupled.

[0009] Step 2, Dynamic Object and Behavior Injection: Assign an initial motion trajectory to each dynamic object represented by a learnable Bézier curve, and use a generative AI model to output adjustment instructions for the behavior of other traffic participants based on the semantic information of the current scene and the vehicle's planned trajectory, and fine-tune the control points of the Bézier curve.

[0010] Step 3: Multi-sensor simulation and data generation: Based on the reconstructed 3D scene and object state, perform high-fidelity simulation of multi-modal sensor data to simulate sensor attenuation, noise and multipath reflection effects under extreme weather conditions;

[0011] Step 4, Closed-loop evaluation and iterative optimization: Use the generated scenarios to test the autonomous driving algorithm, record its performance, and based on the test results, use the conditional generation capabilities of generative AI to generate long-tail scenarios in a targeted manner, forming an automated closed loop of testing, discovering weaknesses, and enhancing training.

[0012] Preferably, in step one, the improved 3D Gaussian splashing technology introduces semantic tags as parameters for Gaussian points, where each Gaussian point is represented as: G i =μ i ,Σ i ,c i ,α i ,s i , where s i For semantic tags;

[0013] The reconstruction loss function adds semantic consistency loss to the original photometric loss:

[0014] L total=L photo+λL semantic .

[0015] Preferably, in step two, the parametric equation of the Bézier curve is:

[0016] Among them, control point C i Encodes position, velocity, and acceleration information.

[0017] Preferably, in step two, the generative AI model is a diffusion model, and its training objective is:

[0018] The conditional information c includes the scene semantic graph St and the vehicle's planned trajectory Tego.

[0019] Preferably, in step three, the rain and snow effect transmittance model simulated by the camera is:

[0020] I out =I in e -βd +L ∞ (1-e -βd ), where β is the scattering coefficient, d is the depth, and I out Used as background light.

[0021] Preferably, in step three, the intensity attenuation model for lidar simulation is as follows:

[0022] I = I0e -μx , where μ is the attenuation coefficient and x is the propagation distance.

[0023] Preferably, in step four, the condition generation is achieved through a conditional diffusion model, and the condition information is a text description or key performance indicators, which are integrated into the generation process through a cross-attention mechanism.

[0024] Preferably, step four also includes a meta-learning optimization step, which adjusts the generation strategy through reinforcement learning, with the objective function being: Where W(s) is the weakness score of scenario s.

[0025] Preferably, in step two, the generative AI model is a world model, which uses a cyclic state-space model to predict the next state of a dynamic object.

[0026] A system for dynamic scene simulation generation method for autonomous driving includes: a static reconstruction module, a behavior injection module, a sensor simulation module, and a closed-loop optimization module.

[0027] Compared with the prior art, the beneficial effects of the present invention are:

[0028] The present invention has the following advantages over the prior art:

[0029] Enhanced realism of interaction: Generative AI simulates human driving behavior, and explicit trajectories ensure physical plausibility;

[0030] Improved testing efficiency: Conditional generation improves the efficiency of generating long-tail scenes by more than 50%;

[0031] High simulation confidence: Multi-sensor physical model reduces the gap between simulation and real vehicle;

[0032] Adaptive optimization: Meta-learning mechanism enables intelligent difficulty enhancement. Attached Figure Description

[0033] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0034] Figure 1 This is a flowchart of the dynamic scene simulation generation method for autonomous driving according to the present invention; Figure 2 This is a block diagram of the dynamic scene simulation generation system for autonomous driving according to the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Please see Figures 1 to 2 The present invention provides a technical solution:

[0038] A method for generating dynamic scene simulations for autonomous driving includes the following steps:

[0039] Step 1: Scene Initialization and Static Background Reconstruction: Utilizing improved 3D Gaussian splashing technology and fusing LiDAR point cloud prior information, a large-scale static road environment is reconstructed, and the scene is semantically decoupled.

[0040] Step 2, Dynamic Object and Behavior Injection: Assign an initial motion trajectory to each dynamic object represented by a learnable Bézier curve, and use a generative AI model to output adjustment instructions for the behavior of other traffic participants based on the semantic information of the current scene and the vehicle's planned trajectory, and fine-tune the control points of the Bézier curve.

[0041] Step 3: Multi-sensor simulation and data generation: Based on the reconstructed 3D scene and object state, perform high-fidelity simulation of multi-modal sensor data to simulate sensor attenuation, noise and multipath reflection effects under extreme weather conditions;

[0042] Step 4, Closed-loop evaluation and iterative optimization: Use the generated scenarios to test the autonomous driving algorithm, record its performance, and based on the test results, use the conditional generation capabilities of generative AI to generate long-tail scenarios in a targeted manner, forming an automated closed loop of testing, discovering weaknesses, and enhancing training.

[0043] Preferably, in step one, the improved 3D Gaussian splashing technology introduces semantic tags as parameters for Gaussian points, where each Gaussian point is represented as: G i =μ i ,Σ i ,c i ,α i ,s i , where s i For semantic tags;

[0044] The reconstruction loss function adds semantic consistency loss to the original photometric loss:

[0045] Ltotal=Lphoto+λL semantic .

[0046] Preferably, in step two, the parametric equation of the Bézier curve is:

[0047] Among them, control point C i Encodes position, velocity, and acceleration information.

[0048] Preferably, in step two, the generative AI model is a diffusion model, and its training objective is:

[0049] The conditional information c includes the scene semantic graph St and the vehicle's planned trajectory Tego.

[0050] Preferably, in step three, the rain and snow effect transmittance model simulated by the camera is:

[0051] I out =I in e -βd +L ∞ (1-e -βd ), where β is the scattering coefficient, d is the depth, and I out Used as background light.

[0052] Preferably, in step three, the intensity attenuation model for lidar simulation is as follows:

[0053] I = I0e -μx , where μ is the attenuation coefficient and x is the propagation distance.

[0054] Preferably, in step four, the condition generation is achieved through a conditional diffusion model, and the condition information is a text description or key performance indicators, which are integrated into the generation process through a cross-attention mechanism.

[0055] Preferably, step four also includes a meta-learning optimization step, which adjusts the generation strategy through reinforcement learning, with the objective function being: Where W(s) is the weakness score of scenario s.

[0056] Preferably, in step two, the generative AI model is a world model, which uses a cyclic state-space model to predict the next state of a dynamic object.

[0057] A system for dynamic scene simulation generation method for autonomous driving includes: a static reconstruction module, a behavior injection module, a sensor simulation module, and a closed-loop optimization module.

[0058] Example 1:

[0059] Step 1: Scene Initialization and Static Background Reconstruction

[0060] This step employs an improved 3D Gaussian splashing (3DGS) technique, integrating prior information such as LiDAR point clouds to achieve high-fidelity static environment reconstruction. Key technical points include:

[0061] 3DGS model with semantic decoupling:

[0062] Each Gaussian point contains a semantic label: G i =μ i ,Σ i ,c i ,α i ,s i}

[0063] Loss function: Ltotal = Lphoto + λL semantic

[0064] Where λL semantic Semantic consistency is calculated using a pre-trained segmentation network.

[0065] Prior information fusion:

[0066] LiDAR point clouds provide geometric constraints, accelerating Gaussian point initialization.

[0067] Multi-view images provide texture and color information

[0068] Step Two: Dynamic Object and Behavior Injection

[0069] A hybrid approach is used to inject dynamic behavior, combining explicit trajectories and generative AI:

[0070] Explicit trajectory modeling:

[0071] Bézier curve equation:

[0072] The control points Ci are learned through an optimization algorithm, and the loss function includes a smoothing term and physical constraints.

[0073] Generative AI-driven:

[0074] Training objective of diffusion model:

[0075] Conditional information c = (St, Tego) is injected through cross-attention.

[0076] Alternative: The world model uses state-space equations for prediction.

[0077] Step 3: Multi-sensor simulation and data generation

[0078] Physics-based sensor simulation:

[0079] Camera simulation:

[0080] Rasterization rendering base image

[0081] Weather effect: Ltotal = Lphoto + λL semantic

[0082] LiDAR Simulation:

[0083] Point cloud generation + noise model

[0084] Attenuation model: I = I0e -μx

[0085] Millimeter-wave radar simulation:

[0086] Ray tracing simulation of multipath effects

[0087] Radar equations for calculating echo intensity

[0088] Step 4: Closed-loop evaluation and iterative optimization

[0089] Forming an intelligent testing closed loop:

[0090] Evaluation module:

[0091] Record metrics such as collision count and rule violation rate.

[0092] Calculate the weakness score W(s)

[0093] Conditional generation:

[0094] Text condition: "Car broken down on highway during a rainstorm night"

[0095] Indicator conditions: High obstruction rate, low visibility

[0096] Meta-learning optimization:

[0097] Objective function:

[0098] The policy gradient updates the generated parameters.

[0099] Example 2: Urban Road Scene Simulation

[0100] Static Reconstruction: Training an Improved 3DGS Model Using LiDAR and Image Data from the KITTI Dataset; Behavior Injection: Pre-training a Diffusion Model on the Waymo Open Dataset.

[0101] Sensor simulation: Simulating camera and lidar data under rainy conditions

[0102] Closed-loop optimization: 1000 variant scenarios were generated to address the weaknesses of cut-in scenarios.

[0103] Example 3: Simulation of Extreme Weather on Highways

[0104] Condition generation: Text condition "Multiple vehicle collisions on highways during blizzard weather"

[0105] Sensor Attenuation: Simulating the Obstruction Effect of Heavy Snow on LiDAR

[0106] Evaluation metric: Braking distance of the test algorithm in low visibility conditions

[0107] Experimental data

[0108] Test results on the nuScenes dataset show:

[0109] Behavioral authenticity score improved by 35%

[0110] Long-tail scene generation speed increased by 5 times

[0111] The correlation between simulation and real vehicle testing reached 0.89.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating dynamic scene simulations for autonomous driving, characterized in that, Includes the following steps: Step 1: Scene Initialization and Static Background Reconstruction: Utilizing improved 3D Gaussian splashing technology and fusing LiDAR point cloud prior information, a large-scale static road environment is reconstructed, and the scene is semantically decoupled. Step 2, Dynamic Object and Behavior Injection: Assign an initial motion trajectory to each dynamic object represented by a learnable Bézier curve, and use a generative AI model to output adjustment instructions for the behavior of other traffic participants based on the semantic information of the current scene and the vehicle's planned trajectory, and fine-tune the control points of the Bézier curve. Step 3: Multi-sensor simulation and data generation: Based on the reconstructed 3D scene and object state, perform high-fidelity simulation of multi-modal sensor data to simulate sensor attenuation, noise and multipath reflection effects under extreme weather conditions; Step 4, Closed-loop evaluation and iterative optimization: Use the generated scenarios to test the autonomous driving algorithm, record its performance, and based on the test results, use the conditional generation capabilities of generative AI to generate long-tail scenarios in a targeted manner, forming an automated closed loop of testing, discovering weaknesses, and enhancing training.

2. The dynamic scene simulation generation method for autonomous driving according to claim 1, characterized in that, In step one, the improved 3D Gaussian splashing technology introduces semantic tags as parameters for Gaussian points, with each Gaussian point represented as: G i =μ i ,Σ i ,c i ,α i ,s i , where s i For semantic tags; The reconstruction loss function adds semantic consistency loss to the original photometric loss: Ltotal=Lphoto+λL semantic 。 3. The dynamic scene simulation generation method for autonomous driving according to claim 1, characterized in that, In step two, the parametric equation of the Bézier curve is: Among them, control point C i Encodes position, velocity, and acceleration information.

4. The dynamic scene simulation generation method for autonomous driving according to claim 1, characterized in that, In step two, the generative AI model is a diffusion model, and its training objective is: The conditional information c includes the scene semantic graph St and the vehicle's planned trajectory Tego.

5. The dynamic scene simulation generation method for autonomous driving according to claim 1, characterized in that, In step three, the rain and snow effect transmittance model simulated by the camera is as follows: I out =I in e -βd +L ∞ (1-e -βd ), where β is the scattering coefficient, d is the depth, and I out Used as background light.

6. The dynamic scene simulation generation method for autonomous driving according to claim 1, characterized in that, In step three, the intensity attenuation model for lidar simulation is as follows: I = I0e -μx , where μ is the attenuation coefficient and x is the propagation distance.

7. The dynamic scene simulation generation method for autonomous driving according to claim 1, characterized in that, In step four, the condition generation is achieved through a conditional diffusion model. The conditional information is either a text description or a key performance indicator, which is incorporated into the generation process through a cross-attention mechanism.

8. The dynamic scene simulation generation method for autonomous driving according to claim 1, characterized in that, Step four also includes a meta-learning optimization step, which adjusts the generation strategy through reinforcement learning. The objective function is: Where W(s) is the weakness score of scenario s.

9. The dynamic scene simulation generation method for autonomous driving according to claim 1, characterized in that, In step two, the generative AI model is a world model, which uses a cyclic state-space model to predict the next state of a dynamic object.

10. A system for implementing the method according to any one of claims 1-9, characterized in that, include: The module includes a static reconstruction module, a behavior injection module, a sensor simulation module, and a closed-loop optimization module.

Images