A system for generating key safety scenarios for autonomous vehicles based on diffusion models

Abstract

The application discloses a kind of key safety scene generation systems of autonomous vehicle based on diffusion model, comprising: traffic module is handled to noise track by diffusion model and generates real multiple traffic scene;Key safety scene generation module is guided to happen collision between opponent intelligent agent and ego intelligent agent by calling mixed guide function in each step of the de-noising process of diffusion model, to generate antagonistic key safety scene;Expert trajectory optimization module carries out multiple iteration optimization to objective function by Adam optimizer, generates safe real expert trajectory;Planner enhancement module is fine-tuned to pre-trained planner by generated key safety scene and its expert trajectory, to improve its ability to cope with extreme scene;Real, multiple and antagonistic key safety scene is generated, and the safety and robustness of autonomous driving planner in extreme environment are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of autonomous vehicles, and particularly relates to a key safety scene generation system for autonomous vehicles based on a diffusion model. BACKGROUND

[0002] Currently, the existing technologies for generating adversarial key safety scenes can be mainly divided into two categories: real-world data collection-based methods and simulation environment generation-based methods.

[0003] The first category is real-world data collection-based methods. These methods extract adversarial key safety scenes from a large amount of real driving data, usually by analyzing historical data through clustering and other techniques to identify sparse key scenes. The advantage of this method is that it can discover potential safety problems from actual driving data, which is highly realistic. However, due to the extremely low proportion of adversarial key safety scenes in daily driving data, it is very difficult and time-consuming to effectively extract such scenes. In addition, the limited nature of real-world data makes it impossible for these methods to cover all potential extreme scenarios, making it difficult to provide comprehensive training data for autonomous driving systems.

[0004] The second category is simulation environment generation-based methods. These methods use simulation tools to design adversarial key safety scenes by manually setting the initial state of autonomous vehicles and other intelligent agents (such as position, speed, etc.), generating scenarios that may lead to collisions. Although this method can make up for the shortcomings of real data to some extent, manually designing paths or parameters is very time-consuming and laborious when generating a large number of complex scenarios, making it difficult to meet the needs of autonomous driving systems for diverse scenarios.

[0005] In order to solve the above problems, in recent years, methods for automatically generating adversarial key safety scenes have emerged. These methods generate more targeted scenarios through the optimization design of adversarial agents. Some studies adjust the initial state of adversarial agents to guide the generation of collision scenarios, while others use diffusion model-based, kinematic optimization, or reinforcement learning techniques to generate more complex adversarial scenarios. These methods have made some progress in improving the efficiency and diversity of scenario generation, but still have the following problems:

[0006] (1) Some methods do not fully consider dynamic factors in complex traffic environments. In multi-vehicle interaction scenarios, adversarial agents may collide with other non-target vehicles in advance, resulting in unrealistic and unrealistic generated scenarios.

[0007] (2) Some methods rely too much on original trajectory data. This dependency not only limits the diversity of generated scenarios, but also cannot run when there is no original trajectory. In addition, the distribution of generated scenario data is relatively single, making it difficult to significantly improve the performance of autonomous driving planners.

[0008] (3) Some methods lack end-to-end testing of generated scenarios on autonomous driving planners. Although these methods can generate targeted scenarios, they fail to verify their actual improvement effect on planner performance, limiting their practical application value in autonomous driving system optimization.

[0009] In recent years, significant progress has been made in natural language processing and computer vision using generative simulation techniques, making controllable traffic simulation possible. Some methods achieve controllable generation of traffic scenarios by combining rule constraints or large model generation loss functions. Although these methods can meet different goals, they often exhibit low efficiency in generating specific adversarial critical safety scenarios and are difficult to generate high-quality adversarial scenarios.

[0010] In summary, although the current technology has made some progress in generating adversarial critical safety scenarios, it still has shortcomings in the rationality, diversity, and practical application effect of generated scenarios, and cannot fully meet the needs of autonomous driving systems for diversified and efficient test scenarios. SUMMARY

[0011] To overcome the shortcomings of the prior art, the present application provides a diffusion model-based key safety scenario generation system for autonomous vehicles, which generates realistic, diverse, and adversarial key safety scenarios to improve the safety and robustness of autonomous driving planners in extreme environments.

[0012] To achieve the above invention purposes, the present application adopts the following technical solutions:

[0013] A diffusion model-based key safety scenario generation system for autonomous vehicles, comprising:

[0014] A traffic module that denoises noise trajectories using a diffusion model and generates realistic and diverse traffic scenarios;

[0015] A key safety scenario generation module that calls a hybrid guide function during each denoising process of the diffusion model to guide the collision between the adversary agent and the self agent, thereby generating adversarial key safety scenarios;

[0016] An expert trajectory optimization module that regenerates the trajectory of the self agent by selecting a solution for the self agent in the generated key safety scenario, enabling the self agent to successfully avoid collision in the key scenario, and iteratively optimizing the objective function using an Adam optimizer to generate safe and realistic expert trajectories;

[0017] The planner enhancement module fine-tunes the pre-trained planner by the generated key safety scenarios and their expert trajectories, so as to improve the ability of the planner to cope with extreme scenarios. The generated key safety scenarios and expert trajectories are used as training data to fine-tune the existing data-driven planner. The expert trajectory is converted into the action of the self-agent to drive the motion of the self-agent in the key scenario.

[0018] Further, the traffic module denoises the noise trajectory by a diffusion model and generates a real and diverse traffic scenario, including:

[0019] The initial condition is set, and the trajectory noise is taken as the input data of the traffic model;

[0020] Uniformly sampling the denoising step from the denoising step [1, K] The noise loaded on the noise trajectory τ k is directly loaded on the real trajectory τ 0 , and a loss function for supervised training is obtained, and the expression is as follows:

[0021]

[0022] Wherein, L is the loss function of supervised training, is the expectation of noise, step, clean trajectory and agent decision context, τ 0 is the clean trajectory, is the model output result,

[0023] The noise trajectory is denoised based on the diffusion model;

[0024] The dynamic interaction information among time, map and agent is fused based on the Transformer architecture to realize accurate simulation of traffic dynamic interaction, and a real and diverse traffic scenario is generated.

[0025] Further, the diffusion model includes a forward noise adding module and a reverse noise removing module. The forward noise adding module performs a forward noise adding process, including: starting from a clean real trajectory τ 0 ~ q(τ 0 ), the forward diffusion process generates a trajectory sequence (τ 0 , τ 1 ,…, τ K ) with gradually increasing noise by adding Gaussian noise at each step, and the expression is as follows:

[0026]

[0027] Wherein, q is the conditional probability of the model, τ k is the noise trajectory, and β kis the noise coefficient for each step, I is an identity matrix, represents the scale of the covariance, τ 0 is the clean trajectory, τ k-1 is the denoised trajectory, is a Gaussian distribution, specifically including the mean and covariance.

[0028] The inverse denoising module performs an inverse denoising process, including: in the model training process, learning an inverse process from the noise trajectory τ k to the clean trajectory τ 0 Each step of the inverse diffusion is conditioned on the agent decision context initial condition C, and its expression is as follows:

[0029]

[0030] where p θ is the parameter of the model, C is the agent decision context, is a Gaussian distribution, ∑ θ is the covariance, which depends on the noise trajectory τ k , the step τ k and the agent decision context C, τ k is the noise trajectory, τ k-1 is the denoised trajectory.

[0031] Further, the Transformer architecture includes a time Transformer module, a map Transformer module, and a social Transformer module. The time Transformer module extracts dynamic interaction information of the agent trajectory in the time dimension, the map Transformer module extracts interaction features between the agent and the surrounding environment based on the map vectorization information, and the social Transformer module takes the query as the center to model the symmetric interaction between multiple agents through the relative relationship, thereby improving the efficiency and authenticity of the traffic flow simulation.

[0032] Further, the hybrid guiding function includes an adversarial guiding function, an environment guiding function, a collision guiding function, and an initialization guiding function. The adversarial guiding function is used to minimize the distance between the self-agent and the opponent agent to increase the possibility of collision. The environment guiding function is used to punish the agent for driving in an un-drivable area, and the collision between the agent and the environment is detected by checking the overlap between the rasterized un-drivable map layer and the rasterized vehicle bounding box. The collision guiding function is used to ensure that only the opponent and the self-agent collide when generating the adversarial key safety scene, avoiding collisions between other agents. The initialization guiding function is used to minimize the influence of the original trajectory on the generated trajectory when generating the adversarial key safety scene, ensuring that the naturalness and rationality of the original trajectory are maximally preserved in the process of creating the adversarial scene.

[0033] Further, the expert trajectory optimization module includes an original guidance sub-module, an environment guidance sub-module, a collision guidance sub-module, and an initialization guidance sub-module. The non-self agent is kept running along the original trajectory of the confrontation key safety scene through the original guidance sub-module, so that the non-self agent complies with the trajectory of the original confrontation key safety scene, and the expression is as follows:

[0034]

[0035] wherein γ is a decay factor, is the original guidance, is the global coordinate of the current agent V i , and is the global coordinate of the confrontation key safety scene agent V i .

[0036] The self agent is ensured to run in the drivable area through the environment guidance sub-module, the collision risk between the self agent and other agents is minimized through the collision guidance sub-module, and the smoothness and authenticity of the self agent trajectory are ensured through the initialization guidance sub-module.

[0037] Optionally, the stability evaluation index, the authenticity evaluation index, and the effectiveness evaluation index are used to evaluate the enhanced ability of the planner.

[0038] Further, the planner enhancement module includes a stability index evaluation sub-module, an authenticity index evaluation sub-module, and an effectiveness index evaluation sub-module. The stability index evaluation sub-module evaluates the planner performance according to the collision rate, the road deviation rate, and the like, the authenticity index evaluation sub-module evaluates according to the distribution similarity between the generated trajectory and the real driving data, and the effectiveness index evaluation sub-module evaluates according to the contribution of the generated collision rate and the expert resolution rate to the optimization of the planner.

[0039] The application has the following beneficial effects: a query-centered multi-agent traffic generation model is realized, which focuses on generating confrontation key safety scenes in a complex dynamic traffic environment. A hybrid guidance function is used to optimize the process of the traffic scene generation model, which not only generates reasonable and feasible confrontation key safety scenes, but also provides expert solutions corresponding thereto, thereby realizing the effective combination of scene generation and coping strategies. The existing automatic driving planner is fine-tuned through the generated expert solutions, which significantly improves the performance and robustness of these algorithms in coping with confrontation key safety scenes.

[0040] Real, diverse and adversarial key safety scenarios are generated, and combined with expert trajectory optimization to provide efficient training and testing data for autonomous driving planners. Through the multi-agent traffic generation model based on the diffusion model of the motion Transformer, efficient generation of key safety scenarios is achieved in complex dynamic traffic environments. This technical breakthrough addresses the shortcomings of existing methods in scenario realism, generation diversity and performance optimization, enabling autonomous driving planners to handle low-frequency high-risk scenarios and significantly improving their safety and robustness.

[0041] A hybrid guidance mechanism combining adversarial guidance and expert trajectory optimization is implemented, which innovatively integrates scenario generation and response strategies. In the adversarial key safety scenario generation phase, a dynamic interaction guidance function is introduced to make the scenario generation both reasonable and realistic. In the expert trajectory optimization phase, the solution trajectory of the self-agent is generated to ensure that the autonomous driving system can effectively avoid collisions in adversarial scenarios and improve the actual performance of the planner.

[0042] A model architecture based on Transformer is implemented to deeply integrate map, time series and dynamic interaction information between agents. Through the collaborative action of time Transformer, map Transformer and social Transformer, the model can accurately simulate traffic flow dynamic interaction, not only overcoming the dependence on original trajectories of existing methods, but also significantly improving the diversity and coverage of key safety scenarios.

[0043] An end-to-end testing and evaluation method is implemented to verify the effective improvement of the generated scenarios on the performance of autonomous driving planners. By generating adversarial key safety scenarios and testing their performance, the planner's performance in extreme scenarios is significantly enhanced, and the practical application value of scenario generation technology in autonomous driving system optimization is demonstrated. This end-to-end testing method meets the core needs of users for the safety and reliability of autonomous driving systems in key safety scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0045] Figure 1 is the overall framework diagram of a key safety scenario generation system for an autonomous driving vehicle based on a diffusion model;

[0046] Figure 2is a framework diagram of a generation model in the present application;

[0047] Figure 3 is an example diagram of a training process and an inference process in the present application. DETAILED DESCRIPTION

[0048] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0049] The above and other advantages and effects of the present application will become readily apparent to those of ordinary skill in the art from the following description thereof taken in conjunction with the accompanying drawings. It is to be understood that the described embodiments are only a part of the embodiments of the present application, and are not all-inclusive of the embodiments of the present application. The present application can be implemented or applied in other different embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0050] Embodiment One:

[0051] A key safety scene generation system for an autonomous vehicle based on a diffusion model, comprising:

[0052] The traffic module denoises the noise trajectory through the diffusion model and generates a real and diverse traffic scene. The traffic module is used to generate stable, real and diverse traffic flow scenes. The diffusion model includes a forward noise adding module and a reverse noise removing module. The forward noise adding module performs a forward noise adding process, which includes starting from a clean real trajectory v 0 ~ q (τ 0 ), the forward diffusion process generates a trajectory sequence (τ 0 , τ 1 , …, τ K ) with gradually increasing noise by adding Gaussian noise at each step, and the expression is as follows:

[0053]

[0054] wherein q is the conditional probability of the model, τ k is the noise trajectory, β k is the noise coefficient at each step, I is the unit matrix, represents the scale of the covariance, τ 0 is the clean trajectory, τ k-1 is the denoised trajectory, is a Gaussian distribution, specifically including a mean and a covariance.

[0055] The reverse denoising sub-module performs a reverse denoising process, which includes learning, in a model training process, a reverse process from a noisy trajectory τ K to a clean trajectory τ 0 . Each step of the reverse diffusion is conditioned on the initial condition C of the agent decision context, which is expressed as follows:

[0056]

[0057] where p θ is the parameter of the model, C is the agent decision context, is a Gaussian distribution, and ∑ θ is a covariance, which depends on the noisy trajectory τ k , the step τ k , and the agent decision context C, τ k is the noisy trajectory, and τ k-1 is the denoised trajectory.

[0058] It should be noted that, in this process, the traffic model directly predicts the clean trajectory τ 0 rather than directly predicting the noisy trajectory mean μ, thereby improving the authenticity and diversity of the generated trajectory.

[0059] The traffic module generates a real and diverse traffic scenario by denoising the noisy trajectory through the diffusion model, which includes:

[0060] Setting the initial condition and taking the trajectory noise as the input data of the traffic model;

[0061] Uniformly sampling the denoising step τ from the denoising steps [1, K] Loading the noise on the noisy trajectory τ k directly onto the real trajectory τ 0 to obtain a supervised training loss function, which is expressed as follows:

[0062]

[0063] where L is the supervised training loss function, is the expectation of the noise, the step, the clean trajectory, and the agent decision context, τ 0 is the clean trajectory, is the model output result,

[0064] Denoising the noisy trajectory based on the diffusion model;

[0065] Fusing the dynamic interaction information among time, map, and agent based on the Transformer architecture to achieve accurate simulation of traffic dynamic interaction and generate a real and diverse traffic scenario.

[0066] The Transformer architecture is used to construct the time sequence information, the map environment information and the interaction relationship between the agents. The Transformer architecture includes a time Transformer module, a map Transformer module and a social Transformer module. The time Transformer module is used to extract dynamic interaction information of the agent trajectory in the time dimension. The map Transformer module is used to extract interaction features of the agent and the surrounding environment based on map vectorization information. The social Transformer module is used to model the symmetric interaction between multiple agents in a query-centered manner through relative relationship, so as to improve the efficiency and authenticity of the traffic flow simulation.

[0067] It should be noted that, in order to guide the network, the network does not directly predict the mean value μ of the next step noise trajectory τ k-1 during training, but directly predicts the clean trajectory τ 0 , uniformly samples the denoising step [1, K] from the denoising step The noise of the noise trajectory τ k step is directly loaded onto the real trajectory τ 0 , and the wherein The direct output of the model is denoted as The loss function of supervised training is obtained.

[0068] The critical safety scene generation module is used to guide the collision between the opponent agent and the self agent by calling the hybrid guidance function in each denoising process of the diffusion model, so as to generate an antagonistic critical safety scene. The critical safety scene generation module includes a guidance sampling sub-module and a guidance function sub-module. By minimizing the distance between the self agent and the opponent agent based on the guidance function of the critical safety scene in each denoising process of the diffusion model, other unreasonable scenes are avoided.

[0069] It should be noted that, in the prior art, when generating a critical safety scene, the complex dynamic interaction between multiple agents cannot be effectively simulated, which easily leads to the collision between the opponent agent and the non-target agent in the generated scene, and the rationality and authenticity of the scene generation are difficult to guarantee. The social Transformer and the decoder module are designed in the present application, the interaction and time sequence understanding ability between the agents are strengthened, and the rationality and authenticity of the scene generation are ensured through the collision guiding mechanism in the hybrid guidance function.

[0070] The guidance sampling sub-module is used to add the guidance function of the critical safety scene in each denoising process The expression of the optimization target is as follows:

[0071]

[0072] wherein, for minimizing the distance between the ego agent and the opponent agent and avoiding other unreasonable scenario generation, for the adversarial guiding function, for the environment guiding function, for the collision guiding function, for the initialization guiding function.

[0073] The hybrid guiding function includes the adversarial guiding function, the environment guiding function, the collision guiding function and the initialization guiding function.

[0074] The adversarial guiding function is used for minimizing the distance between the ego agent and the opponent agent to increase the collision possibility, and its expression is as follows:

[0075]

[0076] wherein, ξ t is defined as the two-dimensional distance between the opponent agent and the ego agent at time step t, the softmin of the two-dimensional distance between the ego agent V0 and the opponent agent at time step t, is the global coordinate of the opponent agent, is the global coordinate of the ego agent.

[0077] The environment guiding function is used for punishing the agent driving in the non-drivable area, and the collision between the agent and the environment is detected by checking the overlap between the rasterized non-drivable map layer and the rasterized vehicle bounding box. Assuming that the collision point between the agent and the environment is c, the average value of the pixel value of the overlapping area, the expression of the environment loss is as follows:

[0078]

[0079] wherein, d is the distance between the collision point c and the center of the vehicle, r i is the agent V i half of the diagonal line of the bounding box.

[0080] The environment guiding function is the cumulative sum of the environment loss of all agents under time decay, and its expression is as follows:

[0081]

[0082] wherein, γ is a decay factor, is the environment loss.

[0083] Collision guiding function, used to ensure that only the adversary and ego agent collide when generating the adversarial critical safety scenario, avoiding collisions between other agents. By using the collision guiding function, it is ensured that a safe distance is maintained between other agents when generating trajectories, thereby avoiding collisions. Optionally, simplified optimization is performed using pair-wise collision loss and efficient differentiable relaxation. For example, 2 circles are used to approximate each agent, and the L2 distance between the centers of the closest circles for each pair of agents is calculated.

[0084] The collision guiding function is the cumulative sum of the collision loss of all agents under time decay, and its expression is as follows:

[0085]

[0086] where γ is the decay factor, r i represents the global coordinates of agent V i The radius of the approximate circle, represents the global coordinates of agent V i at time t. In order to not calculate the collision guiding between the adversary and the ego agent, the corresponding position values are shielded by using masking in the experiment.

[0087] Initialization guiding function, used to minimize the influence of the guiding trajectory on the original trajectory when generating the adversarial critical safety scenario, ensuring that the naturalness and rationality of the original trajectory are maximally preserved in the process of creating the adversarial scenario. The expression of the initialization loss is as follows:

[0088]

[0089] where, is the initialization guiding function, is the global coordinates of agent V i before optimization, represents the global coordinates of agent V i at time t.

[0090] The expert trajectory optimization module is used to select the solution of the ego agent in the generated critical safety scenario, and by regenerating the trajectory of the ego agent, it can successfully avoid collision in the critical scenario. The Adam optimizer is used to perform multiple iterations of optimization on the objective function to generate a safe and realistic expert trajectory.

[0091] The expert trajectory optimization module includes an original guiding submodule, an environment guiding submodule, a collision guiding submodule, and an initialization guiding submodule. The original guiding submodule is used to keep the non-ego agents running along the original trajectory of the adversarial critical safety scenario, so that the non-ego agents comply with the trajectory of the original adversarial critical safety scenario, and its expression is as follows:

[0092]

[0093] wherein, gamma is an attenuation factor, is the original guidance, is the agent V i global coordinates at time t, global coordinates of the agent V i for the critical safety scene. In order to not calculate the original guidance of the ego agent, the corresponding position value is shielded in the experiment in a masked manner.

[0094] The environment guidance sub-module is used to ensure that the ego agent runs within the drivable area. The collision guidance sub-module is used to minimize the collision risk of the ego agent with other agents. The initialization guidance sub-module is used to ensure the smoothness and authenticity of the ego agent trajectory.

[0095] It should be noted that most of the prior art is generated based on the original trajectory, and it is difficult to generate scenes independently of the original trajectory. This dependence limits the diversification of the generated scenes, and it is difficult to generate scenes when the original trajectory is lacking. The present application generates future trajectories only by relying on the initial state, attributes and map information of the agent based on the generation mechanism of the diffusion model, thereby breaking the dependence on the original trajectory and significantly improving the diversity and coverage of the scene.

[0096] The planner enhancement module is used to fine-tune the pre-trained planner through the generated critical safety scene and its expert trajectory, thereby improving its ability to cope with extreme scenes. The generated critical safety scene and expert trajectory are used as training data to fine-tune the existing data-driven planner (such as trajectory prediction algorithm, etc.), convert the expert trajectory into the action of the ego agent, and drive its movement in the critical scene. Optionally, the stability evaluation index, the authenticity evaluation index and the effectiveness evaluation index are used to evaluate the ability of the planner enhancement.

[0097] The planner enhancement module includes a stability index evaluation sub-module, a reality index evaluation sub-module and an effectiveness index evaluation sub-module. The stability index evaluation sub-module is used to evaluate the planner performance through indicators such as collision rate and road deviation rate. The authenticity index evaluation sub-module is used to evaluate the distribution similarity between the generated trajectory and the real driving data. The effectiveness index evaluation sub-module is used to evaluate the contribution of the scene to the optimization of the planner by verifying the generated collision rate and the expert resolution rate.

[0098] It needs to be explained that although part of the prior art can generate adversarial scenes, it fails to assess the impact of generated scenes on the performance of the autonomous driving planner, especially the insufficient contribution to the optimization of planner performance. The present application proposes a two-stage strategy combining adversarial scene generation and expert trajectory optimization, which not only generates adversarial key safety scenes, but also provides expert solutions to deal with the scenes, and fine-tunes the existing planner through expert trajectory optimization, thereby significantly improving the robustness and performance of the planner in extreme scenes.

[0099] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0100] The terms "first", "second", and "third" and the like in the specification of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0101] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0102] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the foregoing embodiments of the present application have been described in detail, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A key safety scenario generation system for autonomous vehicles based on a diffusion model, characterized in that, The method comprises the following steps: a traffic module for denoising the noise trajectory by a diffusion model and generating a real and diverse traffic scene; a key safety scene generation module for generating an adversarial key safety scene by calling a hybrid guide function in each step of the denoising process of the diffusion model to guide the collision between the opponent agent and the self agent; an expert trajectory optimization module for regenerating the trajectory of the self agent in the generated key safety scene to enable the self agent to successfully avoid collision in the key scene, and iteratively optimizing the objective function by an Adam optimizer to generate a safe and real expert trajectory; a planner enhancement module for fine-tuning the pre-trained planner by the generated key safety scene and the expert trajectory to improve the ability of the planner to cope with extreme scenes, and using the generated key safety scene and the expert trajectory as training data to fine-tune the existing data-driven planner, and converting the expert trajectory into the action of the self agent to drive the motion of the self agent in the key scene; the traffic module denoises the noise trajectory by a diffusion model and generates a real and diverse traffic scene, comprising: setting an initial condition and taking the trajectory noise as the input data of the traffic model; From the denoising step Uniformly sampling the denoising step Noise loaded in the noise trajectory Noise loaded directly on the real trajectory The loss function for supervised training is obtained, whose expression is as follows: wherein, is a loss function for supervised training, is an expectation over noise, steps, clean trajectories, and agent decision context, is a clean trajectory, is a model output result, ; denoising the noise trajectory based on the diffusion model; fusing the dynamic interaction information among time, map and agent based on the Transformer architecture to realize accurate simulation of traffic dynamic interaction and generate a real and diverse traffic scene; The diffusion model includes a forward noise adding module and a reverse noise removing module. The forward noise adding module performs a forward noise adding process including: adding Gaussian noise to the clean real trajectory at each step to obtain a trajectory sequence with gradually increasing noise Initially, the forward diffusion process generates a trajectory sequence with gradually increasing noise by adding Gaussian noise at each step ; The inverse denoising sub-module performs an inverse denoising process including: in the model training process, learning an inverse process from a noisy trajectory to a clean trajectory ; each step of the inverse diffusion is conditioned on the initial conditions of the agent decision context . the hybrid guide function comprises an adversarial guide function, an environment guide function, a collision guide function and an initialization guide function; wherein the adversarial guide function is used to minimize the distance between the self agent and the opponent agent to increase the possibility of collision; the environment guide function is used to punish the agent for driving in an un-drivable area, and the collision between the agent and the environment is detected by checking the overlap between the rasterized un-drivable map layer and the rasterized vehicle bounding box; the collision guide function is used to ensure that only the opponent and the self agent collide when generating the adversarial key safety scene, avoiding the collision between other agents; and the initialization guide function is used to minimize the influence of the generated trajectory on the original trajectory when generating the adversarial key safety scene, so as to maximize the preservation of the naturalness and rationality of the original trajectory in the process of creating the adversarial scene. 2.The system for automatic generation of critical safety scenarios for diffusion model based autonomous vehicles according to claim 1, wherein, The in model training process, learning from noisy trajectories Recovering to clean trajectories The inverse process; each step of inverse diffusion conditioned on the agent decision context initial condition whose expression is as follows: wherein, are parameters of the model, is an agent decision context, is a Gaussian distribution, is a covariance, depending on the noise trajectory , steps and the agent decision context , is a noise trajectory, is a denoised trajectory. 3.The diffusion model based automatic driving car critical safety scenario generation system of claim 1, wherein, the Transformer architecture comprises a time Transformer module, a map Transformer module and a social Transformer module, the time Transformer module extracts the dynamic interaction information of the agent trajectory in the time dimension, the map Transformer module extracts the interaction features of the agent and the surrounding environment based on the map vectorization information, and the social Transformer module takes the query as the center to model the symmetric interaction among multiple agents through relative relationship, thereby improving the efficiency and reality of traffic flow simulation.

4. The system for automatic generation of critical safety scenarios for diffusion model based autonomous vehicles of claim 1, wherein, The expert trajectory optimization module comprises an original guidance submodule, an environment guidance submodule, a collision guidance submodule and an initialization guidance submodule; the original guidance submodule is used to keep the non-self agent running along the original trajectory of the key safety scene, so that the non-self agent complies with the trajectory of the original key safety scene; the environment guidance submodule is used to ensure that the self agent runs in the drivable area; the collision guidance submodule is used to minimize the collision risk between the self agent and other agents; and the initialization guidance submodule is used to ensure the smoothness and authenticity of the trajectory of the self agent.

5. The system for automatic generation of critical safety scenarios for diffusion model based autonomous vehicles according to claim 4, wherein, The expression of the original guidance submodule is as follows: wherein, is the original guide, is the decay factor, is the current agent is the global coordinate of the agent, is the global coordinate of the agent against critical security scenarios.

6. The system for automatic generation of critical safety scenarios for diffusion model based autonomous vehicles of claim 1, wherein, The planner enhancement module comprises a stability index evaluation submodule, an authenticity index evaluation submodule and an effectiveness index evaluation submodule; the stability index evaluation submodule is used to evaluate the performance of the planner according to the collision rate and the road deviation rate index; the authenticity index evaluation submodule is used to evaluate the distribution similarity between the generated trajectory and the real driving data; and the effectiveness index evaluation submodule is used to evaluate the contribution of the generated collision rate and the expert resolution rate to the optimization of the planner.

Images