Key safety scene generation system for automatic driving automobile based on diffusion model

Abstract

The invention discloses a key safety scene generation system for an automatic driving automobile based on a diffusion model. The key safety scene generation system is characterized in that a traffic module performs de-noising processing on a noise track through the diffusion model and generates real diversified traffic scenes; the key safety scene generation module calls a mixed guidance function in each step of denoising process of the diffusion model to guide an opponent agent to collide with a self-agent so as to generate an antagonistic key safety scene; the expert trajectory optimization module performs multiple iterative optimization on the target function through an Adam optimizer to generate a safe and real expert trajectory; the planner enhancement module carries out fine adjustment on a pre-trained planner through the generated key safety scene and the expert track thereof, so that the ability of the planner to cope with extreme scenes is improved; the safety and robustness of the automatic driving planner in an extreme environment are improved by generating a real, diversified and antagonistic key safety scene.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving vehicles, and in particular to a key safety scenario generation system for autonomous driving vehicles based on a diffusion model. Background Art

[0002] At present, the existing technologies for generating critical security scenarios can be mainly divided into two types: methods based on real-world data collection and methods based on simulated environment generation.

[0003] The first type of method is based on real-world data collection. These methods extract critical safety scenarios from a large amount of real driving data, usually by analyzing historical data through clustering and other techniques to identify sparse critical scenarios. The advantage of this type of method is that it can discover potential safety issues from actual driving data and has high authenticity. However, since critical safety scenarios account for an extremely low proportion in daily driving data, it becomes very difficult and time-consuming to effectively extract such scenarios. In addition, the limited real-world data makes it impossible for these methods to cover all potential extreme scenarios, making it difficult to provide comprehensive training data for autonomous driving systems.

[0004] The second type of method is based on simulation environment generation. These methods use simulation tools to design critical safety scenarios and generate scenarios that may lead to collisions by manually setting the initial states of autonomous vehicles and other intelligent entities (such as position, speed, etc.). Although this method can make up for the lack of real data to a certain extent, the manual design of paths or parameters is very time-consuming and laborious when generating a large number of complex scenarios, and it is difficult to meet the needs of autonomous driving systems for diverse scenarios.

[0005] In order to solve the above problems, methods for automatically generating critical safety scenarios for adversarial situations have emerged in recent years. These methods generate more targeted scenarios through the optimized design of the opponent agent. Some studies guide the generation of collision scenarios by adjusting the initial state of the opponent agent, while other studies use techniques based on diffusion models, kinematic optimization, or reinforcement learning to generate more complex adversarial scenarios. These methods have made some progress in improving the efficiency and diversity of scenario generation, but the following problems still exist:

[0006] (1) Some methods do not fully consider the dynamic factors in complex traffic environments. In scenarios with multiple vehicles interacting with each other, the opponent agent may collide with other non-target vehicles in advance, resulting in the generated scenarios being unreasonable and unrealistic.

[0007] (2) Some methods are overly dependent on the original trajectory data. This dependence not only limits the diversity of the generated scenarios, but also cannot run without the original trajectory. In addition, the distribution of the generated scenario data is relatively simple, which makes it difficult to significantly improve the performance of the autonomous driving planner.

[0008] (3) Some methods lack end-to-end testing of generated scenarios on autonomous driving planners. Although these methods can generate highly targeted scenarios, they have failed to verify their actual improvement effect on planner performance, limiting their practical application value in autonomous driving system optimization.

[0009] In recent years, generative simulation technology has made important progress in the fields of natural language processing and computer vision, making controllable traffic simulation possible. Some methods achieve controllable generation of traffic scenarios by combining rule constraints or large model generation loss functions. Although such methods can meet different goals, they are often inefficient in generating specific adversarial critical safety scenarios and have difficulty generating high-quality adversarial scenarios.

[0010] In summary, although the existing technologies have made certain progress in generating critical adversarial safety scenarios, they still have shortcomings in terms of the rationality, diversity, and actual application effects of the generated scenarios, and cannot fully meet the needs of autonomous driving systems for diversified and efficient test scenarios. Summary of the invention

[0011] In order to overcome the shortcomings of the prior art, the present invention provides a key safety scenario generation system for autonomous driving vehicles based on a diffusion model, which improves the safety and robustness of the autonomous driving planner in extreme environments by generating realistic, diverse and adversarial key safety scenarios.

[0012] In order to achieve the above-mentioned invention object, the present invention adopts the following technical solutions:

[0013] A key safety scenario generation system for autonomous driving vehicles based on a diffusion model, comprising:

[0014] Traffic module, which denoises noise trajectories through diffusion model and generates realistic and diverse traffic scenes;

[0015] The key safety scenario generation module guides the opponent agent to collide with the self agent to generate adversarial key safety scenarios by calling the hybrid guidance function in each denoising process of the diffusion model;

[0016] The expert trajectory optimization module regenerates the trajectory of the self-agent by selecting the solution of the self-agent in the generated key safety scenarios, so that it can successfully avoid collisions in key scenarios, and optimizes the objective function multiple times through the Adam optimizer to generate safe and realistic expert trajectories;

[0017] The planner enhancement module fine-tunes the pre-trained planner by generating key safety scenarios and their expert trajectories, thereby improving its ability to cope with extreme scenarios. The generated key safety scenarios and expert trajectories are used as training data to fine-tune the existing data-driven planner, convert the expert trajectories into actions of the self-agent, and drive its movement in key scenarios.

[0018] Furthermore, the traffic module denoises the noise trajectory through the diffusion model and generates realistic and diverse traffic scenarios including:

[0019] Set initial conditions and use trajectory noise as input data for the traffic model;

[0020] Uniformly sample denoising steps from denoising steps [1, K] Loading the noise trajectory τ k The noise is directly loaded into the real trajectory τ 0 The loss function of supervised training is obtained, and its expression is as follows:

[0021]

[0022] Among them, L is the loss function of supervised training, To find the expectation for the noise, steps, clean trajectories, and agent decision context, τ 0 For a clean trajectory, Output results for the model.

[0023] De-noising the noise trajectory based on the diffusion model;

[0024] Based on the Transformer architecture, the dynamic interaction information between time, maps and intelligent agents is integrated to accurately simulate dynamic traffic interactions and generate realistic and diverse traffic scenarios.

[0025] Furthermore, the diffusion model includes a forward denoising submodule and a reverse denoising submodule. The forward denoising submodule performs a forward denoising process including: 0 ~q(τ 0 ), the forward diffusion process generates a trajectory sequence with gradually increasing noise by adding Gaussian noise at each step (τ 0 ,τ 1 ,…,τ K ), whose expression is as follows:

[0026]

[0027] Among them, q is the conditional probability of the model, τ k is the noise trajectory, β kis the noise coefficient at each step, I is the unit matrix, indicating the scale of the covariance, τ 0 is a clean trajectory, τ k-1 is the trajectory after denoising, is a Gaussian distribution, including its mean and covariance.

[0028] The reverse denoising submodule performs the reverse denoising process including: during the model training process, learning from the noise trajectory τ k Restore to clean trajectory τ 0 The reverse process of reverse diffusion; each step of the reverse diffusion is conditioned on the initial condition C of the agent's decision context, and its expression is as follows:

[0029]

[0030] Among them, p θ are the parameters of the model, C is the agent decision context, is a Gaussian distribution, ∑ θ is the covariance, which depends on the noise trajectory τ k , step τ k and the agent decision context C,τ k is the noise trajectory, τ k-1 is the trajectory after denoising.

[0031] Furthermore, the Transformer architecture includes a time Transformer module, a map Transformer module, and a social Transformer module. The time Transformer module extracts the dynamic interaction information of the agent's trajectory in the time dimension. The map Transformer module extracts the interaction characteristics between the agent and the surrounding environment based on map vectorization information. The social Transformer module is query-centric and models the symmetrical interactions between multiple agents through relative relationships, thereby improving the efficiency and authenticity of traffic flow simulation.

[0032] Furthermore, the hybrid guidance function includes an adversarial guidance function, an environmental guidance function, a collision guidance function and an initialization guidance function; wherein the adversarial guidance function is used to minimize the distance between the self-agent and the opponent's agent to increase the possibility of collision; the environmental guidance function is used to punish the agent for driving in an un-drivable area, and detects the collision between the agent and the environment by checking the overlap between the rasterized un-drivable map layer and the rasterized vehicle border; the collision guidance function is used to ensure that only the opponent and the self-agent collide when generating adversarial critical safety scenarios, and to avoid collisions between other agents; the initialization guidance function is used to minimize the impact of the guiding trajectory on the original trajectory when generating adversarial critical safety scenarios, thereby ensuring that the naturalness and rationality of the original trajectory are retained to the greatest extent in the process of creating the adversarial scenario.

[0033] Furthermore, the expert trajectory optimization module includes an original guidance submodule, an environment guidance submodule, a collision guidance submodule and an initialization guidance submodule. The original guidance submodule keeps the non-self agent running along the original trajectory of the critical safety scenario, so that the non-self agent follows the trajectory of the original critical safety scenario. The expression is as follows:

[0034]

[0035] Where γ is the attenuation factor, For original guidance, is the current agent V i The global coordinates of To fight against key security scenario agents V i The global coordinates of

[0036] The environment guidance submodule is used to ensure that the self-agent operates within the drivable area; the collision guidance submodule is used to minimize the collision risk between the self-agent and other agents; and the initialization guidance submodule is used to ensure the smoothness and authenticity of the self-agent trajectory.

[0037] Optionally, the enhanced capabilities of the planner are evaluated through stability evaluation indicators, authenticity evaluation indicators, and effectiveness evaluation indicators.

[0038] Furthermore, the planner enhancement module includes a stability index evaluation submodule, a authenticity index evaluation submodule and a validity index evaluation submodule. The stability index evaluation submodule evaluates the planner performance according to indicators such as collision rate and road deviation rate. The authenticity index evaluation submodule evaluates according to the distribution similarity between the generated trajectory and the real driving data. The validity index evaluation submodule evaluates the contribution of the generated collision rate and expert solution rate verification scenario to the planner optimization.

[0039] The beneficial effects of this application are: a query-centric multi-agent traffic generation model is implemented to focus on generating critical adversarial safety scenarios in complex dynamic traffic environments. The hybrid guidance function is used to optimize the process of traffic scenario generation model, which not only generates reasonable and feasible critical adversarial safety scenarios, but also provides corresponding expert solutions, thereby achieving an effective combination of scenario generation and response strategies. The generated expert solutions are used to fine-tune the existing autonomous driving planners, significantly improving the performance and robustness of these algorithms in dealing with critical adversarial safety scenarios.

[0040] It achieves the generation of realistic, diverse and adversarial key safety scenarios, and combines expert trajectory optimization to provide efficient training and test data for the autonomous driving planner. Through the multi-agent traffic generation model based on the diffusion model of the motion Transformer, the efficient generation of key safety scenarios is achieved in complex dynamic traffic environments. This technological breakthrough solves the shortcomings of existing methods in scene realism, generation diversity and performance optimization, enabling the autonomous driving planner to cope with low-frequency high-risk scenarios and significantly improve its safety and robustness.

[0041] The hybrid guidance mechanism combining adversarial guidance and expert trajectory optimization has been realized, which innovatively integrates scenario generation and response strategies. In the generation stage of key safety scenarios for adversarial situations, the dynamic interactive guidance function is introduced to make the scenario generation both reasonable and realistic; in the expert trajectory optimization stage, the solution trajectory of the self-agent is generated to ensure that the autonomous driving system can effectively avoid collisions in adversarial scenarios and improve the actual performance of the planner.

[0042] Through the Transformer-based model architecture, the map, time series and dynamic interaction information between agents are deeply integrated. Through the synergy of time transformer, map transformer and social transformer, the model can accurately simulate the dynamic interaction of traffic flow, which not only overcomes the dependence of existing methods on original trajectories, but also greatly improves the diversity and coverage of key safety scenarios.

[0043] We implemented an end-to-end test and evaluation method to verify the effective improvement of the performance of the autonomous driving planner by generating scenarios. By generating adversarial key safety scenarios and testing and fine-tuning them, we significantly enhanced the performance of the planner in extreme scenarios and demonstrated the practical application value of scenario generation technology in optimizing autonomous driving systems. This end-to-end testing method meets the core needs of users for the safety and reliability of autonomous driving systems in key safety scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is an overall framework diagram of a key safety scenario generation system for an autonomous driving vehicle based on a diffusion model of the present invention;

[0046] Figure 2It is a framework diagram of the generation model in the present invention;

[0047] Figure 3 It is an example diagram of the training process and reasoning process of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0049] The following describes the embodiments of the present invention through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0050] Embodiment 1:

[0051] A key safety scenario generation system for autonomous driving vehicles based on a diffusion model, comprising:

[0052] The traffic module uses a diffusion model to denoise the noise trajectory and generate realistic and diverse traffic scenes. The traffic module is used to generate stable, realistic and diverse traffic flow scenes. The diffusion model includes a forward denoising submodule and a reverse denoising submodule. The forward denoising submodule performs the forward denoising process including: 0 ~q(τ 0 ), the forward diffusion process generates a trajectory sequence with gradually increasing noise by adding Gaussian noise at each step (τ 0 ,τ 1 ,…,τ K ), whose expression is as follows:

[0053]

[0054] Among them, q is the conditional probability of the model, τ k is the noise trajectory, β k is the noise coefficient at each step, I is the unit matrix, indicating the scale of the covariance, τ 0 is a clean trajectory, τ k-1 is the trajectory after denoising, is a Gaussian distribution, including its mean and covariance.

[0055] The reverse denoising submodule performs the reverse denoising process including: during the model training process, learning from the noise trajectory τ K Restore to clean trajectory τ 0 Each step of the reverse diffusion is conditioned on the agent's decision context initial condition C, and its expression is as follows:

[0056]

[0057] Among them, p θ are the parameters of the model, C is the agent decision context, is a Gaussian distribution, ∑ θ is the covariance, which depends on the noise trajectory τ k , step τ k and the agent decision context C,τ k is the noise trajectory, τ k-1 is the trajectory after denoising.

[0058] It should be noted that in this process, the traffic model directly predicts the clean trajectory τ 0 , rather than directly predicting the mean μ of the noise trajectory, thereby improving the authenticity and diversity of the generated trajectory.

[0059] The traffic module denoises the noise trajectory through the diffusion model and generates realistic and diverse traffic scenarios including:

[0060] Set initial conditions and use trajectory noise as input data for the traffic model;

[0061] Uniformly sample denoising steps from denoising steps [1, K] Loading the noise trajectory τ k The noise is directly loaded into the real trajectory τ 0 The loss function of supervised training is obtained, and its expression is as follows:

[0062]

[0063] Among them, L is the loss function of supervised training, To find the expectation for the noise, steps, clean trajectories, and agent decision context, τ 0 For a clean trajectory, Output results for the model.

[0064] De-noising the noise trajectory based on the diffusion model;

[0065] Based on the Transformer architecture, the dynamic interaction information between time, maps and intelligent agents is integrated to accurately simulate dynamic traffic interactions and generate realistic and diverse traffic scenarios.

[0066] The Transformer architecture is used to construct the interaction between time series information, map environment information and agents. The Transformer architecture includes the time Transformer module, the map Transformer module and the social Transformer module. The time Transformer module is used to extract the dynamic interaction information of the agent trajectory in the time dimension. The map Transformer module is used to extract the interaction characteristics between the agent and the surrounding environment based on the map vectorization information. The social Transformer module is used to model the symmetrical interaction between multiple agents through relative relationships with query as the center, thereby improving the efficiency and authenticity of traffic flow simulation.

[0067] It should be noted that in order to guide the network, during the training process, the network will not directly predict the next noise trajectory τ k-1 Instead of the mean μ, we directly predict the clean trajectory τ 0 , uniformly sample denoising steps from denoising steps [1, K] Loading the noise trajectory τ k The noise of the step is directly added to the true trajectory τ 0 On, through in The direct output of the model is denoted as Get the loss function for supervised training.

[0068] The key safety scenario generation module is used to guide the opponent agent to collide with the self agent by calling the hybrid guidance function in each step of the denoising process of the diffusion model to generate adversarial key safety scenarios. The key safety scenario generation module includes a guidance sampling submodule and a guidance function submodule. By minimizing the distance between the self agent and the opponent agent based on the guidance function of the key safety scenario in each step of the denoising process of the diffusion model, it avoids generating other unreasonable scenarios.

[0069] It should be noted that the existing technology often cannot effectively simulate the complex dynamic interactions between multiple agents when generating key safety scenarios, which easily leads to collisions between opponent agents and non-target agents in the generated scenarios, and the irrationality and authenticity of scenario generation are difficult to guarantee. This application strengthens the interaction and timing understanding capabilities between agents by designing social Transformer and decoder modules, and ensures the rationality and authenticity of scenario generation through the collision guidance mechanism in the hybrid guidance function.

[0070] The guidance sampling submodule is used to add guidance functions for key safety scenarios in each step of the denoising process. The expression of the optimization objective is as follows:

[0071]

[0072] in, Used to minimize the distance between the self-agent and the opponent's agent and avoid other unreasonable scene generation, is the adversarial guidance function, is the environmental guidance function, is the collision guidance function, It is the initialization guidance function.

[0073] The hybrid guidance function includes adversarial guidance function, environmental guidance function, collision guidance function and initialization guidance function.

[0074] The adversarial guidance function is used to minimize the distance between the self-agent and the opponent agent to increase the probability of collision. Its expression is as follows:

[0075]

[0076] Among them, ξ t Defined as the opponent agent At time step t and the self-agent V 0 The softmin of the two-dimensional distance, are the global coordinates of the opponent agent, is the global coordinate of the self-agent.

[0077] The environmental guidance function is used to penalize the agent for driving in an undriveable area. It detects the collision between the agent and the environment by checking the overlap between the rasterized undriveable map layer and the rasterized vehicle border. Let the collision point between the agent and the environment be $c$, the average value of the pixel value in the overlapping area of ​​the point, and the expression of the environmental loss is as follows:

[0078]

[0079] Where d is the distance between the collision point c and the center of the vehicle, r i For the agent V i Half the border diagonal.

[0080] The environmental guidance function is the cumulative sum of the environmental losses of all agents under time decay, and its expression is as follows:

[0081]

[0082] Where γ is the attenuation factor, For environmental losses.

[0083] The collision guidance function is used to ensure that only the opponent and the self-agent collide when generating critical safety scenarios, and avoid collisions between other agents. The collision guidance function ensures that other agents maintain a safe distance when generating trajectories, thereby avoiding collisions. Optionally, simplified optimization is performed by using pairwise collision loss and efficient differentiable relaxation. For example, 2 circles are used to approximate each agent, and the L2 distance between the nearest circle centers of each pair of agents is calculated.

[0084] The collision guidance function is the cumulative sum of the collision losses of all agents under time decay, and its expression is as follows:

[0085]

[0086] Among them, γ is the attenuation factor, r i Represents the agent V i The radius of the approximate circle, Represents the agent V i The global coordinates at time t. In order not to calculate the collision guidance between the opponent and the self-agent, the values ​​of the corresponding positions were masked in the experiment.

[0087] The initialization guidance function is used to minimize the impact of the guidance trajectory on the original trajectory when generating the critical security scenario of the adversary, ensuring that the naturalness and rationality of the original trajectory are preserved to the greatest extent in the process of creating the adversarial scenario. The expression of the initialization loss is as follows:

[0088]

[0089] in, To initialize the guidance function, To optimize the previous agent V i The global coordinates of Represents the agent V i The global coordinates at time t.

[0090] The expert trajectory optimization module is used to select the solution of the self-agent in the generated key safety scenarios. By regenerating the trajectory of the self-agent, it can successfully avoid collisions in key scenarios, and optimizes the objective function multiple times through the Adam optimizer to generate safe and realistic expert trajectories.

[0091] The expert trajectory optimization module includes the original guidance submodule, the environment guidance submodule, the collision guidance submodule and the initialization guidance submodule. The original guidance submodule is used to keep the non-self agent running along the original adversarial key safety scenario trajectory, so that the non-self agent follows the trajectory of the original adversarial key safety scenario. Its expression is as follows:

[0092]

[0093] Where γ is the attenuation factor, For original guidance, For the agent V i The global coordinates at time t, To fight against key security scenario agents V i In order not to calculate the original guidance of the self-agent, the values ​​of the corresponding positions were masked in the experiment.

[0094] The environment guidance submodule is used to ensure that the ego-agent operates within the drivable area. The collision guidance submodule is used to minimize the risk of collision between the ego-agent and other agents. The initialization guidance submodule is used to ensure the smoothness and authenticity of the trajectory of the ego-agent.

[0095] It should be noted that most existing technologies are generated based on original trajectories, and it is difficult to generate scenes independently without the original trajectories. This dependence limits the diversified generation of adversarial scenes, and it is difficult to generate scenes in the absence of original trajectories. The present invention uses a generation mechanism based on a diffusion model, which only relies on the initial state, attributes and map information of the intelligent agent to generate future trajectories, thereby getting rid of the dependence on the original trajectory and significantly improving the diversity and coverage of the scenes.

[0096] The planner enhancement module is used to fine-tune the pre-trained planner through the generated key safety scenarios and their expert trajectories, thereby improving its ability to cope with extreme scenarios. The generated key safety scenarios and expert trajectories are used as training data to fine-tune the existing data-driven planner (such as trajectory prediction algorithm, etc.), convert the expert trajectory into the action of the self-agent, and drive its movement in the key scenarios. Optionally, the planner enhancement capability is evaluated through stability evaluation indicators, authenticity evaluation indicators, and effectiveness evaluation indicators.

[0097] The planner enhancement module includes a stability index evaluation submodule, a authenticity index evaluation submodule and a validity index evaluation submodule. The stability index evaluation submodule is used to evaluate the planner performance through indicators such as collision rate and road deviation rate. The authenticity index evaluation submodule is used to evaluate the distribution similarity between the generated trajectory and the real driving data. The validity index evaluation submodule is used to evaluate the contribution of the generated collision rate and expert solution rate verification scenarios to the planner optimization.

[0098] It should be noted that although some existing technologies can generate adversarial scenarios, they have not been able to deeply evaluate the impact of generated scenarios on the performance of autonomous driving planners, especially the insufficient contribution to the optimization of planner performance. This application proposes a two-stage strategy combining adversarial scenario generation and expert trajectory optimization, which not only generates adversarial key safety scenarios, but also provides expert solutions for coping with scenarios, and fine-tunes the existing planner through expert trajectory optimization, thereby significantly improving the robustness and performance of the planner in extreme scenarios.

[0099] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0100] The terms "first", "second" and "third" etc. in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0101] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0102] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A key safety scenario generation system for autonomous driving vehicles based on a diffusion model, characterized in that: include: Traffic module, which denoises noise trajectories through diffusion model and generates realistic and diverse traffic scenes; The key safety scenario generation module guides the opponent agent to collide with the self agent to generate adversarial key safety scenarios by calling the hybrid guidance function in each denoising process of the diffusion model; The expert trajectory optimization module regenerates the trajectory of the self-agent by selecting the solution of the self-agent in the generated key safety scenarios, so that it can successfully avoid collisions in key scenarios, and optimizes the objective function multiple times through the Adam optimizer to generate safe and realistic expert trajectories; The planner enhancement module fine-tunes the pre-trained planner by generating key safety scenarios and their expert trajectories, thereby improving its ability to cope with extreme scenarios. The generated key safety scenarios and expert trajectories are used as training data to fine-tune the existing data-driven planner, convert the expert trajectories into actions of the self-agent, and drive its movement in key scenarios.

2. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 1 is characterized in that: The traffic module denoises the noise trajectory through the diffusion model and generates real and diverse traffic scenes including: Set initial conditions and use trajectory noise as input data for the traffic model; Uniformly sample denoising steps from denoising steps [1, K] Loading the noise trajectory τ k The noise is directly loaded into the real trajectory τ 0 The loss function of supervised training is obtained, and its expression is as follows: Among them, L is the loss function of supervised training, To find the expectation for the noise, steps, clean trajectories, and agent decision context, τ 0 For a clean trajectory, Output results for the model. De-noising the noise trajectory based on the diffusion model; Based on the Transformer architecture, the dynamic interaction information between time, maps and intelligent agents is integrated to accurately simulate dynamic traffic interactions and generate realistic and diverse traffic scenarios.

3. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 1 is characterized in that: The diffusion model includes a forward denoising submodule and a reverse denoising submodule. The forward denoising submodule performs a forward denoising process including: 0 ~q(τ 0 ), the forward diffusion process generates a trajectory sequence with gradually increasing noise by adding Gaussian noise at each step (τ 0 ,τ 1 ,…,τ K ); The reverse denoising submodule performs the reverse denoising process including: during the model training process, learning from the noise trajectory τ K Restore to clean trajectory τ 0 The reverse process of the reverse diffusion; each step of the reverse diffusion is conditioned on the initial condition C of the agent's decision context.

4. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 3 is characterized in that: The denoising step is uniformly sampled from the denoising step [1, K] Loading the noise trajectory τ k The noise is directly loaded into the real trajectory τ 0 The loss function of supervised training is obtained, and its expression is as follows: Among them, L is the loss function of supervised training, To find the expectation for the noise, steps, clean trajectories, and agent decision context, τ 0 For a clean trajectory, Output results for the model.

5. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 3 is characterized in that: In the model training process, learning from the noise trajectory τ K Restore to clean trajectory τ 0 The reverse process of reverse diffusion; each step of the reverse diffusion is conditioned on the initial condition C of the agent's decision context, and its expression is as follows: Among them, p θ are the parameters of the model, C is the agent decision context, is a Gaussian distribution, ∑ θ is the covariance, which depends on the noise trajectory τ k , step τ k and the agent decision context C,τ k is the noise trajectory, τ k-1 is the trajectory after denoising.

6. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 1 is characterized in that: The Transformer architecture includes a time Transformer module, a map Transformer module and a social Transformer module. The time Transformer module extracts dynamic interaction information of the agent's trajectory in the time dimension. The map Transformer module extracts the interaction characteristics between the agent and the surrounding environment based on map vectorization information. The social Transformer module is query-centric and models the symmetrical interaction between multiple agents through relative relationships, thereby improving the efficiency and authenticity of traffic flow simulation.

7. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 1 is characterized in that: The hybrid guidance function includes an adversarial guidance function, an environmental guidance function, a collision guidance function and an initialization guidance function; wherein the adversarial guidance function is used to minimize the distance between the self-agent and the opponent's intelligent agent to increase the possibility of collision; the environmental guidance function is used to punish the intelligent agent for driving in an undriving area, and detects the collision between the intelligent agent and the environment by checking the overlap between the rasterized undriving map layer and the rasterized vehicle border; the collision guidance function is used to ensure that only the opponent and the self-agent collide when the target generates an adversarial key safety scenario, and avoid collisions between other intelligent agents; the initialization guidance function is used to minimize the impact of the guiding trajectory on the original trajectory when generating an adversarial key safety scenario, thereby ensuring that the naturalness and rationality of the original trajectory are retained to the greatest extent in the process of creating the adversarial scenario.

8. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 1 is characterized in that: The expert trajectory optimization module includes an original guidance submodule, an environment guidance submodule, a collision guidance submodule and an initialization guidance submodule; the original guidance submodule is used to keep the non-self-agent running along the original confrontation key safety scenario trajectory, so that the non-self-agent complies with the trajectory of the original confrontation key safety scenario; the environment guidance submodule is used to ensure that the self-agent runs within the drivable area; the collision guidance submodule is used to minimize the collision risk between the self-agent and other agents; and the initialization guidance submodule is used to ensure the smoothness and authenticity of the self-agent trajectory.

9. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 8 is characterized in that: The original guidance submodule is used to keep the non-self agent running along the original trajectory of the confrontation key safety scenario, so that the non-self agent follows the trajectory of the original confrontation key safety scenario. The expression is as follows: in, is the original guidance, γ is the attenuation factor, is the current agent V i The global coordinates of To fight against key security scenario agents V i The global coordinates of .

10. The key safety scenario generation system for autonomous driving vehicles based on a diffusion model according to claim 1, characterized in that: The planner enhancement module includes a stability index evaluation submodule, a authenticity index evaluation submodule and a validity index evaluation submodule. The stability index evaluation submodule evaluates the planner performance according to indicators such as collision rate and road deviation rate. The authenticity index evaluation submodule evaluates according to the distribution similarity between the generated trajectory and the real driving data. The validity index evaluation submodule evaluates the contribution of the generated collision rate and expert solution rate verification scenario to the planner optimization.

Images