Natural data set-oriented safety key scene generalization generation method and system
By constructing a scenario diffusion model and a hierarchical scheme, the problem of insufficient data acquisition for safety-critical scenarios in existing technologies has been solved, enabling the efficient generation of high-quality and diverse safety-critical scenarios and improving the safety assessment of autonomous driving systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2026-04-03
AI Technical Summary
Existing technologies struggle to efficiently acquire high-quality and diverse safety-critical scenario data, resulting in insufficient safety assessments of autonomous driving systems, especially in complex traffic environments.
By constructing a basic scene diffusion model, the map information and traffic participant status are encoded and decoded using a scene encoder and diffusion decoder. The safety criticality score is calculated by combining noise processing and a hierarchical scheme, and safety critical scenes are selected. The trained model is then used to generate future state sequences.
It effectively reduces the time and financial costs of safety-critical scenarios, improves the quality and diversity of generated safety-critical scenarios, and enhances the safety assessment capabilities of autonomous driving systems.
Smart Images
Figure CN121786471A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, and more specifically, to a method and system for generalizing the generation of safety-critical scenarios for natural datasets. Background Technology
[0002] Autonomous driving technology has been developing for a long time and is becoming increasingly mature. However, in the open, complex, and uncertain traffic environments of the real world, the safety of autonomous driving systems cannot yet be fully guaranteed, which limits their further application. Therefore, safety assessment of autonomous driving systems is crucial. For autonomous driving technology at its current stage, most ordinary scenarios no longer pose a significant threat to the safety of autonomous driving systems, but a small number of safety-critical scenarios still pose a significant threat. Therefore, how to obtain rich data on safety-critical scenarios to assess the safety of autonomous driving systems is a current research hotspot.
[0003] Currently, there are two main categories of methods for acquiring safety-critical scenarios. The first category involves identifying and extracting safety-critical scenarios from natural driving scenario data. This type of method collects a large amount of natural driving scenario data and analyzes this data using methods such as anomaly detection and clustering to identify and extract safety-critical scenarios that may lead to accidents or pose a high risk. The second category involves directly generating safety-critical scenarios using deep generative models. This type of method trains the model to learn the inherent characteristics and patterns of driving scenarios, and then uses the trained model to directly generate the required safety-critical scenarios.
[0004] While methods that generate natural datasets from real-world driving scenario data can yield high-quality, realistic safety-critical scenarios, the extreme imbalance between ordinary and safety-critical scenarios, coupled with the scarcity of safety-critical scenarios, makes identifying and extracting safety-critical scenarios from these datasets extremely inefficient. Even with significant time and financial investment, obtaining a sufficient number of safety-critical scenarios remains a challenge. Methods that directly generate safety-critical scenarios using deep generative models offer a significant advantage in generation efficiency. However, the current limitations of deep generative models, combined with the inherent challenges of directly generating safety-critical scenarios, result in severely insufficient quality, particularly in terms of scenario realism. Furthermore, methods using deep generative models to directly generate safety-critical scenarios often suffer from limited diversity in the generated scenarios due to the simplistic and uncontrollable behavioral patterns of traffic participants. Summary of the Invention
[0005] This invention provides a method and system for generalizing the generation of safety-critical scenarios for natural datasets, in order to overcome at least one technical problem existing in the prior art.
[0006] On the one hand, this invention provides a method for generalizing and generating safety-critical scenarios for natural datasets, including:
[0007] Collect scene data from multiple natural driving scenarios;
[0008] All the aforementioned scene data are labeled to obtain a natural scene dataset. Each scene in the natural scene dataset is represented as s = (M, A), where M represents map information and A represents the state sequence A of all traffic participants; the state sequence A of traffic participants includes the historical state sequence A. his and the actual future state sequence
[0009] For the true future state sequence Noise is added to obtain a noisy future state sequence.
[0010] Construct a basic scene diffusion model, which includes a scene encoder and a diffusion decoder;
[0011] The map information M and the historical state sequence A his The input scene encoder is used for encoding to obtain encoding condition c; the encoding condition c includes map encoding and traffic participant historical state encoding.
[0012] The noisy future state sequence The encoding condition c and the noise level σ are input to the diffusion decoder, which performs K-step iterative decoding to obtain the generated future state sequence.
[0013] Using the real future state sequence and the generated future state sequence The basic scene diffusion model is iteratively updated to obtain the trained scene diffusion model;
[0014] A hierarchical scheme is used to calculate the safety criticality score for each scene in the natural scene dataset;
[0015] Compare the safety criticality score with a preset threshold, and select scenarios whose safety criticality score is greater than the preset threshold as safety critical scenarios;
[0016] The trained scenario diffusion model is used to generate a sequence of future states for the safety-critical scenario.
[0017] Optionally, for the true future state sequence Noise addition processing is performed, specifically as follows:
[0018] Sampling noise ∈, using the noise ∈ to obtain the true future state sequence Noise is added.
[0019] Optionally, the actual future state sequence is utilized. and the generated future state sequence The basic scene diffusion model is iteratively updated as follows:
[0020] Based on the true future state sequence and the generated future state sequence Calculate the mean square error loss
[0021] The convergence of the basic scene diffusion model is determined based on the mean squared error loss L. If convergence occurs, the current basic scene diffusion model is used as the trained scene diffusion model. If convergence fails, the parameters θ of the basic scene diffusion model are updated, and the noisy future state sequence is returned. The encoding condition c and the noise level σ are input to the diffusion decoder for a K-step iterative decoding process.
[0022] Optionally, a hierarchical scheme is used to calculate the safety criticality score for each scene in the natural scene dataset, specifically:
[0023] Extract low-level features of traffic participants in the scenario;
[0024] Based on the aforementioned low-level features, calculate the total safety criticality score for each traffic participant i;
[0025] The safety criticality scores of all traffic participants in the scenario are weighted, summed, and normalized to obtain the safety criticality score of the scenario.
[0026] Optionally, low-level features of traffic participants in the scenario are extracted, specifically:
[0027] Extract individual features of each traffic participant i at each time step t. Obtain the set of individual features F ind ;
[0028] Extracting the social characteristics of each traffic participant i at each time step t Obtain the set of social characteristics F soc The social characteristics refer to the interaction between traffic participant i and traffic participant j.
[0029] Optionally, based on the aforementioned low-level features, a total safety criticality score is calculated for each traffic participant i, specifically as follows:
[0030] Based on the individual characteristic set F ind The individual safety criticality score for each traffic participant i was calculated as follows: Among them, w ind f represents the weight vector of individual characteristics. t (i) represents the safety criticality value of a single individual characteristic of traffic participant i at time step t;
[0031] According to the social characteristic set F soc The social safety criticality score for each traffic participant i was calculated as follows: Among them, w soc The weight vector f represents the social characteristics. t (i,j) represents the safety criticality value of a single social characteristic of traffic participant i at time step t;
[0032] The individual safety criticality score and the social safety criticality score of each traffic participant i are added together to obtain the total safety criticality score of traffic participant i: Score(i) = IndScore(i) + ∑ i≠j SocScore(i,j).
[0033] Optionally, the total safety criticality scores of all traffic participants in the scenario are weighted and summed, specifically as follows:
[0034] Set the participation weight w(i) for each traffic participant i in the scenario;
[0035] Based on the participation weight w(i), according to the formula The safety criticality scores of all traffic participants in the scenario are weighted, summed, and normalized to obtain the safety criticality score (Score) for the scenario; where N a This indicates the total number of traffic participants.
[0036] Optionally, it also includes:
[0037] Set a target guiding function J to characterize the behavioral style of traffic participants; target function J = J acc +J vel +J pos +J thw , where J acc J represents the maximum acceleration guided function for traffic participants. vel J represents the target speed guided function for traffic participants. pos J is a guided function representing the relative positions of traffic participants.thw The headway guidance function represents the headway of traffic participants.
[0038] Optionally, the trained scene diffusion model is used to generate the future state sequence of the safety-critical scene, specifically as follows:
[0039] The noisy future state sequence corresponding to the safety-critical scenario Input the trained scene diffusion model and perform K-step iterative denoising; in each iterative denoising process, use the target guiding function J to optimize the result.
[0040] On the other hand, the present invention also provides a generalized generation system for safety-critical scenarios oriented towards natural datasets, comprising:
[0041] The data acquisition module is used to collect scene data from multiple natural driving scenarios;
[0042] The annotation module is used to annotate all the scene data to obtain a natural scene dataset. Each scene in the natural scene dataset is represented as s = (M, A), where M represents map information and A represents the state sequence of all traffic participants. The state sequence A of traffic participants includes the historical state sequence A. his and the actual future state sequence
[0043] A noise-adding module is used to add noise to the real future state sequence. Noise is added to obtain a noisy future state sequence.
[0044] A building module is used to build a basic scene diffusion model, which includes a scene encoder and a diffusion decoder;
[0045] The first input module is used to input the map information M and the historical state sequence A. his The input scene encoder is used for encoding to obtain encoding condition c; the encoding condition c includes map encoding and traffic participant historical state encoding.
[0046] The second input module is used to input the noisy future state sequence. The encoding condition c and the noise level σ are input to the diffusion decoder, which performs K-step iterative decoding to obtain the generated future state sequence.
[0047] An update module is used to utilize the actual future state sequence. and the generated future state sequence The basic scene diffusion model is iteratively updated to obtain the trained scene diffusion model;
[0048] The computation module is used to calculate the safety criticality score for each scene in the natural scene dataset using a hierarchical scheme;
[0049] The comparison module is used to compare the safety criticality score with a preset threshold and filter out scenarios where the safety criticality score is greater than the preset threshold as safety critical scenarios.
[0050] The generation module is used to generate a future state sequence of the safety-critical scenario using the trained scenario diffusion model.
[0051] The innovative aspects of this invention include:
[0052] 1. In this embodiment, a safety criticality score for each scene in the natural scene dataset is calculated using a hierarchical scheme. By comparing the safety criticality score with a preset threshold, safety critical scenes in the natural dataset are extracted. Compared with the existing method of directly identifying and extracting safety critical scenes from the natural dataset, this method can not only extract and generalize to generate a sufficient number of safety critical scenes, but also effectively reduce the time and financial costs of generating safety critical scenes. This is one of the innovations of this embodiment.
[0053] 2. In this embodiment, based on the significant advantages of the samples generated by the diffusion model in terms of fidelity and diversity, the diffusion model is used to generalize and generate safety-critical scenarios identified and extracted from natural driving scenario data. This combines the advantages of both identification and extraction and direct generation methods, and can efficiently generate more high-quality safety-critical scenarios, which is one of the innovations of this embodiment. Attached Figure Description
[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0055] Figure 1 A flowchart of a generation method provided in an embodiment of the present invention;
[0056] Figure 2 Another flowchart of the generation method provided in the embodiments of the present invention;
[0057] Figure 3 A flowchart illustrating the safety criticality score for a computational scenario provided in this embodiment of the invention;
[0058] Figure 4A flowchart for extracting low-level features provided in an embodiment of the present invention;
[0059] Figure 5 A flowchart for calculating the total safety criticality score provided in an embodiment of the present invention;
[0060] Figure 6 A flowchart illustrating the safety criticality score for a computational scenario provided in this embodiment of the invention;
[0061] Figure 7 Another flowchart of the generation method provided in the embodiments of the present invention;
[0062] Figure 8 This is a schematic diagram of a workflow after adding a target guiding function, as provided in an embodiment of the present invention.
[0063] Figure 9 This is a schematic diagram of a generation system provided in an embodiment of the present invention. Detailed Implementation
[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0065] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.
[0066] This invention discloses a method and system for generalizing the generation of safety-critical scenarios for natural datasets. These will be described in detail below.
[0067] Figure 1 A flowchart of a generation method provided in an embodiment of the present invention is provided below. Figure 1 The method for generalizing security-critical scenarios for natural datasets provided in this invention includes:
[0068] Step 1: Collect scene data for multiple natural driving scenarios;
[0069] Step 2: Label all scene data to obtain a natural scene dataset. Each scene in the natural scene dataset is represented as s = (M, A), where M represents map information and A represents the state sequence A of all traffic participants; the state sequence A of traffic participants includes the historical state sequence A. his and the actual future state sequence
[0070] Step 3: Analyze the actual future state sequence Noise is added to obtain a noisy future state sequence.
[0071] Step 4: Construct a basic scene diffusion model, which includes a scene encoder and a diffusion decoder;
[0072] Step 5: Combine map information M and historical state sequence A his The input scene encoder is used for encoding to obtain encoding condition c; encoding condition c includes map encoding and traffic participant historical state encoding.
[0073] Step 6: Sequence of noisy future states The encoding condition c and the noise level σ are input to the diffusion decoder, which performs K-step iterative decoding to obtain the generated future state sequence.
[0074] Step 7: Utilize the real future state sequence and the generated future state sequence Iteratively update the basic scene diffusion model to obtain the trained scene diffusion model;
[0075] Step 8: Calculate the safety criticality score for each scene in the natural scene dataset using a hierarchical scheme;
[0076] Step 9: Compare the safety criticality score with the preset threshold, and select scenarios with a safety criticality score greater than the preset threshold as safety critical scenarios;
[0077] Step 10: Generate the future state sequence of safety-critical scenarios using the trained scenario diffusion model.
[0078] For details, please refer to Figure 1 The safety-critical scenario generalization generation method for natural datasets provided in this embodiment of the invention first collects scenario data for multiple natural driving scenarios in step 1. It should be noted that since the collected scenario data is used for model training, and model training typically requires a large amount of data, the scenario data for multiple natural driving scenarios here refers to a large amount of scenario data for natural driving scenarios.
[0079] Because the data formats of the collected scene data are inconsistent, in order to ensure that the input data format is consistent during model training, this invention performs uniform format annotation processing on all collected scene data in step 2 to form a natural scene dataset. During annotation, each scene needs to be annotated with high-precision map information M and the state sequences A of all traffic participants. Each scene in the resulting natural scene dataset is represented as s = (M, A), where the state sequences A of traffic participants need to be further annotated as historical state sequences Ai. his and the actual future state sequence The high-precision map information M includes road information, such as lane lines, lane connections, and road center lines. The state sequence A includes the state information of each traffic participant at each time step, such as position, speed, and heading angle.
[0080] After annotation is completed, the annotated scene data can be used for model training. First, in step 3, noise ∈ is sampled, and this noise ∈ is used to obtain the true future state sequence of traffic participants. Noise is added to obtain a noisy future state sequence.
[0081] To train the model, a basic scene diffusion model must first be constructed in step 4, and then this basic scene diffusion model is trained. In this invention, the basic scene diffusion model includes a scene encoder and a diffusion decoder.
[0082] Because it is necessary to separately process map information M and historical state sequences of traffic participants A his Encoding is performed; therefore, the scene encoder includes a map encoder and a traffic participant encoder. In step 5, the map information M is input into the map encoder for encoding to obtain the map code; the historical state sequence A is... his The input traffic participant encoder is used for encoding to obtain the traffic participant historical status code. The map code and the traffic participant historical status code together constitute the encoding condition c.
[0083] In step 6, the noisy future state sequence Decoding is then performed. During decoding, the noisy future state sequence is first... The encoding condition c and the noise level σ are input into the diffusion decoder, which performs one decoding operation to obtain a noisy sequence of future states. Then the noisy future state sequence The encoding condition c and the noise level σ are input into the diffusion decoder, which performs one decoding operation to obtain a noisy sequence of future states. Iteratively perform the above decoding operation K times until the generated future state sequence is obtained.
[0084] Obtain the generated future state sequence Then, in step 7, the generated future state sequence is compared. With the actual future state sequence The error between the two is used to iteratively update the basic scene diffusion model until the error between the two meets the requirements. Then, the corresponding scene diffusion model is used as the trained scene diffusion model.
[0085] In this invention, a scene's safety criticality score determines whether it is a safety critical scene. Therefore, step 8 is used to calculate the scene's safety criticality score. In this embodiment, a hierarchical scheme is used to calculate the safety criticality score of each scene in the natural scene dataset. Then, in step 9, safety critical scenes and ordinary scenes are filtered out by comparing the safety criticality score with a preset threshold. When a scene's safety criticality score is greater than the preset threshold, the scene is considered a safety critical scene; otherwise, it is considered an ordinary scene.
[0086] After obtaining the safety-critical scenario, in step 10, the safety-critical scenario is input into the trained scenario diffusion model to generate the corresponding future state sequence of the safety-critical scenario.
[0087] The safety-critical scenario generalization generation method for natural datasets provided by this invention calculates the safety criticality score of each scenario in the natural scenario dataset through a hierarchical scheme, and extracts the safety-critical scenarios in the natural dataset by comparing the safety criticality score with a preset threshold. Compared with the existing method of directly identifying and extracting safety-critical scenarios from natural datasets, this method can not only obtain a sufficient number of safety-critical scenarios, but also effectively reduce the time and financial costs of extracting safety-critical scenarios.
[0088] Furthermore, this invention leverages the significant advantages of diffusion model-generated samples in terms of fidelity and diversity. It utilizes the diffusion model to generalize and generate safety-critical scenarios identified and extracted from natural driving scenario data, combining the advantages of both identification / extraction and direct generation methods. This allows for the efficient generation of more high-quality safety-critical scenarios.
[0089] Optionally, Figure 2 For another flowchart of the generation method provided in the embodiments of the present invention, please refer to... Figure 1 and Figure 2 In step 7, the actual future state sequence is used. and the generated future state sequence Iterative updates to the basic scenario diffusion model are as follows: Step 71, based on the actual future state sequence and the generated future state sequence Calculate the mean square error loss Step 72: Determine whether the basic scene diffusion model has converged based on the mean squared error loss L. If it has converged, use the current basic scene diffusion model as the trained scene diffusion model; if it has not converged, update the parameters θ of the basic scene diffusion model and return the noisy future state sequence. The encoding condition c and the noise level σ are input to the diffusion decoder for a K-step iterative decoding process.
[0090] For details, please refer to Figure 2 In this embodiment, based on the actual future state sequence and the generated future state sequence The mean squared error is used to determine whether the model has converged. Therefore, in step 71, the model is first determined based on the true future state sequence. and the generated future state sequence Calculate the mean square error loss
[0091] After obtaining the mean squared error loss, in step 72, it is determined whether the current basic scene diffusion model has converged based on the mean squared error loss L. For example, when the mean squared error loss L changes very little or hardly changes anymore, it can be regarded as the model has converged.
[0092] If the model converges, the current base scene diffusion model is used as the trained scene diffusion model. Otherwise, if it does not converge, it means the model needs to be updated. Updating the base scene diffusion model mainly involves updating its parameters θ. After updating the parameters θ, return to step 6, that is, re-process the noisy future state sequence. The encoding condition c and noise level σ are input into the diffusion decoder, which performs K-step iterative decoding to recalculate the generated future state sequence. And utilize it with the actual future state sequence Determine if the current model has converged. Repeat the above steps until the model converges.
[0093] Optionally, Figure 3 A flowchart illustrating the safety criticality score for a computational scenario provided in this embodiment of the invention is shown below. Figure 1 and Figure 3 In step 8, a hierarchical scheme is used to calculate the safety criticality score of each scene in the natural scene dataset. Specifically, step 81 is to extract the low-level features of traffic participants in the scene; step 82 is to calculate the total safety criticality score of each traffic participant i based on the low-level features; and step 83 is to perform weighted summation and normalization on the total safety criticality scores of all traffic participants in the scene to obtain the safety criticality score of the scene.
[0094] For details, please refer to Figure 3For each scenario s, the state sequence of traffic participants is represented as A = {A(1), ..., A(N)}. a Let A(i) represent the state sequence of traffic participant i, and N be the number of participants in the traffic flow. a This represents the total number of all traffic participants in the scenario. The state of each traffic participant at time step t is represented as A. t (i). The safety criticality score of a scene is closely related to the characteristics of all traffic participants in the scene. Therefore, when calculating the safety criticality score of a scene, the low-level features of all traffic participants in the scene are first extracted through step 81. The low-level features include individual features and social features. Individual features include, for example, speed and acceleration, while social features include, for example, collision time and headway.
[0095] Then, in step 82, based on the low-level features, the total safety criticality score of the entire state sequence A(i) for each traffic participant i is calculated. After obtaining the total safety criticality score of all traffic participants in the scenario, in step 83, the total safety criticality scores of all traffic participants in the scenario are weighted, summed, and normalized to obtain the safety criticality score of the scenario.
[0096] Optionally, Figure 4 A flowchart for extracting low-level features provided in an embodiment of the present invention is shown below. Figure 3 and Figure 4 In step 81, low-level features of traffic participants in the scene are extracted, specifically: Step 811, extract the individual features f of each traffic participant i at each time step t. t ind (i) , obtaining the set of individual features F ind Step 812: Extract the social characteristics of each traffic participant i at each time step t. Obtain the set of social characteristics F soc Social characteristics refer to the interactive behavior between traffic participant i and traffic participant j.
[0097] For details, please refer to Figure 4 The basic characteristics of traffic participants include individual characteristics and social characteristics. Individual characteristics refer to characteristics that are only related to the behavior of traffic participant i, such as speed, acceleration, and jerk. Social characteristics refer to characteristics that are related to the interaction between traffic participant i and other traffic participants j, such as collision time and headway.
[0098] When extracting features, firstly, in step 811, the individual features of each traffic participant i at each time step t are extracted. The individual characteristics at all time steps form the corresponding set of individual characteristics F for traffic participants. indThen, through step 812, the social characteristics of each traffic participant i at each time step t are extracted. The social characteristics of all time steps form the corresponding set of social characteristics F of traffic participants. soc .
[0099] Optionally, Figure 5 Please refer to the flowchart for calculating the total safety criticality score provided in this embodiment of the invention. Figure 3 and Figure 5 In step 82, based on the low-level features, the total safety criticality score for each traffic participant i is calculated. Specifically, in step 821, based on the individual feature set F... ind The individual safety criticality score for each traffic participant i was calculated as follows: Among them, w ind f represents the weight vector of individual characteristics. t (i) represents the safety criticality value of a single individual characteristic of traffic participant i at time step t; Step 822, based on the social characteristic set F soc The social safety criticality score for each traffic participant i was calculated as follows: Among them, w soc The weight vector f represents the social characteristics. t (i,j) represents the safety criticality value of a single social characteristic of traffic participant i at time step t; in step 823, the individual safety criticality score and the social safety criticality score of each traffic participant i are added together to obtain the total safety criticality score of traffic participant i: Score(i) = IndScore(i) + ∑ i≠j SocScore(i,j).
[0100] For details, please refer to Figure 5 The basic characteristics of traffic participants include individual and social characteristics. Therefore, it is necessary to calculate individual safety criticality scores and social safety criticality scores separately. When calculating individual safety criticality scores, the calculation is performed for each individual characteristic in the individual characteristic set, such as speed, acceleration, and jerk, and then each individual is weighted. Specifically, this can be done in step 821 using the formula... Calculate the individual safety criticality score, where w ind f represents the weight vector of individual characteristics. t (i) represents the safety criticality value of a single individual characteristic (speed, acceleration, jerk, etc.) of traffic participant i at time step t. The maximum safety criticality value generated by the single individual characteristic of traffic participant i in the entire state sequence is taken. For example, for speed, acceleration, etc., the maximum value of f is directly taken.
[0101] Similarly, when calculating the social safety criticality score, it is calculated separately for each social feature in the set of social features, such as collision time and headway, and then weighted according to each social feature. Specifically, this can be seen in step 822, the principality formula. Calculate the social security criticality score, where w soc The weight vector f represents the social characteristics. t (i,j) represents the safety criticality value of a single social characteristic of traffic participant i at time step t. The maximum safety criticality value generated by the single social characteristic of traffic participant i throughout the entire state sequence is taken; for example, for the collision time, the maximum value of 1 / f is taken.
[0102] After obtaining the individual safety criticality score and the social safety criticality score, in step 823, the two are summed. For example, by adding the individual safety criticality score and the social safety criticality score of each traffic participant i, the total safety criticality score of traffic participant i can be obtained as Score(i) = IndScore(i) + ∑ i≠j SocScore(i,j).
[0103] Optionally, Figure 6 A flowchart illustrating the safety criticality score for a computational scenario provided in this embodiment of the invention is shown below. Figure 3 and Figure 6 In step 83, the total safety criticality scores of all traffic participants in the scenario are weighted and summed. Specifically: Step 831, set the participation weight w(i) for each traffic participant i in the scenario; Step 832, based on the participation weight w(i), according to the formula... The safety criticality scores of all traffic participants in the scenario are weighted, summed, and normalized to obtain the scenario's safety criticality score; where N a This indicates the total number of traffic participants.
[0104] For details, please refer to Figure 6 This embodiment calculates the safety criticality score of a scenario by weighted summing of the total safety criticality scores of each traffic participant in the scenario. First, in step 831, the participation weight w(i) of each traffic participant i in the scenario is set. Then, in step 832, based on the set participation weight w(i), the safety criticality score is calculated using the formula... The safety criticality score (Score) for a given scenario is obtained by weighted summation and normalization of the total safety criticality scores for all traffic participants. Where N... a This indicates the total number of traffic participants.
[0105] Optionally, Figure 7For another flowchart of the generation method provided in the embodiments of the present invention, please refer to... Figure 7 It also includes: Step 91, setting a target guiding function J that characterizes the behavioral style of traffic participants; Target guiding function J = J acc +J vel +J pos +J thw , where J acc J represents the maximum acceleration guided function for traffic participants. vel J represents the target speed guided function for traffic participants. pos J is a guided function representing the relative positions of traffic participants. thw This represents the headway guidance function for traffic participants. Preferably, the future state sequence of safety-critical scenarios is generated using the trained scene diffusion model, specifically by generating the noisy future state sequence corresponding to the safety-critical scenarios. Input the trained scene diffusion model and perform K-step iterative denoising; in each iteration of denoising, use the target guiding function J to optimize the result.
[0106] For details, please refer to Figure 7 In order to achieve controllable generalization generation of the selected safety-critical scenarios, this embodiment sets a target guidance function J that represents the behavioral style of traffic participants in step 91. In order to control the behavioral style of traffic participants through the target guidance function J during the K-step iterative denoising process of the trained scenario diffusion model, thereby achieving controllable generalization generation.
[0107] Wherein, the target guiding function J = J acc +J vel +J pos +J thw J acc J represents the maximum acceleration guided function for traffic participants. vel J represents the target speed guided function for traffic participants. pos J is a guided function representing the relative positions of traffic participants. thw The headway guidance function represents the headway of traffic participants.
[0108] When generating future state sequences for safety-critical scenarios using the trained scene diffusion model, the noisy future state sequences corresponding to the safety-critical scenarios are... Input the trained scene diffusion model and perform K-step iterative denoising on it.
[0109] Figure 8 This is a schematic diagram of a workflow after adding the target guiding function according to an embodiment of the present invention. Please refer to it. Figure 8In each iteration of denoising, the gradient information of the target guiding function J is first calculated through a query process. Then, this gradient information is backpropagated through a guided optimization process to optimize the generated state sequence. After completing K iterations of denoising, the generated future state sequence is obtained. This enables controllable generalization of the selected safety-critical scenarios.
[0110] Based on the same inventive concept, this invention also provides a generalized generation system 100 for safety-critical scenarios oriented towards natural datasets. Figure 9 This is a schematic diagram of a generation system provided in an embodiment of the present invention. Please refer to it. Figure 9 The present invention provides a generalized generation system 100 for safety-critical scenarios based on natural datasets, comprising:
[0111] The data acquisition module 101 is used to acquire scene data from multiple natural driving scenarios;
[0112] The annotation module 102 is used to annotate all scene data to obtain a natural scene dataset. Each scene in the natural scene dataset is represented as s = (M, A), where M represents map information and A represents the state sequence of all traffic participants. The state sequence A of traffic participants includes the historical state sequence A. his and the actual future state sequence
[0113] Noise module 103 is used to add noise to the real future state sequence. Noise is added to obtain a noisy future state sequence.
[0114] Module 104 is used to build a basic scene diffusion model, which includes a scene encoder and a diffusion decoder.
[0115] The first input module 105 is used to input map information M and historical state sequence A. his The input scene encoder is used for encoding to obtain encoding condition c; encoding condition c includes map encoding and traffic participant historical state encoding.
[0116] The second input module 106 is used to input the noisy future state sequence. The encoding condition c and the noise level σ are input to the diffusion decoder, which performs K-step iterative decoding to obtain the generated future state sequence.
[0117] Update module 107 is used to utilize the real sequence of future states. and the generated future state sequence Iteratively update the basic scene diffusion model to obtain the trained scene diffusion model;
[0118] The calculation module 108 is used to calculate the safety criticality score of each scene in the natural scene dataset using a hierarchical scheme;
[0119] The comparison module 109 is used to compare the safety criticality score with a preset threshold and filter out scenarios with a safety criticality score greater than the preset threshold as safety critical scenarios.
[0120] The generation module 110 is used to generate a sequence of future states for safety-critical scenarios using the trained scenario diffusion model.
[0121] For details, please refer to Figure 9 The safety-critical scenario generalization generation system for natural datasets provided in this embodiment of the invention first collects scenario data of multiple natural driving scenarios through the acquisition module 101. It should be noted here that since the collected scenario data is used for model training, and model training usually requires a large amount of data, the scenario data of multiple natural driving scenarios here refers to a large amount of scenario data of natural driving scenarios.
[0122] Because the data formats of the collected scene data are inconsistent, in order to ensure that the input data format is consistent during model training, this invention uses annotation module 102 to perform uniform format annotation processing on all collected scene data to form a natural scene dataset. During annotation, each scene needs to be annotated with high-precision map information M and the state sequences A of all traffic participants. Each scene in the resulting natural scene dataset is represented as s = (M, A), where the state sequences A of traffic participants need to be further annotated as historical state sequences Ai. his and the actual future state sequence The high-precision map information M includes road information, such as lane lines, lane connections, and road center lines. The state sequence A contains the state information of each traffic participant at each time step, including position, speed, and heading angle.
[0123] After annotation is completed, the annotated scene data can be used for model training. First, noise ∈ is sampled using the noise-adding module 103, and then the noise ∈ is used to determine the true future state sequence of traffic participants. Noise is added to obtain a noisy future state sequence.
[0124] To train the model, a basic scene diffusion model first needs to be constructed using the construction module 104, and then this basic scene diffusion model is trained. In this invention, the basic scene diffusion model includes a scene encoder and a diffusion decoder.
[0125] Because it is necessary to separately process map information M and historical state sequences of traffic participants Ahis Encoding is performed; therefore, the scene encoder includes a map encoder and a traffic participant encoder. The first input module 105 inputs map information M into the map encoder for encoding, obtaining the map code; the historical state sequence A... his The input traffic participant encoder is used for encoding to obtain the traffic participant historical status code. The map code and the traffic participant historical status code together constitute the encoding condition c.
[0126] The second input module 106 inputs a noisy future state sequence. Decoding is then performed. During decoding, the noisy future state sequence is first... The encoding condition c and the noise level σ are input into the diffusion decoder, which performs one decoding operation to obtain a noisy sequence of future states. Then the noisy future state sequence The encoding condition c and the noise level σ are input into the diffusion decoder, which performs one decoding operation to obtain a noisy sequence of future states. Iteratively perform the above decoding operation K times until the generated future state sequence is obtained.
[0127] Obtain the generated future state sequence Then, the update module 107 compares the generated future state sequence. With the actual future state sequence The error between the two is used to iteratively update the basic scene diffusion model until the error between the two meets the requirements. Then, the corresponding scene diffusion model is used as the trained scene diffusion model.
[0128] In this invention, a scene's safety criticality score determines whether it is a safety critical scene. Therefore, the calculation module 108 calculates the safety criticality score of each scene. In this embodiment, a hierarchical scheme is used to calculate the safety criticality score of each scene in the natural scene dataset. The comparison module 109 filters out safety critical scenes and ordinary scenes by comparing the safety criticality score with a preset threshold. When the safety criticality score of a scene is greater than the preset threshold, the scene is classified as a safety critical scene; otherwise, it is classified as an ordinary scene.
[0129] After obtaining the safety-critical scenario, the generation module 110 inputs the safety-critical scenario into the trained scenario diffusion model to generate the corresponding future state sequence of the safety-critical scenario.
[0130] The safety-critical scenario generalization generation system for natural datasets provided by this invention calculates the safety criticality score of each scenario in the natural scenario dataset through a hierarchical scheme, and extracts the safety-critical scenarios from the natural dataset by comparing the safety criticality score with a preset threshold. Compared with the existing method of directly identifying and extracting safety-critical scenarios from natural datasets, this method can not only obtain a sufficient number of safety-critical scenarios, but also effectively reduce the time and financial costs of extracting safety-critical scenarios.
[0131] Furthermore, this invention leverages the significant advantages of diffusion model-generated samples in terms of fidelity and diversity. It utilizes the diffusion model to generalize and generate safety-critical scenarios identified and extracted from natural driving scenario data, combining the advantages of both identification / extraction and direct generation methods. This allows for the efficient generation of more high-quality safety-critical scenarios.
[0132] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.
[0133] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.
[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for generalizing and generating safety-critical scenarios based on natural datasets, characterized in that, include: Collect scene data from multiple natural driving scenarios; All the aforementioned scene data are labeled to obtain a natural scene dataset. Each scene in the natural scene dataset is represented as s = (M, A), where M represents map information and A represents the state sequence A of all traffic participants; the state sequence A of traffic participants includes the historical state sequence A. his and the actual future state sequence For the true future state sequence Noise is added to obtain a noisy future state sequence. Construct a basic scene diffusion model, which includes a scene encoder and a diffusion decoder; The map information M and the historical state sequence A his The input scene encoder is used for encoding to obtain encoding condition c; the encoding condition c includes map encoding and traffic participant historical state encoding. The noisy future state sequence The encoding condition c and the noise level σ are input to the diffusion decoder, which performs K-step iterative decoding to obtain the generated future state sequence. Using the real future state sequence and the generated future state sequence The basic scene diffusion model is iteratively updated to obtain the trained scene diffusion model; A hierarchical scheme is used to calculate the safety criticality score for each scene in the natural scene dataset; Compare the safety criticality score with a preset threshold, and filter out scenarios where the safety criticality score is greater than the preset threshold as safety critical scenarios; The trained scenario diffusion model is used to generate a sequence of future states for the safety-critical scenario.
2. The method for generalizing security-critical scenarios based on natural datasets according to claim 1, characterized in that, For the true future state sequence Noise addition processing is performed, specifically as follows: Sampling noise ∈, using the noise ∈ to obtain the true future state sequence Noise is added.
3. The method for generalizing security-critical scenarios based on natural datasets according to claim 1, characterized in that, Using the real future state sequence and the generated future state sequence The basic scene diffusion model is iteratively updated as follows: Based on the true future state sequence and the generated future state sequence Calculate the mean square error loss The convergence of the basic scene diffusion model is determined based on the mean square error loss L. If convergence is achieved, the current basic scene diffusion model is used as the trained scene diffusion model. If convergence fails, update the parameters θ of the basic scene diffusion model and return the noisy future state sequence. The encoding condition c and the noise level σ are input to the diffusion decoder for a K-step iterative decoding process.
4. The method for generalizing security-critical scenarios based on natural datasets according to claim 1, characterized in that, A hierarchical scheme is used to calculate the safety criticality score for each scene in the natural scene dataset, specifically as follows: Extract low-level features of traffic participants in the scenario; Based on the aforementioned low-level features, calculate the total safety criticality score for each traffic participant i; The safety criticality scores of all traffic participants in the scenario are weighted, summed, and normalized to obtain the safety criticality score of the scenario.
5. The method for generalizing security-critical scenarios based on natural datasets according to claim 4, characterized in that, Extracting low-level features of traffic participants in the scenario, specifically: Extract individual features of each traffic participant i at each time step t. Obtain the set of individual features F ind ; Extracting the social characteristics of each traffic participant i at each time step t Obtain the set of social characteristics F soc The social characteristics refer to the interaction between traffic participant i and traffic participant j.
6. The method for generalizing security-critical scenarios based on natural datasets according to claim 5, characterized in that, Based on the aforementioned low-level features, the total safety criticality score for each traffic participant i is calculated as follows: Based on the individual characteristic set F ind The individual safety criticality score for each traffic participant i was calculated as follows: Among them, w ind f represents the weight vector of individual characteristics. t (i) represents the safety criticality value of a single individual characteristic of traffic participant i at time step t; According to the social characteristic set F soc The social safety criticality score for each traffic participant i was calculated as follows: Among them, w soc The weight vector f represents the social characteristics. t (i,j) represents the safety criticality value of a single social characteristic of traffic participant i at time step t; The individual safety criticality score and the social safety criticality score of each traffic participant i are added together to obtain the total safety criticality score of traffic participant i: Score(i) = IndScore(i) + ∑ i≠j SocScore(i,j).
7. The method for generalizing security-critical scenarios based on natural datasets according to claim 4, characterized in that, The total safety criticality scores of all traffic participants in the scenario are weighted and summed as follows: Set the participation weight w(i) for each traffic participant i in the scenario; Based on the participation weight w(i), according to the formula The safety criticality scores of all traffic participants in the scenario are weighted, summed, and normalized to obtain the safety criticality score (Score) for the scenario; where N a This indicates the total number of traffic participants.
8. The method for generalizing security-critical scenarios based on natural datasets according to claim 1, characterized in that, Also includes: Set a target guiding function J that represents the behavioral style of traffic participants; Target Guiding function J = J acc +J vel +J pos +J thw , where J acc J represents the maximum acceleration guided function for traffic participants. vel J represents the target speed guided function for traffic participants. pos J is a guided function representing the relative positions of traffic participants. thw The headway guidance function represents the headway of traffic participants.
9. The method for generalizing safety-critical scenarios based on natural datasets according to claim 8, characterized in that, The trained scene diffusion model is used to generate the future state sequence of the safety-critical scene, specifically as follows: The noisy future state sequence corresponding to the safety-critical scenario Input the trained scene diffusion model and perform K-step iterative denoising; in each iterative denoising process, use the target guiding function J to optimize the result.
10. A generalized generation system for safety-critical scenarios based on natural datasets, characterized in that, include: The data acquisition module is used to collect scene data from multiple natural driving scenarios; The annotation module is used to annotate all the scene data to obtain a natural scene dataset. Each scene in the natural scene dataset is represented as s = (M, A), where M represents map information and A represents the state sequence of all traffic participants. The state sequence A of traffic participants includes the historical state sequence A. his and the actual future state sequence A noise-adding module is used to add noise to the real future state sequence. Noise is added to obtain a noisy future state sequence. A building module is used to build a basic scene diffusion model, which includes a scene encoder and a diffusion decoder; The first input module is used to input the map information M and the historical state sequence A. his The input scene encoder is used for encoding to obtain encoding condition c; the encoding condition c includes map encoding and traffic participant historical state encoding. The second input module is used to input the noisy future state sequence. The encoding condition c and the noise level σ are input to the diffusion decoder, which performs K-step iterative decoding to obtain the generated future state sequence. An update module is used to utilize the actual future state sequence. and the generated future state sequence The basic scene diffusion model is iteratively updated to obtain the trained scene diffusion model; The computation module is used to calculate the safety criticality score for each scene in the natural scene dataset using a hierarchical scheme; The comparison module is used to compare the safety criticality score with a preset threshold and filter out scenarios where the safety criticality score is greater than the preset threshold as safety critical scenarios. The generation module is used to generate a future state sequence of the safety-critical scenario using the trained scenario diffusion model.