Automatic driving safety critical scenario generation method, system, device and medium

Abstract

The application relates to the technical field of automatic driving and discloses an automatic driving safety key scene generation method, system, device and medium, which comprises the following steps: performing structural analysis on a real traffic accident sketch and corresponding text abstract to obtain high-level accident semantic information; constructing fixed attributes and floating attributes according to the high-level accident semantic information; searching and assembling scene code fragments in a pre-constructed primitive template library by using the fixed attributes to obtain a basic scene description; modeling the floating attributes into environmental parameters and behavior constraint rules, and performing scene constraint and scene expansion on the basic scene description by using the environmental parameters and the behavior constraint rules to obtain a complete scene description; and performing scene-level semantic consistency verification on the complete scene description to obtain a target safety key scene set. The application can convert real accident information into an executable simulation scene and improve the coverage rate of high-risk safety key scenes.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a method, system, device, and medium for generating key safety scenarios for autonomous driving. Background Technology

[0002] With the continuous development of autonomous driving technology, how to fully verify the safety of autonomous driving systems under limited cost and controllable risk conditions has become a core issue facing the industry. Since real-world road tests are difficult to cover complex, rare, and high-risk traffic situations, simulation-based safety-critical scenario generation technology has been widely used in the research and development and testing of autonomous driving systems.

[0003] Existing methods for generating safety scenarios primarily rely on manual rule modeling, parameter template configuration, or scenario replay based on accident data. While these methods can construct test scenarios to some extent, they still fall short in terms of scenario diversity, long-tail risk coverage, and automation. In recent years, with the development of large language models, some technologies have attempted to directly utilize large language models to generate traffic scenario descriptions or simulation configuration files from accident descriptions or natural language text, thereby reducing the cost of manual modeling and improving scenario generation efficiency.

[0004] However, existing scene generation methods based on large language models typically lack stable structural constraints and semantic verification mechanisms, making them susceptible to model illusions. The generated scenes often fail to guarantee grammatical correctness, physical feasibility, or traffic semantic consistency, limiting their engineering application in autonomous driving safety testing. Therefore, there is an urgent need for a technical solution that can effectively constrain and verify the scene generation process while fully utilizing the semantic understanding capabilities of large language models, thereby improving the accuracy, executability, and reliability of safety-critical scene generation.

[0005] However, when facing application needs such as reproducing real traffic accidents, expanding into high-risk long-tail scenarios, and performing engineering-grade executable simulation tests, the existing technology still has significant shortcomings, specifically in the following aspects.

[0006] 1) At the level of scene modeling and semantic representation, existing methods struggle to characterize the causal structure of accidents and the boundaries of attribute effects:

[0007] Existing methods based on rule templates or parameterized scene descriptions typically rely on predefined traffic rules, behavioral patterns, and scene parameter combinations to generate test scenarios through enumeration or random sampling. While these methods offer some controllability in scenarios with clear rules and simple structures, their scene modeling is highly dependent on human experience, with fixed rule granularity, making it difficult to cover the complex, multi-factor coupled causal structures of real traffic accidents. For example, the paper "A Knowledge-Driven Method for Generating Autonomous Driving Simulation Scenes," presented at the 8th International Conference on Intelligent Information and Biomedical Sciences (ICIIBMS) 2023 (IEEE, 2023, 8: 495-498), proposes a knowledge-driven personalized scene generation method. This method uses a hierarchical model to describe the relationships between scenes and combines OpenSCENARIO format structuring and parameterization techniques to generate specific test scenarios in batches through automated scripts. However, this method still mainly relies on prior human knowledge, and its generation capability is currently only applicable to relatively simple scene types, making it difficult to cover complex, high-risk accident scenarios.

[0008] 2) At the level of scene generation mechanisms, existing optimization and learning methods struggle to balance criticality, controllability, and interpretability: Safety scenario generation methods based on gradient optimization or reinforcement learning typically define collision probability, minimum safe distance, and traffic time-to-continuity (TTC) as optimization objectives to search for the most unfavorable adversarial scenarios for autonomous driving systems in a simulation environment. For example, gradient optimization of adversarial vehicle trajectories can shorten vehicle distances, thereby generating the "most dangerous" collision scenario. However, the generated adversarial trajectories often violate traffic rules (such as driving against traffic or crossing solid lines) or even physical constraints. Alternatively, attempts can be made to generate "near-miss" scenarios that are adversarial but still feasible for autonomous driving systems. Reinforcement learning is used to control key background vehicles to learn adversarial strategies, but their behavioral patterns lack intuitive semantic interpretation, easily generating unnatural scenarios that do not conform to real traffic behavior, which is detrimental to subsequent safety analysis and accident mechanism research. These methods focus more on the extreme value search of numerical indicators and lack explicit expression of the semantics and rule constraints of real traffic accidents. The generated scenarios often exhibit abnormal behavior, are difficult to interpret, or do not conform to common traffic sense in engineering practice, increasing the difficulty of test result analysis and experiment reproduction.

[0009] 3) Shortcomings in the way real accident data is used: In recent years, with the development of natural language understanding capabilities of large language models, some studies have begun to attempt to combine large language models to generate autonomous driving test scenarios. For example, large models are used to generate natural language descriptions of key safety scenarios, and retrieval enhancement techniques are combined to convert the natural language into domain-specific language representations of the test scenarios; [Paper 7] uses large models to inject interference factors from natural language descriptions into basic scenarios to form safety-critical scenarios, but its interference factors mainly come from pre-built datasets. These methods still require human participation in scenario design to varying degrees, and are difficult to cover long-tail risk scenarios in real traffic.

[0010] Furthermore, some studies have begun to utilize traffic accident data or natural driving data for scene generation and risk modeling. However, existing methods mostly focus on learning the overall parameter distribution or temporal evolution patterns from accident data, with the generation goal primarily concentrated on improving scene diversity at the distribution level, rather than on the controllable expansion of the semantic structure of specific accidents. For example, based on real accident databases, TimeGAN is used to learn the temporal risk characteristics during the accident process and synthesize a batch of safety-critical scenarios with statistical similarity. However, such methods typically incorporate all accident elements into the generation model for overall modeling, lacking the ability to distinguish and characterize the stability and perturbation of different types of elements in an accident. It is difficult to systematically abstract reusable behavioral patterns, structural patterns, and risk triggering conditions from accident data, and it is also difficult to generate a series of variant scenarios with adjustable criticality around the same accident while maintaining its causal structure.

[0011] Some methods based on large language models further attempt to enhance the depth of accident data utilization. For example, they use large language models to extract attributes from accident reports, generate structured collision sequences, and transform them into standardized OpenSCENARIO scenarios; they convert accident text reports into corresponding OpenSCENARIO scenario files, and generate derived scenarios through scenario disassembly and reconstruction. However, the above methods mainly focus on the reconstruction or direct derivation of accident scenarios, failing to independently model and reuse the risk factors that play a key role in the accident, thus limiting the application value of accident data in the expansion of critical safety scenarios.

[0012] Therefore, how to transform real accident information into executable simulation scenarios, how to reuse accident semantics in a structured manner, and how to improve the coverage of high-risk safety-critical scenarios have become urgent problems to be solved. Summary of the Invention

[0013] The technical problem to be solved by this invention is how to address the issues in existing traffic scene generation methods, such as the difficulty in converting real accident information into executable simulation scenes, the difficulty in reusing accident semantics in a structured manner, and the insufficient coverage of high-risk safety-critical scenes.

[0014] The present invention solves the above-mentioned technical problems through the following technical means: Structured parsing of real traffic accident sketches and corresponding text summaries yields high-level accident semantic information. Based on the high-level accident semantic information, fixed attributes including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident, are constructed. Using the fixed attributes, the corresponding scene code fragments are retrieved and assembled from the pre-built primitive template library to obtain a basic scene description with basic traffic semantics; The floating attributes are modeled as configurable environmental parameters and behavioral constraint rules. The environmental parameters and behavioral constraint rules are used to constrain and expand the basic scene description to obtain a complete scene description. A scenario-level semantic consistency check is performed on the complete scenario description to obtain the target security critical scenario set.

[0015] Optionally, the step of performing structured parsing on the real traffic accident sketch and the corresponding text summary to obtain high-level accident semantic information includes: Extract visual features from the real traffic accident sketch; The visual features are then aligned across text modalities to obtain visual entity features; A pre-built semantic parsing large language model is used to perform semantic parsing on the visual entity features and the text summary to obtain high-level accident semantic information.

[0016] Optionally, the step of constructing fixed attributes, including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident, based on the high-level accident semantic information includes: Entity recognition and annotation are performed on the semantic information of the high-rise accident. The semantic information of high-level accidents after entity recognition and annotation is subjected to relational extraction of behavioral attributes, road network structure attributes and generation point attributes to obtain fixed attributes containing behavioral attributes, road network structure attributes and generation point attributes. Initial floating attributes are extracted from the high-level accident semantic information using a variable-length attribute list structure; The initial floating attributes are filtered for interference information to obtain the floating attributes related to the occurrence of the accident.

[0017] Optionally, the step of retrieving and assembling corresponding scene code fragments from a pre-built primitive template library using the fixed attributes to obtain a basic scene description with basic traffic semantics includes: Retrieve primitive code fragments corresponding to the behavioral attribute, the road network structure attribute, and the generation point attribute from the pre-built primitive template library; The scene skeleton generation logic is constructed based on the fixed attributes. The primitive code fragments are assembled according to the scene skeleton generation logic to obtain the basic scene skeleton; Information encoding is performed on the basic scene skeleton to obtain a basic scene description with basic traffic semantics.

[0018] Optionally, the scene skeleton generation logic for constructing the basic scene based on the fixed attributes includes: The spatial structure constraints of the traffic scenario are constructed based on the road network structure attribute in the fixed attributes. The initial layout of traffic participants is constructed based on the generation point attribute in the fixed attributes; Assign high-level behavioral rules to the traffic participants based on the behavioral attributes in the fixed attributes; The scene skeleton generation logic is obtained by combining the spatial structure constraints, the initial step distance, and the high-level behavior rules.

[0019] Optionally, the step of using the environmental parameters and the behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description includes: The environmental parameters and behavioral constraint rules are used to constrain the basic scene description to obtain a constrained scene. Based on the environmental parameters, the behavioral constraint rules, and the basic scenario description, key safety scenarios are generated using preset prompt templates. By combining the aforementioned constraint scenarios and key security scenarios, a complete scenario description is obtained.

[0020] Optionally, the step of performing scene-level semantic consistency verification on the complete scene description to obtain a set of target security-critical scenes includes: The logical constraints and semantic conflicts of the complete scene description are verified using preset semantic rules. If the verification fails, return to the step of using the environmental parameters and the behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description. If the verification passes, the complete scenario description will be used as the target security critical scenario set.

[0021] To address the aforementioned problems, this invention also proposes a system for generating key safety scenarios for autonomous driving, the system comprising: The semantic parsing module is used to perform structured parsing of real traffic accident sketches and corresponding text summaries to obtain high-level accident semantic information. The attribute construction module is used to construct fixed attributes, including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident, based on the high-level accident semantic information. The basic scene description generation module is used to retrieve and assemble corresponding scene code fragments in the pre-built primitive template library using the fixed attributes to obtain a basic scene description with basic traffic semantics. The scene constraint and scene expansion module is used to model the floating attributes as configurable environmental parameters and behavioral constraint rules, and to use the environmental parameters and behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description. The semantic verification module is used to perform scene-level semantic consistency verification on the complete scene description to obtain the target security critical scene set.

[0022] The present invention also provides a processing device, characterized in that it includes at least one processor and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor can execute the above-described method for generating key safety scenarios for autonomous driving by calling the program instructions.

[0023] The present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, the computer instructions causing the computer to execute the above-described method for generating key safety scenarios for autonomous driving.

[0024] The advantages of this invention are: This invention performs structured parsing of real traffic accident sketches and corresponding text summaries to obtain high-level accident semantic information. Based on the high-level accident semantic information, fixed attributes and floating attributes are constructed. The fixed attributes can be further abstracted into reusable and composable scene code fragments, and predefined parameterized code templates are provided for them. The scene skeleton is constructed by combining primitive code fragments rather than generating complete scene scripts end-to-end. This significantly reduces the risk of syntax errors and semantic conflicts in the large model generation process and improves the accuracy, stability and consistency of the generated scene representation. By modeling floating attributes as configurable environmental parameters and behavioral constraint rules, and using these parameters and rules to constrain and expand the basic scenario description, a complete scenario description is obtained. Floating attributes are applied to the validated scenario skeleton in an independent and superimposed manner, realizing a scenario generation mechanism that is "structurally invariant and risk-adjustable." This allows the same basic scenario to correspond to multiple scenario instances with different levels of danger, providing more targeted and interpretable test samples for autonomous driving systems. It can significantly expand the risk distribution range of test scenarios without relying on the direct reproduction of large-scale accident data. Instead, it does so by combining and injecting a small number of key floating attributes, improving the coverage and efficiency of autonomous driving testing in long-tail scenarios while ensuring scenario rationality and executability.

[0025] Furthermore, by introducing a hierarchical semantic consistency and executability verification mechanism in multiple stages, such as filtering interference information of floating attributes, verifying primitive code fragments, and verifying scene-level semantic consistency of complete scene descriptions, the generated results are constrained and verified. When semantic conflicts, unreasonable attribute combinations, or unexecutable situations are detected, the system can backtrack to the corresponding steps for correction or regeneration, thereby ensuring that the final output scene meets the requirements of engineering applications in terms of semantic rationality, reality consistency, and simulation executability. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating a method for generating key safety scenarios for autonomous driving according to an embodiment of the present invention; Figure 2 This is a traffic accident draft in one embodiment of the present invention. Figure 1 Schematic diagram; Figure 3 This is a functional module diagram of an autonomous driving safety critical scenario generation system provided in one embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] Reference Figure 1 The diagram shown is a flowchart illustrating a method for generating critical safety scenarios for autonomous driving according to an embodiment of the present invention. In this embodiment, the method for generating critical safety scenarios for autonomous driving includes: S1. Perform structured parsing on real traffic accident sketches and corresponding text summaries to obtain high-level accident semantic information.

[0029] In this embodiment of the invention, the traffic accident sketch and the corresponding text summary are extracted from official accident reports of real traffic accidents, such as the NHTSA CIREN (National Highway Traffic Safety Administration, Crash Injury Research and Engineering Network) database. For each traffic accident record, the input format includes a traffic accident sketch and a text summary of the accident. Figure 2 This represents a sketch of a traffic accident, where V1 represents vehicle number one and V2 represents vehicle number two.

[0030] Figure 2 The text summary corresponding to the traffic accident sketch can be represented as: Collision Summary: Vehicle 1 (V1 – the case vehicle) was a 2002 Subaru Forester four-door SUV, traveling westbound on a two-lane road and making a right turn. Vehicle 2 (V2) was a 2002 Buick four-door SUV, traveling eastbound on the same road and making a left turn. It was daytime, snowing, and the asphalt was icy. The driver of V1 lost control on the icy surface, and the vehicle began to spin counter-clockwise. After spinning approximately 90 degrees, V1 crossed the center line into the eastbound lane. The driver of V2 was unable to avoid V1, and the front of V2 struck the right side of V1 in a T-shape. Both vehicles were towed away due to severe damage. The 36-year-old male driver of V1 (the case occupant) was wearing a three-point seatbelt, but the driver's seat airbag failed to deploy. He was taken by ground ambulance to a regional level one trauma center.

[0031] Causes of injury: The occupant in this case was the driver of a 2002 Subaru Forester SUV, a 36-year-old male, 185 cm tall, and weighing 100 kg. He lost control of the vehicle. After the vehicle spun 90 degrees, its right side (T-shaped impact) was struck from the front by a 2002 Buick SUV. The impact was determined to be severe. The reconstruction procedure calculated a speed change of 55 km / h (34 mph), consistent with the observed damage. The occupant was wearing the vehicle's three-point seatbelt. The airbag in his seat area did not deploy. The side airbag in the right front seatback deployed, but that seat was unoccupied. Under the force at the three o'clock position (primary force direction angle 100 degrees), the driver was moved to the right rear relative to the interior space. His right top scalp was lacerated due to contact with the right side roof beam. He also suffered three rib fractures (the first and second anterolateral ribs on the right side, and the second anterolateral rib on the left side) due to contact with the B-pillar, accompanied by right pneumothorax, lateral sternal fracture, and right acromioclavicular joint separation.

[0032] In detail, since different accident reports differ in their descriptive perspectives, granularity of expression, and completeness of information, they are difficult to use directly as input for scene generation. Through structured parsing, traffic accident sketches and text summaries are converted into a unified semantic representation of accidents. The input is a real traffic accident sketch and its corresponding text summary, and the output is cleaned JSON format attribute fields, i.e., high-level accident semantic information.

[0033] Specifically, the structured parsing of the real traffic accident sketch and its corresponding text summary to obtain high-level accident semantic information includes: Extract visual features from the real traffic accident sketch; The visual features are then aligned across text modalities to obtain visual entity features; A pre-built semantic parsing large language model is used to perform semantic parsing on the visual entity features and the text summary to obtain high-level accident semantic information.

[0034] In this process, a pre-built visual encoder is used to encode real traffic accident sketches into a sequence of visual tokens (such as 256 image blocks) to capture visual features such as vehicle outlines, damaged areas, road markings, and environmental features. The visual features are then mapped to a pre-built text semantic vector space to align the visual features with the text features, resulting in text feature vectors representing visual entities, i.e., visual entity features.

[0035] Furthermore, a pre-built large language model for extracting semantic information (a semantic parsing large language model) is used to jointly parse real accident sketches and accident text summaries. During the parsing process, the model output is formatted to provide a semantic description of the accident in a structured JSON format with fixed fields. This structured representation does not aim for a complete reproduction of low-level physical parameters (such as precise velocity, trajectory, or dynamic state), but focuses on characterizing high-level semantic information that plays a decisive role in the conditions for the accident to occur, thereby reducing the dependence on the accuracy of the original accident data while ensuring information integrity.

[0036] The extracted semantic information of high-level accidents includes at least the following: The types of traffic participants involved in the accident and their relative relationships, such as the interaction between vehicles, pedestrians, or non-motorized vehicles; The main behavioral patterns of various traffic participants before the accident, such as going straight, turning, crossing, and failing to yield; The type of road structure and its spatial constraints at the time of the accident; External or internal conditions that may amplify or trigger accident risks.

[0037] Through the above semantic abstraction and unified representation process, each original accident record is transformed into a high-level accident semantic information that is semantically consistent, has stable fields, and is computable, providing a unified input for the subsequent construction of executable scenario representations.

[0038] Preferably, a small number of examples are manually constructed to utilize the few-shot learning technique in Prompt Engineering. By providing a large model with only one example and setting a specific output format, the illusion of a large model is significantly reduced. Furthermore, based on the characteristics of accident text summaries, when generating high-level accident semantic information, information irrelevant to the generation of key safety scenarios (such as vehicle year, model, whether it was towed away, and the status of injured persons' medical treatment) is systematically removed, retaining only elements that have a causal effect on the collision.

[0039] S2. Based on the high-level accident semantic information, construct fixed attributes including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident.

[0040] In this embodiment of the invention, during the information extraction process from real traffic accident data, the invention does not use the accident semantic information holistically and indiscriminately for scene generation. Instead, it decomposes the accident semantics in a function-oriented manner, considering the executability and risk modeling requirements of autonomous driving test scenarios. Specifically, it proposes a scenario attribute division principle of "fixed attributes - floating attributes" to distinguish the roles of different semantic elements in scenario construction and risk adjustment.

[0041] Among them, fixed attributes are used to define "the minimum set of attributes necessary for a traffic scenario to be semantically valid", including: Behavioral attributes: used to describe the high-level behavioral patterns of traffic participants before an accident occurs; Road network attributes: used to describe the topology and spatial constraints of the road where the accident occurred; Generated point attributes: used to describe the initial position, orientation, and relative layout relationship of each traffic participant.

[0042] The aforementioned fixed attributes collectively determine the basic traffic semantics and executable skeleton of the scene. The absence of any one of these attributes will result in the scene being unable to be constructed properly or being unable to execute correctly in the simulation environment.

[0043] Unlike fixed attributes, floating attributes are used to describe conditional factors that do not necessarily occur in all scenarios but may significantly affect the probability of an accident or the evolution of risk. These attributes include, but are not limited to, weather conditions, visibility conditions, road adhesion status, and driver status.

[0044] Specifically, the step of constructing fixed attributes, including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident, based on the high-level accident semantic information includes: Entity recognition and annotation are performed on the semantic information of the high-rise accident. The semantic information of high-level accidents after entity recognition and annotation is subjected to relational extraction of behavioral attributes, road network structure attributes and generation point attributes to obtain fixed attributes containing behavioral attributes, road network structure attributes and generation point attributes. Initial floating attributes are extracted from the high-level accident semantic information using a variable-length attribute list structure; The initial floating attributes are filtered for interference information to obtain the floating attributes related to the occurrence of the accident.

[0045] In this embodiment of the invention, predefined entity types can be used for entity recognition and annotation. For example, traffic participant entities include vehicles, pedestrians, and cyclists; behavioral entities include going straight, turning left, changing lanes, and overtaking; road network structure entities include lanes, intersections, and traffic facilities; and entities describing key parameters of the scene, such as time, location, and behavioral state. Based on entity recognition and annotation, behavioral attributes describing the dynamic behavioral characteristics of traffic participants, road network structure attributes describing the topology and constraints of the static road environment, and generation point attributes describing the floating range and constraints of key parameters of the scene are extracted. Among them, temporal relationships and causal chains can be used to extract behavioral attributes; then, qualitative accident description entity information (such as behavioral attribute: "collision at an intersection in rainy weather") is transformed into a quantitative parameter space (entities of key parameters of the scene: speed ∈ [40,60], friction coefficient ∈ [0.4,0.6], position ∈ [(10,8)±1.5m]), providing a foundation for the subsequent generation of basic scene descriptions.

[0046] Furthermore, floating attributes are used to describe conditional factors that do not necessarily occur in all scenarios during an accident, but may significantly influence the probability of accident occurrence or the evolution of risk. These attributes include, but are not limited to, weather conditions, visibility conditions, road adhesion status, and driver status. Since the number and specific types of floating attributes vary across different accidents, this invention models them as a variable-length attribute list structure to avoid the information loss or limited expression problems caused by using fixed fields.

[0047] Meanwhile, considering that official accident summaries usually contain a large amount of interfering information that is not directly related to the feasibility of the scenario or the causal relationship of the risk, such as the year and model of the vehicle, the status of the airbags, and the medical treatment of the injured, this invention explicitly excludes the above-mentioned interfering information based on the principle of "whether it affects the establishment of the scenario structure or the formation of risk", and does not allow it to be included in the unified accident semantic representation, thereby improving the conciseness and consistency of attribute representation.

[0048] by Figure 2Taking the accident shown as an example, this accident occurred during the day under snowy conditions, with the road surface in an icy state. The "daytime" aspect did not introduce additional hazards and is generally consistent with the system's default simulation environment settings, therefore it was not identified as a floating attribute. However, "snowfall" and "icy roads" significantly reduced road adhesion, which are important external conditions leading to vehicle loss of control, and were thus identified and extracted as floating attributes. By explicitly modeling these attributes, on the one hand, the key conditions for the accident can be accurately reflected in accident reproduction scenarios, enhancing the interpretability of scenario behavior results; on the other hand, these floating attributes can also be reused in ordinary driving or near-miss scenarios, systematically expanding the test scenario set in the form of a small number of key events, thereby effectively alleviating the problem of insufficient long-tail risk coverage in autonomous driving testing.

[0049] Specifically, after the initial extraction of floating attributes, the pre-built semantically extracted large language model is used to perform the following processing on the floating attributes: determining whether the extracted attributes meet the definition range of floating attributes; evaluating whether the extracted attributes are comprehensive and accurate, and whether they do indeed have an impact on the occurrence or evolution of the accident; and further summarizing the direct consequences or behavioral constraints caused by each floating attribute in the accident, such as "road icing" may lead to vehicle skidding or decreased braking performance. The above results will be stored together with the floating attributes as important inputs for subsequent scene enhancement, condition injection, and consistency verification.

[0050] S3. Using the fixed attributes, retrieve and assemble the corresponding scene code fragments in the pre-built primitive template library to obtain a basic scene description with basic traffic semantics.

[0051] In this embodiment of the invention, the basic scene description is a basic skeleton of the scene built solely based on fixed attributes. These fixed attributes are used to characterize the structural conditions necessary for the semantic validity of the accident, including the high-level behavioral patterns of traffic participants, the structural type of the road where the accident occurred, and the configuration of the generation points of each traffic participant. Constructing the basic scene description does not involve completely reproducing all the details of the accident, but rather, after excluding accidental environmental factors, determining and constructing the minimum executable scene to determine whether the accident still possesses the structural conditions for its occurrence.

[0052] Specifically, the step of retrieving and assembling corresponding scene code fragments from a pre-built primitive template library using the fixed attributes to obtain a basic scene description with basic traffic semantics includes: Retrieve primitive code fragments corresponding to the behavioral attribute, the road network structure attribute, and the generation point attribute from the pre-built primitive template library; The scene skeleton generation logic is constructed based on the fixed attributes. The primitive code fragments are assembled according to the scene skeleton generation logic to obtain the basic scene skeleton; Information encoding is performed on the basic scene skeleton to obtain a basic scene description with basic traffic semantics.

[0053] In this embodiment of the invention, a primitive template library containing historical executable scenario code snippets is constructed by combining Retrieval-Augmented Generation (RAG) technology. When code corresponding to a certain fixed attribute needs to be generated, semantically similar primitive code snippets are first retrieved from the template library based on that attribute, and the retrieval results and attribute descriptions are input into the large language model. Under template constraints, the large language model only needs to perform parameter filling or minor modifications on the retrieved code snippets to generate code descriptions corresponding to the behavioral attributes, road network structure attributes, and generation point attributes in the fixed attributes, significantly reducing the degrees of freedom and illusion risks in the generation process. Specifically, semantically similar primitive code snippets can be retrieved from the primitive template library by calculating semantic similarity.

[0054] This invention further abstracts fixed attributes into scene primitives with explicit semantic functions. Each type of scene primitive corresponds to a type of fixed attribute, such as road structure primitives, generation point primitives, and behavior pattern primitives, and predefines parameterized scene description templates for each type of primitive. The templates limit the syntactic structure of the primitives in the scene, the range of possible parameters, and their compatibility with other primitives, thereby avoiding unreasonable attribute combinations at the template level.

[0055] Furthermore, the scene skeleton generation logic for constructing the basic scene based on the fixed attributes includes: The spatial structure constraints of the traffic scenario are constructed based on the road network structure attribute in the fixed attributes. The initial layout of traffic participants is constructed based on the generation point attribute in the fixed attributes; Assign high-level behavioral rules to the traffic participants based on the behavioral attributes in the fixed attributes; The scene skeleton generation logic is obtained by combining the spatial structure constraints, the initial step distance, and the high-level behavior rules.

[0056] Spatial structural constraints are the geometric and topological rules that all traffic participants in the basic scenario must abide by. These include the spatial topology of the scenario and its constraint relationships, such as: road network structure (roads, intersections, roundabouts, ramps, number of lanes, lane direction); geometric boundaries (road width, length, curve radius, slope); passable areas (lanes, sidewalks, non-motorized vehicle lanes, restricted areas, guardrails, green belts); topological connections (which roads connect to which roads, where turns are possible, where dead ends are); traffic facilities (traffic light locations, stop lines, zebra crossings, guide lines); and static obstacles (bridge piers, roadblocks, barriers, buildings, etc.). Initial layout refers to the initial position, orientation, and relative arrangement of all traffic participants at the initial moment, such as position coordinates, type, and number of traffic participants. High-level behavioral rules are the intention-level and decision-level rules for each traffic participant, such as cruising, lane changing, turning, parking, and overtaking.

[0057] Specifically, for spatial structural constraints, the road network's attributes such as roads, lanes, connections, geometry, and rules are first read from the road network structure properties. Based on road connections and lane flow direction, node-connectivity relationships are constructed to form the spatial topology of the scene. Based on attributes such as road boundaries, lane rules, speed limits, no-entry zones, and intersection rules, position constraints, driving constraints, turning constraints, and conflict constraints are derived to form complete spatial constraint relationships. Then, the spatial topology and constraint relationships together constitute the spatial structural constraints of the traffic scene.

[0058] Furthermore, based on the initial position, quantity, type, and orientation of various traffic participants in the generated point attributes, a static distribution state is constructed at the start of the scene, resulting in the initial layout of the traffic participants. Then, based on the behavioral attributes, corresponding intent-level behavioral rules are matched for each traffic participant, and behavioral parameters such as target speed, safe distance, turning timing, and priority are configured, assigning high-level behavioral rules to each traffic participant.

[0059] Preferably, the scene skeleton generation logic, which is composed of spatial structure constraints, initial step distance, and high-level behavior rules, can maintain consistency in traffic semantics and is executable at the physical constraint level.

[0060] In this embodiment of the invention, before assembling the primitive code fragments according to the scene skeleton generation logic, a pre-built large language model is used. Under the configured template constraints, the large language model can fill parameters or make minor modifications to the primitive code fragments and fixed attributes, and further generate code descriptions corresponding to the fixed attributes, which significantly reduces the degree of freedom and illusion risk in the generation process.

[0061] Preferably, an executable verification mechanism is introduced during the generation process to check the form and parameter completeness of the generated primitive code snippets. This is to confirm that the scene description is syntactically correct, that all required parameters are present, and that it can be correctly parsed and loaded by the target simulation environment, thereby avoiding scene unexecutability issues caused by format errors or missing parameters.

[0062] Furthermore, assembling the primitive code fragments according to the scene skeleton generation logic involves combining the primitive code fragments into a recognizable fixed structure organization. The first layer consists of spatial structural constraints and physical constraints in the scene skeleton generation logic. An initial layout of traffic participants is superimposed on the first layer to ensure that the initial layout conforms to the spatial structural constraints and physical constraints. High-level behavioral rules are then assigned to the initial layout that conforms to the spatial structural constraints and physical constraints to generate a basic scene skeleton description that is semantically consistent and physically executable.

[0063] Preferably, primitive code fragments, their corresponding fixed attributes, and high-level accident semantic information can be input into a pre-built large language model to verify whether the generated result is consistent with the fixed attributes. If the verification fails, the corresponding fixed attributes are regenerated or replaced; if the verification passes, the primitive code fragments are passed to the information encoder for information encoding.

[0064] In this embodiment of the invention, information encoding involves passing a basic scene skeleton composed of primitive code fragments to a pre-built information encoder. The information encoder, based on predefined regular expressions and mapping rules, converts the validated code description into standard element representations in a target domain-specific language, thereby obtaining a complete and standardized basic scene description.

[0065] S4. Model the floating attributes as configurable environmental parameters and behavioral constraint rules, and use the environmental parameters and behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description.

[0066] In this embodiment of the invention, the basic scenario description is constructed based solely on fixed attributes. While a scenario constructed solely on fixed attributes is executable in a simulation environment and can reproduce the basic traffic conflict structure, the causal relationship between its behavioral outcomes and environmental conditions is not clear enough, making it difficult to accurately reflect the true causes of the accident. For example, in Figure 2In the illustrated case, in scenarios reconstructed solely based on fixed attributes, vehicles exhibit trajectory deviation when performing steering or braking operations. This phenomenon, in real accidents, stems from skidding due to decreased road surface adhesion. However, without corresponding environmental constraints, the simulation system struggles to provide a reasonable explanation for this behavior. Furthermore, when one of the traffic participants is designated as the tested vehicle, the lack of relevant environmental constraints means the vehicle may still avoid a collision through normal braking, leading to inconsistencies between the test results and real-world accident scenarios, thus weakening the effectiveness of the scenario test. Therefore, by modeling floating attributes as configurable environmental parameters and incorporating the aforementioned behavioral constraint rules to behaviorally constrain and extend the basic scenario description, a complete scenario description is obtained.

[0067] Specifically, modeling the floating attributes as configurable environmental parameters and behavioral constraint rules allows the floating attributes to be divided into environmental attributes and behavioral rule attributes. Environmental attributes represent environmental information within high-level accident semantics, such as floating attributes like "snowfall" and "road icing." Behavioral rule attributes represent the actions of traffic participants, such as vehicle navigation and turning. By encapsulating environmental attributes and behavioral rule attributes into executable policy parameters, the environmental parameters corresponding to the environmental attributes and the behavioral constraint rules corresponding to the behavioral rule attributes are obtained.

[0068] Furthermore, the step of using the environmental parameters and the behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description includes: The environmental parameters and behavioral constraint rules are used to constrain the basic scene description to obtain a constrained scene. Based on the environmental parameters, the behavioral constraint rules, and the basic scenario description, key safety scenarios are generated using preset prompt templates. By combining the aforementioned constraint scenarios and key security scenarios, a complete scenario description is obtained.

[0069] In this embodiment of the invention, environmental parameters and behavioral constraint rules are injected into the basic scene description in an assimil manner. Each floating attribute corresponds to a conditional transformation in the scene execution process, and its effects include, but are not limited to: changing environmental visibility or perceived quality, introducing additional uncertainty or execution delay, and amplifying or weakening the risk consequences of specific behavioral decisions. Multiple floating attributes can act simultaneously on the same basic scene to jointly characterize risk factors under complex environmental conditions.

[0070] For example, by constraining the environment and behavioral rules in the basic scenario description to floating attributes such as "snowfall" and "road icing," a constrained scenario is obtained. On the one hand, the environmental causes of vehicle skidding and drifting can be clearly explained. On the other hand, the behavior execution process of all participating vehicles in the scenario is subject to corresponding constraints, such as a reduction in braking deceleration or a slight delay in control commands. This more realistically reflects driving conditions under complex road conditions and helps in the testing and evaluation of automated driving systems (ADS) in rare long-tail scenarios.

[0071] Furthermore, in addition to scenario constraints, this invention also saves and reuses floating attributes extracted from real traffic accident sketches and corresponding text summaries for systematic expansion of a large number of normal or low-risk driving scenarios. Specifically, this invention inputs the basic scenario description and floating attribute list into a pre-built autonomous driving scenario generation system, and guides the model to generate key safety scenarios that conform to real-world constraints through preset prompt templates, such as the LangAuto (Language-guided Autonomous Driving) model and GPT-4.

[0072] By injecting floating attributes into the base scenario in a way independent of the scenario structure, this invention achieves a scenario generation mechanism that "maintains the structure while adjusting the risk level," enabling the same scenario skeleton to correspond to a set of scenario instances with different levels of danger. This effectively improves the diversity of test scenarios and the ability to cover long-tail risks. By utilizing risk factors that have occurred in real accidents as priors, the generated test scenarios are statistically closer to the real world in terms of both statistical significance and traffic semantics, thereby improving the effectiveness and relevance of autonomous driving system testing.

[0073] S5. Perform scenario-level semantic consistency verification on the complete scenario description to obtain a set of target security critical scenarios.

[0074] In this embodiment of the invention, scene-level semantic consistency verification is used to determine whether the extended scene conforms to real-world constraints, thereby avoiding unreasonable or inexplicable scene settings caused by attribute superposition.

[0075] Specifically, the step of performing scene-level semantic consistency verification on the complete scene description to obtain a set of target security-critical scenes includes: The logical constraints and semantic conflicts of the complete scene description are verified using preset semantic rules. If the verification fails, return to the step of using the environmental parameters and the behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description. If the verification passes, the complete scenario description will be used as the target security critical scenario set.

[0076] In this embodiment of the invention, semantic rule verification includes traffic rules, physical common sense, and scene logic constraints, judging whether there are semantic conflicts or unreasonable combinations among various attributes. For example, it verifies whether environmental conditions and behavioral patterns match, whether floating attributes have a reasonable impact on relevant traffic participants in the scene, and whether the overall description conforms to the real traffic situation.

[0077] Specifically, verifying whether environmental conditions and behavioral patterns match can be done using a traffic domain knowledge graph to verify logical contradictions and whether environmental parameters and behavioral rule constraints conform to physical logic. For example, if stopping within 10 meters of emergency braking at 120 km / h in rainy weather is physically impossible, the verification will fail. Verifying whether floating attributes have a reasonable impact on relevant traffic participants in the scenario involves verifying whether the floating attributes produce expected traffic participant behaviors. For example, if a vehicle deviates too far from its lane, exceeding the normal range of aggressive driving, the verification will fail. Verifying whether the overall description conforms to real-world traffic situations involves verifying whether the scenario conforms to social norms and traffic regulations. This can be done by converting each complete scenario description into a formalized specification and then using pre-defined semantic rules for verification. For example, the specification: Vehicles must stop at red lights.

[0078] Furthermore, if the verification fails, the process returns to the steps described above: using the environmental parameters and behavioral constraint rules to constrain and expand the basic scene description to obtain a complete scene description. Attribute combinations that cause conflicts are corrected, replaced, or discarded, and the subsequent construction and verification processes are re-executed. Through this multi-level verification and rollback mechanism, it can be ensured that the final generated set of target safety-critical scenarios meets the application requirements for autonomous driving testing in terms of semantic consistency, realistic rationality, and simulation executability.

[0079] In detail, this invention can backtrack to the corresponding attribute level or scene construction stage based on the type of error during filtering of interference information from floating attributes, verification of primitive code snippets, and scene-level semantic consistency verification of the complete scene description. It can then correct, replace, or discard the attribute combinations that cause conflicts and re-execute the subsequent construction and verification processes. Through this multi-level verification and backtracking mechanism, this invention ensures that the final generated scene meets the application requirements for autonomous driving testing in terms of semantic consistency, realistic rationality, and simulation executability.

[0080] In this embodiment of the invention, by using scene-level semantic consistency verification, when semantic conflicts, unreasonable attribute combinations, or unexecutable situations are detected, the system can backtrack to the corresponding steps for correction or regeneration, thereby ensuring that the final output scene meets the engineering application requirements in terms of semantic rationality, reality consistency, and simulation executability.

[0081] like Figure 3 The diagram shown is a functional block diagram of an autonomous driving safety critical scenario generation system provided in an embodiment of the present invention.

[0082] The autonomous driving safety critical scenario generation system 100 of this invention can be installed in a processing device. Depending on the functions implemented, the autonomous driving safety critical scenario generation system 100 may include a semantic parsing module 101, an attribute construction module 102, a basic scenario description generation module 103, a scenario constraint and scenario expansion module 104, and a semantic verification module 105. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0083] In this embodiment, the functions of each module / unit are as follows: The semantic parsing module 101 is used to perform structured parsing on real traffic accident sketches and corresponding text summaries to obtain high-level accident semantic information. The attribute construction module 102 is used to construct fixed attributes, including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident, based on the high-level accident semantic information. The basic scene description generation module 103 is used to retrieve and assemble corresponding scene code fragments in the pre-built primitive template library using the fixed attributes to obtain a basic scene description with basic traffic semantics. The scene constraint and scene expansion module 104 is used to model the floating attribute as configurable environmental parameters and behavioral constraint rules, and use the environmental parameters and behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description. The semantic verification module 105 is used to perform scene-level semantic consistency verification on the complete scene description to obtain the target security critical scene set.

[0084] The specific execution methods for the steps in each of the above modules are the same as the corresponding execution steps in the above method for generating key safety scenarios for autonomous driving.

[0085] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0086] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0087] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating key safety scenarios for autonomous driving, characterized in that, include: Structured parsing of real traffic accident sketches and corresponding text summaries yields high-level accident semantic information. Based on the high-level accident semantic information, fixed attributes including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident, are constructed. Using the fixed attributes, the corresponding scene code fragments are retrieved and assembled from the pre-built primitive template library to obtain a basic scene description with basic traffic semantics; The floating attributes are modeled as configurable environmental parameters and behavioral constraint rules. The environmental parameters and behavioral constraint rules are used to constrain and expand the basic scene description to obtain a complete scene description. A scenario-level semantic consistency check is performed on the complete scenario description to obtain the target security critical scenario set.

2. The method for generating key safety scenarios for autonomous driving as described in claim 1, characterized in that, The structured parsing of real traffic accident sketches and corresponding text summaries yields high-level accident semantic information, including: Extract visual features from the real traffic accident sketch; The visual features are then aligned across text modalities to obtain visual entity features; A pre-built semantic parsing large language model is used to perform semantic parsing on the visual entity features and the text summary to obtain high-level accident semantic information.

3. The method for generating key safety scenarios for autonomous driving as described in claim 1, characterized in that, The construction of fixed attributes, including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident, based on the high-level accident semantic information includes: Entity recognition and annotation are performed on the semantic information of the high-rise accident. The semantic information of high-level accidents after entity recognition and annotation is subjected to relational extraction of behavioral attributes, road network structure attributes and generation point attributes to obtain fixed attributes containing behavioral attributes, road network structure attributes and generation point attributes. Initial floating attributes are extracted from the high-level accident semantic information using a variable-length attribute list structure; The initial floating attributes are filtered for interference information to obtain the floating attributes related to the occurrence of the accident.

4. The method for generating key safety scenarios for autonomous driving as described in claim 1, characterized in that, The step of retrieving and assembling corresponding scene code fragments from a pre-built primitive template library using the fixed attributes to obtain a basic scene description with basic traffic semantics includes: Retrieve primitive code fragments corresponding to the behavioral attribute, the road network structure attribute, and the generation point attribute from the pre-built primitive template library; The scene skeleton generation logic is constructed based on the fixed attributes. The primitive code fragments are assembled according to the scene skeleton generation logic to obtain the basic scene skeleton; Information encoding is performed on the basic scene skeleton to obtain a basic scene description with basic traffic semantics.

5. The method for generating key safety scenarios for autonomous driving as described in claim 4, characterized in that, The scene skeleton generation logic for constructing the basic scene based on the fixed attributes includes: The spatial structure constraints of the traffic scenario are constructed based on the road network structure attribute in the fixed attributes. The initial layout of traffic participants is constructed based on the generation point attribute in the fixed attributes; Assign high-level behavioral rules to the traffic participants based on the behavioral attributes in the fixed attributes; The scene skeleton generation logic is obtained by combining the spatial structure constraints, the initial step distance, and the high-level behavior rules.

6. The method for generating key safety scenarios for autonomous driving as described in claim 1, characterized in that, The process of using the environmental parameters and behavioral constraint rules to constrain and expand the basic scene description to obtain a complete scene description includes: The environmental parameters and behavioral constraint rules are used to constrain the basic scene description to obtain a constrained scene. Based on the environmental parameters, the behavioral constraint rules, and the basic scenario description, key safety scenarios are generated using preset prompt templates. By combining the aforementioned constraint scenarios and key security scenarios, a complete scenario description is obtained.

7. The method for generating key safety scenarios for autonomous driving as described in claim 1, characterized in that, The step of performing scene-level semantic consistency verification on the complete scene description yields a set of target security-critical scenes, including: The logical constraints and semantic conflicts of the complete scene description are verified using preset semantic rules. If the verification fails, return to the step of using the environmental parameters and the behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description. If the verification passes, the complete scenario description will be used as the target security critical scenario set.

8. A system for generating safety-critical scenarios for autonomous driving, characterized in that, include: The semantic parsing module is used to perform structured parsing of real traffic accident sketches and corresponding text summaries to obtain high-level accident semantic information. The attribute construction module is used to construct fixed attributes, including behavioral attributes, road network structure attributes, and generation point attributes, as well as floating attributes related to the occurrence of the accident, based on the high-level accident semantic information. The basic scene description generation module is used to retrieve and assemble corresponding scene code fragments in the pre-built primitive template library using the fixed attributes to obtain a basic scene description with basic traffic semantics. The scene constraint and scene expansion module is used to model the floating attributes as configurable environmental parameters and behavioral constraint rules, and to use the environmental parameters and behavioral constraint rules to perform scene constraints and scene expansion on the basic scene description to obtain a complete scene description. The semantic verification module is used to perform scene-level semantic consistency verification on the complete scene description to obtain the target security critical scene set.

9. A processing device, characterized in that, It includes at least one processor and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor can execute the method as described in any one of claims 1-7 by invoking the program instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the computer to perform the method as described in any one of claims 1-7.

Images