Automatic driving test scene set optimization method and device, equipment and storage medium

Abstract

The invention discloses an automatic driving test scene set optimization method, apparatus and device, and a storage medium. The method comprises the steps of generating a simulation scene file in an OpenSCENARIO format through preprocessing data; analyzing the simulation scene file through a teacher model, outputting a risk description text, receiving a scene feature vector and the risk description text through a student model, and outputting a failure probability; determining a comprehensive value index according to the failure probability, dynamically updating a scene library of automatic driving test scenes, collecting failure data in an AUT test, performing incremental fine tuning on the student model, and obtaining an optimized target test scene set; the problem of gradient estimation variance explosion caused by sparseness disasters can be effectively solved, the stability of model training is improved, the recognition capability of a rare long-tail failure scene is enhanced, the accuracy of failure probability prediction is improved, the judgment accuracy of the model to a boundary scene is improved, and the accuracy and comprehensiveness of scene value quantification are improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a method, apparatus, device, and storage medium for optimizing autonomous driving test scenario sets. Background Technology

[0002] With the rapid development of the automotive industry, autonomous driving technology has gradually become a focus of industry attention. The research and application of autonomous driving technology can not only improve traffic efficiency and reduce traffic accident rates, but also provide users with a more convenient and comfortable travel experience. Therefore, governments around the world attach great importance to the development of autonomous driving technology and have introduced a series of policies to support its research and promotion.

[0003] As an important foundation for the research and development and testing of autonomous driving technology, the autonomous driving simulation test scenario library can provide a rich testing environment for autonomous driving systems, which helps to improve the adaptability and robustness of autonomous driving systems. At the same time, it can also reduce the risks and costs of actual road testing and shorten the research and development cycle of autonomous driving technology. When constructing the autonomous driving simulation test scenario library, it is necessary to follow the principles of comprehensiveness, representativeness, scalability and consistency to ensure that the scenario library can meet the testing needs of autonomous driving systems.

[0004] An existing iterative optimization system for autonomous driving test scenario sets based on closed-loop real-world data includes the following modules: Real data acquisition module: Deployed on mass-produced autonomous vehicles, it collects raw driving data in real time during vehicle operation through multi-source sensors such as cameras, lidar, millimeter-wave radar, Global Positioning System (GPS) / Inertial Measurement Unit (IMU).

[0005] Scene extraction and reconstruction module: Receives preprocessed data from the real data acquisition module, fuses multi-source sensor data through a spatiotemporal alignment algorithm, and automatically parses the data stream based on a preset scene semantic labeling system. This system includes dimensions such as road structure, traffic participant type, behavioral intent, and interaction relationship. It extracts discretized atomic scene units and reconstructs these atomic scene units into reproducible simulation test scene files in a format compatible with OpenSCENARIO / OpenDRIVE for subsequent scene analysis and testing.

[0006] The Scenario Value Quantification Module receives the reconstructed simulation test scenario file and quantifies the test value of the scenario through a series of calculations. Based on the historical accident database and safety boundary model, it calculates the potential collision severity score (SS) of the current scenario. It simulates the motion trajectory conflicts of all traffic participants in the scenario through a multibody dynamics model. If a collision is detected, it queries the damage benchmark table based on the collision speed difference, collision angle, and participant type. It constructs a binary adversarial model with scenario feature vectors as input and the decision errors of the Automated Driving System Under Test (AUT) as output. It trains the model using historical failure scenario data and outputs the failure probability (FP) of the AUT in the scenario. It combines SS and FP to generate the Scenario Risk Value (RV). Based on information entropy theory, it calculates the feature distribution difference degree (DD) between the current scenario and the existing test scenario library, and gives the comprehensive value index (VI) based on the scenario risk value and feature distribution difference degree.

[0007] Test scenario set generation module: Sort all scenarios in descending order according to VI, and dynamically select Top-K high-value scenarios to inject into the target test scenario library.

[0008] The existing "scene value quantification module" uses an adversarial neural network model to predict the failure probability (FP) of the autonomous driving system under test (AUT). This design may face the problem of gradient estimation variance explosion caused by rare events during implementation, which may lead to model training divergence or failure. The core contradiction of this problem lies in the extreme sparsity of long-tail autonomous driving scenarios. Specifically, real failure scenarios, such as "ghost peek" and "illegal entry", account for a very low proportion in the dataset, resulting in a serious imbalance between positive and negative samples.

[0009] From the perspective of the problem generation mechanism, the adversarial neural network takes the scene feature vector as input and outputs the failure probability FP∈[0,1] of the AUT. The training process needs to maximize the ability to distinguish historical failure scenes (FP→1) while minimizing the misjudgment of normal scenes (FP→0). However, since the sample size of rare failure scenes is extremely small, the gradient calculation of the loss function for these samples during backpropagation produces high variance due to insufficient sampling.

[0010] Mathematical analysis shows that the variance of Monte Carlo gradient estimation is inversely proportional to the probability p of the failure scenario. When p approaches 0, the variance approaches infinity, leading to an exponential increase in the instability of gradient estimation. This phenomenon is essentially consistent with the "Catastrophe of Rarity" (CoR) proposed by the Tsinghua team in a Nature sub-journal. In actual training, it will cause the loss curve to fluctuate violently and fail to converge, similar to the instability phenomenon in Bird's-Eye View (BEV) quantization training. To reduce variance, the network tends to ignore rare samples, causing the FP prediction value to always be close to 0, completely losing the ability to identify real risk scenarios. Summary of the Invention

[0011] The main objective of this invention is to provide a method, apparatus, device, and storage medium for optimizing autonomous driving test scenario sets. This invention aims to solve the technical problem in the prior art where the gradient estimation variance of the adversarial neural network model in the scenario value quantification module explodes due to rare events, leading to model training divergence or failure, severe imbalance between positive and negative samples, and poor ability to identify real risk scenarios.

[0012] In a first aspect, the present invention provides a method for optimizing an autonomous driving test scenario set, the method comprising the following steps: Collect raw driving data, preprocess the raw driving data to obtain preprocessed data, and generate a simulation scene file in OpenSCENARIO format based on the preprocessed data; The simulation scenario file is parsed by a preset teacher model, and the risk description text is output. The scenario feature vector and the risk description text are received by a preset student model, and the failure probability is output. The comprehensive value index is determined based on the failure probability. The scenario library for autonomous driving test scenarios is dynamically updated based on the comprehensive value index. Failure data in the AUT test of the autonomous driving system under test is collected. The student model is incrementally fine-tuned based on the failure data to fill in the boundary areas misjudged by the student model and obtain the optimized target test scenario set.

[0013] Optionally, the step of collecting raw driving data, preprocessing the raw driving data to obtain preprocessed data, and generating a simulation scene file in OpenSCENARIO format based on the preprocessed data includes: The original driving data of the current autonomous vehicle is collected through the real data acquisition module of the current autonomous vehicle; The raw driving data is transformed into a structured feature stream, retaining key risk factors, and noise is added to sensitive information in the raw driving data to ensure that the preprocessed data meets the differential privacy definition.

[0014] in, For the perturbation of the first One characteristic, The original value, For function Sensitivity Budget for privacy; The preprocessed data is spatiotemporally aligned to generate atomic scene units, which are then reconstructed into simulation scene files in OpenSCENARIO format.

[0015] Optionally, the step of spatiotemporally aligning the preprocessed data to generate atomic scene units, and reconstructing the atomic scene units into simulation scene files in OpenSCENARIO format, includes: The system obtains the timestamps provided by the Global Positioning System (GPS) and the Inertial Measurement Unit (IMU). Based on the timestamps, it performs joint spatial reprojection on the object recognition results in each frame of the preprocessed data to determine the position, orientation, and speed information of each type of traffic participant in the same coordinate system, forming a frame-level entity state description. The preprocessed data is divided into scene segments in the time dimension, and time windows with scene variability that meet the requirements are extracted as candidate keyframe intervals. Entity behavior patterns are determined based on frame-level entity state descriptions, and similar fragments are clustered based on candidate keyframe intervals. Based on the entity behavior pattern and the environment configuration, cluster similar fragments to generate atomic scene units, convert the atomic scene units into a data format conforming to the OpenSCENARIO standard, and encapsulate them into simulation scene files that can be reproduced by the simulation platform.

[0016] Optionally, the step of parsing the simulation scenario file through a preset teacher model and outputting risk description text, and receiving the scenario feature vector and the risk description text through a preset student model and outputting the failure probability, includes: The simulation scenario file is parsed using a Visual Language Model (VLM) based on a pre-defined teacher model, and the risk description text is output. The system receives scene feature vectors and risk description text through a preset student model, and converts the risk description text into failure probability (FP) soft labels through a semantic-probability mapper. The failure probability is then calculated by combining the FP soft labels with the real labels.

[0017] Optionally, the step of receiving the scene feature vector and the risk description text through a preset student model, converting the risk description text into a failure probability (FP) soft label through a semantic-probability mapper, and calculating the failure probability by combining the FP soft label with the real label includes: The scenario feature vector and the risk description text are received through a preset student model; The risk description text is encoded using the pre-trained text encoding model BERT, and the output is a semantic vector that captures the semantic information and underlying logical structure of the risk description text. A linear mapping matrix is ​​constructed based on the semantic vector using the following formula:

[0018] in, It is a linear mapping matrix. It is a matrix space of 768 rows and 1 column consisting of real numbers; Based on the linear mapping matrix, the FP soft tag corresponding to the risk description text is obtained by projection using the following formula;

[0019] in, This represents the probability value of soft tag failure. For the Sigmoid function, , A semantic vector with dimension 768. It is a linear mapping matrix. , For the set of real numbers, For bias terms; The true labels are obtained from the original driving data, and the binary cross-entropy loss function of the preset student model is determined based on the true labels and the FP soft labels:

[0020] in, For binary cross-entropy loss, For real labels, The probability value predicted by the preset student model; The total loss function is obtained from the binary cross-entropy loss function using the following formula:

[0021] in, For the total loss function, It is a hyperparameter that adjusts the weights of the two monitoring signals. For real labels, The output prediction of the preset student model; The gradient variance penalty term is added to the total loss function by the following formula:

[0022] in, This is the gradient variance penalty coefficient. , For the current training round, For the loss function Find the gradient. express Squaring norms; By constraining the preset student model with the total loss function, the failure probability of the optimized model is obtained.

[0023] Optionally, the step of determining a comprehensive value index based on the failure probability, dynamically updating the scenario library of autonomous driving test scenarios based on the comprehensive value index, collecting failure data in the AUT test of the autonomous driving system under test, incrementally fine-tuning the student model based on the failure data, supplementing the boundary areas misjudged by the student model, and obtaining an optimized target test scenario set includes: The potential collision severity of different scenarios is obtained based on the risk description text, and a scenario risk value is generated based on the potential collision severity and the failure probability. Calculate the data diversity of the current scenario and the scenario library for autonomous driving test scenarios; The comprehensive value index is determined based on the scenario risk value and the data diversity using the following formula:

[0024] in, For the first The comprehensive value index of each scenario. and These are the weighting coefficients. For the first The scenario risk value for each scenario. For the first Data diversity in each scenario; A sliding time window mechanism is adopted. Every time a preset number of real scenes are added, the comprehensive value index is dynamically sorted once. Based on the sorting results, relevant scenes in the scene library are retained, added, or eliminated to obtain a dynamically updated target scene library. Based on the scenario risk value, the sampling weights are determined using a dynamic gradient stabilizer according to the following formula:

[0025] in, For the first Sampling weights for each scenario, Indicates proportional to, For the first The scenario risk value for each scenario; Failure data from the AUT test of the autonomous driving system under test is collected based on the sampling weights and the target scenario library. The failure data is structured to extract high-risk scenario feature vectors; The high-risk scene feature vector is used as a conditional input to the conditional generative adversarial network CGAN. The generator and discriminator are iteratively optimized under the adversarial mechanism to obtain edge failure scene data. The edge failure scenario data is verified by CARLA simulation. The verified synthetic data and real failure data are mixed and injected into the training set according to a preset mixing ratio to obtain the failure dataset. The parameters of the preset proportional layer of the student model are frozen, and the fully connected layer of the student model is trained according to the failure dataset, so that the update of the student model focuses on fine-tuning the output decision boundary. When the backpropagation gradient magnitude of the student model exceeds a preset magnitude threshold, the gradient vector is forcibly truncated to fill in the boundary regions that were misjudged by the student model, thereby obtaining an optimized target test scenario set.

[0026] Optionally, the step of using the high-risk scene feature vector as a conditional input to a conditional generative adversarial network (CGAN), and obtaining edge failure scene data through iterative optimization of the generator and discriminator under the adversarial mechanism, includes: The high-risk scene feature vectors are used as conditional inputs to a Conditional Generative Adversarial Network (CGAN). The generator and discriminator iteratively optimize the loss function under the adversarial mechanism using the following formula to obtain edge failure scene data:

[0027] in, For generator, For discriminator, For real samples, For noise vectors, For high-risk scene feature vectors, Output the synthesized sample.

[0028] Secondly, to achieve the above objectives, the present invention also proposes an autonomous driving test scenario set optimization device, the autonomous driving test scenario set optimization device comprising: The preprocessing module is used to collect raw driving data, preprocess the raw driving data to obtain preprocessed data, and generate simulation scene files in OpenSCENARIO format based on the preprocessed data. The file parsing module is used to parse the simulation scenario file through a preset teacher model, output risk description text, receive the scenario feature vector and the risk description text through a preset student model, and output the failure probability. The scenario library update module is used to determine the comprehensive value index based on the failure probability, dynamically update the scenario library of autonomous driving test scenarios based on the comprehensive value index, collect failure data in the AUT test of the autonomous driving system under test, incrementally fine-tune the student model based on the failure data, fill in the boundary areas misjudged by the student model, and obtain the optimized target test scenario set.

[0029] Thirdly, to achieve the above objectives, the present invention also proposes an autonomous driving test scenario set optimization device, the autonomous driving test scenario set optimization device comprising: a memory, a processor, and an autonomous driving test scenario set optimization program stored in the memory and executable on the processor, the autonomous driving test scenario set optimization program being configured to implement the steps of the autonomous driving test scenario set optimization method described above.

[0030] Fourthly, to achieve the above objectives, the present invention also proposes a storage medium storing an autonomous driving test scenario set optimization program, wherein the autonomous driving test scenario set optimization program, when executed by a processor, implements the steps of the autonomous driving test scenario set optimization method described above.

[0031] The proposed method for optimizing autonomous driving test scenario sets involves collecting raw driving data, preprocessing the raw data to obtain preprocessed data, and generating simulation scenario files in OpenSCENARIO format based on the preprocessed data. A pre-set teacher model parses the simulation scenario files, outputting risk description text. A pre-set student model receives scenario feature vectors and the risk description text, outputting failure probabilities. A comprehensive value index is determined based on the failure probabilities. The scenario library for autonomous driving test scenarios is dynamically updated based on the comprehensive value index. Failure data from the AUT (Autonomous Under Test) testing of the autonomous driving system is also collected. The data is used to incrementally fine-tune the student model, filling in the boundary regions that were misjudged by the student model, and obtaining an optimized target test scenario set. This effectively solves the gradient estimation variance explosion problem caused by the "curse of sparsity," improves the stability of model training, enhances the ability to identify long-tail rare failure scenarios, improves the accuracy of failure probability prediction, actively reduces the sparsity of data distribution, expands the coverage of training data, improves the model's accuracy in identifying boundary scenarios, and improves the accuracy and comprehensiveness of scenario value quantification. This makes the generation of test scenario sets more targeted and effective, and improves the speed and efficiency of optimizing autonomous driving test scenario sets. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the autonomous driving test scenario set optimization method of the present invention; Figure 3 This is a flowchart illustrating the second embodiment of the autonomous driving test scenario set optimization method of the present invention; Figure 4 This is a schematic diagram of the overall architecture of the autonomous driving test scenario set optimization system of the present invention; Figure 5 This is a schematic diagram of the dual-model architecture of the scenario value quantification module in the autonomous driving test scenario set optimization system of the present invention; Figure 6 This is a schematic diagram of the workflow of the hybrid training module in the autonomous driving test scenario set optimization system of the present invention; Figure 7 This is a schematic diagram illustrating the working principle of the dynamic gradient stabilizer in the autonomous driving test scenario set optimization system of the present invention; Figure 8 This is a functional block diagram of the first embodiment of the autonomous driving test scenario set optimization device of the present invention.

[0033] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0034] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0035] Reference Figure 1 , Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention.

[0036] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0037] Those skilled in the art will understand that Figure 1 The device structure shown does not constitute a limitation on the device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0038] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating device, a network communication module, a user interface module, and an autonomous driving test scenario set optimization program.

[0039] The device of the present invention calls the autonomous driving test scenario set optimization program stored in the memory 1005 through the processor 1001, and performs the operations in the autonomous driving test scenario set optimization method embodiment described below.

[0040] Based on the above hardware structure, an embodiment of the autonomous driving test scenario set optimization method of the present invention is proposed.

[0041] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the autonomous driving test scenario set optimization method of the present invention.

[0042] In the first embodiment, the autonomous driving test scenario set optimization method includes the following steps: Step S10: Collect raw driving data, preprocess the raw driving data to obtain preprocessed data, and generate a simulation scene file in OpenSCENARIO format based on the preprocessed data.

[0043] It should be noted that after collecting the raw driving data of the current autonomous vehicle, the raw driving data can be preprocessed to obtain preprocessed data. The preprocessed data can be converted into a format to generate a simulation scene file in OpenSCENARIO format.

[0044] Furthermore, step S10 specifically includes the following steps: The original driving data of the current autonomous vehicle is collected through the real data acquisition module of the current autonomous vehicle; The raw driving data is transformed into a structured feature stream, retaining key risk factors, and noise is added to sensitive information in the raw driving data to ensure that the preprocessed data meets the differential privacy definition.

[0045] in, For the perturbation of the first One characteristic, The original value, For function Sensitivity Budget for privacy; The preprocessed data is spatiotemporally aligned to generate atomic scene units, which are then reconstructed into simulation scene files in OpenSCENARIO format.

[0046] It should be noted that raw driving data can be collected through the real data acquisition module of the current autonomous vehicle. The real data acquisition module is deployed in mass-produced autonomous vehicles and collects raw driving data through multi-source sensors, such as cameras, lidar, millimeter-wave radar, and GPS / IMU.

[0047] It should be understood that the real data acquisition module is deployed on mass-produced autonomous vehicles and has a complete multi-source perception system and onboard computing platform to continuously collect high-fidelity traffic scene information. The perception system includes high-definition cameras in the front, rear, left, and right directions, LiDAR, millimeter-wave radar, and a high-precision positioning module GPS / IMU, which work together to achieve multimodal observation of dynamic and static elements in the surrounding environment. The cameras are used to capture traffic signals, obstacles, and pedestrian appearances. The LiDAR outputs dense point clouds to build high-precision maps and obstacle outlines. The millimeter-wave radar supplements the detection of speed and presence of high-speed, long-distance targets. The GPS / IMU jointly provides the timestamp, pose, and acceleration information of each frame of data for subsequent data alignment and trajectory reconstruction.

[0048] Understandably, compression and privacy protection processing are performed immediately on the local edge device after data collection. The compression strategy is based on inter-frame redundancy analysis and feature extraction dimensionality reduction, transforming the raw data into a structured feature stream to reduce storage and transmission overhead while retaining key risk factors. Privacy protection adopts a differential privacy mechanism, adding noise interference to sensitive information to ensure that individual identities are not identifiable. Its core implementation uses a Laplace mechanism, adding a noise term to each output feature to satisfy the definition of differential privacy. The privacy budget is used to control the balance between perturbation strength and availability.

[0049] In the specific implementation, the feature vectors formed after all data is perturbed will be used as input for subsequent scene extraction and semantic modeling, which ensures data compliance and retains sufficient semantic information for failure scene reconstruction and value quantification. The overall acquisition cycle is controlled at the millisecond level to ensure that the vehicle can still achieve stable data flow closed-loop input in high-speed dynamic environment, providing a high-quality training foundation for the whole system.

[0050] It should be noted that after receiving the preprocessed data, scene extraction and reconstruction can be performed. That is, through spatiotemporal alignment, atomic scene units (ASUs) are generated and reconstructed into simulation scene files in OpenSCENARIO format.

[0051] Furthermore, the step of spatiotemporally aligning the preprocessed data to generate atomic scene units, and reconstructing the atomic scene units into simulation scene files in OpenSCENARIO format, includes: The system obtains the timestamps provided by the Global Positioning System (GPS) and the Inertial Measurement Unit (IMU). Based on the timestamps, it performs joint spatial reprojection on the object recognition results in each frame of the preprocessed data to determine the position, orientation, and speed information of each type of traffic participant in the same coordinate system, forming a frame-level entity state description. The preprocessed data is divided into scene segments in the time dimension, and time windows with scene variability that meet the requirements are extracted as candidate keyframe intervals. Entity behavior patterns are determined based on frame-level entity state descriptions, and similar fragments are clustered based on candidate keyframe intervals. Based on the entity behavior pattern and the environment configuration, cluster similar fragments to generate atomic scene units, convert the atomic scene units into a data format conforming to the OpenSCENARIO standard, and encapsulate them into simulation scene files that can be reproduced by the simulation platform.

[0052] Understandably, the purpose of scene extraction and reconstruction is to transform the multimodal data collected and preprocessed by the vehicle into structured, reproducible simulation scene files for subsequent value quantification and model training.

[0053] It should be understood that the process involves receiving a preprocessed data stream containing information such as image frames, point cloud data, millimeter-wave targets, and positioning trajectories. The time stamps provided by GPS / IMU are used to temporally align all modalities. By performing joint spatial reprojection on the object recognition results in each frame, the position, orientation, and speed information of each type of traffic participant in a unified coordinate system are determined, forming a frame-level entity state description. Subsequently, the scene is segmented along the temporal dimension, and time windows with significant scene variability are extracted as candidate keyframe intervals. Similar segments are then clustered based on entity behavior patterns and environmental configurations, ultimately generating atomic scene units.

[0054] It should be noted that the Atomic Scene Unit (ASU) consists of three core components: environment configuration, participant configuration, and behavioral trajectory. Environment configuration includes road topology, weather conditions, view occlusion, and lighting status; participant configuration includes entity ID, category, initial position, and dynamic attributes; and behavioral trajectory is represented by the position sequence of each entity within a time window, denoted as:

[0055] in, It is a position sequence. For the first An object in time The planar position, For the first An object in time speed, For the first An object in time The heading angle; After the atomic scene unit is constructed, the system converts it into a data format that conforms to the OpenSCENARIO standard and encapsulates it into a scene file that can be reproduced by simulation platforms such as Car Learning to Act (CARLA). Each item in the OpenSCENARIO file corresponds to a key element in ASU. By mapping semantic tags and control scripts, behavior triggering settings are completed, realizing the semantic verification function driven by simulation.

[0056] Step S20: Analyze the simulation scenario file using a preset teacher model, output risk description text, receive the scenario feature vector and the risk description text using a preset student model, and output the failure probability.

[0057] It should be understood that the simulation scenario file can be parsed through a pre-set teacher model, and then risk description text can be output. The scenario feature vector and the risk description text can be received through a pre-set student model, and then failure probability can be output.

[0058] Step S30: Determine the comprehensive value index based on the failure probability, dynamically update the scenario library of autonomous driving test scenarios based on the comprehensive value index, collect failure data in the AUT test of the autonomous driving system under test, incrementally fine-tune the student model based on the failure data, supplement the boundary areas misjudged by the student model, and obtain the optimized target test scenario set.

[0059] It is understood that the comprehensive value index can be determined based on the failure probability, the scenario library of autonomous driving test scenarios can be dynamically updated based on the comprehensive value index, and failure data in the AUT test of the autonomous driving system under test can be collected. The student model can be incrementally fine-tuned based on the failure data, thereby supplementing the boundary areas misjudged by the student model and obtaining the optimized target test scenario set.

[0060] This embodiment effectively solves the gradient estimation variance explosion problem caused by the "curse of sparsity" through the above solution, improves the stability of model training, enhances the ability to identify long-tail rare failure scenarios, improves the accuracy of failure probability prediction, actively reduces the sparsity of data distribution, expands the coverage of training data, improves the model's accuracy in distinguishing boundary scenarios, improves the accuracy and comprehensiveness of scenario value quantification, makes the generation of test scenario sets more targeted and effective, and improves the speed and efficiency of autonomous driving test scenario set optimization.

[0061] Furthermore, Figure 3 This is a flowchart illustrating the second embodiment of the autonomous driving test scenario set optimization method of the present invention, as shown below. Figure 3 As shown, based on the first embodiment, a second embodiment of the autonomous driving test scenario set optimization method of the present invention is proposed. In this embodiment, step S20 specifically includes the following steps: Step S21: Using a preset teacher model, the Visual Language Model (VLM) is used to parse the scene semantics of the simulation scene file and output the risk description text.

[0062] It should be noted that the preset teacher model can use a Vision-Language Model (VLM) to parse the scene semantics of the simulation scene file, thereby outputting risk description text, such as: "The truck on the right is blocking the pedestrian from crossing".

[0063] Step S22: Receive the scene feature vector and the risk description text through a preset student model, convert the risk description text into a failure probability (FP) soft label through a semantic-probability mapper, and calculate the failure probability by combining the FP soft label with the real label.

[0064] Understandably, by using a pre-set student model, a lightweight adversarial neural network can be implemented. It can receive scene feature vectors and risk descriptions output by the teacher model. The text can be converted into FP soft labels through a semantic-probability mapper, and the failure probability FP can be calculated by combining the real labels.

[0065] It should be understood that a two-stage distillation architecture is constructed at the model level. The strong semantic reasoning ability of the Visual Language Model (VLM) is used to generate scenario risk description text, such as "motorcycle illegally cutting in during a nighttime rainstorm". The text is transformed into smooth probability labels through a learnable semantic-probability mapper. The lightweight adversarial neural network serves as the student model, learning both the real failure labels and the soft labels of the teacher model. The teacher model's generalization ability for long-tail events is used to provide low-variance gradient signals.

[0066] It should be noted that the semantic-probability mapper aims to convert the risk description text output by the teacher model into soft-label probability values ​​that can be used by the student model, thereby achieving joint supervision at the semantic and numerical levels.

[0067] Furthermore, step S22 specifically includes the following steps: The scenario feature vector and the risk description text are received through a preset student model; The risk description text is encoded using the pre-trained text encoding model BERT, and the output is a semantic vector that captures the semantic information and underlying logical structure of the risk description text. A linear mapping matrix is ​​constructed based on the semantic vector using the following formula:

[0068] in, It is a linear mapping matrix. It is a matrix space of 768 rows and 1 column consisting of real numbers; Based on the linear mapping matrix, the FP soft tag corresponding to the risk description text is obtained by projection using the following formula;

[0069] in, This represents the probability value of soft tag failure. For the Sigmoid function, , A semantic vector with dimension 768. It is a linear mapping matrix. , For the set of real numbers, For bias terms; The true labels are obtained from the original driving data, and the binary cross-entropy loss function of the preset student model is determined based on the true labels and the FP soft labels:

[0070] in, For binary cross-entropy loss, For real labels, The probability value predicted by the preset student model; The total loss function is obtained from the binary cross-entropy loss function using the following formula:

[0071] in, For the total loss function, It is a hyperparameter that adjusts the weights of the two monitoring signals. For real labels, The output prediction of the preset student model; The gradient variance penalty term is added to the total loss function by the following formula:

[0072] in, This is the gradient variance penalty coefficient. , For the current training round, For the loss function Find the gradient. express Squaring norms; By constraining the preset student model with the total loss function, the failure probability of the optimized model is obtained.

[0073] It should be noted that a pre-trained text encoding model (Bidirectional Encoder Representations from Transformers, BERT) can be used to encode risk description text, outputting a semantic vector with a dimension of 768. This vector captures the semantic information and underlying logical structure of the text.

[0074] Understandably, to achieve the mapping from semantic space to probability space, a learnable linear mapping matrix needs to be constructed. And add a bias term. The failure probability value of the soft tag corresponding to the text is obtained through the projection above. , The Sigmoid function is used to map real numbers to the interval [0,1] as a probabilistic interpretation; this probability represents the teacher model's semantic judgment of the failure risk in the current scenario.

[0075] It should be understood that the training objective of the student model is to simultaneously approximate both the real labels and the soft labels generated by the teacher model. These are hyperparameters that adjust the weights of the two supervisory signals. The first term uses the binary cross-entropy loss function BCE to enhance the model's classification accuracy for labeled samples. The second term is the mean squared error term, which constrains the student model's behavior on unlabeled or soft-labeled samples, ensuring that its semantic judgments remain consistent with those of the teacher model.

[0076] It should be noted that, to further control gradient fluctuations during backpropagation, a gradient variance penalty term is introduced into the loss function. Its weight coefficient λ is dynamically adjusted with the number of training epochs, where λ represents the gradient penalty strength and epoch represents the current training epoch. In the initial training phase, since the model has not yet converged, λ has a relatively large value, effectively suppressing the severe gradient oscillations driven by noise in the early stages. As training progresses, λ gradually decreases, allowing the model to have greater gradient freedom to optimize complex decision boundaries when approaching the optimal solution. This exponential decay strategy prevents over-smoothing while preserving the model's ability to fine-tune for high-risk samples in later stages. Combined with importance sampling and the dynamic penalty term, the student model can maintain a stable gradient update path even when facing a very low proportion of real failure samples, thereby improving the accuracy of judgment in scenarios with blurred boundaries and multiple overlapping factors.

[0077] In the specific implementation, the preset teacher model adopts a distillation-specific architecture, and its backbone network is the Qwen-VL-Max version. This version has powerful cross-modal perception and language generation capabilities, and can perform deep fusion and semantic parsing of multi-source input data.

[0078] At the input end, the teacher model receives multimodal data from the scene extraction module, including images, LiDAR point clouds, and semantic labels. Through the built-in visual-language alignment mechanism, it generates a natural language description with risk orientation. The output layer sets the upper limit of the generated text to 20 words and adopts a structured guidance strategy to force the generated text to explicitly include information on both the conflict subject and environmental elements, thereby ensuring that the output has task relevance and causal integrity.

[0079] A typical output text is: "Trucks obstruct the view at a rainy intersection, pedestrians cross the road," where "trucks" and "pedestrians" are the subjects of the conflict, and "rainy day" and "intersection" are the environmental elements, providing clear semantic supervision objectives for the student model.

[0080] The teacher model is deployed on a heterogeneous computing platform in the cloud and communicates with the student model on the vehicle via the Protobuf protocol for structured data. Protobuf has the characteristics of compact data and fast parsing speed, making it suitable for frequent exchange of structured semantic data in vehicle-cloud collaborative scenarios. At the beginning of each distillation process, the teacher model receives several batches of scene feature vectors, outputs corresponding risk description text through the semantic generation mechanism, and encodes them into semantic vectors through the mapping module before transmitting them to the vehicle. The preset student model then uses these as semantic soft labels for training.

[0081] In test scenarios such as "electric vehicles crossing the lane under backlight conditions at the tunnel exit", the teacher model can stably generate text descriptions such as "backlight at the exit - electric vehicle - obstruction makes it difficult to identify", thereby helping the student model understand the hidden behavioral risk factors in complex environments. Through this cross-modal, structured semantic compression and collaborative reasoning mechanism, the teacher model not only improves the student model's discrimination ability in the case of sparse labels, but also significantly enhances the generalization robustness to sudden heterogeneous scenarios.

[0082] Accordingly, step S30 specifically includes the following steps: The potential collision severity of different scenarios is obtained based on the risk description text, and a scenario risk value is generated based on the potential collision severity and the failure probability. Calculate the data diversity of the current scenario and the scenario library for autonomous driving test scenarios; The comprehensive value index is determined based on the scenario risk value and the data diversity using the following formula:

[0083] in, For the first The comprehensive value index of each scenario. and These are the weighting coefficients. For the first The scenario risk value for each scenario. For the first Data diversity in each scenario; A sliding time window mechanism is adopted. Every time a preset number of real scenes are added, the comprehensive value index is dynamically sorted once. Based on the sorting results, relevant scenes in the scene library are retained, added, or eliminated to obtain a dynamically updated target scene library. Based on the scenario risk value, the sampling weights are determined using a dynamic gradient stabilizer according to the following formula:

[0084] in, For the first Sampling weights for each scenario, Indicates proportional to, For the first The scenario risk value for each scenario; Failure data from the AUT test of the autonomous driving system under test is collected based on the sampling weights and the target scenario library. The failure data is structured to extract high-risk scenario feature vectors; The high-risk scene feature vector is used as a conditional input to the conditional generative adversarial network CGAN. The generator and discriminator are iteratively optimized under the adversarial mechanism to obtain edge failure scene data. The edge failure scenario data is verified by CARLA simulation. The verified synthetic data and real failure data are mixed and injected into the training set according to a preset mixing ratio to obtain the failure dataset. The parameters of the preset proportional layer of the student model are frozen, and the fully connected layer of the student model is trained according to the failure dataset, so that the update of the student model focuses on fine-tuning the output decision boundary. When the backpropagation gradient magnitude of the student model exceeds a preset magnitude threshold, the gradient vector is forcibly truncated to fill in the boundary regions that were misjudged by the student model, thereby obtaining an optimized target test scenario set.

[0085] It should be noted that the scenario library can be dynamically updated based on the comprehensive value index, and failure data in AUT testing can be collected. An importance sampling distribution can be constructed based on the scenario risk value RV. Failure data can be received and conditional generative adversarial network CGAN can be used to synthesize edge failure scenario data with high-risk scenario features as conditions. By mixing the synthesized data with real failure data, incremental fine-tuning of the student model can be performed.

[0086] Understandably, the dynamic gradient stabilizer executes the following during the training phase: (a) Construct an importance sampling distribution based on the scenario risk value RV, with sampling weights. ; (b) Add a gradient variance penalty term to the loss function .

[0087] It should be understood that the core purpose of hybrid training is to improve the accuracy of student models in distinguishing boundary scenarios by enhancing their ability to model rare failure scenarios.

[0088] In the synthetic data generation stage, the feature vector of the high-risk scene is first concatenated with standard Gaussian noise as the input to the generator. The generator network outputs a fake scene sample with a reasonable structure but high-risk characteristics. The discriminator directly adopts the current student model structure and uses its output failure probability as the basis for distinguishing between real and fake. The generator parameters are updated in reverse through adversarial training so that the generated samples are closer to the distribution of real failure samples. The adversarial goal is to make the student model unable to distinguish between real and synthetic high-risk scenes, thereby improving the model's generalization ability.

[0089] During the incremental fine-tuning phase, to avoid disrupting the existing discriminative ability of the student model in normal scenarios, the parameters of the first 80% of its layers are frozen, and only the last fully connected layer is retained for training. This allows model updates to focus primarily on fine-tuning the output decision boundary. This approach significantly reduces the risk of overfitting in cases of uneven sample distribution. During training, synthetic data and real failure data are mixed in a 3:1 ratio. The real failure data comes from actual failure scenarios reverted from the test records of the autonomous driving system in Module 4. The mixing ratio is designed to enhance the model's sensitivity to failure samples while preventing synthetic data from dominating the training process and causing discriminative drift. The entire fine-tuning process employs a gradient pruning mechanism. When the gradient magnitude exceeds a set threshold of 1.0 during backpropagation, it is forcibly truncated to suppress training instability that may be caused by the high diversity of synthetic data.

[0090] For example, when a set of feature vectors representing "a stationary truck blocking the sidewalk on the right" is input at an urban intersection, the generator can output pseudo-samples such as "a fast-moving object is entering the non-motorized vehicle lane on the left front". After fine-tuning, this type of combined high-risk scenario significantly improves the student model's ability to perceive the joint risks of complex occlusion and sudden moving targets.

[0091] Understandably, the design of the dynamic gradient stabilizer aims to improve the learning efficiency of the student model in high-risk scenarios and suppress gradient oscillations caused by sample scarcity. During training, each batch of samples consists of 64 different scenarios, of which at least 40% are high-risk scenarios, i.e., risk values ​​RV are greater than or equal to 7. This risk-value-based sampling mechanism strengthens the model's attention to failure-prone samples, ensuring that high-quality gradient update signals can still be maintained even in cases of severe sample imbalance. Compared with random sampling, this importance sampling method significantly improves training convergence speed and boundary discrimination ability, and is more stable, especially when dealing with failure types such as sudden occlusion and reverse interleaving.

[0092] In practical implementation, the authenticity and effectiveness of synthetic samples can be verified through a failure scenario simulator to ensure that the data entering the training set has sufficient risk representativeness. In each round of training, a large number of synthetic high-risk scenario features output by the generator are first evaluated by the student model. If a synthetic sample is judged by the student model to have a failure probability of less than 0.3, it means that its surface risk features are not perceived as a threat by the model. At this time, the system automatically triggers the CARLA simulation engine to construct a complete virtual scenario based on the scenario feature vector of the sample and reproduce traffic dynamics in the simulation environment. The scenario construction process includes the restoration of key parameters such as the initial position, speed, environmental conditions, and road topology of traffic participants to ensure that the simulation results have sufficient behavioral and semantic consistency.

[0093] During the simulation, the system continuously monitors the response performance of the AUT in the scenario, focusing on detecting key indicators that affect driving safety, such as path deviation, braking delay, and misjudgment of perception obstruction. If the AUT exhibits a functional failure that meets the definition of Level 3 or above, such as failing to recognize pedestrians crossing and causing braking timeout, or entering a dangerous area due to obstruction and failure to detect lane occupancy, the synthetic sample is determined to have the ability to induce real failures. At this point, the system marks the data as a valid synthetic sample and injects it into the training set along with the scene description text and simulation feedback labels for subsequent fine-tuning training of the student model. Through this screening mechanism, the system can identify samples that pose a real threat to the AUT in a real execution environment from a large batch of fake data, thereby effectively improving the risk coverage and generalization strength of the training samples. For example, if a synthetic scene is described as "a non-motorized vehicle enters the main lane from the right rear in reverse under twilight light," the student model initially judges its FP to be low, but the simulation results show that the behavior caused the emergency avoidance failure. In this case, the scene will be recycled by the system and used to strengthen the model's ability to identify reverse intrusion-type abnormal behaviors. The overall process ensures that the synthetic data is not only sufficient in quantity, but also has reasonable scene causes and decision-making challenges, further promoting the evolution of the student model towards high robustness.

[0094] It should be understood that representative and challenging samples are selected from the constructed atomic scene units to dynamically build a test scene library that can be used to verify the robustness of the autonomous driving system. The module evaluates the value of each atomic scene unit and calculates its comprehensive value index to measure the contribution of the scene to training effectiveness and failure detection capability. The comprehensive value index consists of two core indicators: scene risk value and data diversity. The scene risk value measures the potential conflict level in the scene and is derived from the semantic risk assessment results output by the teacher model. Data diversity reflects the degree of difference between the scene and samples in the existing scene library in terms of spatial behavioral structure, often measured by the entropy or divergence of feature distributions. These two are combined with weighted coefficients to form a unified indicator. , and The weighting coefficient is used to adjust the system's preference for high-risk samples and low-overlapping samples. During each round of updates, a subset of samples with the highest comprehensive value are selected from all candidate scenarios and injected into the test set to ensure that the test coverage includes both high failure probability scenarios and rare samples in the feature space boundary region, thereby achieving a verification effect that balances breadth and depth.

[0095] During testing, the autonomous driving system operates in these scenarios, where its perception, decision-making, planning, and control modules are all affected by multidimensional disturbances and behavioral challenges. The system simultaneously records key indicators such as trajectory deviation, response timeout, and obstacle misjudgment. Once an abnormal system performance is detected, it is judged as a failure, and the scenario along with contextual information is fed back to the training pipeline for retraining the student model and semantic updating of the teacher model, forming a closed-loop improvement. For example, in a scenario of "encountering an oncoming vehicle accelerating and cutting off the right-turn lane while turning left," if the vehicle control module exhibits slow deceleration and insufficient avoidance, this scenario will be marked as a valid failure sample and entered into the data flow loop, continuously strengthening the system's adaptability to complex interactive dynamics.

[0096] Furthermore, the step involves using the high-risk scene feature vector as a conditional input to a conditional generative adversarial network (CGAN). Based on the iterative optimization of the generator and discriminator under the adversarial mechanism, edge failure scene data is obtained. This specifically includes the following steps: The high-risk scene feature vectors are used as conditional inputs to a Conditional Generative Adversarial Network (CGAN). The generator and discriminator iteratively optimize the loss function under the adversarial mechanism using the following formula to obtain edge failure scene data:

[0097] in, For generator, For discriminator, For real samples, For noise vectors, For high-risk scene feature vectors, Output the synthesized sample.

[0098] It should be noted that the core task of the hybrid training module is to incrementally fine-tune the student model by combining synthesized and real failure data, thereby enhancing its discrimination ability and failure identification stability in edge scenarios. First, this module receives failure samples returned by the test scenario set generation module. After structured processing, high-dimensional scene feature vectors are extracted from these data, including the positions of traffic participants, speed covariance matrices, road structure codes, and environmental attribute parameters. Subsequently, these high-risk scene features are fed as conditional inputs into the Conditional Generative Adversarial Network (CGAN). The generator and discriminator iteratively optimize under the adversarial mechanism, aiming to generate samples with failure tendency but whose distribution is not fully covered, thus filling in the boundary regions in the feature space that are easily misjudged by the student model. The generation process of CGAN is based on the conditional input, guiding the generator to synthesize samples within a specific risk semantic structure; its loss function is shown above.

[0099] Understandably, through this mechanism, the generator continuously optimizes its output structure, making it impossible for the discriminator to distinguish between real and synthetic data, thus achieving data diversity and semantic consistency. After synthetic samples are generated, they are mixed with real failure data in proportion and injected into the training set to form a representative and challenging failure dataset for incremental fine-tuning of the student model. During the fine-tuning phase, the parameters of the entire model are not updated; only limited parameters of the final decision layer are optimized to preserve the model's original generalization ability in non-failure scenarios and improve its adaptability to boundary samples. This fine-tuning process can effectively enhance the model's discrimination accuracy in scenarios such as occlusion-type sudden behavior and reverse abnormal movement, and eliminate prediction blind spots in sparse feature distribution areas round by round. For example, for a synthetic scenario of "non-motorized vehicles crossing a T-junction at high speed in the rain," the original model outputs an insufficient failure probability. After training with synthetic samples, the student model's sensitivity to this type of semantic structure is significantly improved, enabling it to actively identify it as a high-risk behavior and output a failure probability close to 1, thereby achieving expanded coverage and accurate perception of rare and high-risk events in reality.

[0100] In the specific implementation, see Figure 4 , Figure 4 This is a schematic diagram of the overall architecture of the autonomous driving test scenario set optimization system of the present invention, as shown below. Figure 4 As shown, the system includes a real data acquisition module, a scene extraction and reconstruction module, a scene value quantification module, a test scene set generation module, and a hybrid training module; among them, the scene value quantification module is as follows: Figure 5 As shown, Figure 5 This is a schematic diagram of the dual-model architecture of the scene value quantification module in the autonomous driving test scene set optimization system of the present invention; the scene value quantification module includes a dynamic gradient stabilizer, a teacher model, a student model, and a semantic-probability mapper; the training process of the hybrid training module is as follows: Figure 6 As shown, Figure 6 This is a schematic diagram of the workflow of the hybrid training module in the autonomous driving test scenario set optimization system of the present invention. After receiving failure data, CGAN synthesizes edge failure scenario data. The synthesized data is mixed with real failure data, and the student model is incrementally fine-tuned. CARLA simulation verification is performed. If FP < 0.3, it is marked as a valid synthesized sample; otherwise, the process ends. Figure 7 This is a schematic diagram illustrating the working principle of the dynamic gradient stabilizer in the autonomous driving test scenario set optimization system of the present invention, as shown below. Figure 7 As shown, an importance sampling distribution is constructed, with each batch of training containing ≥40% high-risk scenarios. A gradient variance penalty term is added to the loss function, and λ decays exponentially with the training rounds. Stable gradient updates are achieved through the above operations. Taking an autonomous driving test project in a first-tier city as an example, the practical application of this embodiment improves the testing effectiveness and safety of the autonomous driving system in the city's complex traffic environment: In practical implementation, the real data acquisition module can be deployed on 50 mass-produced autonomous vehicles. These vehicles are distributed in different areas of the city, covering main roads, secondary roads, expressways, and some complex side roads. The vehicles are equipped with multiple high-definition cameras, 128-line LiDAR, millimeter-wave radar, and high-precision GPS / IMU combined sensors. On the on-board edge device, a keyframe extraction algorithm is used to compress the video data volume to 1 / 10 of the original, effectively reducing the data transmission pressure. The LiDAR point cloud is downsampled using voxel grids to reduce the data dimensionality, and differential privacy technology is used to add noise to the GPS coordinates to protect user privacy. The collected data is uploaded to the cloud server in real time via 5G vehicle-to-everything (V2X) network, with an average of about 50GB of data uploaded per vehicle per day.

[0101] In its implementation, the system fuses multi-source sensor data using a spatiotemporal alignment algorithm. Based on a pre-defined scene semantic labeling system, it can quickly and accurately extract discretized atomic scene units. For example, in one data processing operation, the system successfully extracted a complex scene containing "intersection, vehicles turning left, pedestrians running red lights, and oncoming vehicles going straight" and reconstructed it into a simulation test scene file in OpenSCENARIO format. In this scene, the road structure label is "intersection, four lanes, high traffic sign density"; the participant behavior label is "pedestrians running red lights"; and the interaction relationship label is "conflict between vehicles turning left and oncoming vehicles going straight, and conflict between vehicles turning left and pedestrians crossing".

[0102] In the specific implementation, the teacher model can use the Qwen-VL-Max version of the visual language large model, deployed on a cloud server. When receiving the aforementioned complex scene file, the teacher model, through its powerful reasoning ability, outputs a risk description text: "At the intersection, a pedestrian is running a red light, oncoming vehicles are going straight, and there is a risk of collision if a vehicle turns left." The student model is a lightweight adversarial neural network that receives the scene feature vector and the risk description output by the teacher model. Through a semantic-probability mapper, the risk description text is first encoded into a semantic vector using a BERT model. Then through a learnable mapping matrix The semantic vector is linearly projected into probability values, i.e. The student model, together with the real labels, calculates the failure probability FP.

[0103] In its implementation, the dynamic gradient stabilizer plays a crucial role during the training phase, constructing an importance sampling distribution based on the scene risk value RV, and sampling weights. Each training batch contains 64 scenarios, with high-risk scenarios (RV>7) accounting for 45%; a gradient variance penalty term is added to the loss function. Gradient variance penalty coefficient The value decays exponentially with each training epoch. The initial value is 0.1, and in this training iteration, after 50 epochs... The value was reduced to 0.06. These operations effectively stabilized the model training process and improved the accuracy of scene value quantification.

[0104] In its implementation, the test scenario set generation module uses a comprehensive value index. , The value is 0.6. The value is set to 0.4; the scenario library is dynamically updated; a sliding time window mechanism is adopted, and a VI sorting update is triggered every 1000 new real scenarios; in this test, after a period of data accumulation, the system added 1000 scenarios. After calculating and sorting the VI of these scenarios, the top 500 scenarios with the highest VI in the original scenario library are retained, and 50 new high-VI scenarios are added, while the 50 scenarios with the lowest VI are eliminated; at the same time, failure data of the tested autonomous driving system (AUT) is collected during the test, including the types of functional violations such as emergency braking mis-triggered and lane keeping deviation when the AUT faces pedestrian red light crossing scenarios, as well as the corresponding violation severity level and scenario feature fingerprint.

[0105] In the specific implementation, the hybrid training module receives failure data from the test scenario set generation module. Using Conditional Generative Adversarial Network (CGAN), edge failure scenario data is synthesized based on high-risk scene features. The generator input consists of a high-risk scene feature vector and a Gaussian noise vector, while the discriminator is a student model. The generator weights are updated using adversarial loss. For example, for the high-risk scenario of pedestrians running red lights, the generator synthesizes multiple scenarios of pedestrians running red lights under different times and weather conditions. The student model is incrementally fine-tuned, freezing the weights of the first 80% of the layers and only fine-tuning the last fully connected layer. The training data ratio is synthetic data:real failure data = 3:1. Gradient clipping (threshold = 1.0) is used to constrain the backpropagation amplitude. In addition, the hybrid training module also includes a failure scenario simulator. When the synthetic data is judged by the student model to have FP < 0.3, the CARLA simulation engine is automatically triggered to reproduce the scenario. In one simulation, the synthesized "pedestrian running red lights at night in rainy weather" scenario triggered the L4 level failure of the AUT. This data was marked as a valid synthetic sample and injected into the training set. Through a series of operations in the hybrid training module, the performance of the student model is significantly improved, and its ability to identify and process complex scenarios is enhanced.

[0106] Actual operational results: After a period of operation, the system has achieved remarkable results in the city's autonomous driving tests; the safety and reliability of the autonomous driving system in complex traffic scenarios have been significantly improved, the misjudgment rate has been reduced by 30%, the accuracy of emergency braking has been increased by 25%, and the stability of lane keeping function has been improved by 20%; at the same time, through the analysis and optimization of failure scenarios, the system can better cope with various potential risks, providing strong support for the application of autonomous driving technology in complex urban traffic environments.

[0107] Taking a highway autonomous driving test scenario as an example, this embodiment demonstrates its adaptability and effectiveness in a specific scenario, optimizes the testing of the autonomous driving system in a highway scenario, and improves the safety and stability of the system under high-speed driving conditions: System Deployment and Data Acquisition: In the highway test project, 30 autonomous vehicles equipped with the system's real data acquisition module were deployed. These vehicles were equipped with high-resolution forward-facing cameras, medium-to-long-range millimeter-wave radar, and high-precision GPS / IMU sensors. On the vehicle-mounted edge devices, keyframe extraction algorithms were used to compress video data, achieving a compression ratio of 1:8. Voxel grid downsampling was performed on the LiDAR point cloud, and differential privacy technology was used to process GPS coordinates. Due to the relatively simple driving scenario on highways and the relatively smaller data volume compared to urban areas, each vehicle uploaded approximately 30GB of data per day. The collected data was uploaded to the cloud server via a 4G / 5G hybrid network to ensure data transmission stability. Scene Extraction and Reconstruction: The scene extraction and reconstruction module processed the received highway driving data. Based on a scene semantic tagging system, this study focuses on highway scenarios, paying particular attention to road structure (number of lanes, radius of curvature, gradient), traffic participant behaviors such as lane changing, overtaking, and emergency braking, as well as interactive relationships such as following and lane merging conflicts. A scenario of "a three-lane highway, a vehicle traveling at 120 km / h in the leftmost lane, a vehicle on the right suddenly accelerating to overtake and abruptly cutting in" is extracted and reconstructed into an OpenSCENARIO format file. The road structure tag is "three-lane highway, large radius of curvature, small gradient"; the participant behavior tag is "the vehicle on the right suddenly accelerating to overtake and abruptly cutting in"; and the interactive relationship tag is "lane merging conflict between following and overtaking vehicles".

[0108] Scene Value Quantification: The dual-model architecture of the scene value quantification module plays a role in highway scenarios. The teacher model (Qwen-VL-Max) outputs a risk description based on the scene file: "On the highway, a vehicle on the right suddenly overtakes and cuts in abruptly, posing a rear-end collision risk to following vehicles." The student model receives the scene feature vector and the teacher model's output, converts the risk description into FP soft labels through a semantic-probability mapper, and calculates the failure probability FP by combining the real labels. During training, the dynamic gradient stabilizer operates according to preset rules, sampling 64 scenes per batch based on importance, with high-risk scenes (RV>7) accounting for 40%, and the gradient variance penalty coefficient decaying exponentially from the initial value of 0.1. After multiple rounds of training, the model's risk assessment of highway scenarios becomes more accurate, effectively identifying potentially dangerous scenes.

[0109] Test Scenario Set Generation: The test scenario set generation module dynamically updates highway scenarios based on the Comprehensive Value Index (VI). A sliding time window mechanism is adopted, triggering a VI sorting update every 1000 new real scenarios. In actual testing, when the number of new scenarios reaches a threshold, the scenario library is updated, retaining the top 500 scenarios with high VI, adding 50 new high VI scenarios, and eliminating 50 low VI scenarios. At the same time, failure data of the AUT in highway testing is collected, such as the response delay of the Automatic Emergency Braking (AEB) system in emergency vehicle entry scenarios and abnormal adjustments of the Adaptive Cruise Control (ACC) function.

[0110] Hybrid Training Module: The hybrid training module addresses highway failure scenarios. It utilizes CGAN to synthesize edge failure scenarios relevant to highway conditions, simulating emergency situations for vehicles on highways under different weather conditions. The generator takes high-risk scenario feature vectors and Gaussian noise vectors as input, while the discriminator (student model) updates the generator weights using adversarial loss. Incremental fine-tuning is performed on the student model, freezing the weights of the first 80% of layers and only fine-tuning the last fully connected layer. The training data ratio is synthetic data:real failure data = 3:1, and gradient clipping (threshold = 1.0) is used. When the failure scenario simulator detects synthetic data with FP < 0.3, it triggers the CARLA simulation engine to reproduce the scenario. If the AUT exhibits L3+ level failure, the data is marked as a valid synthetic sample and injected into the training set. Through these operations, the student model's understanding and ability to respond to highway scenarios are improved, enabling it to better adapt to the complex and ever-changing driving conditions on highways.

[0111] Comparison and Advantages: Compared to the complex urban traffic scenario in Example 1, this example focuses more on the unique characteristics and risk factors of highways. In terms of data acquisition, the data volume and transmission pressure are relatively small due to the simpler scenario, but higher requirements are placed on the accuracy and reliability of sensors, especially the ability to detect distant targets. In scenario value quantification, the teacher model accurately captures key risk points in highway scenarios, providing more targeted semantic information to the student model, making the student model more accurate in calculating failure probabilities. In the test scenario set generation and hybrid training modules, the update strategy and training data generation method for highway scenarios better optimize the highway scenario library, improving the testing effect and safety of the autonomous driving system on highways. Through the comparison of the two examples, it can be seen that this system has good adaptability and effectiveness in different application scenarios, and can be optimized and improved according to scenario characteristics, providing strong support for the comprehensive testing and optimization of autonomous driving systems.

[0112] It should be noted that this embodiment has the following beneficial effects: First, it effectively solves the gradient estimation variance explosion problem caused by the "curse of sparsity" and improves the stability of model training: This is because the solution introduces a dynamic importance sampling strategy at the data level, constructs a non-uniform sampling distribution based on safety risk values, and makes the proportion of high-risk scenarios in the training batch ≥40%, so that gradient estimation focuses on high-value samples. At the same time, it adds an L2 norm penalty term to the loss function to suppress the oscillation amplitude of backpropagation.

[0113] Second, it enhances the ability to identify long-tail rare failure scenarios and improves the accuracy of failure probability prediction: This solution constructs a two-stage distillation architecture at the model level, uses a large visual language model to generate scenario risk description text, and transforms it into smooth probability labels through a semantic-probability mapper. A lightweight adversarial neural network, as a student model, learns both the real failure labels and the soft labels of the teacher model. By leveraging the teacher model's generalization ability for long-tail events, it provides low-variance gradient signals, avoiding the problem of the model ignoring rare samples due to the scarcity of samples.

[0114] Third, it actively reduces the sparsity of data distribution, expands the coverage of training data, and improves the model's accuracy in distinguishing boundary scenarios: At the system level, this solution deploys generative-adversarial hybrid training, using high-risk scenario features as conditions to drive the conditional generative adversarial network to synthesize edge failure scenarios, and then injects them into the training set after verifying their effectiveness through CARLA simulation. The student model alternately optimizes the discrimination ability of real data and synthetic data in incremental fine-tuning, which solves the problem of extremely small sample size of real failure scenarios in existing technologies.

[0115] Fourth, it improves the accuracy and comprehensiveness of scenario value quantification, making the generation of test scenario sets more targeted and effective: The scenario value quantification module of this solution adopts a parallel dual-model architecture. The risk description text output by the teacher model contains conflict subjects and environmental elements, providing richer semantic supervision information for the student model. Combined with the optimization of the dynamic gradient stabilizer, the calculated scenario risk value and comprehensive value index can better reflect the real test value of the scenario, thus enabling more accurate screening of high-value scenarios when generating test scenario sets.

[0116] In practical implementation, decision-making models for autonomous vehicles in different scenarios can be constructed, and reinforcement learning algorithms can be used to allow the vehicle to conduct extensive trials and learning in a virtual environment. During the learning process, the vehicle continuously adjusts its decision-making strategy based on reward signals from the environment, aiming to achieve safe and efficient driving in various scenarios. The advantage of this method is that it can make full use of the virtual environment for rapid iteration and optimization, without requiring a large amount of real data collection, thus reducing costs and time consumption. It allows the autonomous driving system to continuously try new decisions in the virtual environment and explore more possible ways to deal with scenarios, thereby improving the system's adaptability and robustness. However, this method relies on accurate environmental modeling and reward function design. If the model and function design are unreasonable, the learning results may deviate from reality and fail to effectively guide autonomous driving tests in real scenarios. Moreover, the training process of reinforcement learning algorithms usually requires a large amount of computing resources and time, placing high demands on hardware equipment.

[0117] This embodiment, through the above-described scheme, uses a pre-set teacher model to parse the scene semantics of the simulation scene file using a Visual Language Model (VLM) and outputs risk description text. A pre-set student model receives the scene feature vector and the risk description text, and uses a semantic-probability mapper to convert the risk description text into failure probability (FP) soft labels. The failure probability is then calculated by combining the FP soft labels with the real labels. This approach can generate scene risk description text using a large visual language model, convert it into smooth probability labels using a semantic-probability mapper, and use a lightweight adversarial neural network as the student model to simultaneously learn the real failure labels and the soft labels from the teacher model. By leveraging the teacher model's generalization ability for long-tail events, it provides low-variance gradient signals, avoiding the problem of the model ignoring rare samples due to scarce samples. This enhances the ability to identify long-tail rare failure scenarios, improves the accuracy of failure probability prediction, and increases the speed and efficiency of optimizing autonomous driving test scene sets.

[0118] Accordingly, the present invention further provides an optimization device for autonomous driving test scenario sets.

[0119] Reference Figure 8 , Figure 8This is a functional block diagram of the first embodiment of the autonomous driving test scenario set optimization device of the present invention.

[0120] In a first embodiment of the autonomous driving test scenario set optimization device of the present invention, the autonomous driving test scenario set optimization device includes: The preprocessing module 10 is used to collect raw driving data, preprocess the raw driving data to obtain preprocessed data, and generate a simulation scene file in OpenSCENARIO format based on the preprocessed data.

[0121] The file parsing module 20 is used to parse the simulation scenario file through a preset teacher model, output risk description text, receive the scenario feature vector and the risk description text through a preset student model, and output the failure probability.

[0122] The scenario library update module 30 is used to determine the comprehensive value index based on the failure probability, dynamically update the scenario library of autonomous driving test scenarios based on the comprehensive value index, collect failure data in the AUT test of the autonomous driving system under test, incrementally fine-tune the student model based on the failure data, fill in the boundary areas misjudged by the student model, and obtain the optimized target test scenario set.

[0123] The steps for implementing each functional module of the autonomous driving test scenario set optimization device can be referred to in the various embodiments of the autonomous driving test scenario set optimization method of the present invention, and will not be repeated here.

[0124] Furthermore, this embodiment of the invention also proposes a storage medium storing an autonomous driving test scenario set optimization program. When the autonomous driving test scenario set optimization program is executed by a processor, it implements the operations described in the above-described autonomous driving test scenario set optimization method embodiment.

[0125] Those skilled in the art will understand that all or part of the steps in the methods described above can be implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium is a computer-readable storage medium, including: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program code.

[0126] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0127] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0128] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for optimizing an autonomous driving test scenario set, characterized in that, The optimization method for the autonomous driving test scenario set includes: Collect raw driving data, preprocess the raw driving data to obtain preprocessed data, and generate a simulation scene file in OpenSCENARIO format based on the preprocessed data; The simulation scenario file is parsed by a preset teacher model, and the risk description text is output. The scenario feature vector and the risk description text are received by a preset student model, and the failure probability is output. The comprehensive value index is determined based on the failure probability. The scenario library for autonomous driving test scenarios is dynamically updated based on the comprehensive value index. Failure data in the AUT test of the autonomous driving system under test is collected. The student model is incrementally fine-tuned based on the failure data to fill in the boundary areas misjudged by the student model and obtain the optimized target test scenario set.

2. The method for optimizing autonomous driving test scenario sets as described in claim 1, characterized in that, The process involves collecting raw driving data, preprocessing the raw driving data to obtain preprocessed data, and generating a simulation scene file in OpenSCENARIO format based on the preprocessed data, including: The original driving data of the current autonomous vehicle is collected through the real data acquisition module of the current autonomous vehicle; The raw driving data is transformed into a structured feature stream, retaining key risk factors, and noise is added to sensitive information in the raw driving data to ensure that the preprocessed data meets the differential privacy definition. in, For the perturbation of the first One characteristic, The original value, For function Sensitivity Budget for privacy; The preprocessed data is spatiotemporally aligned to generate atomic scene units, which are then reconstructed into simulation scene files in OpenSCENARIO format.

3. The autonomous driving test scenario set optimization method as described in claim 2, characterized in that, The process of spatiotemporally aligning the preprocessed data to generate atomic scene units, and reconstructing the atomic scene units into simulation scene files in OpenSCENARIO format, includes: The system obtains the timestamps provided by the Global Positioning System (GPS) and the Inertial Measurement Unit (IMU). Based on the timestamps, it performs joint spatial reprojection on the object recognition results in each frame of the preprocessed data to determine the position, orientation, and speed information of each type of traffic participant in the same coordinate system, forming a frame-level entity state description. The preprocessed data is divided into scene segments in the time dimension, and time windows with scene variability that meet the requirements are extracted as candidate keyframe intervals. Entity behavior patterns are determined based on frame-level entity state descriptions, and similar fragments are clustered based on candidate keyframe intervals. Based on the entity behavior pattern and the environment configuration, cluster similar fragments to generate atomic scene units, convert the atomic scene units into a data format conforming to the OpenSCENARIO standard, and encapsulate them into simulation scene files that can be reproduced by the simulation platform.

4. The method for optimizing autonomous driving test scenario sets as described in claim 1, characterized in that, The process of parsing the simulation scenario file using a preset teacher model and outputting risk description text, and receiving the scenario feature vector and the risk description text using a preset student model to output the failure probability, includes: The simulation scenario file is parsed using a Visual Language Model (VLM) based on a pre-defined teacher model, and the risk description text is output. The system receives scene feature vectors and risk description text through a preset student model, and converts the risk description text into failure probability (FP) soft labels through a semantic-probability mapper. The failure probability is then calculated by combining the FP soft labels with the real labels.

5. The method for optimizing autonomous driving test scenario sets as described in claim 4, characterized in that, The process of receiving scene feature vectors and risk description text through a preset student model, converting the risk description text into failure probability (FP) soft labels through a semantic-probability mapper, and calculating the failure probability by combining the FP soft labels with the real labels includes: The scenario feature vector and the risk description text are received through a preset student model; The risk description text is encoded using the pre-trained text encoding model BERT, and the output is a semantic vector that captures the semantic information and underlying logical structure of the risk description text. A linear mapping matrix is ​​constructed based on the semantic vector using the following formula: in, It is a linear mapping matrix. It is a matrix space of 768 rows and 1 column consisting of real numbers; Based on the linear mapping matrix, the FP soft tag corresponding to the risk description text is obtained by projection using the following formula; in, This represents the probability value of soft tag failure. For the Sigmoid function, , A semantic vector with dimension 768. It is a linear mapping matrix. , For the set of real numbers, For bias terms; The true labels are obtained from the original driving data, and the binary cross-entropy loss function of the preset student model is determined based on the true labels and the FP soft labels: in, For binary cross-entropy loss, For real labels, The probability value predicted by the preset student model; The total loss function is obtained from the binary cross-entropy loss function using the following formula: in, For the total loss function, It is a hyperparameter that adjusts the weights of the two monitoring signals. For real labels, The output prediction of the preset student model; The gradient variance penalty term is added to the total loss function by the following formula: in, This is the gradient variance penalty coefficient. , For the current training round, For the loss function Find the gradient. express Squaring norms; By constraining the preset student model with the total loss function, the failure probability of the optimized model is obtained.

6. The method for optimizing autonomous driving test scenario sets as described in claim 1, characterized in that, The process involves determining a comprehensive value index based on the failure probability, dynamically updating the scenario library for autonomous driving test scenarios based on the comprehensive value index, collecting failure data from the AUT (Autonomous Under Test) testing of the autonomous driving system, incrementally fine-tuning the student model based on the failure data, supplementing boundary regions misjudged by the student model, and obtaining an optimized target test scenario set, including: The potential collision severity of different scenarios is obtained based on the risk description text, and a scenario risk value is generated based on the potential collision severity and the failure probability. Calculate the data diversity of the current scenario and the scenario library for autonomous driving test scenarios; The comprehensive value index is determined based on the scenario risk value and the data diversity using the following formula: in, For the first The comprehensive value index of each scenario. and These are the weighting coefficients. For the first The scenario risk value for each scenario. For the first Data diversity in each scenario; A sliding time window mechanism is adopted. Every time a preset number of real scenes are added, the comprehensive value index is dynamically sorted once. Based on the sorting results, relevant scenes in the scene library are retained, added, or eliminated to obtain a dynamically updated target scene library. Based on the scenario risk value, the sampling weights are determined using a dynamic gradient stabilizer according to the following formula: in, For the first Sampling weights for each scenario, Indicates proportional to, For the first The scenario risk value for each scenario; Failure data from the AUT test of the autonomous driving system under test is collected based on the sampling weights and the target scenario library. The failure data is structured to extract high-risk scenario feature vectors; The high-risk scene feature vector is used as a conditional input to the conditional generative adversarial network CGAN. The generator and discriminator are iteratively optimized under the adversarial mechanism to obtain edge failure scene data. The edge failure scenario data is verified by CARLA simulation. The verified synthetic data and real failure data are mixed and injected into the training set according to a preset mixing ratio to obtain the failure dataset. The parameters of the preset proportional layer of the student model are frozen, and the fully connected layer of the student model is trained according to the failure dataset, so that the update of the student model focuses on fine-tuning the output decision boundary. When the backpropagation gradient magnitude of the student model exceeds a preset magnitude threshold, the gradient vector is forcibly truncated to fill in the boundary regions that were misjudged by the student model, thereby obtaining an optimized target test scenario set.

7. The method for optimizing autonomous driving test scenario sets as described in claim 6, characterized in that, The step of inputting the high-risk scene feature vector as a conditional input to the Conditional Generative Adversarial Network (CGAN), and obtaining edge failure scene data through iterative optimization of the generator and discriminator under the adversarial mechanism, includes: The high-risk scene feature vectors are used as conditional inputs to a Conditional Generative Adversarial Network (CGAN). The generator and discriminator iteratively optimize the loss function under the adversarial mechanism using the following formula to obtain edge failure scene data: in, For generator, For discriminator, For real samples, For noise vectors, For high-risk scene feature vectors, Output the synthesized sample.

8. An optimization device for autonomous driving test scenario sets, characterized in that, The autonomous driving test scenario set optimization device includes: The preprocessing module is used to collect raw driving data, preprocess the raw driving data to obtain preprocessed data, and generate simulation scene files in OpenSCENARIO format based on the preprocessed data. The file parsing module is used to parse the simulation scenario file through a preset teacher model, output risk description text, receive the scenario feature vector and the risk description text through a preset student model, and output the failure probability. The scenario library update module is used to determine the comprehensive value index based on the failure probability, dynamically update the scenario library of autonomous driving test scenarios based on the comprehensive value index, collect failure data in the AUT test of the autonomous driving system under test, incrementally fine-tune the student model based on the failure data, fill in the boundary areas misjudged by the student model, and obtain the optimized target test scenario set.

9. An optimization device for autonomous driving test scenario sets, characterized in that, The autonomous driving test scenario set optimization device includes: a memory, a processor, and an autonomous driving test scenario set optimization program stored in the memory and executable on the processor, wherein the autonomous driving test scenario set optimization program is configured to implement the steps of the autonomous driving test scenario set optimization method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores an autonomous driving test scenario set optimization program, which, when executed by a processor, implements the steps of the autonomous driving test scenario set optimization method as described in any one of claims 1 to 7.

Images