An autonomous driving safety test system and method

Abstract

The application provides an automatic driving safety test system and method, and relates to the technical field of automatic driving.The system comprises a conventional traffic scene generation subsystem, which is used for generating a conventional driving scene dataset in a simulation platform; a safety key scene generation subsystem, which is used for generating a safety key scene by using an automatic driving scene generation network SAGA based on an adversarial disturbance according to the conventional driving scene dataset; and a safety key scene clustering analysis subsystem, which is used for obtaining a scene analysis result by using a progressive unsupervised clustering method to dynamically discover new categories and optimize a clustering structure according to the generated safety key scene.The application solves the problems of insufficient test scene coverage, weak adversarial test capability and low key scene analysis efficiency in the safety verification of an existing automatic driving system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of automatic driving, and particularly relates to an automatic driving safety test system and method. BACKGROUND

[0002] With the rapid development of automatic driving technology, it shows great potential in improving traffic efficiency and reducing traffic accident rate. However, the robustness and safety of the automatic driving system in the actual complex traffic environment still have many challenges, and safety problems have become an important factor restricting its large-scale application. Therefore, it is a key task for research and industry to comprehensively and efficiently test and verify the safety of the automatic driving system. At present, the test methods of the automatic driving system mainly include actual road test and simulation platform test, the former has problems such as high cost, long cycle and insufficient safety, and the latter depends on the simulation platform and its scene generation ability. Although there are related researches on the architecture of the automatic driving system, the generation of key scenes and the test analysis method, there are still many deficiencies in the aspects of automatic scene generation, diversity test and dynamic risk analysis. The related technical background is described as follows.

[0003] (1) Architecture of automatic driving system and existing challenges

[0004] Most of the current mainstream automatic driving systems adopt a modular design strategy. The modular system divides functions such as perception, prediction and planning into independent modules for development and integration, which has good interpretability and module independent optimization ability. However, due to the weak coupling between modules, the overall optimization of the system is insufficient, and problems such as information redundancy and decision efficiency decline may occur in the interaction process of the modules. In order to make up for the shortcomings of the modular architecture, end-to-end automatic driving systems have gradually emerged in recent years. The end-to-end system directly maps the sensor input to the control output through a single neural network, avoiding information loss between modules and improving the overall optimization ability of the system. End-to-end automatic driving systems mainly adopt two methods of imitation learning (IL) and reinforcement learning (RL). IL can achieve efficient and stable automatic driving strategy training by learning expert behavior demonstration, but its generalization ability is limited, especially in safety-critical scenarios, where input disturbances can easily cause system failure. RL has the ability to actively explore the environment and optimize complex strategies, but the training process requires a huge amount of data, usually relying on simulation environments for high-risk decision training. RL methods are sensitive to reward function design, and existing systems often use simple objective functions, which can easily lead to risky behavior. In addition, RL has difficulty in optimizing in high-dimensional environments, and has problems such as insufficient gradient and sparse feedback.

[0005] In summary, the robustness and stability of the existing automatic driving system in safety-critical scenarios still need to be improved. Therefore, designing an automatic driving system test and analysis method for complex scenarios has become a problem that needs to be solved at present.

[0006] (2) Safety-critical scenario generation technology

[0007] Existing methods for generating safety-critical scenarios can be broadly categorized into three types: data-driven methods, which use large-scale traffic datasets for scenario sampling or distribution learning to generate high-risk traffic scenarios. While these methods improve scenario diversity, they are limited by the scarcity of data sources, making it difficult to cover long-tail scenarios. Furthermore, data-driven generation lacks flexibility and struggles to dynamically adapt to different testing needs. Knowledge-based scenario generation methods utilize traffic rules and expert knowledge to generate safety-critical test scenarios through rule reasoning or model constraints. Although they can ensure the logical rationality and safety of the generated scenarios, they rely excessively on manual design, resulting in insufficient innovation and difficulty in capturing potential system risks. Adversarial generation methods generate challenging test scenarios by simulating adversarial behavior. While adversarial generation methods increase scenario complexity and challenge, they suffer from the following problems: high computational complexity for large-scale optimization, easily leading to the curse of dimensionality; difficulty in guaranteeing the realism of generated scenarios, relying on strong rule constraints to improve rationality; and reliance on adversarial model design, limiting adaptability and generalization capabilities. Existing generation methods have not yet formed an integrated solution for automated, dynamic, and diversified scenario generation and system robustness assessment.

[0008] (3) Current Status of Clustering and Analysis Technology for Safety-Critical Scenarios

[0009] The generation of large-scale autonomous driving test scenarios has led to a massive amount of test data, necessitating effective scenario clustering and analysis techniques to uncover risk patterns and locate system vulnerabilities. Currently commonly used methods include:

[0010] Traditional clustering algorithms, such as K-means and DBSCAN, have been applied to the classification and analysis of autonomous driving test data. These methods rely on preset parameters (such as the number of categories and density thresholds), and suffer from problems such as unstable clustering results and blurred category boundaries in high-dimensional complex data and dynamic environments.

[0011] Spectral clustering and anchor graph clustering algorithms: Spectral clustering constructs a graph structure using a similarity matrix and performs feature decomposition to achieve cluster analysis, making it suitable for classifying complex data scenarios. However, this method has high computational complexity, especially in large-scale data environments where its Laplacian matrix decomposition consumes enormous computational resources. Anchor graph clustering simplifies data similarity calculation by selecting representative anchor points, reducing the computational complexity of spectral clustering, but it faces challenges such as imbalanced anchor point selection and insufficient stability of the cluster structure.

[0012] Progressive unsupervised clustering method: the progressive clustering method combines anchor optimization and label propagation algorithm to realize dynamic class discovery and adaptive clustering. This method has the ability to handle dynamically growing data and new class discovery by gradually optimizing and adjusting the class boundary, improving the adaptability and scalability of the clustering algorithm. However, the existing progressive clustering method still has problems such as insufficient computational efficiency, weak robustness, etc. in large-scale automatic driving test environment, which makes it difficult to realize real-time and efficient dynamic risk analysis.

[0013] Main problems existing in the prior art: Although existing research has made some progress in automatic driving system architecture, scene generation and test data analysis, there are still the following deficiencies:

[0014] The scene generation method has low automation, high cost, and is difficult to cover complex and extreme safety critical scenes, and has insufficient generation efficiency and authenticity; the test data analysis method has high computational complexity, insufficient dynamic class recognition and risk analysis capability, and is difficult to realize real-time analysis and risk warning of large-scale test data; there is a lack of a unified automatic driving safety test platform, which cannot realize the closed-loop integration of scene generation, system vulnerability mining and risk analysis, affecting the automation and efficiency of the test process. Summary of the application

[0015] In view of the above deficiencies in the prior art, the automatic driving safety test system and method provided by the present application solve the problems of insufficient test scene coverage, weak adversarial testing capability, and low key scene analysis efficiency in existing automatic driving system safety verification.

[0016] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows: an automatic driving safety test system, comprising:

[0017] A conventional traffic scene generation subsystem for generating a conventional driving scene dataset in a simulation platform;

[0018] A safety critical scene generation subsystem for generating safety critical scenes using an adversarial perturbation-based automatic driving scene generation network SAGA based on the conventional driving scene dataset;

[0019] A safety critical scene clustering analysis subsystem for dynamically discovering new classes and optimizing clustering structure by introducing a progressive unsupervised clustering method based on the generated safety critical scenes to obtain scene analysis results and complete automatic driving safety testing.

[0020] Further, the conventional traffic scene generation subsystem comprises:

[0021] A road network topology modeling module for extracting road topology information by analyzing the road network in the simulation platform and modeling the road network topology based on the road topology information.

[0022] a path planning and optimization module, configured to generate a smooth driving path from the starting point to the target point based on the constructed road network topology by using a path planning algorithm, and to optimize the smooth driving path by using an interpolation method;

[0023] a multi-modal data acquisition module, configured to acquire multiple types of scene data based on the optimized smooth driving path by using multi-modal sensors, and to construct a conventional driving scene dataset.

[0024] Further, the safety-critical scene generation subsystem comprises:

[0025] a conventional training module, configured to train an automatic driving agent by using the conventional driving scene dataset, and to form a benchmark model with driving decision-making and environmental perception capabilities;

[0026] an adversarial scene generation module, configured to generate an adversarial scene for a victim model in an initial scene by using an attacker's behavior trajectory based on a gradient-based adversarial perturbation strategy according to a training result, wherein an optimal perturbation sequence is found by minimizing an objective function to generate a safety-critical scene;

[0027] a victim model module, configured to provide real-time feedback in the perturbation generation process to optimize the adversarial perturbation strategy, wherein the victim model is an automatic driving decision-making system that is an attack target in the adversarial scene generation process.

[0028] Further, the training of the automatic driving agent by using the conventional driving scene dataset specifically comprises:

[0029] observed data O 𝑡 is combined with target trajectory point G to train an automatic driving strategy network π 𝜃 , and a benchmark model with driving decision-making and environmental perception capabilities is formed.

[0030] Further, the expression of the optimal perturbation sequence is as follows:

[0031]

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039] in, This represents the optimal perturbation sequence. Represents the set of all action sequences of adversarial agents. This represents minimizing the objective function. Indicates the first A driving strategy based on an adversarial agent's action sequence. This refers to the autonomous driving agent associated with the victim model. Indicates that, except for External transportation intelligent entities, This indicates the number of adversarial proxies in the adversarial sequence. Indicates the first i An autonomous driving agent in t Actions at any moment T Indicates the number of simulation steps. t Indicates time, This represents the collision distance cost of an autonomous driving agent. , and All of these represent adjustable hyperparameters. This represents the cost of speed differences for autonomous driving agents. This represents the cost of safe distances between adversarial agents. This indicates the cost of exceeding the boundary constraint. This represents the Euclidean distance operation. This represents the state corresponding to the victim model. Indicates the first i An autonomous driving agent in t The state at any given moment, Indicates the speed difference, Indicates the minimum safety threshold. Indicates the first j An autonomous driving agent in t The state at any given moment, L This indicates the number of corner points contained in each vehicle's bounding box. m This represents the index of the currently processed corner point, used to iterate through all corner positions of the vehicle bounding box at each time step. This represents the Gaussian potential function around the diagonal point. Representing adversarial agents At any moment t The bounding box at that time, Indicates the corner points of the bounding box. This represents a convolution operation with an impassable area in the map.

[0040] Further, according to the generated safety critical scene, a progressive unsupervised clustering method is introduced to dynamically discover new categories and optimize the clustering structure, and a scene analysis result is obtained, which is specifically:

[0041] The generated safety critical scene is hierarchically divided, wherein a representative sample is selected in each division process to construct an anchor point set;

[0042] Pseudo-labels are generated based on the anchor point set;

[0043] According to the pseudo-labels, an anchor point graph is constructed based on data similarity, and representative samples are used to depict the distribution characteristics of the data;

[0044] Based on the distribution characteristics, high confidence information is diffused through a label propagation mechanism to make the clustering boundary clear;

[0045] When a new sample different from the existing categories is detected, a new category is created, and the clustering result is dynamically updated until the clustering result meets the requirements, and a scene analysis result is obtained.

[0046] The application also provides an automatic driving safety test method, which is used to execute an automatic driving safety test system, including the following steps:

[0047] A conventional driving scene dataset is generated in a simulation platform;

[0048] According to the conventional driving scene dataset, an automatic driving scene generation network SAGA based on adversarial perturbation is used to generate safety critical scenes;

[0049] According to the generated safety critical scene, a progressive unsupervised clustering method is introduced to dynamically discover new categories and optimize the clustering structure, and a scene analysis result is obtained, and the automatic driving safety test is completed.

[0050] The beneficial effects of the application are:

[0051] (1) Compared with the prior art, the application proposes a closed-loop automatic driving algorithm safety test platform framework, which has significant innovation and superiority. The existing automatic driving system test method mainly depends on actual road collection or rule preset scene, which not only has high test cost, but also is difficult to fully cover long tail and extreme conditions under complex traffic environment, resulting in insufficient system safety verification. In order to solve the above problems, the application is based on CARLA simulation platform, and functions such as automatic generation of conventional traffic scene, generation of adversarial safety critical scene and dynamic clustering analysis of large-scale test data are designed and realized, so as to build a unified automatic driving algorithm safety test platform, and realize the automation and closed-loop integration of the test process.

[0052] (2) The application significantly improves the safety verification capability and algorithm robustness optimization efficiency of the automatic driving system through the innovative scene generation method, adversarial testing mechanism and dynamic data analysis strategy, has the advantages of high automation, strong system integration, rich scene diversity, accurate risk mining and the like, breaks through the technical bottleneck of the existing testing method, and provides solid technical support for the safety improvement and industrial application landing of the automatic driving system.

[0053] (3) The application automatically generates a conventional traffic scene by road topology modeling and JPS path planning algorithm, combines B-spline interpolation to optimize path smoothness, significantly reduces the testing data acquisition cost, improves the data acquisition efficiency and the diversity of traffic scenes. At the same time, the privileged expert agent model is used to drive multi-modal data acquisition, ensuring the high precision and stability of data acquisition, and providing rich and reliable basic data for automatic driving algorithm training and evaluation. Compared with the traditional real vehicle testing method, the application significantly improves the automation degree of data generation and the realism of simulation scenes, greatly expands the coverage range of the testing environment.

[0054] (4) In terms of safety-critical scene generation, the application proposes an adversarial scene generation method based on a generator-victim model architecture, which automatically generates challenging and risky test scenes through a policy gradient optimization algorithm. Through trajectory disturbance of the attacker and multi-dimensional target function optimization, the potential decision-making vulnerabilities of the victim model are accurately mined, significantly improving the comprehensiveness and effectiveness of the robustness testing of the automatic driving system. In addition, traffic rules and physical constraints are introduced during the generation process to ensure that the generated scenes not only have challenges, but also meet the rationality and feasibility of the real traffic environment, solving the problems of scene distortion and strategy failure in existing adversarial generation methods.

[0055] (5) For testing data analysis requirements, the application uses a progressive unsupervised clustering analysis method to realize preliminary hierarchical clustering based on the BKHK algorithm, dynamically discovers new categories and optimizes the clustering structure through anchor point graph construction and label propagation mechanism. This method significantly improves the processing efficiency of large-scale high-dimensional test data, enhances the automatic identification and clustering ability of high-risk scenes, and overcomes the problem that existing static clustering methods cannot adapt to data dynamic evolution. Through continuous iteration optimization, the system can efficiently extract typical high-risk scene patterns, providing reliable data support for dynamic risk warning and safety optimization of the automatic driving system.

[0056] (6) The application builds a unified closed-loop automatic driving algorithm safety test platform, integrates conventional scene generation, adversarial scene test and risk analysis functions, forms a full-process automatic processing mechanism of data generation-vulnerability mining-dynamic analysis. Compared with the existing dispersed test system, the test process of the application is more systematic and automated, greatly improves the test efficiency and algorithm optimization iteration speed, shortens the development cycle of the automatic driving system, and improves the overall safety and robustness of the system. The platform is designed based on the open source CARLA simulation environment, has good expansibility and compatibility, supports the access of multiple types of automatic driving algorithms, can adapt to different sensor configurations and test requirements, and is convenient for integration and practical application promotion. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 FIG. 1 is a system structure diagram of the application.

[0058] Figure 2 FIG. 2 is a platform framework diagram of the application.

[0059] Figure 3 FIG. 3 is a conventional traffic scene data set generation process diagram in the application.

[0060] Figure 4 FIG. 4 is a schematic diagram of the interaction relationship between the overall network architecture and each module in the application.

[0061] Figure 5 FIG. 5 is a method flowchart of the application. DETAILED DESCRIPTION

[0062] The specific embodiments of the application are described below to facilitate those skilled in the art to understand the application, but it should be clear that the application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the application defined and determined by the appended claims, and all applications utilizing the concept of the application are within the scope of protection.

[0063] Example 1

[0064] In view of the problems of insufficient test scene coverage, weak adversarial test capability and low key scene analysis efficiency in the safety verification of existing automatic driving systems, the application provides an automatic driving algorithm test platform framework. The framework realizes the functions of automatic generation of safety critical scenes, vulnerability mining of system robustness and dynamic clustering analysis through the test and analysis of the automatic driving system in the simulation environment, and forms a closed-loop automatic driving safety verification scheme.

[0065] The application focuses on solving the following technical problems:

[0066] (1) Safety-critical scenario generation efficiency and diversity are insufficient: existing data-driven and knowledge rule scenario generation methods have problems such as sample scarcity, limited coverage, and insufficient challenges, making it difficult to generate diversified and challenging long-tail extreme test scenarios. The present application proposes an automated scenario generation method based on a generative adversarial mechanism, which automatically exploits system vulnerabilities through adversarial optimization strategies, enabling the automatic generation of complex extreme traffic scenarios and significantly improving test coverage and challenge.

[0067] (2) Safety-critical scenario generation authenticity and rationality are insufficient: current adversarial generation methods often ignore the rationality and explainability of traffic scenarios when improving scenario challenge, resulting in a lack of physical constraints and behavior norms in real traffic environments. The present application designs a fusion mechanism based on expert knowledge and traffic rules, combining traffic behavior norms with optimization processes to ensure that generated scenarios are both challenging and meet the rationality and feasibility of real traffic environments.

[0068] (3) Large-scale test data analysis efficiency is low, and class discovery ability is insufficient: as the number and complexity of test scenarios continue to grow, existing test result analysis methods struggle to effectively handle large-scale high-dimensional data and dynamically discover new high-risk scenario patterns. The present application proposes a gradual unsupervised clustering analysis method that uses anchor optimization and label propagation mechanisms to dynamically classify and analyze test data, support automatic discovery of new classes, and significantly improve the efficiency and accuracy of automated driving system test result analysis.

[0069] (4) Lack of a unified closed-loop automated driving algorithm safety testing platform: current scenario generation, system testing, and data analysis processes are fragmented, making it difficult to form a systematic and automated safety testing platform. The present application integrates simulation testing, adversarial scenario generation, dynamic clustering analysis, and other key technologies to build a unified automated driving algorithm testing platform, enabling a closed-loop automated processing flow from data generation to vulnerability exploitation to dynamic analysis.

[0070] As shown in Figure 1 and Figure 2 , the present application provides an automated driving safety testing system, comprising:

[0071] A conventional traffic scenario generation subsystem is used to generate conventional driving scenario datasets in a simulation platform, including:

[0072] A road network topology modeling module is used to extract road topology information by analyzing the road network in the simulation platform and model the road network topology based on the road topology information;

[0073] A path planning and optimization module is used to generate a smooth driving path from the starting point to the target point based on the constructed road network topology using a path planning algorithm, and optimize the smooth driving path using an interpolation method.

[0074] a multi-modal data acquisition module, configured to acquire multiple types of scene data based on the optimized smooth driving path and using multi-modal sensors to construct a conventional driving scene dataset;

[0075] a safety-critical scene generation subsystem, configured to generate safety-critical scenes based on the conventional driving scene dataset and using an automatic driving scene generation network SAGA based on adversarial perturbation, including:

[0076] a conventional training module, configured to train an automatic driving agent using the conventional driving scene dataset to form a benchmark model with driving decision-making and environmental perception capabilities, including:

[0077] according to the conventional driving scene dataset, observation data O 𝑡 combined with the target trajectory point G to train an automatic driving strategy network π 𝜃 to form a benchmark model with driving decision-making and environmental perception capabilities;

[0078] an adversarial scene generation module, configured to generate an adversarial scene for a victim model in an initial scene by using an attacker's behavior trajectory based on a gradient-based adversarial perturbation strategy according to the training result, wherein an optimal perturbation sequence is found by minimizing an objective function to generate a safety-critical scene;

[0079] a victim model module, configured to provide real-time feedback during the perturbation generation process to optimize the adversarial perturbation strategy, wherein the victim model is an automatic driving decision-making system that is the attack target during the adversarial scene generation process;

[0080] a safety-critical scene clustering analysis subsystem, configured to dynamically discover new classes and optimize clustering structure using an introduced progressive unsupervised clustering method based on the generated safety-critical scenes to obtain scene analysis results and complete automatic driving safety testing, including:

[0081] hierarchical division is performed on the generated safety-critical scenes, wherein representative samples are selected in each division process to construct an anchor point set;

[0082] pseudo-labels are generated based on the anchor point set;

[0083] based on the pseudo-labels, an anchor point graph is constructed based on data similarity, and representative samples are used to depict the distribution characteristics of the data;

[0084] based on the distribution characteristics, high-confidence information is diffused through a label propagation mechanism to make the clustering boundary clear;

[0085] When a new sample different from the existing categories is detected, a new category is created, and the clustering result is dynamically updated until the clustering result meets the requirements, and the scene analysis result is obtained.

[0086] In this embodiment, the application proposes a closed-loop automatic driving algorithm safety test platform framework, which is based on the CARLA simulation platform, integrates functions such as conventional traffic scene generation, automatic driving system behavior training and evaluation, adversarial scene generation and testing, risk clustering analysis, and forms a full-process automatic test system from data generation to system testing, and then to risk mining and analysis. The platform includes the following three core technical subsystems: a conventional traffic scene generation subsystem, a safety critical scene generation subsystem, and a clustering analysis subsystem, and the platform framework diagram is as shown in Figure 2

[0087] In this embodiment, the conventional traffic scene generation subsystem: in the actual road environment, the data of the conventional driving scene usually needs to be collected through real vehicle testing, and the running data of the vehicle under different traffic conditions is recorded to support the development and verification of automatic driving technology. However, the actual road test is high in cost and long in cycle, and it is difficult to effectively cover a variety of traffic situations. In order to solve the above problems, the application selects to generate a conventional driving scene dataset in the CARLA simulation platform to break through the limitations of the traditional data collection method. First, by analyzing the road network of the simulation town in the CARLA platform, detailed road topology information is extracted, including road points, road segments and their corresponding geometric and semantic attributes, and then a directed graph model is constructed to accurately describe the road structure. On this basis, a path planning algorithm is applied to generate a smooth driving path from the starting point to the target point, and an interpolation method is further used to optimize the continuity and feasibility of the path to ensure that the generated path meets the vehicle dynamics constraints and the actual road running conditions. Subsequently, a privileged expert agent model is deployed, and multiple types of scene data such as RGB images, depth images, semantic segmentation images, point cloud data and vehicle dynamic state information are synchronously collected by using multi-modal sensors to construct a conventional traffic scene dataset with high fidelity and diversity. The conventional traffic scene dataset not only provides reliable and rich data support for the development and performance evaluation of automatic driving algorithms, but also provides a solid data foundation for the generation of subsequent safety critical scenes. The specific design flow chart is as shown in Figure 3

[0088] In this embodiment, a path planning algorithm based on the Jump Point Search (JPS) algorithm and B-spline interpolation is designed and implemented, and the specific implementation process of the JPS algorithm is as follows:

[0089] ​​(1) Node initialization: Initialize open set (openSet) and closed set (closedSet), the starting point is added to openSet and the cost gScore=0, and the cost of the remaining nodes is set to infinity.

[0090] (2) Jump expansion: Jump in the current search direction until an obstacle is encountered or a forced neighbor is encountered, which is marked as a key node.

[0091] (3) Cost update: Update the cost of the key node after jumping and add it to the open set to ensure the optimality of the path.

[0092] (4) Path backtracking: When the target node is added to the closed set, the final path is generated by backtracking from the target node to the starting point.

[0093] In this embodiment, the B-spline interpolation algorithm is used to optimize the path generated by JPS, and the specific steps are as follows:

[0094] (1) Path point filtering: Remove dense and redundant points in the path to reduce the complexity of subsequent interpolation calculation.

[0095] (2) B-spline curve generation: Generate a B-spline curve using the filtered path points as control points to make the path smoother and more continuous.

[0096] (3) Kinetic constraint adjustment: Properly adjust the distance between control points to ensure that the turning radius of the path meets the constraints of vehicle dynamics and avoid large steering angle changes.

[0097] In this embodiment, the safety critical scenario generation subsystem: the present application proposes an automatic driving scene generation network SAGA based on adversarial perturbation, aiming to solve the problem of insufficient testing of high-risk scenarios of automatic driving system. The network designs a perturbation strategy for the trajectory of the attacker, specifically excavates the decision-making vulnerabilities of the victim model, generates extreme and diverse safety critical scenarios, and expands the distribution range and diversity of the training data. Specifically, the automatic driving scene generation network SAGA is based on a conventional scene dataset, extracts the geographical structure, traffic participant behavior and environmental conditions as initial input information, and provides support for the evolution of adversarial scenes. The automatic driving scene generation network SAGA consists of a conventional training module, an adversarial scene generation module, a victim model module (Victim Models) and a target function optimization module, which work together to gradually complicate the scene and approach extreme conditions. The conventional training module is based on a conventional scene dataset The automatic driving strategy model is initially trained to ensure its basic driving and decision-making capabilities, thereby forming a benchmark model under attack. This module is mainly used to train the victim model and the adversarial agent sequence. The adversarial scenario generation module is the core part of the framework. Through a gradient-based adversarial perturbation strategy, the attacker's behavior trajectory is used to generate an adversarial scenario for the victim model in the initial scenario. The perturbation generation process guides the evolution of the scenario to a high-risk state, simulating an extreme driving environment. The victim model module is composed of specific automatic driving models (AIM-BEV and TF++). These models serve as attack targets and provide real-time feedback during the generation process, assisting in optimizing the adversarial perturbation strategy. Finally, the objective function optimization module quantifies the extremity and diversity of the scenario through a multi-dimensional cost function (such as collision distance, speed difference, and boundary crossing penalty), ensuring that the generated scenario is more challenging and representative. The generation process of the automatic driving scenario generation network SAGA starts with an initial scenario. Through the perturbation strategy, the attacker's behavior trajectory is optimized, and a new adversarial scenario is gradually generated. In each iteration, the framework evaluates the perturbation effect through the objective function and optimizes the perturbation strategy based on the feedback until the generated adversarial scenario triggers a collision or other high-risk behavior between the attack target and the victim model, completing the optimization process. As shown in FIG. 8, Figure 4 Figure 4 FIG. 8 shows the overall network architecture of the algorithm framework and the interaction between the modules.

[0098] In this embodiment, the adversarial scenario is generated as follows:

[0099] ​The adversarial scenario generation module aims to generate high-risk adversarial scenarios that can induce the failure of autonomous driving systems by carefully designed disturbance strategies. The specific process is as follows: First, the adversarial scenario generation module selects a regular scenario as the initial environment and deploys the victim model (such as AIM-BEV, TF++) and the attacker agent in it. The attacker assumes the role of a disturber in the environment, and its behavior parameters (such as position, speed, acceleration, etc.) will be used as optimization variables for continuous adjustment. Next, the module initializes the attacker's action trajectory and defines the disturbance range. In each frame of simulation, the attacker performs actions according to the current disturbance parameters, and the environment state is updated accordingly. The victim model responds to the environmental changes based on its strategy. Then, the system evaluates the disturbance effect according to the risk situation caused by the disturbance, and optimizes the disturbance trajectory according to the feedback. The evaluation process relies on a set of pre-defined risk indicators, such as the minimum distance between the two vehicles, the speed difference, whether a collision occurs, whether there is an out-of-boundary, etc., to comprehensively measure the extremity of the current scenario. The optimization process uses an iterative approach, adjusting the attacker's trajectory based on the victim's behavior feedback after each disturbance attempt, gradually guiding the scenario to evolve towards a higher risk state. Finally, when a collision or significant abnormal behavior is successfully induced in the scenario, the disturbance process terminates, and the generated adversarial scenario is output as a test sample. The module emphasizes the goal-oriented nature of the disturbance and the diversity of the scenario, effectively exploiting the potential decision-making vulnerabilities of the victim model, and expanding the coverage of extreme cases in the scenario dataset, thereby improving the robustness testing ability of the autonomous driving system in high-risk environments.

[0100] In this embodiment, the regular training module is the first step in the entire safety-critical scenario generation framework, and its core goal is to train the autonomous driving agent through imitation learning strategies to improve its driving ability and robustness in regular traffic environments. The module uses the regular traffic scenario dataset Dreg to supervise the learning and training of the autonomous driving strategy model, to form a baseline model with basic driving decision-making and environmental perception capabilities. During the training process, the regular training module combines the input observation data O 𝑡 , including vehicle-mounted camera images, laser radar point clouds, and vehicle state information and target trajectory points G, to train the autonomous driving strategy network π 𝜃。

[0101] In this embodiment, the core goal of the adversarial scenario generation module is to generate extreme driving scenarios by iteratively optimizing the disturbance sequence, thereby testing and challenging the driving strategy of the target victim model. The design of this module includes the following parts: definition of state and action, adversarial disturbance control sequence and scenario generation, and corresponding cost function design.

[0102] (1) States and actions in traffic simulation

[0103] First, in traffic simulation, define the state variables for each agent: planar position. , towards speed magnitude Among them, the index This represents the autonomous driving agent associated with the victim model; the rest... This represents other traffic agents (including adversarial agents) in the scene. At time... At that time, the states of all agents can be represented by a set. Let it be represented as . The state corresponding to the victim model is denoted as . The entire traffic scenario in time sequence The set of states on is then represented as:

[0104]

[0105] in, T This refers to a fixed number of simulation steps or simulation duration. During initialization, X It will be set as the starting point of a normal, non-collision traffic scenario. In each simulation step, if the time is known... t status and the actions of all agents (including victim models and adversarial agents). Then the state at the next moment The following formula:

[0106]

[0107] in, K This represents a differentiable kinematic model of a bicycle. Because this model can well characterize the physical motion characteristics of a non-omnidirectional vehicle and is differentiable, it is beneficial for subsequent gradient propagation and adversarial perturbation optimization.

[0108] During the simulation, the victim model will adjust based on the observational information provided by the environment. 𝑡 To decide one's own actions These observation data are provided by the rendering function, as shown in the following formula:

[0109]

[0110] in, This represents a map of the simulation environment (such as road information, drivable areas, etc.). Rendering function. This is used to convert simulation states and map information into input data that can be used for decision-making by the attacked model. The rendering method may differ depending on the type of attacked model. The AIM-BEV model (a simulated BEV-view autonomous driving model) uses a differentiable rasterization method to convert vehicle and road information into a bird's-eye view (BEV) format, ensuring the differentiability of the data processing. The TF++ model (an improved TransFuser fusion model) uses the non-differentiable graphics engine of the CARLA simulator to generate LiDAR point clouds and then uses PointNet technology to convert the point clouds into a differentiable form, thus supporting subsequent cost function and gradient calculations.

[0111] (2) Action sequences and scene generation of adversarial agents

[0112] To discover and simulate safety-critical traffic scenarios, this invention focuses on optimizing adversarial agents (i.e., Action sequences of intelligent agents By continuously updating the action sequences of the adversarial agent, extreme scenarios can be gradually approximated during the simulation process, allowing for more rigorous testing of the victim model.

[0113] Let parameter set Represents the set of all action sequences of adversarial agents, i.e. , Since the action of each agent at each moment may consist of two control variables (such as steering angle and acceleration), therefore The dimension is N×T×2. We minimize the objective function. To find the optimal perturbation order As shown in the following formula:

[0114]

[0115] Wherein, objective function Taking into account factors such as scenario security, feasibility, and attack targets, S This represents the set of spatiotemporal evolution sequences of the entire scene. During the adversarial scenario generation process, the adversarial intelligence observes and utilizes the victim model's control actions and environmental reactions from the previous step, gradually adjusting its own actions to generate situations detrimental to the victim model, such as collisions, sudden braking, and dangerous lane changes. At the end of each iteration cycle, the algorithm is updated using gradient descent. This causes the adversarial scenarios to gradually converge to extreme situations that pose a significant threat to autonomous driving systems.

[0116] (3) Cost function design

[0117] objective function The design is crucial, and the invention comprehensively considers key factors such as collision induction, safe distance between opposing vehicles, and boundary crossing penalties, and sets the total objective function as the following weighted sum form:

[0118]

[0119] wherein, , and all represent adjustable hyperparameters for controlling the weight of each loss term in the overall objective. The specific loss term definitions are as follows:

[0120] 1) Collision distance cost of the autonomous driving agent :

[0121] In order to quantify the collision risk between the autonomous driving agent and the opposing agent , define this term as the average value of the shortest distance between the bounding box of the agent and the closest opposing agent over the simulation step T , as shown in the following formula:

[0122]

[0123] By minimizing this distance, the opposing agent is guided to choose actions that are easy to cause collisions or extremely close interactions.

[0124] 2) Speed difference cost of the autonomous driving agent :

[0125] Equally important as the collision distance is the influence of the speed difference between vehicles on the collision risk. The greater the speed difference, the higher the risk and destructive potential of potential collisions, as shown in the following formula:

[0126]

[0127] 3) Safe distance cost between opposing agents :

[0128] To avoid unreasonable mutual collisions within the opposing agents, the invention sets a minimum safety threshold . If the distance between any two opposing agents is less than , a corresponding penalty is generated. This constraint helps to maintain the physical reasonableness of the opposing scenario and avoid overly extreme (or stacked) results, as shown in the following formula:

[0129]

[0130] 4) Boundary crossing constraint cost :

[0131] In addition to the relative collision between the adversarial agents, the invention also needs to consider the road boundary constraints. If the adversarial vehicle crosses the boundary and enters the non-passable area, it means that the vehicle behavior lacks authenticity or rationality. Therefore, a boundary crossing penalty is defined as shown in the following formula:

[0132]

[0133] wherein, represents the optimal disturbance sequence, represents the set of all adversarial agent action sequences, represents the minimization objective function, represents the driving strategy of the adversarial agent action sequence, represents the autonomous driving agent associated with the victim model, represents the traffic agents in the scene except for the adversarial agent and the victim model (the measured autonomous driving model), represents the number of adversarial agents in the adversarial sequence, represents the i autonomous driving agent action at time t , T represents the number of simulation steps, t represents the time, represents the collision distance cost of the autonomous driving agent, , and all represent adjustable hyperparameters, represents the speed difference cost of the autonomous driving agent, represents the safety distance cost between the adversarial agents, represents the boundary crossing constraint cost, represents the Euclidean distance operation, represents the state corresponding to the victim model, represents the state of the i autonomous driving agent at time t , represents the speed difference, represents the minimum safety threshold, represents the state of the j autonomous driving agent at time t , L represents the number of corner points contained in each vehicle bounding box. Usually, a rectangular bounding box has 4 corner points, the bounding box is rectangular, L=3 (because from m=0 to L=, there are a total of 4 corner points), mrepresents the index of the current processed corner point, used to traverse all the corner points of the vehicle bounding box at each time step , represents the Gaussian potential function around the corner point, used to measure the distance to the map boundary or impassable area, represents the adversarial agent (vehicle) at time t , the bounding box, represents the corner point of the bounding box, represents the convolution operation with the impassable area in the map (such as sidewalks, green belts, etc.). If the adversarial vehicle crosses the boundary, this item will increase significantly, corresponding to a high penalty or directly leading to the termination of the simulation.

[0134] In this embodiment, the traffic agent includes an adversarial agent and a victim model, and each adversarial agent in the adversarial sequence has state and action contents.

[0135] The above cost function comprehensively considers multiple constraint conditions, generates high-risk scenarios while ensuring the rationality and physical authenticity of the scenarios, thereby realizing more effective safety-critical scenario generation.

[0136] In this embodiment, the victim model refers to an autonomous driving decision system that serves as an attack target in the adversarial scenario generation process. The victim model in the present application adopts an autonomous driving agent trained based on an end-to-end imitation learning method, aiming to comprehensively test and verify the safety and robustness of the autonomous driving system in the simulation environment. The overall structure of the model includes three core functional modules: perception, decision, and control. The structures in these three blocks are different for different autonomous driving models. The perception module collects environmental data in real time through sensors (such as cameras, lidar), identifies roads, obstacles, and traffic rules, and provides the system with "eyes"; the decision module plans a path, predicts risks, and formulates driving strategies (such as lane changing, braking) based on perception information, acts as the "brain" to complete intelligent judgment; the control module converts decisions into vehicle execution instructions (such as steering, throttle, braking), accurately controls hardware to realize actions, and completes closed-loop operations like "hands and feet".

[0137] In this embodiment, the safety-critical scene clustering analysis subsystem: in view of the diversity, dynamics and complexity characteristics of the data in the safety-critical scene, the application introduces a progressive unsupervised clustering method to realize adaptive clustering update and new class discovery. The method optimizes the clustering structure through iteration, fully utilizes the pseudo-label generated by the preliminary clustering result, and deeply mines the similarity relationship between the data, thereby continuously optimizing the class boundary, so as to improve the accuracy and adaptability of the clustering effect. Specifically, first, the BKHK algorithm (binary K-Means) is used to preliminarily hierarchically divide the data. BKHK performs adaptive layering on the data based on a binary tree structure, selects representative samples to construct an anchor set during each division process, and generates preliminary pseudo-labels based on the anchor set; subsequently, an anchor graph is constructed based on data similarity, and the distribution characteristics of the data are described using representative samples; then, high-confidence information is diffused through a label propagation mechanism, so that the clustering boundary gradually becomes clear; when new samples that are significantly different from existing classes are detected, the algorithm can adaptively identify and create new classes, and dynamically update the clustering structure until the clustering result tends to be stable.

[0138] The application presents an adversarial scene generation method based on a generator-victim model architecture. Through a policy gradient optimization algorithm, the method generates adversarial scenes with high risk and challenge, and realizes automatic mining of potential vulnerabilities of the automatic driving system. The method effectively breaks through the bottleneck of the existing test method that the extreme complex scene is not covered enough, and guarantees the robustness and safety of the automatic driving system. This technical point is one of the important protection contents of the application. For the generated large-scale test scene data, the application designs a progressive unsupervised clustering analysis method, combines anchor graph construction and label propagation mechanism, realizes dynamic discovery of new classes and automatic clustering of high-risk scenes. Through this method, the test data analysis efficiency is improved, the weak scene mode of the system is accurately identified, and the dynamic risk warning capability is enhanced. This technical point is the core method of test data analysis, and is an important protection content of the application. The application uses the CARLA simulation platform, combines road topology modeling, JPS path planning and B-spline interpolation, realizes automatic generation of conventional traffic scenes. Through the privileged expert agent model, multi-modal data synchronous collection is completed, and high-quality data are provided for training and evaluation of the automatic driving system. This module effectively improves the data generation efficiency and the authenticity of the simulation scene, and is the basic technical content of the application, and the protection range covers the complete data generation and collection process. The application integrates the modules of conventional scene generation, adversarial testing and dynamic risk analysis, constructs a unified automatic driving algorithm safety test platform, realizes the closed-loop automatic process of data generation-system testing-risk mining-dynamic analysis. The platform has high integration, strong expansibility and automation, and is the architecture basis of the overall technical scheme of the application. The protection points cover the platform architecture design, module cooperation mode and system integration mechanism.

[0139] In summary, the present application takes the safety critical scene generation method as the core, cooperates with dynamic clustering analysis and conventional scene automatic generation technology, forms a closed-loop automatic driving system safety test platform. Each technical scheme complements each other, comprehensively improves the test coverage and risk mining ability of the automatic driving system, and the related technical methods and system architecture are the main protection content of the present application. The present application significantly improves the safety verification ability and algorithm robustness optimization efficiency of the automatic driving system through the innovative scene generation method, the adversarial test mechanism and the dynamic data analysis strategy, has the advantages of high automation, strong system integration, rich scene diversity, accurate risk mining and the like, breaks through the technical bottleneck of the existing test method, and provides a solid technical support for the safety improvement and industrial application landing of the automatic driving system.

[0140] Embodiment 2

[0141] As Figure 5 shown, the present application provides an automatic driving safety test method, which is used for executing the automatic driving safety test system in embodiment 1, and comprises the following steps:

[0142] S1, generating a conventional driving scene dataset in a simulation platform;

[0143] S2, generating a safety critical scene by using an automatic driving scene generation network SAGA based on adversarial disturbance according to the conventional driving scene dataset;

[0144] S3, obtaining a scene analysis result by using a progressive unsupervised clustering method to dynamically discover new categories and optimize the clustering structure according to the generated safety critical scene, and completing the automatic driving safety test.

[0145] The purpose of the present application is to design and implement an automatic driving algorithm test platform framework with efficient scene generation, vulnerability mining and risk analysis capability in view of the deficiencies of the existing automatic driving algorithm safety test means, specifically including the following aspects:

[0146] 1) An automatic safety critical scene generation method is provided: based on an adversarial optimization algorithm, a generator-victim model architecture is adopted, and a policy gradient optimization method is used to realize automatic and efficient generation of safety critical scenes, covering different types of traffic participant behaviors and diversified extreme environmental conditions.

[0147] 2) An adversarial generation mechanism with traffic rationality constraints is provided: by introducing traffic rules, expert experience and physical constraints, the adversarial scene generation process is optimized to ensure that the generated test scenes conform to the actual traffic logic and behavior specifications, and the credibility and explainability of the test data are improved.

[0148] 3) Provide a progressive unsupervised clustering analysis method: anchor point graph construction and label propagation mechanism are adopted to support dynamic classification of automatic driving test data, optimization of class boundary and discovery of new class, realize efficient analysis of large-scale test data and identification of key risk scenarios, and improve the system risk assessment capability.

[0149] 4) Provide a closed-loop automatic driving algorithm safety test platform framework: based on the CARLA simulation platform, the invention integrates functions such as automatic generation of conventional traffic scenes, automatic driving system behavior training and evaluation, generation of adversarial scenes, automated testing and risk clustering analysis, realizes the whole process safety verification of the automatic driving system, and provides reliable basis for algorithm optimization and robustness improvement.

[0150] In summary, the invention realizes the automation, systematization and intelligentization of automatic driving algorithm testing, significantly improves the test scene generation efficiency, system vulnerability discovery capability and intelligent level of test data analysis, and provides a solid technical guarantee for the safety improvement and wide application of automatic driving system.

[0151] Those skilled in the art will realize that the embodiments described herein are for the purpose of helping the reader to understand the principles of the present application, and should be understood as not limiting the scope of protection of the present application to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the present application without departing from the essence of the present application, and these modifications and combinations are still within the scope of protection of the present application.

Claims

1. An automated driving safety testing system, characterized by, The method comprises the following steps: a conventional traffic scene generation subsystem is used to generate a conventional driving scene dataset in a simulation platform; the conventional traffic scene generation subsystem comprises: a road network topology modeling module is used to extract road topology information by analyzing a road network in the simulation platform, and model the road network topology based on the road topology information; a path planning and optimization module is used to generate a smooth driving path from a starting point to a target point based on the constructed road network topology, and optimize the smooth driving path by using an interpolation method; a multi-modal data acquisition module is used to acquire multiple types of scene data based on the optimized smooth driving path by using multi-modal sensors, and construct the conventional driving scene dataset; a safety-critical scene generation subsystem is used to generate safety-critical scenes by using an automatic driving scene generation network SAGA based on adversarial perturbation according to the conventional driving scene dataset; the safety-critical scene generation subsystem comprises: a conventional training module is used to train an automatic driving agent by using the conventional driving scene dataset, and form a benchmark model with driving decision and environment perception capabilities; an adversarial scene generation module is used to generate an adversarial scene for a victim model in an initial scene by using an attacker's behavior trajectory based on a gradient-based adversarial perturbation strategy according to the training result, wherein an optimal perturbation sequence is found by minimizing an objective function to generate a safety-critical scene; a victim model module is used to provide real-time feedback during the perturbation generation process to optimize the adversarial perturbation strategy, wherein the victim model is an automatic driving decision system that is the attack target during the adversarial scene generation process; a safety-critical scene clustering analysis subsystem is used to dynamically discover new categories and optimize the clustering structure by using an introduced progressive unsupervised clustering method according to the generated safety-critical scenes, obtain a scene analysis result, and complete the automatic driving safety test; the safety-critical scene clustering analysis subsystem is used to dynamically discover new categories and optimize the clustering structure by using an introduced progressive unsupervised clustering method according to the generated safety-critical scenes, and obtain a scene analysis result, which specifically comprises: hierarchical division is performed on the generated safety-critical scenes, wherein representative samples are selected in each division process to construct an anchor point set; pseudo-labels are generated based on the anchor point set; an anchor point graph is constructed based on data similarity based on the pseudo-labels, and representative samples are used to depict the distribution characteristics of the data; based on the distribution characteristics, high-confidence information is diffused through a label propagation mechanism to make the clustering boundary clear; when a new sample different from the existing categories is detected, a new category is created, and the clustering result is dynamically updated until the clustering result meets the requirements, and the scene analysis result is obtained.

2. The automated safety testing system of claim 1, wherein, the automatic driving agent is trained by using the conventional driving scene dataset, which specifically comprises: According to the conventional driving scene dataset, the observation data An autonomous driving strategy network π is trained in combination with the target trajectory point G 𝜃 , forming a benchmark model with driving decision and environment perception capabilities.

3. The automated safety testing system of claim 1, wherein, the expression of the optimal perturbation sequence is as follows: in, This represents the optimal perturbation sequence. Represents the set of all action sequences of adversarial agents. This represents minimizing the objective function. Indicates the first A driving strategy based on an adversarial agent's action sequence. This refers to the autonomous driving agent associated with the victim model. Indicates that, except for External transportation intelligent entities, This indicates the number of adversarial proxies in the adversarial sequence. Indicates the first i An autonomous driving agent in t Actions at any moment T Indicates the number of simulation steps. t Indicates time, This represents the collision distance cost of an autonomous driving agent. , and All of these represent adjustable hyperparameters. This represents the cost of speed differences for autonomous driving agents. This represents the cost of safe distances between adversarial agents. This indicates the cost of exceeding the boundary constraint. This represents the Euclidean distance operation. This represents the state corresponding to the victim model. Indicates the first i An autonomous driving agent in t The state at any given moment, Indicates the speed difference, Indicates the minimum safety threshold. Indicates the first j An autonomous driving agent in t The state at any given moment, L This indicates the number of corner points contained in each vehicle's bounding box. m This represents the index of the currently processed corner point, used to iterate through all corner positions of the vehicle bounding box at each time step. This represents the Gaussian potential function around the diagonal point. Representing adversarial agents At any moment t The bounding box at that time, Indicates the corner points of the bounding box. This represents a convolution operation with an impassable area in the map.

4. An automated driving safety test method characterized by, the automatic driving safety test method is used to execute the automatic driving safety test system of any one of claims 1-3, comprising the following steps: a conventional driving scene dataset is generated in a simulation platform; a safety-critical scene is generated by using an automatic driving scene generation network SAGA based on adversarial perturbation according to the conventional driving scene dataset; According to the generated safety-critical scene, a progressive unsupervised clustering method is introduced to dynamically discover new categories and optimize the clustering structure, obtain the scene analysis result, and complete the automatic driving safety test.

Images