Quantitative assessment method for safety performance of planning module of autonomous driving system, electronic device, storage medium, and computer program product

Abstract

Disclosed in the present invention are a quantitative assessment method for the safety performance of a planning module of an autonomous driving system, an electronic device, a storage medium, and a computer program product. The quantitative assessment method for the safety performance of a planning module of an autonomous driving system comprises: decomposing hazard events of a planning module into events caused by planning uncertainties of different algorithm modules of the planning module; performing scenario modeling on the probability of the hazard events caused by the planning uncertainties of the planning module to obtain driver models under different hazard events; and at least on the basis of the driver models under different hazard events, obtaining quantization objectives corresponding to different algorithm modules of the planning module. The embodiments of the present application provide a theoretical basis-based complete quantitative assessment method for the safety performance of a planning module of an autonomous driving system.

Description

[Amended according to Rule 26, dated September 2024] Methods for quantitative evaluation of safety performance of autonomous driving system planning modules, electronic devices and storage media, and computer program products. [Amended according to Rule 26, dated September 2024] Technical Field

[0001] [Amended according to Rule 26, 25.09.2024] This application relates to the field of intelligent driving technology, and in particular to a method for quantitative evaluation of safety performance of an autonomous driving system planning module, an electronic device and storage medium, and a computer program product. [Amended according to Rule 26, September 25, 2024] Background Technology

[0002] [Amended according to Detailed Rules 26, September 2024] In related technologies, the core of autonomous driving systems / driverless systems can be summarized into three parts: perception, planning, and control. Planning is the process by which a driverless system makes purposeful decisions for a specific goal. For driverless vehicles, this goal typically refers to reaching the destination from the origin while avoiding obstacles and continuously optimizing the driving trajectory and behavior to ensure passenger safety and comfort. The national standard classifies driving automation systems into six levels: L0 (emergency assistance), L1 (partial driving assistance), L2 (combined driving assistance), L3 (conditional automated driving), L4 (highly automated driving), and L5 (fully automated driving).

[0003] [Revised from Rule 26, September 2024] Currently, the safety design of autonomous driving system planning modules either sets an overall quantitative safety target at the vehicle level, or sets some experience-based quantitative safety targets or some safety requirements based on experience or theoretical analysis at the system level. However, these designs lack specific quantitative indicators and have poor theoretical basis, failing to provide reasonable theoretical derivations. This results in the planning module lacking quantifiable safety objectives. On the one hand, most qualitative safety designs for autonomous driving system planning modules lack quantitative design references. On the other hand, a small portion of quantitative autonomous driving system safety designs rely on estimates based on engineering or expert experience, lacking theoretical basis. Current autonomous driving system performance development, due to the lack of safety objectives, can only guarantee performance availability, without clarifying the extent to which performance needs to be developed and perfected.

[0004] [Amended according to Rule 26, dated September 2024] Summary of the Invention

[0005] [Amended according to Rule 26, 25.09.2024] This embodiment of the invention provides a method for quantitative evaluation of the safety performance of an autonomous driving system planning module, an electronic device and a storage medium, and a computer program product, for at least solving one of the above-mentioned technical problems.

[0006] [Amended according to Rule 26, 25.09.2024] In a first aspect, embodiments of the present invention provide a method for quantitatively evaluating the safety performance of a planning module in an autonomous driving system, comprising: decomposing a hazardous event of the planning module into events caused by planning uncertainties of different algorithm modules of the planning module; performing scenario modeling on the probability of hazardous events caused by planning uncertainties of the planning module to obtain driver models under different hazardous events; and obtaining quantitative targets corresponding to different algorithm modules of the planning module based at least on the driver models under different hazardous events.

[0007] [Amended according to Rule 26, 25.09.2024] In a second aspect, embodiments of the present invention provide a method for quantitatively assessing the safety of an autonomous driving system, comprising: obtaining quantitative targets corresponding to different algorithm modules of the autonomous driving system planning module obtained according to the method of the first aspect; and performing a quantitative assessment of the safety performance of the autonomous driving system based on the quantitative targets to obtain a safety quantitative assessment result.

[0008] [Amended according to Rule 26, 25.09.2024] In a third aspect, embodiments of the present invention provide an electronic device comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute any of the above-described methods for quantitative evaluation of safety performance in an autonomous driving system planning module of the present invention.

[0009] [Amended according to Rule 26, 25.09.2024] In a fourth aspect, embodiments of the present invention provide a storage medium storing one or more programs including execution instructions, the execution instructions being readable and executable by electronic devices (including but not limited to computers, servers, or network devices, etc.) to perform any of the above-described methods for quantitative evaluation of safety performance in an autonomous driving system planning module.

[0010] [Amended according to Rule 26, 25.09.2024] In a fifth aspect, embodiments of the present invention provide a computer program product, the computer program product including a computer program stored on a storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to execute any of the above-mentioned methods for quantitative evaluation of safety performance of an autonomous driving system planning module.

[0011] [Amended according to Rule 26, 25.09.2024] In a sixth aspect, embodiments of the present invention also provide a mobile platform, comprising: a vehicle body; a power system mounted on the vehicle body for providing power to the mobile platform; and electronic equipment of the mobile platform according to the third aspect.

[0012] [Amended according to Rule 26, September 2024] This application provides a theoretically sound and complete method for quantitatively evaluating the safety performance of an autonomous driving system planning module. Using the method of this application, quantitative safety design indicators for the autonomous driving system planning module can be obtained, providing a clear quantitative reference for the safety design and development of autonomous driving systems. Using the method of this application, performance indicators for each algorithm of the autonomous driving system planning module can be obtained, which can be used to evaluate whether the current safety performance of the autonomous driving system meets the requirements. [Revised according to Rule 26, dated September 25, 2024] Attached Figure Description

[0013] [Amended according to Rule 26, 25.09.2024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] [Amended according to Rule 26, 25.09.2024] Figure 1 is a flowchart of a safety performance quantitative evaluation for an autonomous driving system planning module provided in an embodiment of this application;

[0015] [Amended according to Rule 26, 25.09.2024] Figure 2 is a schematic diagram illustrating the scheme decomposition of a specific example of Embodiment 1 of this application;

[0016] [Amended according to Rule 26, 25.09.2024] Figure 3 is an example of a scenario where a vehicle cuts into an adjacent lane corresponding to a predicted missed detection, provided by an embodiment of this application;

[0017] [Amended according to Rule 26, 25.09.2024] Figure 4 is an example of a scenario corresponding to a predicted false detection provided in an embodiment of this application;

[0018] [Amended according to Rule 26, 2005.09.2024] Figures 5a-5d are specific examples of the derivation of the scene modeling process and quantification target corresponding to the minimum TTC and minimum THW provided in an embodiment of this application;

[0019] [Amended according to Rule 26, 25.09.2024] Figure 6 is an example of a scenario corresponding to an erroneous active lane change provided in an embodiment of this application;

[0020] [Amended according to Rule 26, 25.09.2024] Figure 7 is a scene modeling and specific derivation example of the quantification target of trajectory lateral offset error provided in an embodiment of this application;

[0021] [Amended according to Rule 26, 25.09.2024] Figure 8 is a flowchart of a safety quantification assessment method for an autonomous driving system provided in an embodiment of this application;

[0022] [Amended according to Rule 26, 25.09.2024] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. [Revised according to Rule 26, September 2024] Detailed Implementation Method

[0023] [Amended according to Rule 26, 25.09.2024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] [Amended according to Rule 26, 25.09.2024] This invention is applicable to the field of safety design and evaluation of autonomous driving system planning modules. As part of autonomous driving safety, it can be applied to the safety design and evaluation of planning modules for various levels of autonomous driving systems.

[0025] [Amended according to Rule 26, dated September 2024] This invention designs a method for quantitatively evaluating the safety performance of an autonomous driving system planning module. It employs mathematical modeling, theoretical analysis, and natural driving data analysis to design a quantitative scheme for the safety performance indicators of the planning module, obtaining the quantitative safety targets for the planning module and providing a reference for the safety design and evaluation of the autonomous driving system planning module. Mathematical modeling mainly involves kinematic model analysis and modeling of hazardous scenarios. Theoretical analysis primarily examines the hazards generated by the algorithm module and the rationality of the mathematical modeling. Natural driving data analysis mainly obtains driver behavior through statistical analysis of natural driving data. Based on reasonable mathematical modeling and utilizing natural driving data, this invention obtains a quantitative scheme for the safety performance targets of the planning module for safety design and evaluation.

[0026] [Amended according to Rule 26, dated September 2024] Please refer to Figure 1, which shows a flowchart of a method for quantitatively evaluating the safety performance of an autonomous driving system planning module according to an embodiment of this application. The above method can be applied to the field of safety design and evaluation of autonomous driving system planning modules or the field of autonomous driving system testing. As part of autonomous driving safety, it can be applied to the safety design and evaluation of various levels of autonomous driving system planning modules or to autonomous driving system testing; this application does not limit this application. The entity executing the above method can be a computer device, server, or other device capable of data processing.

[0027] [Revised according to Rule 26, 25.09.2024] As shown in Figure 1, in step 101, the hazard events of the planning module are decomposed into those caused by the planning uncertainties of different algorithm modules of the planning module;

[0028] [According to Rule 26, amended 25.09.2024] In step 102, the probability of a hazardous event caused by the planning uncertainty of the planning module is modeled to obtain driver models under different hazardous events;

[0029] [Revised according to Rule 26, 25.09.2024] In step 103, the quantitative targets corresponding to different algorithm modules of the planning module are obtained at least based on the driver model under different hazardous events.

[0030] [Amended according to Rule 26, 25.09.2024] In this embodiment, for step 101, the hazardous events of the autonomous driving system planning module are decomposed into events caused by the planning uncertainties of different algorithm modules of the planning module. The hazardous events can be events that cause vehicle failure. Different algorithm modules of the planning module can include obstacle prediction, vehicle planning, and intelligent obstacle avoidance. Planning uncertainties can include some or all of the following: predicted missed detections, predicted false detections, planned minimum distance collision time, planned minimum headway, planned minimum safe distance, erroneous active lane changes, and trajectory lateral offset errors. Furthermore, predicted missed detections and predicted false detections belong to the planning uncertainties of obstacle prediction; the planned minimum distance collision time, planned minimum headway, and the planned minimum safe distance belong to the planning uncertainties of vehicle planning; and erroneous active lane changes and trajectory lateral offset errors belong to the planning uncertainties of intelligent obstacle avoidance. This embodiment only models and analyzes some common hazardous scenarios corresponding to planning uncertainties. It is understood that planning uncertainties are not limited to the above examples. Next, in step 102, scenario modeling is performed on the probability of hazardous events caused by various planning uncertainties to obtain driver models under different hazardous events. For example, for predicted missed detections, a model can be built for the hazardous scenarios caused by predicted missed detections, thereby obtaining a driver model for predicted missed detections. Finally, in step 103, the quantitative targets corresponding to different algorithm modules of the planning module can be obtained based on the driver models under different hazardous events. In some embodiments, in addition to using driver models, some driving datasets may also be needed. Driving datasets can be natural driving datasets or driving simulation data (such as data obtained through simulation by a driving simulator). Natural driving datasets may include aerial survey datasets, CitySim datasets, HighD datasets, VisDrone datasets, etc., and this application does not limit this. For example, after obtaining the driver model for predicted missed detections, parameters corresponding to relevant hazardous scenarios can be extracted from the relevant driving datasets. Then, by assigning values ​​to some of these parameters, which can be changed according to the actual situation, the quantitative target corresponding to the predicted missed detections can be obtained. In other embodiments, in addition to using driver models, some vehicle and road parameters may also be needed. For example, after obtaining the driver model of the trajectory lateral offset error, the relationship between the trajectory lateral offset error and vehicle parameters and road parameters can be obtained. Subsequently, by substituting specific vehicles and specific roads, the quantitative target of the trajectory lateral offset error can be obtained.

[0031] [Amended according to Rule 26, September 2024] This application provides a theoretically sound and complete method for quantitatively evaluating the safety performance of an autonomous driving system planning module. Using the method of this application, quantitative safety design indicators for the autonomous driving system planning module can be obtained, providing a clear quantitative reference for the safety design and development of autonomous driving systems. Using the method of this application, performance indicators for each algorithm of the autonomous driving system planning module can be obtained, which can be used to evaluate whether the current safety performance of the autonomous driving system meets the requirements.

[0032] [Amended according to Rule 26 25.09.2024] Please refer to Figure 2, which shows an exploded view of an embodiment of this application.

[0033] [Amended according to Rule 26, September 2024] As shown in Figure 2, this application embodiment adopts a layer-by-layer decomposition method to decompose the vehicle failure problem layer by layer into various algorithm modules. The core idea is that the safety of the autonomous driving system planning module should not be lower than human safety, or not lower than the safety of a better autonomous driving system. First, the causes of vehicle failure are analyzed. Vehicle failure is caused by both hazardous behaviors and related scenarios. For related scenarios, scenario modeling can be performed to obtain regions of interest (specific hazardous scenarios) for different scenarios. Then, hazardous behaviors are analyzed to list specific hazardous behaviors. After that, the causes of hazardous behaviors are analyzed to identify the performance limitations that lead to hazardous behaviors as planning-related performance limitations.

[0034] [Amended according to Rule 26, dated September 2024, 2025] Furthermore, planning-related performance limitations can include planning uncertainty. Planning uncertainty mainly includes the predicted false negative and false positive rates, the minimum planned TTC (Time to Collision), minimum THW (Time Headway), and minimum planned safe distance, the probability of erroneous active lane changes, and performance limitations such as trajectory lateral offset error. TTC, applied to FCW (Forward Collision Warning), represents the distance between two vehicles / the relative speed between the two vehicles. THW represents the distance between two vehicles / the vehicle's speed. In a specific example, when the relative distance is 20m, the speed of the preceding vehicle is 60km / h, and the speed of the following vehicle is 80km / h, THW = 20 / (80 / 3.6) = 0.9s, and TTC = 20 / ((80-60) / 3.6) = 3.6s. The minimum safe distance can range from 0.3m to 3m. The minimum safe distance can also be determined through statistical analysis, which will not be elaborated here.

[0035] [Amended according to Rule 26, dated September 2024] All performance limitations are caused by different algorithm modules, which may include obstacle prediction, vehicle planning, and intelligent obstacle avoidance. By analyzing and modeling performance limitations and calculating them using driving datasets, quantitative indicators of the performance limitations corresponding to different algorithm modules can be obtained, serving as quantitative targets for the safety performance of the corresponding algorithm modules.

[0036] [Amended according to Rule 26, dated September 2024] In some optional embodiments, the hazard event of the planning module is decomposed into events caused by the planning uncertainty of different algorithm modules of the planning module, including: when different algorithm modules of the planning module have planning uncertainty, the planning uncertainty leads to the exposure of hazardous behavior, and the hazardous behavior is uncontrollable, and the resulting accident has a preset severity. In such cases, the planning uncertainty of the planning module is considered to have caused a hazard event. Here, the exposure of hazardous behavior refers to the occurrence of a hazardous behavior, which is an act that will cause harm, and may include unexpected braking, unexpected steering, and late braking. Uncontrollable hazardous behavior means that the hazardous behavior cannot be controlled by humans (e.g., the specific actions of the driver in response to the hazardous behavior cannot be predicted, or in L3 and above autonomous driving, humans do not participate in monitoring). Preset severity refers to the possibility that the accident may cause property damage, personal injury, or death, and may also be other severity levels, which will not be elaborated here. In the context of planning uncertainty in different algorithm modules of the planning module, factors such as the exposure of hazardous behaviors due to planning uncertainty, the uncontrollability of hazardous behaviors after exposure, accidents caused by uncontrollable hazardous behaviors, and the predetermined severity of accidents can all be characterized by corresponding quantitative parameters. This allows the planning uncertainty to be decomposed into specific, quantifiable scenarios. The probability of each scenario occurring can then be obtained from relevant statistical data. Finally, by assigning parameter values ​​and adjusting certain parameters according to actual needs, the probability of hazardous events caused by planning uncertainty in different algorithm modules of the planning module under different practical requirements can be obtained.

[0037] [Amended according to Rule 26, 25.09.2024] Furthermore, by using the method of the embodiments of this application, some specific dangerous scenarios and driver models under dangerous scenarios corresponding to the various planning uncertainties mentioned above can be obtained, providing input and reference for the development of autonomous driving systems. For details, please refer to the following embodiments.

[0038] [Revised from Rule 26 to Rule 25.09.2024] In some optional embodiments, when the planning uncertainty is a predicted missed detection, scenario modeling is performed on the probability of a hazardous event caused by the planning uncertainty of the planning module to obtain driver models under different hazardous events. This includes: when the vehicle in front decelerates in the road or a vehicle in an adjacent lane cuts in, scenario modeling is performed on the collision between the vehicle and the vehicle in front or the vehicle cutting in caused by the predicted missed detection, to obtain a driver model corresponding to the predicted missed detection. Scenario modeling is mainly mathematical modeling, implemented through a program. For example, the corresponding driver model parameters (such as vehicle position changes, speed changes, etc. in the scenario) can be defined by the program. The driver model obtained after modeling can represent the corresponding scenario. For example, the deceleration of the vehicle in front in the road or the cutting in of a vehicle in an adjacent lane can be represented by mathematical modeling; furthermore, a collision between two vehicles can also be considered as the two vehicles being too close, which can be mathematically expressed as TTC < 3s (where 3s is only an example and can be set according to the actual situation). Such scenarios can be extracted from the dataset by writing a program. The quantitative targets for obtaining different algorithm modules of the planning module based on driver models under different hazardous events include: extracting scenarios from the driving dataset corresponding to the predicted missed detections of the driver model; and obtaining the quantitative target corresponding to the predicted missed detection rate of the autonomous driving system based on the premise that the safety of the autonomous driving system is not lower than a safety reference benchmark. In this application, by modeling and analyzing the hazardous scenarios corresponding to the hazardous events caused by predicted missed detections, and then extracting relevant scenarios, the quantitative target corresponding to the predicted missed detections can be obtained based on a set safety reference benchmark, such as the safety of human drivers or the safety of other well-performing autonomous driving systems.

[0039] [According to Rule 26, amended 25.09.2024] Further optionally, the step of extracting the scene of the driver model corresponding to the predicted missed detection from the driving dataset, based on the safety of the autonomous driving system being no less than the safety reference benchmark, to obtain the quantitative target corresponding to the predicted missed detection rate of the autonomous driving system includes: when the vehicle in front of the vehicle decelerates in the road or a vehicle in the adjacent lane cuts in, if the vehicle does not decelerate in advance, it is considered that the vehicle has experienced a predicted missed detection; extracting the scene of the vehicle in front of the vehicle decelerating in the road or the scene of a vehicle in the adjacent lane cutting in from the driving dataset, if the TTC is less than a preset threshold or the distance between the vehicle and the vehicle in front is less than a preset distance, it is considered that the distance between the two vehicles is too close (too close distance is used to characterize a collision, in this scenario it can be considered that the collision is caused by the vehicle not decelerating in advance), and a hazardous scene is considered to exist, and the probability of predicted missed detection is calculated as the quantitative target corresponding to the predicted missed detection rate of the autonomous driving system. In this embodiment of the application, after determining the scenario type of the predicted missed detection, the scenario data of the hazardous scenario corresponding to the predicted missed detection can be extracted. Based on all the cut-in or front vehicle deceleration scenario data, the minimum TTC is assigned a value. Some scenarios that meet the minimum TTC requirement are extracted from the scenario data, and the proportion of the extracted scenarios to all cut-in and front vehicle deceleration scenarios is calculated. The missed detection rate can be obtained, and the quantitative target corresponding to the predicted missed detection can be obtained.

[0040] [Revised according to Rule 26, dated September 2024] Please refer to Figure 3, which shows an example of a scenario where a vehicle cuts into an adjacent lane corresponding to a predicted missed detection.

[0041] [Revised according to Rule 26, September 2024] As shown in Figure 3, when a missed detection is predicted, the relevant hazardous scenarios considered are the vehicle in front slowing down in the road, or a vehicle in the adjacent lane cutting in. Here, "cut in" indicates that the target vehicle is cutting in. Specifically, when another vehicle suddenly cuts into the lane ahead, the autonomous driving system will select the cutting vehicle as the new target vehicle and automatically adjust its speed and distance to maintain a safe distance.

[0042] [Amended according to Rule 26, dated September 2024, 2005] In this embodiment, it is assumed that when a human driver or a well-performing autonomous driving system responds to a cut-in or deceleration scenario, if the driver or system predicts correctly, it will decelerate in advance and determine the speeds of the vehicle and the vehicle in front to maintain a safe distance. If the driver or system predicts incorrectly, it will not take corresponding measures in advance for the cut-in or deceleration behavior, resulting in a hazardous scenario. If the safety reference benchmark is the safety of a human driver or a well-performing autonomous driving system, i.e., the safety of the vehicle's autonomous driving system must be no less than the safety reference benchmark, then the scenario can be directly extracted from the driver dataset (such as an aerial survey dataset). Besides aerial survey datasets, the driver dataset can also use CitySim, HighD datasets, VisDrone datasets, etc. This application does not impose any restrictions on these datasets and will not elaborate further. Subsequently, all cut-in scenarios and preceding vehicle deceleration scenarios can be extracted based on the aerial survey dataset. TTC (Total Traffic Concentration) can be used as the criterion for determining hazardous scenarios (minimum safe distance can also be used, which can be preset, for example, to 0.5m, but will not be elaborated here). If, during a cut-in or preceding vehicle deceleration, the minimum TTC between the vehicle and the preceding vehicle is less than ts (t can be set according to actual needs), or the minimum distance between the vehicle and the preceding vehicle is less than the minimum safe distance, a hazardous scenario exists. This indicates that the human driver or system incorrectly predicted the preceding vehicle's cut-in or deceleration (the human driver or system's prediction of a cut-in or deceleration was missed). The probability of a missed prediction by the human driver can be calculated. For example, as shown in the table below, P... fn Let P represent the probability of missing the predicted cut-in and deceleration of the vehicle in front, which is used as the missed detection rate predicted by the autonomous driving system. When TTC is less than 3s and less than 4s, the missed detection rate P is... fn As shown in the table below.

[0043] [Amended according to Rule 26, dated September 2024] In some optional embodiments, when the planning uncertainty is a prediction misdetection, the driver model under different hazard events is obtained by scenario modeling of the probability of a hazard event caused by the planning uncertainty of the planning module. This includes: when there is a vehicle behind the vehicle on the road, scenario modeling is performed for the collision with the vehicle behind caused by the prediction misdetection, to obtain the driver model corresponding to the prediction misdetection. The corresponding driver model parameters (such as vehicle position changes, speed changes, etc. in the scenario) can be defined programmatically, and the driver model obtained after modeling can represent the corresponding scenario. For example, the presence of a vehicle behind the vehicle on the road can be represented by mathematical modeling, and such scenarios can be extracted from the dataset by writing a program; a collision with a vehicle behind can also be represented by mathematical modeling, which will not be elaborated here. The quantitative target corresponding to different algorithm modules of the planning module based at least on the driver model under different hazard events includes: extracting the scenario of the driver model corresponding to the prediction misdetection from the driving dataset, and obtaining the quantitative target corresponding to the prediction misdetection rate of the autonomous driving system based on the safety of the autonomous driving system being no less than a safety reference benchmark. In this embodiment of the application, by modeling and analyzing the hazardous scenarios corresponding to the hazardous events caused by the predicted false detection, and then extracting the relevant scenarios, the quantitative target corresponding to the predicted false detection can be obtained based on the set safety reference benchmark, such as the safety of human drivers or the safety of other well-performing autonomous driving systems.

[0044] [Amended according to Rule 26, dated September 2024] Further optionally, the step of extracting the scene of the driver model corresponding to the predicted false detection from the driving dataset, based on the premise that the safety of the autonomous driving system is not lower than the safety reference benchmark, to obtain the quantitative target corresponding to the predicted false detection rate of the autonomous driving system includes: when there is a vehicle behind the vehicle on the road, and the vehicle suddenly decelerates without collision risk, it is considered that the vehicle has experienced a predicted false detection; extracting the scene from the driving dataset where there is a vehicle behind the vehicle on the road and the vehicle in front has not cut in while the vehicle decelerates. If the TTC is greater than the preset threshold, it is considered that the distance between the vehicle and the vehicle in front is too large (for example, a large distance between the vehicle and the vehicle in front may indicate that no vehicle is cutting in front). In this scenario, the large distance between the vehicle and the vehicle in front can be attributed to the sudden deceleration of the vehicle. If there is a vehicle behind the vehicle, the sudden deceleration of the vehicle will lead to the vehicle being too close to the vehicle behind, which may cause a collision. Therefore, TTC or minimum safe distance can be used as the indicator for judging false detection. If the TTC is greater than the preset threshold or the distance between the vehicle and the vehicle behind is less than the preset distance, it can be considered that there is a hazardous scenario. The probability of predicted false detection is calculated as the quantitative target corresponding to the prediction miss rate of the autonomous driving system. In this embodiment, after determining the type of predicted false detection scenario, the scenario data of the hazard scenario corresponding to the predicted false detection is extracted. All scenario data of the vehicle having a following vehicle behind it and no vehicle cutting in front while the vehicle decelerates are obtained. In the scenario data, the data of TTC>ts between the vehicle and the vehicle in front (t can be set according to actual needs) is extracted, or the data of the distance between the vehicle and the following vehicle being less than a preset distance (the preset distance can be set according to actual needs) is extracted. The t or preset distance is assigned a value, and the proportion of the extracted scenario to all scenarios of the vehicle having a following vehicle behind it and no vehicle cutting in front while the vehicle decelerates is calculated. The false detection rate can be obtained as the quantitative target corresponding to the predicted false detection.

[0045] [Revised according to Rule 26, dated September 2024, 2005] Please refer to Figure 4, which shows an example of a scenario corresponding to a false positive prediction.

[0046] [Revised according to Rule 26, September 2024] As shown in Figure 4, when a false detection occurs during prediction, the relevant hazardous scenario is considered as follows: This vehicle (car_3) is on the road, and there is a vehicle (car_1) behind this vehicle. False detection can lead to the vehicle braking incorrectly. If this vehicle brakes incorrectly, a rear-end collision will only occur if there is a vehicle behind it; otherwise, a rear-end collision will not occur. Therefore, the hazardous scenario is that there is a vehicle behind this vehicle.

[0047] [Amended according to Rule 26, 25.09.2024] In this embodiment of the application, taking the safety of the human driver as a safety reference benchmark, it is assumed that during normal straight driving, the human driver will maintain a safe relative distance from the vehicle in front. If there is no risk of collision, but the driver misjudges the vehicle in front cutting in, the driver will suddenly decelerate, which may lead to being rear-ended by the vehicle behind. This application embodiment assumes that the safety of an autonomous driving system should be no less than that of a human (or no less than that of a better autonomous driving system). Therefore, scene extraction is performed directly from driver datasets (such as aerial survey datasets). All scenarios where the vehicle is traveling straight and the vehicle in front has not cut in are extracted from the aerial survey dataset. Time-to-Cost (TTC) is used as the criterion for judging hazardous scenarios (minimum safe distance can also be used, which can be preset). It is considered that during normal driving, if the TTC between the vehicle and the vehicle in front is <0s or >ts (t can be set according to actual needs), there is no hazardous scenario, i.e., the TTC is sufficiently large. Sufficiently large TTC has two possibilities: either the distance between the two vehicles is large enough, or the relative speed between the two vehicles is large enough (the vehicle in front is sufficiently faster than the vehicle). In both cases, a collision will not occur. If a human driver decelerates in the absence of a hazardous scenario, it is considered a false prediction, and the probability P of the human driver's false prediction can be calculated. fp For example, as shown in the table below, when TTC is set to greater than 6 seconds, the false detection rate P predicted by the autonomous driving system is... fp Less than 0.097.

[0048] [Amended according to Rule 26, September 2024] In some optional embodiments, it is considered that the safety of the autonomous driving system should be no less than that of human safety (or no less than that of a better autonomous driving system). Therefore, scene extraction is performed directly from the driver dataset (such as an aerial survey dataset). Based on the aerial survey dataset, all scenarios where the vehicle is driving straight and the vehicle in front has not cut in while the vehicle is decelerating are extracted. The minimum safe distance is used as the judgment index for hazardous scenarios (the minimum safe distance can be preset, for example, 0.7m). If, during normal driving, the distance between the vehicle and the vehicle behind is less than the minimum safe distance, a hazardous scenario exists. If the human driver decelerates (the vehicle decelerates) when there is no vehicle in front cutting in, it is considered that an incorrect prediction has occurred. The probability P of the human driver's false detection can be calculated by the proportion of scenarios where the distance between the vehicle and the vehicle behind is less than the minimum safe distance in the extracted scenarios. fp .

[0049] [Amended according to Rule 26, dated September 2024] In some optional embodiments, when the planning uncertainty is any one of the minimum TTC, minimum THW, or minimum safe distance of the plan, scenario modeling is performed on the probability of a hazardous event caused by the planning uncertainty of the planning module to obtain driver models under different hazardous events. This includes: when there is a vehicle ahead in the road, scenario modeling is performed on the collision with the vehicle ahead caused by the minimum TTC, minimum THW, or minimum safe distance of the plan, to obtain a driver model corresponding to the minimum TTC, minimum THW, or minimum safe distance of the plan. The corresponding driver model parameters (such as vehicle position changes, speed changes, etc. in the scenario) can be defined programmatically, and the driver model obtained after modeling can represent the corresponding scenario. For example, the presence of a vehicle ahead in the road can be represented by mathematical modeling, and such scenarios can be extracted from the dataset by writing a program; a collision with the vehicle ahead can also be represented by mathematical modeling, which will not be elaborated here. The quantitative targets corresponding to different algorithm modules of the planning module, based at least on driver models under different hazardous events, include: based on the premise that the safety of the autonomous driving system is not lower than a safety reference benchmark, analyzing the minimum TTC, the planned minimum THW, or the planned minimum safe distance in the driving dataset to obtain the quantitative targets corresponding to the planned minimum TTC, the planned minimum THW, or the planned minimum safe distance of the autonomous driving system. In this embodiment, by modeling potential hazardous scenarios when the planned minimum TTC, the planned minimum THW, or the planned minimum safe distance are set unreasonably, and then extracting relevant data from the driving dataset, the quantitative targets corresponding to the planned minimum TTC, the planned minimum THW, or the planned minimum safe distance can be obtained.

[0050] [Amended according to Rule 26, dated September 2024] Further optionally, the step of analyzing driving data based on the premise that the safety of the autonomous driving system is not lower than a safety reference benchmark, and obtaining the quantitative targets corresponding to the minimum TTC, the planned minimum THW, or the planned minimum safe distance, includes: when there is a vehicle ahead in the road, if the planned minimum TTC, the planned minimum THW, or the planned minimum safe distance causes a collision between the vehicle and the vehicle ahead, then the planned minimum TTC is considered to be... If the planned minimum THW or planned minimum safe distance is incorrectly set, the boundary values ​​corresponding to TTC, THW, or safe distance for each speed range are extracted from the driving dataset. If the TTC, THW, or safe distance is less than the corresponding boundary value, a hazardous scenario is considered to exist. The boundary value corresponding to TTC, THW, or safe distance for each speed range is then used as the planned minimum TTC, THW, or safe distance for that speed range. In this embodiment, when a hazardous scenario occurs due to the planned minimum TTC, THW, or safe distance, it indicates that the relevant data settings are unreasonable. The boundary values ​​corresponding to TTC, THW, or safe distance for each speed range can be extracted from the driving dataset. During this process, some parameters can be assigned values ​​to ensure that the safety of the autonomous driving system is not less than that of a human driver or other better autonomous driving systems, thereby obtaining the planned minimum TTC, THW, or safe distance for each speed range of the autonomous driving system.

[0051] [Revised according to Rule 26, September 2024] Please refer to Figures 5a-5d, which show the specific derivation examples of the scene modeling process and quantification targets corresponding to the minimum TTC and minimum THW.

[0052] [Amended according to Rule 26, 2024] In this embodiment of the application, taking the safety of a human driver as a safety reference benchmark, when the planning has performance limitations, the minimum TTC and minimum THW capabilities that the autonomous vehicle can handle are considered. The relevant scenario is that there is a vehicle in front in the lane. Considering that the capabilities of autonomous vehicles are higher than those of human drivers, the TTC and THW of the autonomous driving system should not be less than the boundary TTC and THW of humans (or not less than the boundary TTC and boundary THW of a better autonomous driving system) to ensure comfortable driving and safety compared to humans. Based on the above principles, by analyzing the driving dataset, based on the threshold (relative safety) of human drivers' TTC and THW, statistical analysis is performed on the minimum TTC and THW of human drivers (refer to Figures 5a and 5b). The minimum TTC and THW at different speed ranges are extracted, and the 10th percentile is taken as the boundary value (a statistical method used to remove outliers). The statistical data of TTC and THW are shown in Figures 5a and 5b.

[0053] [Amended according to Rule 26, 25.09.2024] For example, in this embodiment of the application, a box plot is used to perform statistical analysis on TTC and THW. It is found that the mean and median of TTC and THW do not change much with speed, but the 10th percentile value changes significantly with speed, which is consistent with actual traffic conditions. Moreover, the data distribution of THW and TTC is relatively uniform and reasonable. At the same time, outliers in the data are filtered out. Therefore, the 10th percentile value is taken as the threshold reference value. Its physical meaning is that 90% of drivers will not be less than this threshold. Therefore, if TTC or THW is less than the threshold corresponding to the 10th percentile, the scenario can be considered relatively dangerous. The meaning of each node in the box plot is shown in Figure 5c. In addition to using a box plot, conventional statistical charts such as scatter plots and bar charts can also be used to analyze the data patterns. This application does not limit this.

[0054] [Amended according to Rule 26, 2024] Therefore, if 90% of the drivers' TTC and THW are greater than the boundary value, this scenario is considered a hazardous scenario. The TTC and THW under each speed range in this scenario are taken as the safety design target of the planning module. As shown in the table in Figure 5d, the minimum TTC value and minimum THW value of this vehicle under each speed range in the hazardous scenario are provided by the driving dataset extracted and analyzed in the embodiment of this application.

[0055] [Revised according to Rule 26, 25.09.2024] In other embodiments, similar to the above embodiments, statistical analysis schemes such as box plots can be used to statistically analyze the boundary values ​​corresponding to the safety distance in each speed range as the planned minimum safety distance, which will not be elaborated here.

[0056] [Amended according to Rule 26, dated September 2024] In some optional embodiments, when the planning uncertainty is an erroneous active lane change, scenario modeling is performed to obtain driver models under different hazard events based on the probability of a hazardous event caused by the planning uncertainty of the planning module. This includes: when there is a vehicle behind the target lane, scenario modeling is performed for a collision with the vehicle behind caused by the erroneous active lane change, to obtain a driver model corresponding to the erroneous active lane change. The corresponding driver model parameters (such as vehicle position changes, speed changes, etc. in the scenario) can be defined programmatically, and the driver model obtained after modeling can represent the corresponding scenario. For example, a vehicle behind the target lane can be represented by mathematical modeling, and such scenarios can be extracted from the dataset by writing a program; a collision with a vehicle behind can also be represented by mathematical modeling, which will not be elaborated here. The step of obtaining the quantitative targets corresponding to different algorithm modules of the planning module based at least on driver models under different hazard events includes: extracting the scenario of the driver model corresponding to the erroneous active lane change from the driving dataset, and obtaining the quantitative targets corresponding to the erroneous active lane change of the autonomous driving system based on the safety of the autonomous driving system not being lower than a safety reference benchmark. This application embodiment obtains a driver model corresponding to the erroneous active lane change by performing scenario modeling on the hazardous scenarios caused by erroneous active lane changes. Then, by extracting relevant scenario data from the driving dataset and using statistical analysis to analyze and calculate the extracted data, the quantitative target corresponding to the erroneous active lane change can be obtained.

[0057] [Amended according to Rule 26, 25.09.2024] Further optionally, the step of extracting the scene of the driver model corresponding to the erroneous active lane change from the driving dataset, based on the premise that the safety of the autonomous driving system is not lower than the safety reference benchmark, to obtain the quantitative target corresponding to the erroneous active lane change of the autonomous driving system includes: when there is a vehicle behind in the target lane, if the lane change of this vehicle causes the deceleration of the vehicle behind in the target lane to be greater than or equal to a preset deceleration threshold, then it is considered that the vehicle has performed an erroneous active lane change; extracting first data where the deceleration of the vehicle behind in the target lane is less than the preset deceleration threshold and second data where there is a vehicle behind in the target lane during the lane change of this vehicle from the driving dataset, and obtaining the ratio of erroneous lane changes based on the ratio of the first data and the second data as the quantitative target corresponding to the erroneous active lane change of the autonomous driving system. In the embodiments of this application, when it is determined that the current hazardous scenario is caused by an erroneous active lane change, by comparing the number of times the deceleration of the vehicle behind in the target lane is less than or equal to the preset deceleration threshold and the number of times there is a vehicle behind in the target lane, the ratio can be used as the quantitative target of the erroneous active lane change.

[0058] [Revised according to Rule 26, dated September 2024] Please refer to Figure 6, which shows an example of a scenario corresponding to an erroneous active lane change.

[0059] [Corrected according to Rule 26, 25.09.2024] As shown in Figure 6, considering the limitations of the planning module's erroneous active lane change performance and the potential hazards of vehicles behind the target lane, an erroneous lane change may result in a collision with vehicles behind the target lane.

[0060] [Amended according to Rule 26, dated September 2024] In this embodiment of the application, assuming that during a lane change, the human driver in car_1 causes excessive deceleration of the following car_2 in the target lane, this is considered an erroneous lane change. This type of lane change is considered analogous to erroneous active lane changes by autonomous driving systems. Therefore, the number of erroneous active lane changes by autonomous driving systems should not exceed the number of erroneous lane changes by humans (or not exceed the number of erroneous lane changes by a better autonomous driving system). For example, based on the aerial survey dataset used in this embodiment of the application, data extraction and calculation are performed to extract the events during the lane change of car_1 that cause the deceleration of car_2 to be less than am / s. 2 The data (number of times) is collected, and the data (number of times) of car_2 being behind the target lane during the lane change process of car_1 is extracted. The two are compared to obtain the ratio of wrong lane changes (e.g., 15 times / 1000 times), which is used as a safety indicator for wrong active lane changes. The calculation results are: a=2, 15 times / 1000 times; a=1, 58 times / 1000 times; a=3, 9 times / 1000 times.

[0061] [Revised from Rule 26 to Rule 25.09.2024] In some optional embodiments, when the planning uncertainty is a trajectory lateral offset error, scenario modeling is performed to obtain driver models under different hazard events based on the probability of a hazardous event caused by the planning uncertainty of the planning module. This includes: when there is an obstacle in front of the vehicle, scenario modeling is performed for the collision with the obstacle caused by the trajectory lateral offset error, resulting in a driver model corresponding to the trajectory lateral offset error. The corresponding driver model parameters (such as vehicle position changes, speed changes, etc. in the scenario) can be defined programmatically, and the driver model obtained after modeling can represent the corresponding scenario. For example, an obstacle in front of the vehicle can be represented by mathematical modeling; furthermore, a collision with an obstacle can also be considered as being too close to the obstacle, which can be mathematically represented as TTC < 3s with the obstacle, and such scenarios can be extracted from the dataset by writing a program. The quantification target corresponding to different algorithm modules of the planning module based at least on the driver models under different hazard events includes: obtaining the quantification target corresponding to the trajectory lateral offset error based on the vehicle boundary not encroaching on the obstacle. In this embodiment of the application, by performing scenario modeling on the scene corresponding to the hazardous event caused by the trajectory lateral offset error, the quantitative target corresponding to the trajectory lateral offset error can be obtained.

[0062] [According to Rule 26, amended 25.09.2024] Further optionally, the quantification target corresponding to the lateral offset error of the trajectory based on the vehicle boundary not encroaching on the obstacle includes: when there is an obstacle in front of the vehicle, if the vehicle boundary encroaches on the obstacle, causing the vehicle to collide with the obstacle, then it is considered that a lateral offset error of the trajectory has occurred; based on the vehicle width, lane width, lateral offset distance quantification index and obstacle boundary representation, the constraint that the vehicle boundary cannot encroach on the obstacle is applied, and the relationship between the lateral offset distance quantification index and the vehicle width, lane width and obstacle boundary representation is obtained, and the lateral offset distance quantification index is used as the quantification target of the lateral offset error of the trajectory. In the embodiments of this application, when the vehicle collides with the obstacle due to the vehicle boundary encroaching on the obstacle, it is considered that the collision is caused by the lateral offset error of the trajectory. Then, based on the relationship between the lateral offset error of the trajectory and the vehicle width, lane width and obstacle boundary representation, the actual parameters are substituted into it to obtain the specific quantification target of the lateral offset error of the trajectory.

[0063] [Revised according to Rule 26, dated September 2024] Please refer to Figure 7, which provides a scenario example corresponding to the quantization target of the trajectory lateral offset error.

[0064] [Corrected according to Rule 26, 25.09.2024] As shown in Figure 7, regarding the lateral offset error of the trajectory in the planning module, considering the scenario where there is an obstacle in front of the lane and obstacle avoidance is required, if a lateral deviation of the trajectory occurs, it may lead to a collision with the obstacle.

[0065] [Revised according to Rule 26, 25.09.2024] In this embodiment of the application, the constraint condition of the safety target can be set as the vehicle boundary cannot encroach on obstacles. Therefore, assuming d is the offset error, if d becomes larger, it will collide with other obstacles. Not colliding with other obstacles is the constraint condition of the safety target. The vehicle boundary cannot encroach on obstacles, which needs to satisfy: (Lx)-(Lx) / 2-d>w / 2. Assuming the vehicle width is w, the lane line width is L, x is the obstacle boundary output by freespace (detection), and d is the lateral offset distance quantification index, we simplify to: (Lx) / 2-w / 2>d. d is used as the safety target of the trajectory lateral offset error of the planning module.

[0066] [Amended according to Rule 26, 25.09.2024] Please refer to Figure 8, which shows a flowchart of an embodiment of the present application of a method for energy-based safety assessment of an autonomous driving system.

[0067] [Revised according to Rule 26, 25.09.2024] As shown in Figure 8, in step 801, the quantitative targets corresponding to different algorithm modules of the autonomous driving system planning module are obtained; wherein, the quantitative targets are obtained through the above-mentioned safety performance quantitative evaluation method for the autonomous driving system planning module;

[0068] [According to Rule 26, amended 25.09.2024] In step 802, the safety performance of the autonomous driving system is quantitatively evaluated according to the quantification target to obtain the safety quantification evaluation result.

[0069] [Amended according to Rule 26, 25.09.2024] This application embodiment obtains the quantitative targets obtained by the above-mentioned method for quantitative evaluation of safety performance of the autonomous driving system planning module, and then evaluates the safety performance of the autonomous driving system based on each quantitative target to obtain the safety performance evaluation result. For example, in the testing phase, a vehicle equipped with an autonomous driving system can be used for real-vehicle testing. After obtaining various test data, the quantitative target value is calculated using the test data. When the calculated quantitative target value meets the quantitative target obtained by the above method, the evaluation result is that the system meets the safety requirements (e.g., the false negative rate or false positive rate obtained from the test is less than or equal to the quantitative target of the false negative rate or false positive rate; the planned minimum TTC or planned minimum THW or planned minimum safe distance is greater than or equal to the corresponding quantitative target; the probability of erroneous active lane change or the lateral offset error of the trajectory is less than or equal to the corresponding quantitative target). Otherwise, it does not meet the safety requirements, and the system performance needs to be optimized. In the development phase, simulation software can be used to establish various scenarios to verify the safety performance of the autonomous driving system through simulation. The specific verification process is similar to that in the testing phase. Afterwards, the results can be used to evaluate whether further optimization of the system performance is needed. Furthermore, the evaluation results can also be displayed in a visual chart format to make it easier for users to clearly obtain the system performance evaluation results. For example, highlighting (e.g., highlighting) the data that does not meet the standards or giving textual conclusions about the data that does not meet the standards (e.g., displaying the data that does not meet the standards in a pop-up window), etc., will not be elaborated here.

[0070] [Amended according to Rule 26, 25.09.2024] In some optional embodiments, the method further includes: outputting system improvement suggestions for the autonomous driving system based on the quantitative assessment results of the safety performance. In some specific examples, based on the assessment results (e.g., some values ​​are not up to standard), the planning uncertainty type causing the value to be not up to standard can be determined based on the value that is not up to standard, thereby locating the corresponding algorithm module and outputting prompt information for improving the algorithm module. For example, prediction misses and prediction false detections correspond to the obstacle prediction algorithm module, the planned minimum TTC, the planned minimum THW, and the planned minimum safe distance correspond to the autonomous vehicle planning algorithm module, and erroneous active lane changes and trajectory lateral offset errors correspond to the intelligent obstacle avoidance algorithm module, etc. Based on the data that is not up to standard, it can be matched to one of the above-mentioned algorithm modules, thereby generating prompt information for improving the module, so that the performance of the current autonomous driving system algorithm module can be improved based on the system improvement suggestions (such as prompt information).

[0071] [Amended according to Rule 26, 25.09.2024] The above-described evaluation method of this application embodiment can provide reasonable quantitative indicators for the performance evaluation of autonomous driving systems and provide clear quantitative references for the safety design and development of autonomous driving systems by utilizing the safety design indicators quantified by the autonomous driving system planning module in various scenarios required in the development stage, testing stage, etc.

[0072] [Amended according to Rule 26, 25.09.2024] In some other embodiments, the present invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the safety performance quantitative evaluation method for the planning module of an autonomous driving system in any of the above method embodiments;

[0073] [Amended according to Rule 26, 2005.09.2024] As one embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0074] [Amended according to Rule 26, dated September 2024] The hazardous events of the planning module are decomposed into those caused by the planning uncertainties of different algorithm modules of the planning module;

[0075] [According to Rule 26, amended 25.09.2024] The probability of hazardous events caused by the planning uncertainty of the planning module is modeled in a scenario to obtain driver models under different hazardous events;

[0076] [Amended according to Rule 26, dated September 2024] The quantitative targets corresponding to different algorithm modules of the planning module are obtained at least based on the driver model under different hazardous events.

[0077] [Amended according to Rule 26, September 2024] As another embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0078] [Amended according to Rule 26, dated September 2024] Obtain the quantitative targets corresponding to different algorithm modules of the autonomous driving system planning module obtained according to the safety performance quantitative evaluation method used for the autonomous driving system planning module;

[0079] [Revised according to Rule 26, 25.09.2024] The safety performance of the autonomous driving system is quantitatively evaluated based on the quantitative target to obtain the safety quantitative evaluation result.

[0080] [Amended according to Rule 26, September 2024] A non-volatile computer-readable storage medium may include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, and the stored data area may store data created based on the use of the safety performance quantification evaluation device for the autonomous driving system planning module. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely configured relative to a processor, which can be connected via a network to the safety performance quantification evaluation device for the autonomous driving system planning module. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0081] [Amended according to Rule 26, 25.09.2024] This embodiment of the invention also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to execute any of the above-mentioned methods for quantitative evaluation of safety performance of an autonomous driving system planning module.

[0082] [Amended according to Rule 26, 25.09.2024] Figure 9 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. As shown in Figure 9, the device includes one or more processors 910 and a memory 920. Figure 9 shows an example of one processor 910. The device for the safety performance quantitative evaluation method of the autonomous driving system planning module may further include an input device 930 and an output device 940. The processor 910, memory 920, input device 930, and output device 940 can be connected via a bus or other means. Figure 9 shows an example of connection via a bus. The memory 920 is the aforementioned non-volatile computer-readable storage medium. The processor 910 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 920, thereby implementing the safety performance quantitative evaluation method for the autonomous driving system planning module described in the above-described method embodiment. The input device 930 can receive input digital or character information and generate key signal inputs related to user settings and function control of the safety performance quantitative evaluation device for the autonomous driving system planning module. The output device 940 may include a display screen or other display device.

[0083] [Amended according to Rule 26, dated September 2024] This application also provides a mobile platform, which includes: a vehicle body, a power system, and electronic equipment as described in the above embodiments. The power system is installed on the vehicle body and provides power; the principle and implementation of the electronic equipment are consistent with those described in the above embodiments, and will not be repeated here. The electronic equipment may be a controller or other computing device installed on the mobile platform. Optionally, the mobile platform includes at least one of the following: a vehicle, a mobile robot, or an unmanned vehicle.

[0084] [Amended according to Rule 26, 25.09.2024] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0085] [Amended according to Rule 26, September 2024] As one embodiment, the above-described electronic device is applied in a safety performance quantitative evaluation device for an autonomous driving system planning module, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0086] [Amended according to Rule 26, 2024] The hazard events of the planning module are decomposed into those caused by the planning uncertainty of different algorithm modules of the planning module; the probability of the hazard events caused by the planning uncertainty of the planning module is modeled in the scenario to obtain driver models under different hazard events; and at least based on the driver models under different hazard events, the quantitative targets corresponding to the different algorithm modules of the planning module are obtained.

[0087] [Amended according to Rule 26, September 2024] As another embodiment, the above-described electronic device is applied in a safety performance quantitative evaluation device for an autonomous driving system planning module, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0088] [Amended according to Rule 26, 2024] Obtain the quantitative targets corresponding to different algorithm modules of the autonomous driving system planning module obtained according to the safety performance quantitative evaluation method for the autonomous driving system planning module; and obtain the safety quantitative evaluation result by quantitatively evaluating the safety performance of the autonomous driving system according to the quantitative targets.

[0089] [Amended according to Rule 26, September 2024] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0090] [Amended according to Rule 26, 25.09.2024] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments.

[0091] [Revised according to Rule 26, 25.09.2024] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. [Amended according to Rule 26, dated September 2024] A method for quantitatively evaluating the safety performance of an autonomous driving system planning module, comprising: The hazardous events of the planning module are decomposed into those caused by the planning uncertainties of different algorithm modules of the planning module; The probability of hazardous events caused by the planning uncertainty of the planning module is modeled in a scenario to obtain driver models under different hazardous events; The quantitative targets corresponding to different algorithm modules of the planning module are obtained based on driver models under different hazardous events.

2. [Amended according to Rule 26, 25.09.2024] The method according to claim 1, characterized in that, The decomposition of hazardous events in the planning module into those caused by planning uncertainties in different algorithm modules of the planning module includes: If planning uncertainty occurs in different algorithm modules of the planning module, and the planning uncertainty leads to the exposure of harmful behaviors, and the harmful behaviors are uncontrollable, and the resulting accident has a preset severity, then the planning uncertainty of the planning module is considered to have caused a harmful event.

3. [Amended according to Rule 26, 25.09.2024] The method according to claim 1, characterized in that, The planning uncertainties of the different algorithm modules of the planning module include predicted missed detections, predicted false detections, the planned minimum distance collision time, the planned minimum headway, the planned minimum safe distance, erroneous active lane changes, and trajectory lateral offset errors.

4. [Amended according to Rule 26, 25.09.2024] The method according to claim 3, characterized in that, When the planning uncertainty results in a predicted missed detection, scenario modeling is performed on the probability of a hazardous event caused by the planning uncertainty of the planning module to obtain driver models under different hazardous events, including: When the vehicle in front of the vehicle slows down or a vehicle in an adjacent lane cuts in, a scenario model is performed to model the collision with the vehicle in front or the vehicle cutting in caused by the predicted missed detection, and a driver model corresponding to the predicted missed detection is obtained. The quantitative objectives corresponding to the different algorithm modules of the planning module, which are derived from driver models under at least different hazardous events, include: Scenes corresponding to the driver models in the predicted missed detections are extracted from the driving dataset. Based on the premise that the safety of the autonomous driving system is not lower than the safety reference benchmark, a quantitative target corresponding to the predicted missed detection rate of the autonomous driving system is obtained.

5. [Amended according to Rule 26, 25.09.2024] The method according to claim 4, characterized in that, The step of extracting the scene of the driver model corresponding to the predicted missed detection from the driving dataset, and obtaining the quantitative target corresponding to the predicted missed detection rate of the autonomous driving system based on the safety of the autonomous driving system being no less than the safety reference benchmark, includes: If the vehicle in front of it slows down or a vehicle in an adjacent lane cuts in, and the vehicle fails to slow down in advance, it is considered that the vehicle has missed a prediction. The system extracts scenarios from the driving dataset where the vehicle in front slows down or where a vehicle in an adjacent lane cuts in. If the time to collision is less than a preset threshold or the distance between the vehicle and the vehicle in front is less than a preset distance, the system considers the scenario to be hazardous and calculates the probability of missed detection as the quantitative target corresponding to the missed detection rate of the autonomous driving system.

6. [Amended according to Rule 26, 25.09.2024] The method according to claim 3, characterized in that, When the planning uncertainty results in a false positive, scenario modeling is performed on the probability of a hazardous event caused by the planning uncertainty of the planning module to obtain driver models under different hazardous events, including: When there is a vehicle behind the vehicle on the road, a scenario model is performed for the collision with the vehicle behind caused by the predicted false detection, and a driver model corresponding to the predicted false detection is obtained. The quantitative objectives corresponding to the different algorithm modules of the planning module, which are derived from driver models under at least different hazardous events, include: Scenes corresponding to the driver models in the predicted false detections are extracted from the driving dataset. Based on the premise that the safety of the autonomous driving system is not lower than the safety reference benchmark, a quantitative target corresponding to the predicted false detection rate of the autonomous driving system is obtained.

7. [Amended according to Rule 26, 25.09.2024] The method according to claim 6, characterized in that, The step of extracting the scene of the driver model corresponding to the predicted false detection from the driving dataset, and obtaining the quantitative target corresponding to the predicted false detection rate of the autonomous driving system based on the safety of the autonomous driving system being no less than the safety reference benchmark, includes: If a vehicle suddenly decelerates when there is a vehicle behind it on the road and there is no risk of collision, it is considered that a false alarm has occurred. The system extracts scenarios from the driving dataset where there is a vehicle behind the vehicle on the road and the vehicle in front does not cut in while the vehicle slows down. If the time between the vehicle and the vehicle in front is greater than a preset threshold or the distance between the vehicle and the vehicle behind is less than a preset distance, the system considers the scenario to be hazardous. The probability of false positives is calculated as the quantitative target corresponding to the false negative rate of the autonomous driving system.

8. [Amended according to Rule 26, 25.09.2024] The method according to claim 3, characterized in that, When the planning uncertainty is any one of the planned minimum distance collision time, the planned minimum headway, and the planned minimum safe distance, scenario modeling is performed on the probability of a hazardous event caused by the planning uncertainty of the planning module to obtain driver models under different hazardous events, including: When there is a vehicle in front of the vehicle in the road, the scenario modeling is performed on the collision with the vehicle in front caused by the planned minimum distance collision time, the planned minimum headway, or the planned minimum safe distance, to obtain the driver model corresponding to the planned minimum distance collision time, the planned minimum headway, or the planned minimum safe distance. The quantitative objectives corresponding to the different algorithm modules of the planning module, which are derived from driver models under at least different hazardous events, include: Based on the premise that the safety of the autonomous driving system is no less than the safety reference benchmark, the minimum distance collision time, minimum headway, or minimum safe distance in the driving data are analyzed to obtain the quantitative targets corresponding to the minimum distance collision time, minimum headway, or minimum safe distance planned by the autonomous driving system.

9. [Amended according to Rule 26, 25.09.2024] The method according to claim 8, characterized in that, Based on the premise that the safety of the autonomous driving system is not lower than a safety reference benchmark, the analysis of the minimum distance collision time, minimum headway, or minimum safe distance in the driving dataset yields the quantitative targets corresponding to the minimum distance collision time, minimum headway, or minimum safe distance planned by the autonomous driving system, including: If a collision occurs between the vehicle and the vehicle in front of it due to the planned minimum collision time, the planned minimum headway, or the planned minimum safe distance, then the planned minimum collision time, the planned minimum headway, or the planned minimum safe distance is considered to be incorrectly set. Extract the boundary values ​​corresponding to the time of collision, the time distance to the front of the vehicle, or the safety distance for each speed range from the driving dataset. If the time of collision is less than the corresponding boundary value, the time distance to the front of the vehicle is less than the corresponding boundary value, or the safety distance is less than the corresponding boundary value, then a hazardous scenario is considered to exist. The boundary value corresponding to the time of collision in each speed range is used as the minimum planned time of collision in each speed range, or the boundary value corresponding to the time distance to the front of the vehicle is used as the minimum planned time distance to the front of the vehicle.

10. [Amended according to Rule 26, 25.09.2024] The method according to claim 3, characterized in that, When the planning uncertainty is an erroneous lane change, scenario modeling is performed on the probability of a hazardous event caused by the planning uncertainty of the planning module to obtain driver models under different hazardous events, including: When there is a vehicle behind the target lane, a scenario model is performed for the collision with the vehicle behind caused by the erroneous active lane change, and a driver model corresponding to the erroneous active lane change is obtained. The quantitative objectives corresponding to the different algorithm modules of the planning module, which are derived from driver models under at least different hazardous events, include: Scenes corresponding to the driver models of the erroneous active lane changes are extracted from the driving dataset. Based on the premise that the safety of the autonomous driving system is not lower than the safety reference benchmark, the quantitative target corresponding to the erroneous active lane changes of the autonomous driving system is obtained.

11. [Amended according to Rule 26, 25.09.2024] The method according to claim 10, characterized in that, The process of extracting the scene of the driver model corresponding to the erroneous lane change from the driving dataset, based on the premise that the safety of the autonomous driving system is not lower than the safety reference benchmark, yields the following quantitative targets for the erroneous lane change of the autonomous driving system: When there is a vehicle behind in the target lane, if the lane change of this vehicle causes the deceleration of the vehicle behind in the target lane to be greater than or equal to a preset deceleration threshold, it is considered that the vehicle has made an erroneous active lane change. The system extracts first data (the deceleration of the vehicle behind it in the target lane is less than the preset deceleration threshold) and second data (the presence of a vehicle behind it in the target lane) from the driving dataset. The ratio of the first data and the second data is used as the quantitative target for the erroneous lane change of the autonomous driving system.

12. [Amended according to Rule 26, 25.09.2024] The method according to claim 3, characterized in that, When the planning uncertainty is a lateral trajectory offset error, scenario modeling is performed on the probability of hazardous events caused by the planning uncertainty of the planning module to obtain driver models under different hazardous events, including: When there is an obstacle in front of the vehicle, a scene model is performed on the collision with the obstacle caused by the lateral offset error of the trajectory, and a driver model corresponding to the lateral offset error of the trajectory is obtained. The quantitative objectives corresponding to the different algorithm modules of the planning module, which are derived from driver models under at least different hazardous events, include: Based on the premise that the vehicle boundary cannot encroach on the obstacle, the quantized target corresponding to the lateral offset error of the trajectory is obtained.

13. [Amended according to Rule 26, 25.09.2024] The method according to claim 12, characterized in that, The quantization target for obtaining the lateral offset error of the trajectory based on the premise that the vehicle boundary cannot encroach on the obstacle includes: If there is an obstacle in front of the vehicle on the road, and the vehicle's boundary encroaches on the obstacle, causing the vehicle to collide with the obstacle, then it is considered that a lateral deviation error has occurred. Based on the vehicle width, lane width, lateral offset distance quantification index, and obstacle boundary representation, the vehicle boundary is constrained to not encroach on the obstacle. The relationship between the lateral offset distance quantification index and the vehicle width, lane width, and obstacle boundary representation is obtained, and the lateral offset distance quantification index is used as the quantification target of the trajectory lateral offset error.

14. [Amended according to Rule 26, dated September 2024] A method for quantitatively assessing the safety of an autonomous driving system, comprising: Obtain the quantization targets corresponding to different algorithm modules of the autonomous driving system planning module obtained by the method according to any one of claims 1-13; The safety performance of the autonomous driving system is quantitatively evaluated based on the stated quantitative objectives to obtain a quantitative safety evaluation result.

15. [Amended according to Rule 26, 25.09.2024] The method according to claim 14, characterized in that, The method further includes: Based on the safety quantification assessment results, system improvement suggestions for the autonomous driving system are output.

16. [Amended according to Rule 26, September 25, 2024] An electronic device comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1-15.

17. [Amended according to Rule 26, September 2024] A storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method according to any one of claims 1-15.

18. [Amended according to Rule 26, September 2024] A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1-15.

19. [Amended according to Rule 26, dated September 2024] A mobile platform, characterized in that, include: Body; A power system, installed on the vehicle body, is used to provide power to the mobile platform; An electronic device with a mobile platform according to claim 16.

Images