A scenario test evaluation method and system for expected functional safety
By collecting and processing autonomous driving scenario data, 7-layer factor segmentation and simulation of random parameter method to evaluate the scene subset and generate test cases, the problem of inability to effectively evaluate the expected functional safety of the autonomous driving system in the existing technology is solved, and more efficient development and testing.
Patent Information
- Application Number
- CN202210589873.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-26
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2042-05-26
AI Technical Summary
The prior art cannot effectively cover accident scenarios caused by performance limitations of autonomous driving systems, and lacks effective expected functional safety testing and evaluation methods and systems.
Acquisition and processing of expected functional safety scenario data, including theoretical analysis and natural driving data, through 7-layer feature segmentation and sample augmentation, evaluate subsets of scenes using simulated random parameter method, generate test cases and perform virtual or physical tests to evaluate or correct automatic Driving system.
It improves the efficiency of the expected functional safety development of autonomous driving, provides an effective scenario testing and evaluation method and system, and improves the evaluation and testing capabilities of potential hazard scenarios.
Smart Images

Figure CN114896325B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to automobile intelligent driving technology, and in particular to a scenario testing and evaluation method and system for expected functional safety. Background Art
[0002] Nowadays, autonomous driving is developing towards a higher level of driver assistance systems. In order to meet the safety of road vehicles, the traditional functional safety standard ISO 26262 can no longer cover the analysis of driving scenarios caused by performance limitations. ISO21448 supplements this and makes up for the parts that ISO 26262 cannot cover. ISO21448 defines SOTIF (Safety Of The Intended Functionality) as hazards caused by insufficient intended functionality or reasonably foreseeable misuse. To analyze the causes that may lead to hazards, an effective and operational method and system are needed to effectively test and evaluate scenarios related to the causes of hazards. Summary of the invention
[0003] The technical problem to be solved by the present invention is to provide a scenario testing and evaluation method and system for expected functional safety in view of the defects in the prior art.
[0004] The technical solution adopted by the present invention to solve the technical problem is: a scenario test evaluation method for expected functional safety, comprising the following steps:
[0005] 1) Collecting expected functional safety scenario data and storing it in a database, the scenario data includes: logical scenario data obtained by theoretical analysis and scenario data during natural driving of the driver, the scenario data during natural driving of the driver includes traffic monitoring, accident data, collected driver data, maps and inherent data of road structure;
[0006] 2) Scene data processing:
[0007] 2.1) For the scene data of the driver's natural driving, extract driving scene element information and vehicle automatic driving information from the raw data of the real scene data source, and convert the extracted information into a format that can be analyzed by the database as scene source data;
[0008] The scene source data is segmented based on the 7-layer elements to obtain a series of scene sub-elements A1, A2, ..., A n , and subdivide the sub-elements into a series of subsets based on the defined scene set structure, such as static structure data, weather environment, etc., denoted as A = {A1, A2, A3, ..., A n1},B={B1,B2,B3,……,B n2},…,Z={Z1,Z2,Z3,…,Z ni}, ..., that is, the set of all scene source spaces Ω = {A, B, ..., Z, ...};
[0009] The 7-layer elements include vehicle elements, target elements, traffic facility elements, road elements, weather elements, and communication elements;
[0010] 2.2) For the logical scene data obtained by theoretical analysis, the corresponding scene subsets K1, K2, ..., K are combined in all scene source space sets according to the same division of the scene data in natural driving. n ; K i ={A m ,B t ,……,Z s ,……};
[0011] 3) Expand the sample of the scene source space set;
[0012] Export the data scenarios stored in the database to obtain a single test case subclass, expand and evaluate the subclass using the simulated random parameter method, and retain the samples that pass the evaluation;
[0013] The details are as follows:
[0014] In the scene source set Ω containing all relevant parameter spaces, all relevant key parameter subsets are found in each scene space. The parameters in all relevant scene spaces are randomly changed to simulate all relevant scene subspaces. The change of random parameters is evaluated by the collision probability model with confidence 3σ. If the change of random parameters satisfies the confidence interval of the 3σ model, then the sample is accepted, otherwise it is discarded.
[0015] 4) Based on the expanded data scenario test cases, virtual or physical testing methods are used to evaluate or correct the design of the autonomous driving system.
[0016] According to the above method, the present invention also provides a scenario test evaluation system for expected functional safety, comprising:
[0017] A data acquisition module is used to collect expected functional safety scenario data and store it in a database, wherein the scenario data includes: logical scenario data obtained by theoretical analysis and scenario data during natural driving of the driver;
[0018] The data processing module is used to process the collected scene data: the details are as follows:
[0019] 1) extracting driving scene element information and vehicle automatic driving information from raw data of a real scene data source for scene data of the driver's natural driving, and converting the extracted information into a format that can be analyzed by a database as scene source data;
[0020] The scene source data is segmented based on the 7-layer elements to obtain a series of scene sub-elements, and the sub-elements are subdivided into a series of subsets based on the defined scene set structure, denoted as
[0021] A={A1,A2,A3,……,A n1},B={B1,B2,B3,……,B n2},……,
[0022] Z={Z1,Z2,Z3,……,Z ni}, ..., that is, the set of all scene source spaces Ω = {A, B, ..., Z, ...};
[0023] The 7-layer elements include vehicle elements, target elements, traffic facility elements, road elements, weather elements, and communication elements;
[0024] 2) For the logical scene data obtained by theoretical analysis, the corresponding scene subsets K1, K2, ..., K are combined in all scene source space sets according to the same division of the scene data in natural driving. n ; K i ={A m ,B t ,……,Z s ,……};
[0025] The extension module is used to expand the sample of the scene source space set to expand the data scene test cases;
[0026] The scenario test evaluation module uses virtual or physical testing methods to evaluate or correct the design of the autonomous driving system based on the expanded data scenario test cases.
[0027] According to the above scheme, the scene data of the driver's natural driving in the data acquisition module includes traffic monitoring, accident data, collected driver data, maps and road structure inherent data.
[0028] According to the above scheme, the scene source space set is sampled and expanded in the expansion module, as follows:
[0029] Export the data scenarios stored in the database to obtain a single test case subclass, expand and evaluate the subclass using the simulated random parameter method, and retain the samples that pass the evaluation;
[0030] The details are as follows:
[0031] In the scene source set Ω containing all relevant parameter spaces, all relevant key parameter subsets are found in each scene space. The parameters in all relevant scene spaces are randomly changed to simulate to cope with all relevant scene subspaces. The change of random parameters is evaluated by the collision probability model with confidence level 3σ. If the change of random parameters satisfies the confidence interval of the 3σ model, then the sample is accepted, otherwise it is discarded.
[0032] The beneficial effect of the present invention is as follows: a method for testing and evaluating safety scenarios of expected functions of autonomous driving is provided by utilizing the method of the scenario database system, thereby improving the safety development efficiency of expected functions of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:
[0034] Figure 1 It is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0035] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0036] like Figure 1 As shown, a scenario test evaluation method for expected functional safety includes the following steps:
[0037] 1) Collecting expected functional safety scenario data and storing it in a database, the scenario data includes: logical scenario data obtained by theoretical analysis and scenario data during natural driving of the driver, the scenario data during natural driving of the driver includes traffic monitoring, accident data, collected driver data, maps and inherent data of road structure;
[0038] 2) Scenario data processing: extracting required information from the raw data of the real scenario data source, including driving scenario element information and vehicle travel information, of the scenario data of the driver's natural driving, and converting the information into a format that can be analyzed by the database;
[0039] A scenario to be tested, for example: the vehicle is driving on a city expressway, in moderate rain, at a speed of 70kph, with the cruise function enabled, following the vehicle in front.
[0040] For this scenario, the information that needs to be extracted is the scene slice, which is: urban expressway, moderate rain, and following a vehicle;
[0041] First, refer to the definition of the autonomous driving system ODD to reduce the scene space of the test, and then segment the scene source data based on the 7-layer elements to obtain a series of scene sub-elements, and then subdivide the sub-elements into a series of subsets based on the defined scene set structure, such as static structure data, weather environment, etc., recorded as
[0042] A={A1,A2,A3,……,A n1},B={B1,B2,B3,……,B n2},…,Z={Z1,Z2,Z3,…,Z ni}, ..., that is, the set of all scene source spaces Ω = {A, B, ..., Z, ...};
[0043] The 7-layer elements include vehicle elements, target elements, traffic facility elements, road elements, weather elements, and communication elements;
[0044] 2.2) For the logical scene data obtained by theoretical analysis, the corresponding scene subsets K1, K2, ..., K are combined in all scene source space sets according to the same division of the scene data in natural driving. n ; K i ={A m ,B t ,……,Z s ,……};
[0045] Taking the scenario of "the vehicle is driving on a city expressway, in moderate rain, at a speed of 70 kph, with the cruise function enabled, following the vehicle in front" as an example, the segmentation description of the scene source data is given as follows:
[0046] Step 1: Split the test scene, and focus on the following scene elements: "moderate rain", "urban expressway", and "following vehicle".
[0047] Step 2: The elements collected by the database are identified and extracted from all scenes involving the above scene elements "moderate rain", "urban expressway", and "following a car". In layman's terms, if there are 100 scenes involving moderate rain, then all scenes related to moderate rain are extracted, and the scenes that lead to accidents due to the single factor of "moderate rain" are extracted; similarly, the scenes that lead to accidents due to the single factor of "following a car" are extracted; similarly, the scenes that lead to accidents due to the dual factors of "following a car" and "moderate rain" are extracted.
[0048] Step 3: The final collection of all results is the final test case.
[0049] 3) Expand the sample of the scene source space set;
[0050] Export the data scenarios stored in the database to obtain a single test case subclass, expand and evaluate the subclass using the simulated random parameter method, and retain the samples that pass the evaluation;
[0051] The details are as follows:
[0052] In the scene source set Ω containing all relevant parameter spaces, all relevant key parameter subsets are found in each scene space. The parameters in all relevant scene spaces are randomly changed to simulate all relevant scene subspaces. The change of random parameters is evaluated by the collision probability model with confidence 3σ. If the change of random parameters satisfies the confidence interval of the 3σ model, then the sample is accepted, otherwise it is discarded.
[0053] For example, for "following a car", the following distance can be randomly generated at different distances; for "moderate rain", different wind direction / rain speed / rainfall amount and other parameters can be randomly generated. The purpose is to use this system to randomly generate test examples (the actual collected data must be limited).
[0054] Therefore, for the scenario test evaluation, taking different following distances as an example, the probability of an accident occurring at different following distances is different. If the following distance is 20m, if the collision probability satisfies the 3σ model (confidence interval), then this following distance is considered OK.
[0055] 4) Based on the expanded data scenario test cases, virtual or physical testing methods are used to evaluate or correct the design of the autonomous driving system.
[0056] According to the above method, the present invention also provides a scenario test evaluation system for expected functional safety, comprising:
[0057] A data acquisition module is used to collect expected functional safety scenario data and store it in a database, wherein the scenario data includes: logical scenario data obtained by theoretical analysis and scenario data of the driver's natural driving; the scenario data of the driver's natural driving in the data acquisition module includes traffic monitoring, accident data, collected driver data, maps and inherent data of road structure;
[0058] The data processing module is used to process the collected scene data: the details are as follows:
[0059] 1) extracting driving scene element information and vehicle automatic driving information from raw data of a real scene data source for scene data of the driver's natural driving, and converting the extracted information into a format that can be analyzed by a database as scene source data;
[0060] The scene source data is segmented based on the 7-layer elements to obtain a series of scene sub-elements, and the sub-elements are subdivided into a series of subsets based on the defined scene set structure, denoted as
[0061] A={A1,A2,A3,……,A n1},B={B1,B2,B3,……,B n2},…,Z={Z1,Z2,Z3,…,Z ni}, ..., that is, the set of all scene source spaces Ω = {A, B, ..., Z, ...};
[0062] The 7-layer elements include vehicle elements, target elements, traffic facility elements, road elements, weather elements, and communication elements;
[0063] 2) For the logical scene data obtained by theoretical analysis, the corresponding scene subsets K1, K2, ..., K are combined in all scene source space sets according to the same division of the scene data in natural driving. n ; K1={A|A m ,B|B t ,……,Z|Z s ,……};
[0064] The extension module is used to expand the sample of the scene source space set to expand the data scene test cases;
[0065] The details are as follows:
[0066] Export the data scenarios stored in the database to obtain a single test case subclass, expand and evaluate the subclass using the simulated random parameter method, and retain the samples that pass the evaluation;
[0067] The details are as follows:
[0068] In the scene source set Ω containing all relevant parameter spaces, all relevant key parameter subsets are found in each scene space. The parameters in all relevant scene spaces are randomly changed to simulate to cope with all relevant scene subspaces. The change of random parameters is evaluated by the collision probability model with confidence level 3σ. If the change of random parameters satisfies the confidence interval of the 3σ model, then the sample is accepted, otherwise it is discarded.
[0069] The scenario test evaluation module uses virtual or physical testing methods to evaluate or correct the design of the autonomous driving system based on the expanded data scenario test cases.
[0070] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.
Claims
1. A scenario test evaluation method for expected functional safety, characterized in that: The following steps are involved: 1) Collecting expected functional safety scenario data and storing it in a database, wherein the scenario data includes: logical scenario data obtained by theoretical analysis and scenario data during natural driving of the driver; 2) Scene data processing: 2.1) For the scene data of the driver's natural driving, extract driving scene element information and vehicle automatic driving information from the raw data of the real scene data source, and convert the extracted information into a format that can be analyzed by the database as scene source data; The scene source data is segmented based on the 7-layer elements to obtain a series of scene sub-elements, and the sub-elements are subdivided into a series of subsets based on the defined scene set structure, denoted as A={A1,A2,A3,……,A n1 },B={B1,B2,B3,……,B n2 },...,Z={Z1,Z2,Z3,...,Z ni }, ..., that is, the set of all scene source spaces Ω = {A, B, ..., Z, ...}; The 7-layer elements include vehicle elements, target elements, traffic facility elements, road elements, weather elements, and communication elements; 2.2) For the logical scene data obtained by theoretical analysis, the corresponding scene subsets K1, K2, ..., K are combined in all scene source space sets according to the same division of the scene data in natural driving. n ; K i ={A m ,B t ,……,Z s ,……}; 3) Expand the sample of the scene source space set; 4) Based on the expanded data scenario test cases, virtual or physical testing methods are used to evaluate or correct the design of the autonomous driving system.
2. The scenario test and evaluation method for expected functional safety according to claim 1, characterized in that: The scene data of the driver's natural driving in step 1) includes traffic monitoring, accident data, collected driver data, maps and road structure inherent data.
3. The scenario test and evaluation method for expected functional safety according to claim 1, characterized in that: In step 3), the scene source space set is expanded by samples, specifically as follows: Export the data scenarios stored in the database to obtain a single test case subclass, expand and evaluate the subclass using the simulated random parameter method, and retain the samples that pass the evaluation; The details are as follows: In the scene source set Ω containing all relevant parameter spaces, all relevant key parameter subsets are found in each scene space. The parameters in all relevant scene spaces are randomly changed to simulate to cope with all relevant scene subspaces. The change of random parameters is evaluated by the collision probability model with confidence level 3σ. If the change of random parameters satisfies the confidence interval of the 3σ model, then the sample is accepted, otherwise it is discarded.
4. A scenario test and evaluation system for expected functional safety, characterized in that: include: The data acquisition module is used to collect the expected functional safety scenario data and store it in the database. The scenario data includes: the logical scenario data obtained by theoretical analysis and the scenario data of the driver's natural driving; the data processing module is used to process the collected scenario data: the details are as follows: 1) extracting driving scene element information and vehicle automatic driving information from raw data of a real scene data source for scene data of the driver's natural driving, and converting the extracted information into a format that can be analyzed by a database as scene source data; The scene source data is segmented based on the 7-layer elements to obtain a series of scene sub-elements, and the sub-elements are subdivided into a series of subsets based on the defined scene set structure, denoted as A={A1,A2,A3,……,A n1 },B={B1,B2,B3,……,B n2 },...,Z={Z1,Z2,Z3,...,Z ni }, ..., that is, the set of all scene source spaces Ω = {A, B, ..., Z, ...}; The 7-layer elements include vehicle elements, target elements, traffic facility elements, road elements, weather elements, and communication elements; 2) For the logical scene data obtained by theoretical analysis, the corresponding scene subsets K1, K2, ..., K are combined in all scene source space sets according to the same division of the scene data in natural driving. n ; K i ={A m ,B t ,……,Z s ,……}; The extension module is used to expand the sample of the scene source space set to expand the data scene test cases; The scenario test evaluation module uses virtual or physical testing methods to evaluate or correct the design of the autonomous driving system based on the expanded data scenario test cases.
5. The scenario test and evaluation system for expected functional safety according to claim 4, characterized in that: The scene data of the driver's natural driving in the data acquisition module includes traffic monitoring, accident data, collected driver data, maps and road structure inherent data.
6. The scenario test and evaluation system for expected functional safety according to claim 4, characterized in that: The extension module performs sample expansion on the scene source space set, as follows: Export the data scenarios stored in the database to obtain a single test case subclass, expand and evaluate the subclass using the simulated random parameter method, and retain the samples that pass the evaluation; The details are as follows: In the scene source set Ω containing all relevant parameter spaces, all relevant key parameter subsets are found in each scene space. The parameters in all relevant scene spaces are randomly changed to simulate to cope with all relevant scene subspaces. The change of random parameters is evaluated by the collision probability model with confidence level 3σ. If the change of random parameters satisfies the confidence interval of the 3σ model, then the sample is accepted, otherwise it is discarded.
