Safety evaluation method of autonomous driving system based on behavior-result dual agent model
Through the safety evaluation method of the dual-agent model of behavioral results, the problem of low safety evaluation efficiency of the autonomous driving system in logical scenarios is solved, and the efficient and reliable safety evaluation of the autonomous driving system is achieved. It is suitable for high-level autonomous driving systems. safety testing and evaluation.
Patent Information
- Application Number
- CN202210431404.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-22
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-04-22
AI Technical Summary
The scene-based safety assessment method of existing autonomous driving systems cannot effectively handle the high complexity and uncertainty of the object being tested and its operating environment. The traditional testing methods are inefficient and cannot conduct efficient logical scenario safety assessment.
Using the security assessment method based on the behavioral result dual agent model, accelerated testing is carried out through the behavioral agent model and the result agent model, obtain historical test sample sets, build and update the agent model, divide the logical scene space, calculate the observation value, and conduct adaptive accelerated testing. , identify hazardous subspaces and modalities, and conduct a comprehensive safety assessment.
It improves the efficiency and reliability of safety assessment of autonomous driving systems, and can be effectively applied to safety testing and evaluation of high-level autonomous driving systems, ensuring the accuracy and reliability of evaluation results.
Smart Images

Figure CN114970318B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving technology, and in particular to a safety evaluation method for an autonomous driving system based on a behavior-result dual-agent model. Background Art
[0002] Safety is a difficult problem in the implementation of autonomous driving technology. Safety assessment and demonstration are necessary ways to provide assurance and confidence in the safety of autonomous driving systems. The draft standard ISO 21448 proposes the concept of intended functional safety, which refers to the safety of autonomous driving vehicles in the event of external environmental disturbances, insufficient expected functions, or human misoperation. The "Opinions on Strengthening the Access Management of Intelligent Connected Vehicle Manufacturers and Products" issued by the Ministry of Industry and Information Technology in August 2021 emphasized the importance of strengthening safety management in aspects such as the expected functional safety of autonomous driving vehicles to ensure product quality and production consistency. Therefore, scenario-based testing and evaluation are an important part of the demonstration of expected functional safety.
[0003] At present, scenario-based safety evaluation of high-level autonomous driving systems faces two major problems: First, it ignores the high complexity and uncertainty of the object under test and its operating environment, and follows the evaluation idea of advanced driving assistance system (ADAS), replacing the evaluation of functional and logical scenarios with typical specific scenarios. This "point-for-surface" approach brings misleading evaluation results; second, the complexity of the object under test and its operating environment causes a sharp increase in the parameter dimension describing the test scenario, and traversing the logical scenario parameter space is not realistic. Traditional testing methods (such as grid testing, random testing, orthogonal experiments, etc.) are inefficient and cannot handle high-dimensional test scenarios. Summary of the invention
[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a safety assessment method for an autonomous driving system based on a behavior-result dual-agent model, which can perform a comprehensive safety assessment under certain logical scenarios.
[0005] The purpose of the present invention can be achieved by the following technical solution: A safety evaluation method for an autonomous driving system based on a behavior-result dual-agent model, comprising the following steps:
[0006] S1. Test sampling: Accelerated testing is performed based on the behavior proxy model and the result proxy model. Through test sampling, a historical test sample set of the safety performance of the tested autonomous driving vehicle in logical scenarios is obtained;
[0007] S2. Safety assessment: Based on the historical test sample set, the overall safety of the autonomous driving vehicle under test in the logical scenario is assessed to obtain a comprehensive safety assessment result.
[0008] Furthermore, the behavioral proxy model is specifically a proxy model that performs predictive proxy on the driving trajectory of the autonomous driving vehicle under test in the test scenario, the driving trajectory including the trajectory geometry and the time for the autonomous driving vehicle under test to reach each position on the trajectory, and the input of the behavioral proxy model is an initial scene frame or an initial scene frame sequence; the output is the driving trajectory of the autonomous driving vehicle under test in a future period of time.
[0009] Furthermore, the result proxy model is specifically a proxy model that performs a mapping proxy on the scenario risk of the autonomous driving vehicle under test in the test scenario, and the input of the result proxy model is the initial scene frame; the output is the scenario risk of the test scenario.
[0010] Furthermore, the scene frame refers to the state information of the test scene at a certain instant, and the state information includes the state information of static scene elements and dynamic scene elements.
[0011] Furthermore, the step S1 specifically includes the following steps:
[0012] S11, initializing sampling, collecting a set number of sample points from the logical scene space for testing, and adding the test results to the historical test sample set to obtain initial information for starting the accelerated test cycle;
[0013] S12, construction and update of agent model, based on the historical test sample set, build or update the behavior agent model and the result agent model, and use the result agent model to divide the logical scene space into multiple subspaces;
[0014] S13, proxy sampling, performing fast proxy sampling on the behavior proxy model and the result proxy model to obtain a batch of temporary proxy sampling data;
[0015] S14, value calculation, calculate the observation value of each subspace according to the proxy sampling data, and sort the subspaces from high to low according to the observation value;
[0016] S15, actual test, select several subspaces ranked in the top N positions as target test spaces, sample specific scenarios from them for actual testing, and add the test results to the historical test sample set. If the test cost has been exhausted, exit the loop and output the current historical test sample set as the result; otherwise return to step S12.
[0017] Furthermore, the step S12 specifically includes the following steps:
[0018] S121. Construct a behavior agent model. Before the evaluation begins, use domain data and deep learning models to learn a general driving behavior agent model as the initial model of the behavior agent model.
[0019] S122, updating the behavior proxy model. During the actual evaluation process, calculating the proxy error of the behavior proxy model to the behavior of the autonomous driving vehicle under test, and using the proxy error to fine-tune the model parameters so that the proxy result of the behavior proxy model gradually becomes consistent with the specific autonomous driving vehicle under test;
[0020] S123, constructing a result proxy model, using the logical scene space as the root node to call the unit space division step, recursively calling the unit space division step for the divided subspaces in the same way, and setting a stop condition for the recursive division, creating a hierarchical division tree of the logical scene space as the result proxy model;
[0021] S124, updating the result proxy model, and rebuilding the result proxy model according to the set period.
[0022] Furthermore, the unit space division step in step S123 is specifically as follows: using the mother space as the root node, learning the division surface according to the historical test samples falling in the mother space, dividing the mother space into a dangerous subspace and a non-dangerous subspace, and creating two corresponding subspace nodes on the root node of the mother space.
[0023] Furthermore, the observation value in step S14 consists of two parts: a "development" value represented by the danger level of the scene and an "exploration" value represented by the uncertainty of the observation information.
[0024] Furthermore, the calculation process of the "development" value is as follows: the "development" value of a specific scenario is measured by the scenario risk in a test process, and the "development" value of a scenario subspace is characterized and calculated by the expectation of all specific scenario risks in the subspace;
[0025] The calculation process of the "exploration" value is as follows: the "exploration" value of a scene subspace is measured by the agent model's information uncertainty measure of the subspace, which is composed of uncertainties from two sources: "unobserved" and "unobserved accurate". The uncertainty caused by "unobserved" is calculated and represented by the sampling density calculated based on the distribution of historical test samples in the logical scene space. The "unobserved accurate" uncertainty is calculated and represented by the behavior agent model's agent error of the actual tested autonomous driving vehicle.
[0026] Furthermore, the step S2 specifically includes the following steps:
[0027] S21, construct a tree model, and binarize the labels of the sample points in the historical test sample set obtained in the test sampling phase of step S1 according to the given scenario risk threshold, and divide them into dangerous sample points and non-dangerous sample points; then use the decision tree algorithm to construct a partition tree of the logical scenario space, and divide the logical scenario space into several dangerous and non-dangerous subspaces;
[0028] S22, calculating the hypervolume of the dangerous subspace, according to the decision tree model constructed in step S21, extracting the leaf nodes representing the dangerous subspace, calculating the hypervolume of each dangerous subspace leaf node and summing them up, obtaining the hypervolume of all dangerous subspaces, and calculating the ratio of the hypervolume of the dangerous subspace to the total volume of the entire logical scenario space, obtaining the overall risk of the tested autonomous driving vehicle in the logical scenario;
[0029] S23, dangerous mode identification, based on the decision tree model constructed in step S21, construct an undirected graph according to the spatial connectivity of dangerous leaf nodes, and merge the connected dangerous subspace leaf nodes on the undirected graph to obtain one or more connected components, and use each connected component to represent a dangerous mode in the logical scenario;
[0030] S24, evaluating the reliability of the results, using the behavior proxy model updated in the last iteration of the test sampling phase in step S1, and calculating the proxy error of each subspace;
[0031] Use the sample distribution of the historical test sample set updated in the last iteration of the test sampling phase in step S1 to estimate the sampling density of each subspace;
[0032] By integrating the proxy error and sampling density, we can get the observation value of each subspace. We use the subspace scenario risk expectation as the weight to perform weighted summation on the observation value and get the overall residual exploration value of the logical scenario space to characterize the reliability of the evaluation results.
[0033] Compared with the prior art, the present invention has the following advantages:
[0034] (1) The present invention proposes a behavior proxy model and a result proxy model, and by combining the behavior proxy model and the result proxy model, a scenario-based testing-based autonomous driving system safety assessment method is implemented, which can effectively assess the expected functional safety of a high-level autonomous driving system and comprehensively evaluate the safety of the autonomous driving system in logical scenarios. The use of the behavior proxy model and the result proxy model can improve the efficiency and reliability of safety assessments.
[0035] (2) The present invention aims at the goal of comprehensive safety assessment of autonomous driving systems in a logical scene space. Through adaptive acceleration testing, the dangerous subspace in the logical scene space is efficiently identified, and then the volume ratio and dangerous mode of the dangerous subspace are analyzed and calculated. The reliability of the evaluation results is estimated by calculating the weighted residual exploration value, and a comprehensive safety assessment result is formed. The present invention can be well applied to the safety testing, evaluation and demonstration of autonomous driving systems.
[0036] (3) By introducing the behavioral proxy model and the "pre-training-fine-tuning" technology, the present invention introduces additional test process information and external domain information on the basis of the traditional optimization-based accelerated test that only uses the test result information, thereby achieving a higher accelerated test effect.
[0037] (4) The present invention proposes a comprehensive value model that includes three parts of value: scenario risk, "unobserved" and "unobserved accurate", which can effectively balance "exploration" and "development", accurately guide test sampling, avoid falling into local optimality, and ensure the accuracy of historical test sample sets obtained during the test sampling stage. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 It is a schematic diagram of the method flow of the present invention;
[0039] Figure 2 Schematic diagram of the application process of the embodiment. DETAILED DESCRIPTION
[0040] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0041] Example
[0042] like Figure 1 As shown, a safety evaluation method for an autonomous driving system based on a behavior-result dual-agent model includes the following steps:
[0043] S1. Test sampling: Accelerated testing is performed based on the behavior proxy model and the result proxy model. Through test sampling, a historical test sample set of the safety performance of the tested autonomous driving vehicle in logical scenarios is obtained;
[0044] S2. Safety assessment: Based on the historical test sample set, the overall safety of the autonomous driving vehicle under test in the logical scenario is assessed to obtain a comprehensive safety assessment result.
[0045] The definition of the behavior proxy model is: to perform predictive proxy on the driving trajectory (including trajectory geometry and the time to reach each position on the trajectory) of the autonomous driving vehicle under test (hereinafter referred to as the object under test) in the test scenario. The input of the behavior proxy model is the initial scene frame or the initial scene frame sequence; the output is the driving trajectory of the object under test in the future period of time.
[0046] The definition of the result proxy model is: to perform a mapping proxy on the scenario risk exhibited by the object under test in the test scenario. The input of the result proxy model is the initial scenario frame; the output is the scenario risk index of the test scenario.
[0047] A scene frame refers to the state information of a scene at a certain instant. The state information includes but is not limited to static roads, traffic signs and markings, buildings, etc., as well as dynamic information such as the position, posture, and speed of traffic participants.
[0048] This embodiment applies the above technical solution, such as Figure 2 As shown, it mainly includes two stages: test sampling stage 1 and safety assessment stage 2.
[0049] 1. Test sampling stage. Accelerated testing is performed based on the behavior proxy model and the result proxy model. Through test sampling, relevant information about the safety performance of the object under test in the logical scenario is obtained, that is, the historical test sample set finally obtained by the accelerated test;
[0050] 2. Security assessment phase: Based on the results of the test sampling phase (historical test sample set), the overall security of the object under test in this logical scenario is evaluated.
[0051] The sampling and testing phase mainly includes four steps: initial sampling step, cyclically connected proxy model construction / update, proxy sampling, value calculation, and actual testing. Details are as follows:
[0052] 11. Initialize sampling. Use sampling methods including but not limited to random sampling and low-difference sequence sampling to collect a small number of sample points from the logical scene space for testing, and add the test results to the historical test sample set to obtain the initial information for starting the accelerated test cycle.
[0053] 12. Proxy model construction / update. Build or update the behavior proxy model and result proxy model based on the historical test sample set. Use the result proxy model to divide the logical scene space into several subspaces.
[0054] 13. Proxy sampling. Use sampling methods including but not limited to random sampling, low-discrepancy sequence sampling, or optimization-based sampling methods to perform fast proxy sampling on the behavior proxy model and the result proxy model to obtain a batch of temporary proxy sampling data.
[0055] 14. Value calculation. Based on the proxy sampling data, calculate the observation value of each subspace, and sort the subspaces from high to low according to the observation value.
[0056] 15. Actual test. According to actual needs, select the top-ranked subspace as the target test space, sample specific scenarios from it, conduct actual tests, and add the test results to the historical test sample set. If the test cost has been exhausted, exit the loop and output the final historical test sample set as the result; if not exhausted, jump to step 12 to build / update the proxy model.
[0057] Among them, in the agent model construction / update step 12, the construction and update methods of the behavior agent model and the result agent model are detailed as follows:
[0058] 121. Construction of the behavior proxy model. Before the evaluation begins, use domain data including but not limited to natural driving data, combined with deep learning neural network models including but not limited to CNN, RNN, Transformer, etc., to learn a general driving behavior proxy model as the initial model of the behavior proxy model.
[0059] 122. Update of the behavior proxy model. In the actual evaluation process, the proxy error of the behavior proxy model for the driving trajectory of the measured object is calculated, and the proxy error is used to fine-tune the model parameters so that the proxy results of the behavior proxy model gradually become consistent with the specific measured object.
[0060] 123. Construction of the result proxy model. Use the logical scene space as the root node to call the unit space division step, recursively call the unit space division step in the same way for the divided subspace, and set the stop condition of the recursive division (including but not limited to the minimum number of historical test samples in the subspace, the maximum depth of the division tree branch, etc.), and create a hierarchical division tree of the logical scene space as the result proxy model
[0061] Among them, the steps of dividing a unit space are: using the mother space as the root node, learning the partition surface according to the historical test samples falling in the mother space, dividing the mother space into dangerous subspace and non-dangerous subspace, and creating two corresponding subspace nodes on the root node of the mother space.
[0062] 124. Update of the result proxy model. The result proxy model is rebuilt at a certain period (i.e., at a certain interval of test sampling times), and the rebuilding method is the same as the method of building the result proxy model in step 123.
[0063] In the value calculation step 14, the composition and calculation method of the observation value are as follows:
[0064] 141. Composition of observation value. Observation value consists of two parts: "exploitation" value represented by the degree of danger of the scene and "exploration" value represented by the uncertainty of the observation information.
[0065] 142. Calculation method of “development” value. The “development” value of a specific scenario is measured by the scenario risk in a test process, and can be calculated according to the actual situation using methods including but not limited to TTC, THW, feasible domain, etc.; the “development” value of a scenario subspace is represented and calculated by the expected risk of all specific scenarios in the subspace.
[0066] 143. Calculation method of “exploration” value. The “exploration” value of a scene subspace is measured by the agent model’s information uncertainty measure of the subspace, which is divided into two sources: “unobserved” and “unobserved quasi”. Among them, the uncertainty caused by “unobserved” is characterized by calculating the sampling density based on the distribution of historical test samples in the logical scene space using techniques including but not limited to the kernel density estimator (KDE); the “unobserved quasi” uncertainty is characterized by the agent error of the behavioral agent model on the actual measured object.
[0067] In the safety assessment stage, it mainly includes three steps: the first step of tree model construction, the subsequent step of dangerous subspace hypervolume calculation, dangerous mode identification and evaluation result reliability estimation:
[0068] 21. Tree model construction. According to the given scenario risk threshold, the labels of the sample points in the historical test sample set obtained during the test sampling phase are binarized and divided into dangerous sample points and non-dangerous sample points; then, a decision tree algorithm including but not limited to C4.5 and CART is used to construct a partition tree of the logical scenario space, dividing it into dangerous and non-dangerous subspaces, where each dimension of the divided subspace is parallel to a dimension of the logical scenario space.
[0069] 22. Calculate the hypervolume of the dangerous subspace. Based on the decision tree model obtained in step 21 of the tree model construction, extract the leaf nodes representing the dangerous subspace, calculate the hypervolume of each leaf node of the dangerous subspace and sum them up to obtain the hypervolume of all dangerous subspaces. Calculate the ratio of the hypervolume of the dangerous subspace to the total volume of the entire logical scenario space as the overall risk of the object under test in the logical scenario.
[0070] 23. Dangerous mode identification. Based on the decision tree model obtained in the tree model construction step 21, an undirected graph is constructed according to the spatial connectivity of dangerous leaf nodes; on the undirected graph, algorithms including but not limited to Kosaraju and union-find are used to merge the connected dangerous subspace leaf nodes to obtain one or more connected components, each of which represents a dangerous mode in the logical scenario.
[0071] 24. Evaluation result reliability estimation. Calculate the surrogate error of each subspace using the behavior surrogate model updated in the last iteration round of the test sampling phase, estimate the sampling density of each subspace using the sample distribution of the historical test sample set updated in the last iteration round of the test sampling phase, fuse the surrogate error and the sampling density using methods including but not limited to weighted average, etc., to obtain the observed value of each subspace. Use the subspace scenario risk expectation as the weight to perform a weighted sum of its observed values to obtain the overall residual exploration value of the logical scenario space, and characterize the reliability of the evaluation result. The larger the residual exploration value, the lower the reliability of the evaluation result; the smaller the residual exploration value, the higher the reliability of the evaluation result.
[0072] In this embodiment, the logical scenario is a three-lane straight road. The vehicle controlled by the autonomous driving system under test (hereinafter referred to as the ego vehicle) travels at a certain speed following another vehicle in front (hereinafter referred to as the leading vehicle). At a certain moment, the leading vehicle brakes with a certain deceleration. This logical scenario space is spanned by the following parameters: the initial speeds V0 of the ego vehicle and the leading vehicle, the braking deceleration A of the leading vehicle, and the braking duration T of the leading vehicle. The value ranges of each parameter are as follows: 70 km / h < V0 < 100 km / h, 1 m / s 2 < A < 6 m / s 2 , 0.5 s < T < 3 s. The safety evaluation of the ego vehicle in this logical scenario includes the following two stages:
[0073] Stage 1. Test sampling stage. Use the pre-trained behavior surrogate model and the real-time constructed result surrogate model to guide the test sampling positions and obtain the historical test sample set of the ego vehicle in this logical scenario. Specifically, it includes the following steps:
[0074] Step 11. Initialize sampling. Use the Sobolev low-discrepancy sequence to generate 64 initial sampling point positions, and use the specific scenarios corresponding to these sampling points for testing. Obtain the historical test sample set D;
[0075] Step 12. Surrogate model construction / updating. Calculate the surrogate error of the behavior surrogate model using the historical test sample set D, and use this error to fine-tune and update the behavior surrogate model through the gradient backpropagation algorithm; recursively divide the logical scenario space according to the historical test sample set D. According to the stopping condition, the logical scenario space is sliced into 8 subspaces;
[0076] Step 13. Surrogate sampling. Use a uniform distribution to generate 1000 sampling point positions in each subspace, and perform numerical calculations on these sampling points on the behavior surrogate model and the result surrogate model to obtain 1000 temporary surrogate sampling results. Each result includes the scenario risk of this point, the sampling density of this point, and the surrogate error of this point;
[0077] Step 14, value calculation. Calculate the expected mean of the scenario risk, sampling density and proxy error of all temporary proxy sampling points in each subspace to obtain the overall scenario risk, sampling density and proxy error of each subspace. Use the weighted average method to combine the overall scenario risk, sampling density and proxy error of each subspace to obtain the observation value of each subspace. Sort each subspace according to its observation value, and select the two subspaces with the highest observation value as the target test subspaces;
[0078] Step 15, actual test. Use the Sobolev sequence combined with acceptance-rejection sampling to generate 10 sampling points in each of the two subspaces, use these sampling points for actual testing, obtain 20 new test sample data, and add them to the historical test sample set. If the number of samples in the historical test sample set exceeds 1000 at this time, enter stage 2; otherwise, jump to step 12;
[0079] Phase 2: Safety assessment phase. Based on the more than 1,000 historical test samples obtained in Phase 1, an overall assessment of the safety of the vehicle in the following vehicle logic scenario is conducted. Specifically, the following steps are included:
[0080] Step 21: Tree model construction. Set the threshold of the scenario risk index to 1, screen the historical test samples, mark the scenario risk below 1 as not dangerous, and mark the scenario risk above 1 as dangerous, and obtain the marked historical test sample set D label . Use C4.5 decision tree as model algorithm, D label As training data, a partition tree of the logical scene space is learned;
[0081] Step 22, calculate the hypervolume of the dangerous subspace. Traverse all leaf nodes of the partition tree and select nodes with more than 50% of dangerous marked samples as dangerous subnodes. Calculate the hypervolume of the hypercube represented by each dangerous subnode and sum them up to get the total volume of the dangerous subspace. Calculate the ratio of the total volume of the dangerous subspace to the total volume of the logical scene space, which is 42%. The overall risk index of the vehicle in this logical scene is 0.42;
[0082] Step 23, dangerous mode identification. Investigate the spatial connectivity of all dangerous sub-nodes in the partition tree, build edges between directly connected dangerous sub-nodes, and construct an undirected graph. Apply the Kosaraju algorithm on the undirected graph and finally obtain two non-intersecting connected components. There are two dangerous modes of the vehicle in this logical scenario, which are represented by the identified connected components.
[0083] Step 24: Reliability estimation of the evaluation results. Use the Monte Carlo sampling method to sample and estimate the proxy error of each subspace on the behavioral proxy model. Use the kernel density estimator to calculate the sample density distribution of the historical test sample set, estimate the sampling density of each subspace, and use the weighted average method to fuse the proxy error and sampling density to obtain the observation value of each subspace. Use the risk expectation of each subspace scenario as the weight to perform a weighted summation of its observation value, and obtain a small overall residual exploration value of the logical scenario space, which proves that the reliability of the safety assessment results is acceptable.
[0084] To sum up, this technical solution proposes an autonomous driving system safety assessment method based on a behavior-result dual-agent model, which is used to comprehensively evaluate the safety of the autonomous driving system in logical scenarios. This technical solution uses a behavior agent model and a result agent model to improve the efficiency and reliability of safety assessment, and can be well applied to the safety testing, evaluation and demonstration of the autonomous driving system, accelerating the implementation of autonomous driving technology.
Claims
1. A safety evaluation method for autonomous driving systems based on a behavior-result dual-agent model. It is characterized in that The following steps are involved: S1. Test sampling: Accelerated testing is performed based on the behavior proxy model and the result proxy model. Through test sampling, a historical test sample set of the safety performance of the tested autonomous driving vehicle in logical scenarios is obtained; S2. Safety assessment: Based on the historical test sample set, the overall safety of the tested autonomous driving vehicle in the logical scenario is assessed to obtain a comprehensive safety assessment result; The step S1 specifically includes the following steps: S11, initializing sampling, collecting a set number of sample points from the logical scene space for testing, and adding the test results to the historical test sample set to obtain initial information for starting the accelerated test cycle; S12, construction and update of agent model, based on the historical test sample set, build or update the behavior agent model and the result agent model, and use the result agent model to divide the logical scene space into multiple subspaces; S13, proxy sampling, performing fast proxy sampling on the behavior proxy model and the result proxy model to obtain a batch of temporary proxy sampling data; S14, value calculation, calculate the observation value of each subspace according to the proxy sampling data, and sort the subspaces from high to low according to the observation value; S15, actual test, select several subspaces ranked in the top N as target test spaces, sample specific scenarios from them for actual test, and add the test results to the historical test sample set. If the test cost has been exhausted, exit the loop and output the current historical test sample set as the result; otherwise, return to step S12; Step S12 specifically includes the following steps: S121. Construct a behavior agent model. Before the evaluation begins, use domain data and deep learning models to learn a general driving behavior agent model as the initial model of the behavior agent model. S122, updating the behavior proxy model. During the actual evaluation process, calculating the proxy error of the behavior proxy model to the behavior of the autonomous driving vehicle under test, and using the proxy error to fine-tune the model parameters so that the proxy result of the behavior proxy model gradually becomes consistent with the specific autonomous driving vehicle under test; S123, constructing a result proxy model, using the logical scene space as the root node to call the unit space division step, recursively calling the unit space division step for the divided subspaces in the same way, and setting a stop condition for the recursive division, creating a hierarchical division tree of the logical scene space as the result proxy model; S124, updating the result proxy model, and rebuilding the result proxy model according to the set period; The unit space division step in step S123 is specifically as follows: using the mother space as the root node, learning the partition surface according to the historical test samples falling in the mother space, dividing the mother space into a dangerous subspace and a non-dangerous subspace, and creating two corresponding subspace nodes on the root node of the mother space; In step S14, the observation value consists of two parts: the "exploitation" value represented by the danger level of the scene and the "exploration" value represented by the uncertainty of the observation information; The calculation process of the "development" value is as follows: the "development" value of a specific scenario is measured by the scenario risk in a test process, and the "development" value of a scenario subspace is characterized and calculated by the expected risk of all specific scenarios in the subspace; The calculation process of the "exploration" value is as follows: the "exploration" value of a scene subspace is measured by the agent model's information uncertainty of the subspace, which is composed of uncertainties from two sources: "unobserved" and "unobserved accurate". The uncertainty caused by "unobserved" is calculated and represented by the sampling density calculated based on the distribution of historical test samples in the logical scene space; the "unobserved accurate" uncertainty is calculated and represented by the agent error of the actual tested autonomous driving vehicle by the behavior agent model; Step S2 specifically includes the following steps: S21, construct a tree model, and binarize the labels of the sample points in the historical test sample set obtained in the test sampling phase of step S1 according to the given scenario risk threshold, and divide them into dangerous sample points and non-dangerous sample points; then use the decision tree algorithm to construct a partition tree of the logical scenario space, and divide the logical scenario space into several dangerous and non-dangerous subspaces; S22, calculating the hypervolume of the dangerous subspace, according to the decision tree model constructed in step S21, extracting the leaf nodes representing the dangerous subspace, calculating the hypervolume of each dangerous subspace leaf node and summing them up, obtaining the hypervolume of all dangerous subspaces, and calculating the ratio of the hypervolume of the dangerous subspace to the total volume of the entire logical scenario space, obtaining the overall risk of the tested autonomous driving vehicle in the logical scenario; S23, dangerous mode identification, based on the decision tree model constructed in step S21, construct an undirected graph according to the spatial connectivity of dangerous leaf nodes, and merge the connected dangerous subspace leaf nodes on the undirected graph to obtain one or more connected components, and use each connected component to represent a dangerous mode in the logical scenario; S24, evaluating the reliability of the results, using the behavior proxy model updated in the last iteration of the test sampling phase in step S1, and calculating the proxy error of each subspace; Use the sample distribution of the historical test sample set updated in the last iteration of the test sampling phase in step S1 to estimate the sampling density of each subspace; By integrating the proxy error and sampling density, we can get the observation value of each subspace. We use the subspace scenario risk expectation as the weight to perform weighted summation on the observation value and get the overall residual exploration value of the logical scenario space to characterize the reliability of the evaluation results.
2. According to claim 1, a method for safety evaluation of an autonomous driving system based on a behavior-result dual-agent model, It is characterized in that The behavior proxy model is specifically a proxy model that performs predictive proxy for the driving trajectory of the tested autonomous driving vehicle in the test scenario, wherein the driving trajectory includes the trajectory geometry and the time for the tested autonomous driving vehicle to reach each position on the trajectory, and the input of the behavior proxy model is an initial scene frame or a sequence of initial scene frames; The output is the driving trajectory of the autonomous vehicle under test over a period of time in the future.
3. The method for safety evaluation of an autonomous driving system based on a behavior-result dual-agent model according to claim 1, It is characterized in that The result proxy model is specifically a proxy model that performs a mapping proxy on the scenario risk of the tested autonomous driving vehicle in the test scenario, and the input of the result proxy model is an initial scene frame; The output is the scenario risk for that test scenario.
4. A method for safety evaluation of an autonomous driving system based on a behavior-result dual-agent model according to claim 2 or 3, It is characterized in that The scene frame refers to the state information of the test scene at a certain instant, and the state information includes the state information of static scene elements and dynamic scene elements.
