A method and device for estimating failure probability of an automatic driving system

Abstract

The present application provides a method and device for estimating the failure probability of an autonomous driving system, which relates to the field of autonomous driving technology. The method includes: sampling a uniformly distributed space to obtain initial sampling results, and constructing multiple initial test scenarios based on the initial sampling results; performing simulation tests on the target autonomous driving system in the multiple initial test scenarios to obtain initial simulation test results, and constructing an evaluation index prediction model based on the initial simulation test results and the initial sampling results; determining a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solving the solution target using an optimization algorithm to obtain a reference failure sample set; sampling in a proposed distribution space using the reference failure sample set, and determining a final failure probability based on the final sampling results. By adopting the above-mentioned method and device for estimating the failure probability of an autonomous driving system, the problems of low autonomous driving system verification efficiency and inaccurate failure probability estimation in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more specifically, to a method and device for estimating the failure probability of an autonomous driving system. Background Art

[0002] With the rapid development of autonomous driving systems, verifying and validating their reliability has become crucial. Simulation testing has become a crucial tool for verifying the reliability of autonomous driving systems. During simulation testing, a logical scenario (i.e., the test space) can be generalized into multiple specific scenarios to test the autonomous driving system based on testing needs. Engineers assess the failure probability of the autonomous driving system based on its performance in these specific scenarios. Therefore, generalizing a logical scenario to obtain failure scenarios—that is, exploring the specific scenarios in the test space that could cause the autonomous driving system to fail—has become a crucial issue in verifying autonomous driving systems.

[0003] In the prior art, the first method for obtaining test scenarios is to use a fixed-step generalization method, which generalizes each parameter in the logical scenario with a fixed-step generalization. However, this method has difficulty determining the granularity of generalization, and the number of generalized test scenarios increases exponentially with the increase in scenario parameters, reducing the verification efficiency of the autonomous driving system. Furthermore, this method lacks consideration of the probability distribution of parameters, which may lead to inaccurate failure probability estimates. A second method for obtaining test scenarios is to use Monte Carlo sampling. This method samples according to the probability distribution or joint distribution of each parameter and has the advantage of high robustness, but the system verification efficiency is low and cannot meet the requirements of rapidly verifying the failure probability of autonomous driving systems. A third method for obtaining test scenarios is reliability analysis, which mainly uses algorithms such as directional sampling or importance sampling. While this method has the advantage of high efficiency, it is difficult to set and select algorithm parameters in the absence of prior data experience, which may lead to local optimality and reduce the accuracy of failure probability estimates. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and device for estimating the failure probability of an autonomous driving system, so as to solve the problems of low efficiency in autonomous driving system verification and inaccurate failure probability estimation in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a method for estimating the failure probability of an autonomous driving system, comprising:

[0006] Sampling is performed in the uniform distribution space corresponding to the current logical scenario to obtain initial sampling results, so as to construct multiple initial test scenarios based on the initial sampling results. The uniform distribution space is a parameter value space in which the scenario parameters obey a uniform distribution.

[0007] Conduct simulation tests on the target autonomous driving system in multiple initial test scenarios, obtain initial simulation test results, and build an evaluation index prediction model based on the initial simulation test results and initial sampling results;

[0008] Determine the solution goal for obtaining reference failure samples based on the evaluation index prediction model, and use the optimization algorithm to solve the solution goal to obtain a reference failure sample set. The reference failure sample refers to the sample with the highest probability density among the failure scenarios determined based on the evaluation index prediction model.

[0009] The reference failure sample set is used to sample in the proposed distribution space to obtain the final sampling result, so as to determine the final failure probability of the target autonomous driving system according to the final sampling result.

[0010] Optionally, a solution target for obtaining a reference failure sample is determined based on the evaluation index prediction model, including: using the evaluation index prediction model to construct constraints for limiting failure scenarios; and constructing a solution target that meets the constraints and is used to determine the sample point closest to the origin of the standard normal distribution space.

[0011] Optionally, an optimization algorithm is used to solve the solution target, including: performing high-coverage sampling on the standard normal distribution space to obtain a first sampling result, and applying an optimization algorithm to solve the first sampling result to obtain a solution set; based on the number of elements in the solution set, determining a reference failure sample set that meets the constraint conditions.

[0012] Optionally, the reference failure sample set includes at least one reference failure sample, and the reference failure sample set that meets the constraint conditions is determined based on the number of elements in the solution set, including: if the number of elements in the solution set is one, then the element is used as the reference failure sample; if the number of elements in the solution set is multiple, then the elements in the solution set are clustered using a clustering algorithm to select an element from each clustering result as a reference failure sample; if the number of elements in the solution set is empty, then a heuristic algorithm is used to solve to obtain a reference failure sample.

[0013] Optionally, a final sampling result is obtained by sampling the proposal distribution space using the reference failure sample set, including: if the number of elements in the reference failure sample set is one, determining the proposal distribution as a normal distribution whose expected vector is the element in the reference failure sample set and whose covariance is the identity matrix, and using adaptive importance sampling to sample the proposal distribution space corresponding to the proposal distribution. If the number of elements in the reference failure sample set is multiple, determining the proposal distribution as a mixed Gaussian distribution whose expected vector is each element in the reference failure sample set and whose covariance is the identity matrix, and using adaptive importance sampling to sample the proposal distribution space corresponding to the proposal distribution.

[0014] Optionally, an evaluation index prediction model is constructed based on the initial simulation test results and the initial sampling results, including: determining the initial sampling results as independent variables, the initial simulation test results as dependent variables, and fitting the relationship between the independent variables and the dependent variables to construct the evaluation index prediction model.

[0015] Optionally, after constructing the evaluation index prediction model, the method also includes: evaluating the evaluation index prediction model to determine whether the evaluation index prediction model meets the prediction accuracy requirements; if it does not meet the prediction accuracy requirements, selecting a target value space that affects the model accuracy, and sampling within the target value space to obtain a second sampling result, so as to optimize the evaluation index prediction model according to the second sampling result.

[0016] Optionally, the evaluation index prediction model is optimized in the following manner, including: performing a simulation test on the target autonomous driving system according to the second sampling result to obtain a second simulation test result; and optimizing the evaluation index prediction model using the second sampling result and the second simulation test result.

[0017] Optionally, sampling is performed in the uniform distribution space corresponding to the current logical scenario, including: sampling in the uniform distribution space using a Latin hypercube sampling method based on maximizing minimum distance optimization.

[0018] In a second aspect, an embodiment of the present application further provides a device for estimating the failure probability of an autonomous driving system, the device comprising:

[0019] The initial scenario construction module is used to sample in the uniform distribution space corresponding to the current logical scenario to obtain initial sampling results, so as to construct multiple initial test scenarios based on the initial sampling results. The uniform distribution space is a parameter value space in which the scenario parameters obey a uniform distribution.

[0020] A prediction model building module is used to simulate and test the target autonomous driving system in multiple initial test scenarios, obtain initial simulation test results, and build an evaluation index prediction model based on the initial simulation test results and initial sampling results;

[0021] An optimization solution module is used to determine a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solve the solution target using an optimization algorithm to obtain a reference failure sample set. The reference failure sample refers to the sample with the highest probability density among the failure scenarios determined based on the evaluation index prediction model.

[0022] The failure probability estimation module is used to use the reference failure sample set to sample in the proposed distribution space to obtain a final sampling result, so as to determine the final failure probability of the target autonomous driving system according to the final sampling result.

[0023] The embodiments of the present application bring the following beneficial effects:

[0024] The present invention provides a method and device for estimating the failure probability of an autonomous driving system. These methods utilize an optimization algorithm to solve a target problem and, through adaptive importance sampling, acquire test scenarios to specifically identify scenarios that could cause the target autonomous driving system to fail. This improves the verification efficiency of the autonomous driving system and enables efficient and accurate failure probability estimation. Furthermore, an evaluation index prediction model is used to explore the parameter value space, providing prior experience and avoiding the inefficiency associated with direct simulation testing. This simplifies the difficulty of setting and selecting algorithm parameters, further enhances the ability to break through local optima during autonomous driving system verification, and improves the accuracy of failure probability estimation. Compared to existing autonomous driving system failure probability estimation methods, this method addresses the issues of low autonomous driving system verification efficiency and inaccurate failure probability estimation.

[0025] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A flowchart of a method for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application is shown;

[0028] Figure 2 A flow chart of a target solving method provided in an embodiment of the present application is shown;

[0029] Figure 3 A schematic diagram showing the structure of a device for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application is shown;

[0030] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0032] It is worth noting that prior to the filing of this application, with the rapid development of autonomous driving systems, verifying and validating their reliability became crucial. Autonomous driving simulation testing has become an important means of verifying the reliability of autonomous driving systems. During simulation testing, a logical scenario (i.e., the test space) can be generalized into multiple specific scenarios for testing the autonomous driving system based on testing needs. Engineers then assess the failure probability of the autonomous driving system based on the performance of the tested autonomous driving system in these specific scenarios. Therefore, generalizing a logical scenario to obtain failure scenarios—that is, exploring specific scenarios in the test space that cause the tested autonomous driving system to fail—has become a top priority for verifying autonomous driving systems. In the prior art, the first method for obtaining test scenarios is to use a fixed-step generalization method, which generalizes each parameter in the logical scenario with a fixed step. However, this method has difficulty determining the generalization granularity, and the number of generalized test scenarios increases exponentially with the increase in scenario parameters, reducing the verification efficiency of the autonomous driving system. Furthermore, this method lacks consideration of the probability distribution of parameters. This may result in finding scenarios that cause the autonomous driving system to fail, but it is difficult to estimate the failure probability and the safe operating range of the system, resulting in inaccurate failure probability estimates. The second method for obtaining test scenarios is Monte Carlo sampling. This method samples according to the probability distribution or joint distribution of each parameter. This method has the advantage of high robustness, but the verification efficiency is low and it cannot meet the requirements of quickly verifying the failure probability of autonomous driving systems. The third method for obtaining test scenarios is reliability analysis. This method mainly involves using algorithms such as directional sampling or importance sampling. Its advantage is high efficiency, but in the absence of prior data experience, the algorithm parameters are difficult to set and select, which may lead to local optimality and reduce the accuracy of failure probability estimation.

[0033] The terms involved in the embodiments of this application are explained below.

[0034] Scenario parameters: Elements in the simulation scenario, whose values ​​obey certain specified distribution functions, that is, the space composed of all parameters can be understood as a Euclidean space composed of a set of random variables.

[0035] Logical scenarios: These express entity characteristics and relationships by defining parameter ranges and distribution functions for variables within a state space. Logical scenarios are detailed descriptions of functional scenarios based on state space variables and are used to generate requirements during the project development phase. For each logical scenario with a range of values, any number of specific scenarios (test scenarios) can be derived.

[0036] Test scenario: A scenario that can be directly simulated and tested, that is, a scenario obtained after all scenario parameters in the logical scenario are assigned values.

[0037] Test failure: The performance of an autonomous driving system in an environment variable X can be represented by an evaluation function g(X). An autonomous driving system test failure can be represented as g(X) ≤ T, where T = 0 for the sake of generality. Therefore, the variable value that causes an autonomous driving system test failure can be defined as {X|g(X) ≤ 0}. Furthermore, {X|g(X) > 0} represents the range of the autonomous driving system's safe operating capabilities.

[0038] Failure probability: In many fields such as autonomous driving, aerospace, and equipment manufacturing, an important task is to strictly verify whether the probability of system failure is less than a specified threshold, that is, to estimate the probability of system failure p in a specified probability distribution space. f =P(g(X)≤0). Let f(x) and μ(x) represent the density function and measure function of X in the probability distribution space, respectively. Then we have:

[0039] p f =P(g(X)≤0)=∫ g(x)≤0 f(x)dμ(x).

[0040] Due to the complexity and unknown nature of the black box function g(X), the above integral cannot be calculated directly, and statistics are often constructed And the failure probability of the system is estimated through sampling results. Statistics The quality of an estimate can be measured by its accuracy (bias, relative bias, whether it is unbiased or asymptotically unbiased), precision (standard deviation, coefficient of variation, or relative error), and efficiency. Accuracy measures whether the estimate is equal to the true value over the long term; precision measures the variability of the estimate, that is, the variability between estimates; and efficiency measures the sample size required to achieve the same accuracy.

[0041] The most common method for the above problem is to use the Monte Carlo method to estimate, that is:

[0042]

[0043] Among them, the Monte Carlo method is an unbiased estimate, and its relative error is

[0044] Based on this, an embodiment of the present application provides a method for estimating the failure probability of an autonomous driving system to improve the verification efficiency of the autonomous driving system and the accuracy of the failure probability estimation.

[0045] See also Figure 1 , Figure 1 This is a flow chart of a method for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application. Figure 1 As shown, the method for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application includes:

[0046] Step S101: sampling in a uniformly distributed space corresponding to the current logical scenario to obtain initial sampling results, and constructing multiple initial test scenarios based on the initial sampling results;

[0047] Step S102: performing simulation tests on the target autonomous driving system in a plurality of initial test scenarios, obtaining initial simulation test results, and constructing an evaluation index prediction model based on the initial simulation test results and the initial sampling results;

[0048] Step S103: determining a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solving the solution target using an optimization algorithm to obtain a reference failure sample set;

[0049] Step S104 : Sampling is performed in the proposed distribution space using the reference failure sample set to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

[0050] The method for estimating the failure probability of an autonomous driving system provided in the embodiments of the present application utilizes an optimization algorithm to solve the target solution and, through adaptive importance sampling, acquires test scenarios to specifically identify scenarios that could cause the target autonomous driving system to fail. This improves the verification efficiency of the autonomous driving system and enables efficient and accurate failure probability estimation. Furthermore, the method utilizes an evaluation indicator prediction model to explore the parameter value space, providing prior experience and avoiding the inefficiency of direct simulation testing. This simplifies the difficulty of setting and selecting algorithm parameters and further enhances the ability to break through local optima during autonomous driving system verification, improving the accuracy of failure probability estimation and addressing the issues of low autonomous driving system verification efficiency and inaccurate failure probability estimation.

[0051] To facilitate understanding of this embodiment, each of the above exemplary steps provided in the embodiment of the present application is described below.

[0052] In step S101 , sampling is performed in a uniformly distributed space corresponding to the current logical scenario to obtain initial sampling results, so as to construct a plurality of initial test scenarios according to the initial sampling results.

[0053] In this step, the current logical scenario may refer to a logical scenario used to test the target autonomous driving algorithm. Multiple test scenarios can be generalized through the current logical scenario, and the current logical scenario includes multiple scenario parameters.

[0054] The uniform distribution space may refer to a parameter value space in which scene parameters obey a uniform distribution. The uniform distribution space is a multidimensional space, the dimension of which is determined by the number of scene parameters, and the range of the uniform distribution is determined by the range of values ​​in the original distribution.

[0055] In an embodiment of the present application, before testing the performance of the target autonomous driving system in the current logical scenario, it is first necessary to generalize multiple test scenarios corresponding to the current logical scenario. In order to better explore the entire value space of the logical scenario, this sampling does not consider the true distribution function and correlation coefficient of the scene parameters, but only considers the value range of the scene parameters. To this end, a high-coverage sampling can be performed in the uniform distribution space using a Latin hypercube sampling method based on maximizing the minimum distance optimization. This sampling method can maximize the minimum distance between sample points so that the sampling points are evenly distributed in the entire uniform distribution space, so as to more comprehensively explore the entire value space. Among them, the amount of samples collected is a set parameter, which can be determined according to the actual computing power and the dimension of the uniform distribution space; the value range of each scene parameter is determined based on real data, for example: the value range of each scene parameter can be determined based on road sampling data.

[0056] In one example, when the value range of a scene parameter is infinite, a subinterval is set as the value range of the scene parameter, for example, the value of the distribution function is 10 -8 to 1-10 -8 The corresponding value interval is used as the value range of the scene parameter.

[0057] After sampling the uniform distribution space, the initial sampling results are obtained, which are recorded as: X1,...,X n Here, each element X in the initial sampling result i Represents a sample point. Each sample point represents a scenario parameter combination and the value of each scenario parameter in the scenario parameter combination. A test scenario can be constructed based on each sample point. In this way, a total of n test scenarios can be constructed based on the initial sampling results. Among them, each scenario parameter combination includes all scenario parameters under the current logical scenario.

[0058] In the embodiment of the present application, the high coverage sampling of the uniform distribution space is actually the use of importance sampling, that is, using the uniform distribution as the proposed distribution and ignoring the sampling under the correlation between the scene parameters.

[0059] In step S102, simulation tests are performed on the target autonomous driving system in multiple initial test scenarios to obtain initial simulation test results, and an evaluation index prediction model is constructed based on the initial simulation test results and the initial sampling results.

[0060] In this step, for each initial test scenario, the target autonomous driving system is simulated in that initial test scenario to obtain an initial simulation test result for the target autonomous driving system in that initial test scenario. The initial simulation test result is the value of the evaluation index of the target autonomous driving system in that initial test scenario. The evaluation index is pre-set based on the scenario test requirements.

[0061] The initial simulation test results include multiple ones, the number of which corresponds to the number of initial test scenarios. The initial simulation test results are recorded as: g(X1),...,g(X n ).

[0062] In an embodiment of the present application, after constructing multiple initial test scenarios, it is also possible to determine whether the number of failure scenarios in the initial test scenarios meets the scenario quantity condition, for example: determining whether the number of failure scenarios in the initial test scenarios is greater than or equal to a set number threshold. If it is greater than or equal to the set number threshold, it means that there are enough failure scenarios to estimate the failure probability, and it is determined that the scenario quantity condition is met. At this time, these initial test scenarios can be directly used to simulate the target autonomous driving algorithm, and the failure probability can be determined based on the initial simulation test results. The formula for determining the failure probability is:

[0063]

[0064] In the above formula, represents an estimator of the failure probability. It is also an unbiased estimator; V represents the volume or area of ​​the uniform distribution space (when the uniform distribution space is two-dimensional, it is called area, and when the uniform distribution space is multi-dimensional, it is called volume); f(X i ) represents the scenario parameter combination (test scenario) X i The probability density under the original probability distribution, the original probability distribution can refer to the true probability distribution of the scene parameters, and the original probability distribution can reflect the correlation between different scene parameters; n represents the sample size.

[0065] If the number is less than the set threshold, it means that there are not enough failure scenarios to estimate the failure probability, then it is determined that the scenario quantity condition is not met, and the step of constructing an evaluation index prediction model based on the initial simulation test results and the initial sampling results is executed.

[0066] In one example, when constructing an evaluation index prediction model, the initial sampling result can be determined as an independent variable, the initial simulation test result can be used as a dependent variable, and a curve fitting is performed on the relationship between the independent variable and the dependent variable to construct an evaluation index prediction model. The evaluation index prediction model is used to characterize the relationship between the initial sampling result X and the evaluation index g(X).

[0067] In another example, when constructing an evaluation index prediction model, the initial evaluation index prediction model can be trained using the initial sampling results and the initial simulation test results, so that the trained initial evaluation index prediction model is determined as the final constructed evaluation index prediction model. The initial evaluation index prediction model can be a neural network model.

[0068] In one example, in order to determine whether the prediction results of the evaluation index prediction model meet the accuracy requirements, after constructing the evaluation index prediction model, the evaluation index prediction model can also be evaluated to determine whether the evaluation index prediction model meets the prediction accuracy requirements. Here, the evaluation index prediction model can be measured based on the prediction coefficient (Coefficient of Prognosis, CoP) of the combined test to determine whether it meets the prediction accuracy requirements. For example: the initial sampling result is input into the evaluation index prediction model to obtain the predicted evaluation index value output by the evaluation index prediction model, and the predicted evaluation index value and the initial simulation test result obtained by the actual simulation test are input into the Cop calculation formula to obtain the prediction coefficient, and the prediction coefficient is compared with the accuracy threshold. If the prediction coefficient is less than or equal to the accuracy threshold, it is determined that it does not meet the prediction accuracy requirements. If the prediction coefficient is greater than the accuracy threshold, it is determined that it meets the prediction accuracy requirements. Among them, the calculation formula of the prediction coefficient is as follows:

[0069]

[0070] In the above formula, CoP represents the prediction coefficient; g(X i ) represents the initial simulation test results; Represents the prediction evaluation index value; μ g represents the mean of the initial simulation test results; test represents the test set, that is, all initial sampling results.

[0071] If the prediction accuracy requirements are met, the evaluation indicator prediction model is directly used to construct the solution target. If the prediction accuracy requirements are not met, a target value space that affects the model accuracy is selected, and sampling is performed within the target value space to obtain a second sampling result. The evaluation indicator prediction model is then optimized based on the second sampling result. The target value space can refer to the space corresponding to the point with the largest error selected in the original value space. The original value space refers to the original space that only considers the value range and does not consider the probability distribution.

[0072] In an embodiment of the present application, based on the output results of the evaluation index prediction model and the initial simulation test results, a sample point with a larger error can be determined. If the standard deviation corresponding to the sample point is greater than the set error threshold, it means that the space near the sample point has a greater impact on the model accuracy. In this case, the sample point is selected as the target sample point, and the space within a preset range around the target sample point in the original value space is selected as the target value space. For example: in the original value space, a spherical space with the target sample point as the center and a radius within the set value is selected as the target value space.

[0073] Then, a second sampling result is obtained through adaptive sampling within the target value space. In one example, Bayesian estimation can be used for adaptive sampling. A simulation test is performed on the target autonomous driving system based on the second sampling result to obtain a second simulation test result. Finally, the evaluation index prediction model is optimized using the second sampling result and the second simulation test result. For example, the initial sampling result and the second sampling result are used as the latest inputs, and the initial simulation test result and the second simulation test result are used as the latest outputs. The evaluation index prediction model is refitted using the latest inputs and the latest outputs to obtain an optimized evaluation index prediction model.

[0074] After obtaining the optimized evaluation index prediction model, it is determined again whether the optimized evaluation index prediction model meets the prediction accuracy requirements. If it still does not meet the requirements, the above steps are repeated to continue optimizing the evaluation index prediction model until the optimized evaluation index prediction model meets the prediction accuracy requirements.

[0075] In one example, after the evaluation index prediction model meets the prediction accuracy requirements, since the subsequent reliability analysis algorithm will be calculated based on the standard normal distribution space, the optimized evaluation index prediction model needs to be converted to the standard normal distribution space, that is, to obtain the evaluation index prediction model under the standard normal distribution space. The evaluation index prediction model under the standard normal distribution space is recorded as: Here, X = d(Z) represents converting a random number Z in the standard normal distribution space into a random number X in the original distribution space. The original distribution space is the original space that considers the probability distribution function. The original space is a space with correlations between different scenario parameters constructed based on real road data. Both the random number Z and the random number X are vectors.

[0076] In step S103, a solution target for obtaining reference failure samples is determined based on the evaluation index prediction model, and the solution target is solved using an optimization algorithm to obtain a reference failure sample set.

[0077] In this step, the reference failure sample refers to the sample with the largest probability density in the failure scenario determined based on the evaluation index prediction model, that is, the reference failure sample is the failure sample closest to the origin in the standard normal distribution space. The closer to the origin, the greater the probability density, and the larger the corresponding probability likelihood function value; the farther from the origin, the smaller the probability density, and the smaller the corresponding probability likelihood function value. One failure sample corresponds to one failure scenario. The reference failure sample is denoted as: Z * .

[0078] The reference failure sample set may refer to a set of reference failure samples, and the reference failure samples are also called the design point (Most Probable Point, MPP).

[0079] The algorithm for obtaining reference failure samples can be expressed as: Z * =argmin Z {||Z|| 2 ,h(Z)≤0}, and h(·) represents the evaluation metric of the target autonomous driving system. Although placing the center of the importance sampling proposal distribution at the MPP often allows for direct convergence to obtain an accurate failure probability, using optimization algorithms to calculate the MPP often requires a high computational load for the black-box function h(x) in complex spaces, and optimization algorithms cannot be parallelized. These issues make it impossible to obtain h(x) through simulation testing.

[0080] Below through Figure 2 Let’s introduce the target solving process.

[0081] Figure 2 A flow chart of the target solving method provided in the embodiment of the present application is shown as follows: Figure 2 As shown in Figure 2, the target solution method includes:

[0082] Step S201: using the evaluation index prediction model, constructing constraint conditions for limiting failure scenarios.

[0083] Here, the constraint condition is used to limit the failure samples corresponding to the failure scenario determined by the evaluation index prediction model. The constraint condition can be expressed as:

[0084] Step S202 : constructing a solution target that meets the constraint conditions and is used to determine the sample point closest to the origin of the standard normal distribution space.

[0085] The solution goal can be expressed as: Find Z * to min{||Z|| 2}, Such that

[0086] It should be noted that The nature of is unknown, and the above solution target may have multiple solution intervals, that is, the target autonomous driving system has multiple failure domains. Therefore, a simple gradient algorithm cannot solve the above solution target. To this end, this application uses the evaluation index prediction model Solve the problem instead of performing simulation tests and use the simulation test results to solve the problem to improve the calculation speed of the constraints.

[0087] Step S203 , performing high coverage sampling on the standard normal distribution space to obtain a first sampling result, and applying an optimization algorithm to solve the first sampling result to obtain a solution set.

[0088] Specifically, a high coverage sampling can be performed on a subspace of the standard normal distribution space to obtain a first sampling result. The subspace can refer to a distribution function with a value of 10 -10 to 1-10 -10 The corresponding interval, and the sample size in the sampling process is determined by the dimension of the standard normal distribution space. The higher the spatial dimension, the larger the sample size, and the lower the spatial dimension, the smaller the sample size. Then, based on the first sampling result, a gradient-based nonlinear optimization algorithm is used to solve the solution target to determine the set of solutions that meet the constraints as the solution set, and each element in the solution set is a solution. Among them, the standard normal distribution space may refer to the space in which the scenario parameters obey the standard normal distribution, and the gradient-based nonlinear optimization algorithm may refer to the NLPQL algorithm. It should be noted that when there is a failure scenario in the initial sampling result, the sample points corresponding to the failure scenario are also added to the solution set.

[0089] Step S204: Determine a reference failure sample set that meets the constraint conditions according to the number of elements in the solution set.

[0090] If the number of elements in the solution set is one, then the element is used as a reference failure sample; if the number of elements in the solution set is multiple, then a clustering algorithm is used to cluster the elements in the solution set, for example: an adaptive DBSCAN algorithm is used to cluster the elements in the solution set, and an element is selected from each clustering result as a reference failure sample, for example: if three categories are obtained by clustering, then the sample point closest to the origin of the standard normal distribution space is selected from each category as a reference failure sample, and a total of three reference failure samples are obtained; if the number of elements in the solution set is empty, then a heuristic algorithm is used to solve the solution target to obtain a reference failure sample, for example: a genetic algorithm and a hippopotamus algorithm are used for the solution, and the input of the heuristic algorithm is the solution target and the standard normal distribution space.

[0091] In step S104 , a reference failure sample set is used to perform sampling in the proposed distribution space to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

[0092] In this step, the proposed distribution space may refer to the parameter value space in which the scene parameters obey the proposed distribution function.

[0093] If the number of elements in the reference failure sample set is one, that is, there is only one Z * , the proposed distribution function is determined as a normal distribution with the expected vector as an element in the reference failure sample set and the covariance as the identity matrix. Adaptive importance sampling is used to sample the proposal distribution space corresponding to the proposed distribution function. Test scenarios are constructed based on the sample points obtained by sampling, and the target autonomous driving algorithm is simulated in these test scenarios to determine the failure probability based on the simulation test results. It should be noted that the failure region has been found at this point. The update strategy for the proposed distribution function during the adaptive importance sampling process is to update all parameters, including the covariance, to improve algorithm accuracy.

[0094] If there are multiple elements in the reference failure sample set, the proposed distribution function is determined as a mixed Gaussian distribution whose expected vector is each element in the reference failure sample set and whose covariance is the identity matrix, and the proposal distribution space corresponding to the proposed distribution is sampled using adaptive importance sampling. For example: the multiple elements in the reference failure sample set are That is, there are multiple failure domains, and the proposed distribution function is respectively is the expected vector, and the covariance is a mixed Gaussian distribution of the identity matrix. At this time, the update strategy of the proposed distribution function in the adaptive importance sampling process is to update some parameters. This update strategy includes updating the variance but not the covariance. Similarly, a test scenario is constructed based on the sample points obtained by sampling, and the target autonomous driving algorithm is simulated and tested in the constructed test scenario to determine the failure probability based on the simulation test results. Among them, the initial m mixed Gaussian distributions have the same weight. The calculation formula for the failure probability is the same as the calculation formula when the number of failure scenarios is met, and it will not be repeated here.

[0095] Compared to traditional reliability analysis algorithms, this application can better break through local optimality and is suitable for solving scenarios with multiple failure domains, multiple local optimalities, and when testers are unable to select algorithms or configure algorithm parameters due to a lack of understanding of reliability analysis algorithms. Furthermore, this method is more efficient than traditional reliability analysis algorithms (simulation test calls are very few). The computational efficiency of this method is also much higher than generalization methods such as Monte Carlo and fixed-step length.

[0096] Based on the same inventive concept, an embodiment of the present application also provides an autonomous driving system failure probability estimation device corresponding to the autonomous driving system failure probability estimation method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned autonomous driving system failure probability estimation method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0097] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of an automatic driving system failure probability estimation device provided by an embodiment of the present application. Figure 3 As shown in , the automatic driving system failure probability estimation device 300 includes:

[0098] An initial scenario construction module 301 is configured to perform sampling in a uniformly distributed space corresponding to the current logical scenario to obtain initial sampling results, and then construct multiple initial test scenarios based on the initial sampling results. The uniformly distributed space is a parameter value space in which scenario parameters follow a uniform distribution.

[0099] A prediction model building module 302 is configured to perform simulation tests on the target autonomous driving system in a plurality of initial test scenarios, obtain initial simulation test results, and build an evaluation index prediction model based on the initial simulation test results and initial sampling results;

[0100] An optimization solution module 303 is configured to determine a solution objective for obtaining reference failure samples based on the evaluation index prediction model, and to solve the solution objective using an optimization algorithm to obtain a reference failure sample set. The reference failure sample is a sample with the highest probability density among the failure scenarios determined based on the evaluation index prediction model.

[0101] The failure probability estimation module 304 is configured to perform sampling in the proposed distribution space using the reference failure sample set to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

[0102] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in FIG, the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0103] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The steps of the method for estimating the failure probability of the autonomous driving system in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.

[0104] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the method for estimating the failure probability of the autonomous driving system in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.

[0105] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0107] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0108] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0109] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0110] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for estimating the failure probability of an autonomous driving system, characterized in that: include: Performing importance sampling in a uniformly distributed space corresponding to the current logical scenario to obtain initial sampling results, and constructing multiple initial test scenarios based on the initial sampling results, wherein the uniformly distributed space is a parameter value space in which scenario parameters obey a uniform distribution; Performing simulation tests on the target autonomous driving system in each of the multiple initial test scenarios to obtain initial simulation test results, and constructing an evaluation index prediction model based on the initial simulation test results and the initial sampling results, where the initial simulation test results are values ​​of the evaluation index of the target autonomous driving system in the initial test scenarios; Determining a solution objective for obtaining reference failure samples based on the evaluation index prediction model, and solving the solution objective using a gradient-based nonlinear optimization algorithm to obtain a reference failure sample set, wherein the reference failure sample refers to a sample with the highest probability density among the failure scenarios determined based on the evaluation index prediction model; Sampling is performed in the proposed distribution space using the reference failure sample set to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

2. The method according to claim 1, characterized in that The determining of a solution target for obtaining a reference failure sample based on the evaluation index prediction model includes: Using the evaluation index prediction model, constructing constraint conditions for limiting failure scenarios; A solution objective is constructed that satisfies the constraints and is used to determine the sample point closest to the origin of the standard normal distribution space.

3. The method according to claim 2, characterized in that The method further comprises: solving the target by using a gradient-based nonlinear optimization algorithm; Performing high-coverage sampling on the standard normal distribution space to obtain a first sampling result, and applying an optimization algorithm to solve the first sampling result to obtain a solution set; A reference failure sample set that meets the constraint condition is determined according to the number of elements in the solution set.

4. The method according to claim 3, characterized in that The step of determining a reference failure sample set that satisfies the constraint condition according to the number of elements in the solution set includes: If the number of elements in the solution set is one, then the element is used as a reference failure sample; If there are multiple elements in the solution set, clustering the elements in the solution set using a clustering algorithm, so as to select an element from each clustering result as a reference failure sample; If the number of elements in the solution set is empty, a heuristic algorithm is used to solve the problem to obtain a reference failure sample.

5. The method according to claim 1, wherein The method of using the reference failure sample set to perform sampling in the proposed distribution space to obtain a final sampling result includes: If the number of elements in the reference failure sample set is one, the proposal distribution is determined as a normal distribution whose expected vector is an element in the reference failure sample set and whose covariance is the identity matrix, and the proposal distribution space corresponding to the proposal distribution is sampled using adaptive importance sampling; If there are multiple elements in the reference failure sample set, the proposed distribution is determined as a mixed Gaussian distribution whose expected vector is each element in the reference failure sample set and whose covariance is the identity matrix, and the proposal distribution space corresponding to the proposed distribution is sampled using adaptive importance sampling.

6. The method according to claim 1, characterized in that The evaluation index prediction model is constructed based on the initial simulation test results and the initial sampling results, including: The initial sampling result is determined as an independent variable, the initial simulation test result is determined as a dependent variable, and the relationship between the independent variable and the dependent variable is fitted to construct an evaluation index prediction model.

7. The method according to claim 1, characterized in that After constructing the evaluation index prediction model, the method further includes: Evaluating the evaluation index prediction model to determine whether the evaluation index prediction model meets the prediction accuracy requirements; If the prediction accuracy requirement is not met, a target value space that affects the model accuracy is selected, and sampling is performed within the target value space to obtain a second sampling result, so as to optimize the evaluation index prediction model according to the second sampling result.

8. The method according to claim 7, characterized in that The evaluation index prediction model is optimized by the following methods, including: performing a simulation test on the target autonomous driving system according to the second sampling result to obtain a second simulation test result; The evaluation index prediction model is optimized using the second sampling result and the second simulation test result.

9. The method according to claim 1, characterized in that The sampling in the uniformly distributed space corresponding to the current logical scene includes: In the uniform distribution space, sampling is performed using a Latin hypercube sampling method based on maximizing minimum distance optimization.

10. An automatic driving system failure probability estimation device, characterized in that: include: An initial scenario construction module is used to perform importance sampling in a uniformly distributed space corresponding to the current logical scenario to obtain initial sampling results, so as to construct multiple initial test scenarios based on the initial sampling results. The uniformly distributed space is a parameter value space in which scenario parameters obey a uniform distribution; a prediction model construction module, configured to perform simulation tests on the target autonomous driving system in each of the plurality of initial test scenarios, obtain initial simulation test results, and construct an evaluation index prediction model based on the initial simulation test results and the initial sampling results, wherein the initial simulation test results are values ​​of the evaluation index of the target autonomous driving system in the initial test scenarios; an optimization solution module, configured to determine a solution objective for obtaining reference failure samples based on the evaluation index prediction model, and solve the solution objective using a gradient-based nonlinear optimization algorithm to obtain a reference failure sample set, wherein the reference failure sample refers to a sample with the highest probability density among the failure scenarios determined based on the evaluation index prediction model; A failure probability estimation module is used to use the reference failure sample set to sample in the proposed distribution space to obtain a final sampling result, so as to determine the final failure probability of the target autonomous driving system according to the final sampling result.

Images