Method and device for estimating failure probability of automatic driving system

Abstract

The invention provides a failure probability estimation method and device for an automatic driving system, and relates to the technical field of automatic driving, and the method comprises the steps: carrying out the sampling of a uniform distribution space, obtaining an initial sampling result, and constructing a plurality of initial testing scenes according to the initial sampling result; respectively performing simulation testing on the target automatic driving system in the plurality of initial testing scenes to obtain initial simulation testing results, and constructing an evaluation index prediction model according to the initial simulation testing results and the initial sampling results; determining a solving target for obtaining a reference failure sample based on the evaluation index prediction model, and solving the solving target by using an optimization algorithm to obtain a reference failure sample set; and sampling in the proposal distribution space by using the reference failure sample set, and determining a final failure probability according to a final sampling result. By adopting the method and the device for estimating the failure probability of the automatic driving system, the problems of low verification efficiency and inaccurate failure probability estimation of the automatic driving system in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more specifically, to a method and device for estimating the failure probability of an autonomous driving system. Background Art

[0002] With the rapid development of autonomous driving systems, how to test and verify the reliability of autonomous driving systems has become crucial. Autonomous driving simulation testing has become an important means to verify the reliability of autonomous driving systems. In the simulation test process, a logical scenario (i.e., test space) can be generalized into multiple specific scenarios for testing autonomous driving systems based on test needs. Engineers evaluate the failure probability of the autonomous driving system based on the performance of the autonomous driving system under test in these specific scenarios. Therefore, how to generalize a logical scenario to obtain a failure scenario, that is, how to explore the specific scenarios in the test space that cause the autonomous driving system under test to fail, has become a top priority in verifying the autonomous driving system.

[0003] In the prior art, the first method of obtaining the test scene is to adopt a fixed-step generalization method, that is, to generalize the various parameters in the logical scene with a fixed step length. However, this method is difficult to determine the generalization granularity, and the number of generalized test scenes will increase exponentially with the increase of scene parameters, which reduces the verification efficiency of the autonomous driving system. At the same time, this method lacks consideration of the probability distribution of parameters, which may lead to inaccurate estimation of the failure probability. The second method of obtaining the test scene is to adopt Monte Carlo sampling. This method samples according to the probability distribution or joint distribution of each parameter, and has the advantage of high robustness, but the system verification efficiency is low and cannot meet the requirements of rapid verification of the failure probability of the autonomous driving system. The third method of obtaining the test scene is reliability analysis. This method mainly refers to the use of algorithms such as directional sampling or importance sampling. Its advantage is high efficiency, but in the absence of data prior experience, the algorithm parameters are difficult to set and select, which may lead to the problem of falling into the local optimum, reducing the accuracy of the failure probability estimation. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method and device for estimating the failure probability of an autonomous driving system, so as to solve the problems of low verification efficiency and inaccurate failure probability estimation of the autonomous driving system in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a method for estimating the failure probability of an autonomous driving system, comprising:

[0006] Sampling is performed in the uniform distribution space corresponding to the current logical scenario to obtain an initial sampling result, so as to construct multiple initial test scenarios according to the initial sampling result. The uniform distribution space is a parameter value space in which the scenario parameters obey a uniform distribution.

[0007] Conduct simulation tests on the target autonomous driving system in multiple initial test scenarios, obtain initial simulation test results, and build an evaluation index prediction model based on the initial simulation test results and initial sampling results;

[0008] Determine a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solve the solution target using an optimization algorithm to obtain a reference failure sample set. The reference failure sample refers to a sample with the largest probability density in the failure scenario determined based on the evaluation index prediction model.

[0009] The reference failure sample set is used to sample in the proposed distribution space to obtain the final sampling result, so as to determine the final failure probability of the target autonomous driving system according to the final sampling result.

[0010] Optionally, determining a solution target for obtaining a reference failure sample based on the evaluation index prediction model includes: using the evaluation index prediction model to construct constraints for limiting failure scenarios; and constructing a solution target that meets the constraints and is used to determine the sample point closest to the origin of the standard normal distribution space.

[0011] Optionally, an optimization algorithm is used to solve the solution target, including: performing high-coverage sampling on the standard normal distribution space to obtain a first sampling result, and applying an optimization algorithm to solve the first sampling result to obtain a solution set; based on the number of elements in the solution set, determining a reference failure sample set that meets the constraint conditions.

[0012] Optionally, the reference failure sample set includes at least one reference failure sample, and the reference failure sample set that satisfies the constraint conditions is determined according to the number of elements in the solution set, including: if the number of elements in the solution set is one, then the element is used as the reference failure sample; if the number of elements in the solution set is multiple, then the elements in the solution set are clustered using a clustering algorithm to select an element from each clustering result as the reference failure sample; if the number of elements in the solution set is empty, then a heuristic algorithm is used to solve to obtain a reference failure sample.

[0013] Optionally, the reference failure sample set is used to sample in the proposal distribution space to obtain the final sampling result, including: if the number of elements in the reference failure sample set is one, the proposal distribution is determined as a normal distribution whose expected vector is an element in the reference failure sample set and whose covariance is the unit matrix, and the proposal distribution space corresponding to the proposal distribution is sampled using adaptive importance sampling. If the number of elements in the reference failure sample set is multiple, the proposal distribution is determined as a mixed Gaussian distribution whose expected vector is each element in the reference failure sample set and whose covariance is the unit matrix, and the proposal distribution space corresponding to the proposal distribution is sampled using adaptive importance sampling.

[0014] Optionally, an evaluation index prediction model is constructed based on the initial simulation test results and the initial sampling results, including: determining the initial sampling results as independent variables, using the initial simulation test results as dependent variables, and fitting the relationship between the independent variables and the dependent variables to construct the evaluation index prediction model.

[0015] Optionally, after constructing the evaluation index prediction model, the method also includes: evaluating the evaluation index prediction model to determine whether the evaluation index prediction model meets the prediction accuracy requirements; if it does not meet the prediction accuracy requirements, selecting a target value space that affects the model accuracy, and sampling in the target value space to obtain a second sampling result, so as to optimize the evaluation index prediction model according to the second sampling result.

[0016] Optionally, the evaluation index prediction model is optimized in the following ways, including: performing a simulation test on the target autonomous driving system according to the second sampling result to obtain a second simulation test result; and optimizing the evaluation index prediction model using the second sampling result and the second simulation test result.

[0017] Optionally, sampling is performed in a uniform distribution space corresponding to the current logical scenario, including: sampling in the uniform distribution space using a Latin hypercube sampling method based on maximizing minimum distance optimization.

[0018] In a second aspect, an embodiment of the present application further provides a device for estimating the failure probability of an autonomous driving system, the device comprising:

[0019] An initial scene construction module is used to perform sampling in a uniform distribution space corresponding to the current logical scene to obtain an initial sampling result, so as to construct multiple initial test scenes according to the initial sampling result. The uniform distribution space is a parameter value space in which the scene parameters obey a uniform distribution.

[0020] A prediction model building module is used to perform simulation tests on the target autonomous driving system in multiple initial test scenarios, obtain initial simulation test results, and build an evaluation index prediction model based on the initial simulation test results and initial sampling results;

[0021] An optimization solution module is used to determine a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solve the solution target using an optimization algorithm to obtain a reference failure sample set. The reference failure sample refers to a sample with the largest probability density in the failure scenario determined based on the evaluation index prediction model.

[0022] The failure probability estimation module is used to obtain a final sampling result by sampling in the proposed distribution space using a reference failure sample set, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

[0023] The embodiments of the present application bring the following beneficial effects:

[0024] The embodiment of the present application provides a method and device for estimating the failure probability of an autonomous driving system, which can solve the solution target using an optimization algorithm, and obtain test scenarios through adaptive importance sampling to achieve targeted search for scenarios that cause the failure of the target autonomous driving system, thereby improving the verification efficiency of the autonomous driving system and being able to efficiently and accurately estimate the failure probability. At the same time, an evaluation index prediction model is used to explore the parameter value space to provide prior experience, avoiding the problem of low solution efficiency caused by directly using simulation tests. While simplifying the difficulty of setting and selecting algorithm parameters, it also further enhances the ability to break through local optimality during the verification process of the autonomous driving system, improves the accuracy of the failure probability estimation, and solves the problems of low verification efficiency and inaccurate failure probability estimation of the autonomous driving system compared to the prior art.

[0025] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A flow chart showing a method for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application is shown;

[0028] Figure 2 A flow chart of a target solving method provided in an embodiment of the present application is shown;

[0029] Figure 3 A schematic diagram showing the structure of a device for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application is shown;

[0030] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0031] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0032] It is worth noting that before the present application was filed, with the rapid development of autonomous driving systems, how to test and verify the reliability of autonomous driving systems has become crucial, and autonomous driving simulation testing has become an important means to verify the reliability of autonomous driving systems. In the simulation test process, a logical scenario (i.e., test space) can be generalized into multiple specific scenarios for testing the autonomous driving system based on the test needs. The engineers evaluate the failure probability of the autonomous driving system based on the performance of the autonomous driving system under test in these specific scenarios. Therefore, how to generalize a logical scenario to obtain a failure scenario, that is, how to explore the specific scenario in the test space that causes the failure of the autonomous driving system under test, has become the top priority for verifying the autonomous driving system. In the prior art, the first method for obtaining the test scenario is to use a fixed-step generalization method, that is, to generalize the various parameters in the logical scenario with a fixed step length, but this method is difficult to determine the generalization granularity, and the number of generalized test scenarios will increase exponentially with the increase of scenario parameters, reducing the verification efficiency of the autonomous driving system. At the same time, this method lacks consideration of the probability distribution of parameters, which may lead to the problem that although the scenario that causes the failure of the autonomous driving system is found, it is difficult to estimate the failure probability and the safe operation range of the system, resulting in inaccurate failure probability estimation. The second method to obtain the test scenario is to use Monte Carlo sampling, which is to sample according to the probability distribution or joint distribution of each parameter. It has the advantage of high robustness, but the verification efficiency is low and cannot meet the requirements of rapid verification of the failure probability of the autonomous driving system. The third method to obtain the test scenario is reliability analysis, which mainly refers to the use of algorithms such as directional sampling or importance sampling. Its advantage is high efficiency, but in the absence of data prior experience, the algorithm parameters are difficult to set and select, which may lead to the problem of falling into the local optimum and reduce the accuracy of the failure probability estimation.

[0033] The terms involved in the embodiments of the present application are explained below.

[0034] Scene parameters: Elements in the simulation scene, whose values ​​obey certain specified distribution functions, that is, the space composed of all parameters can be understood as a Euclidean space composed of a set of random variables.

[0035] Logical scenario: expresses entity features and relationships between entities by defining parameter ranges and distribution functions of variables in the state space. Logical scenarios are further detailed descriptions of functional scenarios based on state space variables and are used to generate requirements during the project development phase. For each logical scenario with a value range, any number of specific scenarios (test scenarios) can be derived.

[0036] Test scenario: A scenario that can be directly simulated and tested, that is, a scenario obtained after all scenario parameters in the logical scenario are assigned values.

[0037] Test failure: The performance of the autonomous driving system in the environment variable X can be represented by the evaluation index function g(X). The failure of the autonomous driving system test can be represented by g(X)≤T, and T=0 without loss of generality. Therefore, the variable value that causes the autonomous driving system test failure can be defined as {X|g(X)≤0}. And {X|g(X)>0} indicates the range of capabilities within which the autonomous driving system can operate safely.

[0038] Failure probability: In many fields such as autonomous driving, aerospace, and equipment manufacturing, an important task is to strictly verify whether the probability of system failure is less than a specified threshold, that is, to estimate the probability of system failure p in a specified probability distribution space. f =P(g(X)≤0). Let f(x) and μ(x) represent the density function and measure function of X in the probability distribution space respectively, then:

[0039] p f =P(g(X)≤0)=∫ g(x)≤0 f(x)dμ(x).

[0040] Due to the complexity and unknown nature of the black box function g(X), the above integral cannot be calculated directly, and statistics are often constructed The failure probability of the system is estimated through sampling results. The quality of an estimate can be measured by accuracy (bias, relative deviation, whether it is unbiased or asymptotically unbiased), precision (standard deviation, coefficient of variation or relative error), and efficiency. Accuracy is used to measure whether the estimate is equal to the true value in the long run; precision is used to measure the change of the estimate, that is, the difference between each estimate; efficiency is used to measure the sample size required to achieve the same precision.

[0041] The most commonly used method for the above problem is to use the Monte Carlo method to estimate, that is:

[0042]

[0043] Among them, the Monte Carlo method is an unbiased estimate, and its relative error is

[0044] Based on this, an embodiment of the present application provides a method for estimating the failure probability of an autonomous driving system to improve the verification efficiency of the autonomous driving system and the accuracy of the failure probability estimation.

[0045] See also Figure 1 , Figure 1 This is a flow chart of a method for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application. Figure 1 As shown, the method for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application includes:

[0046] Step S101, sampling is performed in a uniformly distributed space corresponding to the current logical scenario to obtain an initial sampling result, so as to construct a plurality of initial test scenarios according to the initial sampling result;

[0047] Step S102, performing simulation tests on the target autonomous driving system in a plurality of initial test scenarios respectively, obtaining initial simulation test results, and constructing an evaluation index prediction model according to the initial simulation test results and the initial sampling results;

[0048] Step S103, determining a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solving the solution target using an optimization algorithm to obtain a reference failure sample set;

[0049] Step S104, sampling is performed in the proposed distribution space using the reference failure sample set to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

[0050] The method for estimating the failure probability of an autonomous driving system provided in the embodiment of the present application can solve the solution target using an optimization algorithm, and obtain test scenarios through adaptive importance sampling to achieve targeted search for scenarios that cause the failure of the target autonomous driving system, thereby improving the verification efficiency of the autonomous driving system and being able to efficiently and accurately estimate the failure probability. At the same time, the evaluation index prediction model is used to explore the parameter value space to provide prior experience and avoid the problem of low solution efficiency caused by directly using simulation tests. While simplifying the difficulty of setting and selecting algorithm parameters, it also further enhances the ability to break through the local optimum during the verification process of the autonomous driving system, improves the accuracy of the failure probability estimation, and solves the problems of low verification efficiency of the autonomous driving system and inaccurate failure probability estimation.

[0051] To facilitate understanding of this embodiment, each of the above exemplary steps provided in the embodiment of the present application is described below.

[0052] In step S101, sampling is performed in a uniform distribution space corresponding to the current logical scenario to obtain initial sampling results, so as to construct a plurality of initial test scenarios according to the initial sampling results.

[0053] In this step, the current logical scenario may refer to a logical scenario used to test the target autonomous driving algorithm. Multiple test scenarios may be generalized through the current logical scenario, and the current logical scenario includes multiple scenario parameters.

[0054] The uniform distribution space may refer to the parameter value space in which the scene parameters obey the uniform distribution. The uniform distribution space is a multidimensional space, the dimension of which is determined by the number of scene parameters, and the range of the uniform distribution is determined by the range of values ​​in the original distribution.

[0055] In an embodiment of the present application, before testing the performance of the target autonomous driving system in the current logical scenario, it is first necessary to generalize multiple test scenarios corresponding to the current logical scenario. In order to better explore the entire value space of the logical scenario, this sampling does not consider the true distribution function and correlation coefficient of the scene parameters, but only considers the value range of the scene parameters. To this end, a high-coverage sampling can be performed in the uniform distribution space using a Latin hypercube sampling method based on maximizing the minimum distance optimization. This sampling method can maximize the minimum distance between sample points so that the sampling points are evenly distributed throughout the uniform distribution space, so as to more comprehensively explore the entire value space. Among them, the amount of samples collected is a set parameter, which can be determined based on the actual computing power and the dimension of the uniform distribution space; the value range of each scene parameter is determined based on real data, for example: the value range of each scene parameter can be determined based on road sampling data.

[0056] In one example, when the value range of a scene parameter is infinite, a sub-interval is set as the value range of the scene parameter, for example, the value of the distribution function is 10 -8 To 1-10 -8 The corresponding value interval is used as the value range of the scene parameter.

[0057] After sampling the uniform distribution space, the initial sampling result is obtained, which is recorded as: X1,...,X n Here, each element X in the initial sampling result i Represents a sample point, each sample point represents a scene parameter combination and the value of each scene parameter in the scene parameter combination. A test scene can be constructed according to each sample point. In this way, a total of n test scenes can be constructed through the initial sampling results. Among them, each scene parameter combination includes all scene parameters under the current logical scene.

[0058] In the embodiment of the present application, the high coverage sampling of the uniform distribution space is actually the importance sampling, that is, the uniform distribution is used as the proposed distribution, and the correlation between the scene parameters is ignored.

[0059] In step S102, simulation tests are performed on the target autonomous driving system in multiple initial test scenarios, initial simulation test results are obtained, and an evaluation index prediction model is constructed based on the initial simulation test results and initial sampling results.

[0060] In this step, for each initial test scenario, a simulation test is performed on the target autonomous driving system in the initial test scenario to obtain an initial simulation test result of the target autonomous driving system in the initial test scenario. The initial simulation test result is the value of the evaluation index of the target autonomous driving system in the initial test scenario, and the evaluation index is pre-set according to the scenario test requirements.

[0061] The initial simulation test results include multiple ones, the number of which corresponds to the number of initial test scenarios, and the initial simulation test results are recorded as: g(X1),...,g(X n ).

[0062] In an embodiment of the present application, after constructing multiple initial test scenarios, it is also possible to determine whether the number of failure scenarios in the initial test scenarios meets the scenario quantity condition, for example: determine whether the number of failure scenarios in the initial test scenarios is greater than or equal to a set quantity threshold. If it is greater than or equal to the set quantity threshold, it means that there are enough failure scenarios to estimate the failure probability, and it is determined that the scenario quantity condition is met. At this time, these initial test scenarios can be directly used to simulate the target autonomous driving algorithm, and the failure probability is determined based on the initial simulation test results. Among them, the formula for determining the failure probability is:

[0063]

[0064] In the above formula, represents an estimator of the failure probability. It is also an unbiased estimator; V represents the volume or area of ​​the uniform distribution space (when the uniform distribution space is two-dimensional, it is called area, and when the uniform distribution space is multi-dimensional, it is called volume); f(X i ) represents the scenario parameter combination (test scenario) X i The probability density under the original probability distribution, the original probability distribution can refer to the true probability distribution of the scene parameters, and the original probability distribution can reflect the correlation between different scene parameters; n represents the sample size.

[0065] If it is less than the set quantity threshold, it means that there are not enough failure scenarios to estimate the failure probability, then it is determined that the scenario quantity condition is not met, and the step of constructing an evaluation index prediction model based on the initial simulation test results and the initial sampling results is executed.

[0066] In one example, when constructing an evaluation index prediction model, the initial sampling result can be determined as an independent variable, the initial simulation test result can be used as a dependent variable, and a curve fitting is performed on the relationship between the independent variable and the dependent variable to construct an evaluation index prediction model, which is used to characterize the relationship between the initial sampling result X and the evaluation index g(X).

[0067] In another example, when constructing the evaluation index prediction model, the initial sampling results and the initial simulation test results can be used to train the initial evaluation index prediction model, so as to determine the trained initial evaluation index prediction model as the final constructed evaluation index prediction model. The initial evaluation index prediction model can be a neural network model.

[0068] In one example, in order to determine whether the prediction results of the evaluation index prediction model meet the accuracy requirements, after constructing the evaluation index prediction model, the evaluation index prediction model can also be evaluated to determine whether the evaluation index prediction model meets the prediction accuracy requirements. Here, the evaluation index prediction model can be measured based on the prediction coefficient (Coefficient of Prognosis, CoP) of the combined test to measure whether it meets the prediction accuracy requirements. For example: the initial sampling results are input into the evaluation index prediction model to obtain the predicted evaluation index value output by the evaluation index prediction model, and the predicted evaluation index value and the initial simulation test results obtained by the actual simulation test are input into the Cop calculation formula to obtain the prediction coefficient, and the prediction coefficient is compared with the accuracy threshold. If the prediction coefficient is less than or equal to the accuracy threshold, it is determined that it does not meet the prediction accuracy requirements. If the prediction coefficient is greater than the accuracy threshold, it is determined that it meets the prediction accuracy requirements. Among them, the calculation formula of the prediction coefficient is as follows:

[0069]

[0070] In the above formula, CoP represents the prediction coefficient; g(X i ) represents the initial simulation test result; Represents the prediction evaluation index value; μ g Represents the mean of the initial simulation test results; test represents the test set, that is, all initial sampling results.

[0071] If the prediction accuracy requirement is met, the evaluation index prediction model is directly used to construct the solution target. If the prediction accuracy requirement is not met, the target value space that affects the model accuracy is selected, and sampling is performed in the target value space to obtain a second sampling result, so as to optimize the evaluation index prediction model according to the second sampling result. Among them, the target value space may refer to the space corresponding to the point with a large error selected in the original value space, and the original value space refers to the original space that only considers the value range and does not consider the probability distribution.

[0072] In an embodiment of the present application, based on the output results of the evaluation index prediction model and the initial simulation test results, a sample point with a large error can be determined. If the standard deviation corresponding to the sample point is greater than the set error threshold, it means that the space near the sample point has a greater impact on the model accuracy. In this case, the sample point is selected as the target sample point, and the space within a preset range around the target sample point is selected in the original value space as the target value space. For example, in the original value space, a spherical space with the target sample point as the center and a radius within a set value is selected as the target value space.

[0073] Then, a second sampling result is obtained by adaptive sampling in the target value space. In one example, Bayesian estimation can be used for adaptive sampling. The target autonomous driving system is simulated and tested according to the second sampling result to obtain a second simulation test result. Finally, the evaluation index prediction model is optimized using the second sampling result and the second simulation test result. For example, the initial sampling result and the second sampling result are used as the latest input, and the initial simulation test result and the second simulation test result are used as the latest output. The evaluation index prediction model is refitted using the latest input and the latest output to obtain an optimized evaluation index prediction model.

[0074] After obtaining the optimized evaluation index prediction model, determine again whether the optimized evaluation index prediction model meets the prediction accuracy requirements. If it still does not meet the requirements, repeat the above steps and continue to optimize the evaluation index prediction model until the optimized evaluation index prediction model meets the prediction accuracy requirements.

[0075] In one example, after the evaluation index prediction model meets the prediction accuracy requirements, since the subsequent reliability analysis algorithm will be calculated based on the standard normal distribution space, the optimized evaluation index prediction model needs to be transformed into the standard normal distribution space, that is, to obtain the evaluation index prediction model under the standard normal distribution space. The evaluation index prediction model under the standard normal distribution space is recorded as: Among them, X = d (Z) means converting the random number Z in the standard normal distribution space into the random number X in the original distribution space. The original distribution space is the original space considering the probability distribution function. The original space is a space with correlation between different scene parameters constructed based on real road sampling data. Among them, the random number Z and the random number X are both vectors.

[0076] In step S103, a solution target for obtaining reference failure samples is determined based on the evaluation index prediction model, and the solution target is solved using an optimization algorithm to obtain a reference failure sample set.

[0077] In this step, the reference failure sample refers to the sample with the largest probability density in the failure scenario determined based on the evaluation index prediction model, that is, the reference failure sample is the failure sample closest to the origin in the standard normal distribution space. The closer to the origin, the greater the probability density, and the greater the corresponding probability likelihood function value; the farther from the origin, the smaller the probability density, and the smaller the corresponding probability likelihood function value. One failure sample corresponds to one failure scenario. The reference failure sample is denoted as: Z * .

[0078] The reference failure sample set may refer to a set of reference failure samples, and the reference failure samples are also referred to as a design point (Most Probable Point, MPP).

[0079] The algorithm for obtaining reference failure samples can be expressed as: * =argmin Z {||Z|| 2 ,h(Z)≤0}, h(·) represents the evaluation index of the target autonomous driving system. Although placing the center of the proposed distribution of importance sampling at the MPP can often directly converge to obtain an accurate failure probability, in the process of using the optimization algorithm to calculate the MPP, the black box function h(x) in the complex space often requires a high amount of calculation, and the optimization algorithm cannot be calculated in parallel. These problems make it impossible to obtain h(x) through simulation testing.

[0080] Below through Figure 2 Let’s introduce the target solving process.

[0081] Figure 2 A flow chart of the target solving method provided in the embodiment of the present application is shown as follows: Figure 2 As shown, the target solution method includes:

[0082] Step S201, using the evaluation index prediction model, constructing constraint conditions for limiting failure scenarios.

[0083] Here, the constraint condition is used to limit the failure samples corresponding to the failure scenarios determined by the evaluation index prediction model. The constraint condition can be expressed as:

[0084] Step S202, constructing a solution target that meets the constraint conditions and is used to determine the sample point closest to the origin of the standard normal distribution space.

[0085] The solution goal can be expressed as: Find Z * to min{||Z|| 2}, Such that

[0086] It should be noted that The nature of is unknown, and the above solution target may have multiple solution intervals, that is, the target autonomous driving system has multiple failure domains. Therefore, a simple gradient algorithm cannot solve the above solution target. To this end, this application uses the evaluation index prediction model Solve the problem instead of running a simulation test and use the simulation test results to solve the problem to speed up the calculation of the constraints.

[0087] Step S203, performing high coverage sampling on the standard normal distribution space to obtain a first sampling result, and applying an optimization algorithm to solve the first sampling result to obtain a solution set.

[0088] Specifically, a high coverage sampling can be performed on a subspace of the standard normal distribution space to obtain a first sampling result. The subspace can refer to a distribution function with a value of, 10 -10 To 1-10 -10 The corresponding interval, and the sample size in the sampling process is determined by the dimension of the standard normal distribution space. The higher the spatial dimension, the larger the sample size, and the lower the spatial dimension, the smaller the sample size. Then, based on the first sampling result, a gradient-based nonlinear optimization algorithm is used to solve the solution target to determine the set of solutions that meet the constraints as a solution set, and each element in the solution set is a solution. Among them, the standard normal distribution space may refer to the space where the scenario parameters obey the standard normal distribution, and the gradient-based nonlinear optimization algorithm may refer to the NLPQL algorithm. It should be noted that when there is a failure scenario in the initial sampling result, the sample points corresponding to the failure scenario are also added to the solution set.

[0089] Step S204: determining a reference failure sample set that meets the constraint conditions according to the number of elements in the solution set.

[0090] If the number of elements in the solution set is one, the element is used as a reference failure sample; if the number of elements in the solution set is multiple, a clustering algorithm is used to cluster the elements in the solution set, for example: an adaptive DBSCAN algorithm is used to cluster the elements in the solution set, and an element is selected from each clustering result as a reference failure sample, for example: three categories are obtained by clustering, then the sample point closest to the origin of the standard normal distribution space is selected from each category as a reference failure sample, and a total of three reference failure samples are obtained; if the number of elements in the solution set is empty, a heuristic algorithm is used to solve the solution target to obtain a reference failure sample, for example: a genetic algorithm and a hippopotamus algorithm are used for the solution, and the input of the heuristic algorithm is the solution target and the standard normal distribution space.

[0091] In step S104, a reference failure sample set is used to perform sampling in the proposed distribution space to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

[0092] In this step, the proposed distribution space may refer to the parameter value space of the scene parameters that obey the proposed distribution function.

[0093] If the number of elements in the reference failure sample set is one, that is, there is only one Z * , the proposed distribution function is determined as the expected vector as the element in the reference failure sample set, the covariance is the normal distribution of the unit matrix, and the proposed distribution space corresponding to the proposed distribution function is sampled using adaptive importance sampling. Test scenarios are constructed based on the sample points obtained by sampling, and the target autonomous driving algorithm is simulated and tested in these test scenarios to determine the failure probability based on the simulation test results. It should be noted that the vicinity of the failure domain has been found at this time. The update strategy of the proposed distribution function in the adaptive importance sampling process is to update all parameters. This update strategy includes updating the covariance to improve the accuracy of the algorithm.

[0094] If there are multiple elements in the reference failure sample set, the proposed distribution function is determined as a mixed Gaussian distribution whose expected vector is each element in the reference failure sample set and whose covariance is the identity matrix, and the proposed distribution space corresponding to the proposed distribution is sampled using adaptive importance sampling. For example: multiple elements in the reference failure sample set are That is, there are multiple failure domains, so the proposed distribution function is is the expected vector, and the covariance is a mixed Gaussian distribution of the unit matrix. At this time, the update strategy of the proposed distribution function in the adaptive importance sampling process is to update some parameters, which includes updating the variance but not the covariance. Similarly, a test scenario is constructed based on the sample points obtained by sampling, and a simulation test is performed on the target autonomous driving algorithm in the constructed test scenario to determine the failure probability based on the simulation test results. Among them, the initial m mixed Gaussian distributions have the same weights. The calculation formula for the failure probability is the same as the above calculation formula when the number of failure scenarios is met, and will not be repeated here.

[0095] Compared with traditional reliability analysis algorithms, this application can better break through local optimality and is suitable for solving multiple failure domains, multiple local optimalities, and scenarios where testers cannot select algorithms or configure algorithm parameters due to lack of understanding of reliability analysis algorithms. In addition, this method is more efficient than traditional reliability analysis algorithms (the number of calls to simulation tests is very small). The computational efficiency of this method is also much higher than generalization methods such as Monte Carlo and fixed step size.

[0096] Based on the same inventive concept, an autonomous driving system failure probability estimation device corresponding to the autonomous driving system failure probability estimation method is also provided in an embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned autonomous driving system failure probability estimation method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0097] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a device for estimating the failure probability of an autonomous driving system provided in an embodiment of the present application. Figure 3 As shown in , the automatic driving system failure probability estimation device 300 includes:

[0098] The initial scenario construction module 301 is used to perform sampling in the uniform distribution space corresponding to the current logical scenario to obtain initial sampling results, so as to construct multiple initial test scenarios according to the initial sampling results. The uniform distribution space is a parameter value space in which the scenario parameters obey uniform distribution;

[0099] The prediction model building module 302 is used to perform simulation tests on the target autonomous driving system in multiple initial test scenarios, obtain initial simulation test results, and build an evaluation index prediction model based on the initial simulation test results and initial sampling results;

[0100] The optimization solution module 303 is used to determine a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solve the solution target using an optimization algorithm to obtain a reference failure sample set. The reference failure sample refers to a sample with the largest probability density in the failure scenario determined based on the evaluation index prediction model;

[0101] The failure probability estimation module 304 is used to perform sampling in the proposed distribution space using the reference failure sample set to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

[0102] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0103] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The specific implementation methods of the steps of the method for estimating the failure probability of the autonomous driving system in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0104] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The specific implementation methods of the steps of the method for estimating the failure probability of the autonomous driving system in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0106] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0109] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0110] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for estimating the failure probability of an autonomous driving system, characterized in that: include: Sampling is performed in a uniform distribution space corresponding to the current logical scenario to obtain an initial sampling result, so as to construct a plurality of initial test scenarios according to the initial sampling result, wherein the uniform distribution space is a parameter value space in which scenario parameters obey a uniform distribution; Performing simulation tests on the target autonomous driving system in the multiple initial test scenarios respectively, obtaining initial simulation test results, and constructing an evaluation index prediction model based on the initial simulation test results and the initial sampling results; Determine a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solve the solution target using an optimization algorithm to obtain a reference failure sample set, wherein the reference failure sample refers to a sample with the largest probability density in the failure scenario determined based on the evaluation index prediction model; The reference failure sample set is used to perform sampling in the proposed distribution space to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

2. The method according to claim 1, characterized in that The determining of a solution target for obtaining a reference failure sample based on the evaluation index prediction model includes: Using the evaluation index prediction model, constructing constraint conditions for limiting failure scenarios; A solution objective is constructed that satisfies the constraints and is used to determine the sample points closest to the origin of the standard normal distribution space.

3. The method according to claim 2, characterized in that The method of solving the target by using an optimization algorithm comprises: Performing high coverage sampling on the standard normal distribution space to obtain a first sampling result, and applying an optimization algorithm to solve the first sampling result to obtain a solution set; A reference failure sample set satisfying the constraint condition is determined according to the number of elements in the solution set.

4. The method according to claim 3, characterized in that The step of determining a reference failure sample set satisfying the constraint condition according to the number of elements in the solution set includes: If the number of elements in the solution set is one, then the element is used as a reference failure sample; If there are multiple elements in the solution set, clustering the elements in the solution set using a clustering algorithm, so as to select an element from each clustering result as a reference failure sample; If the number of elements in the solution set is empty, a heuristic algorithm is used to solve the problem to obtain a reference failure sample.

5. The method according to claim 1, characterized in that The using the reference failure sample set to perform sampling in the proposed distribution space to obtain a final sampling result includes: If the number of elements in the reference failure sample set is one, the proposal distribution is determined as a normal distribution whose expected vector is an element in the reference failure sample set and whose covariance is an identity matrix, and the proposal distribution space corresponding to the proposal distribution is sampled using adaptive importance sampling; If there are multiple elements in the reference failure sample set, the proposed distribution is determined as a mixed Gaussian distribution whose expected vector is each element in the reference failure sample set and whose covariance is the unit matrix, and the proposed distribution space corresponding to the proposed distribution is sampled using adaptive importance sampling.

6. The method according to claim 1, characterized in that The evaluation index prediction model is constructed based on the initial simulation test results and the initial sampling results, including: The initial sampling result is determined as an independent variable, the initial simulation test result is determined as a dependent variable, and the relationship between the independent variable and the dependent variable is fitted to construct an evaluation index prediction model.

7. The method according to claim 1, characterized in that After constructing the evaluation index prediction model, the method further includes: Evaluating the evaluation index prediction model to determine whether the evaluation index prediction model meets the prediction accuracy requirements; If the prediction accuracy requirement is not met, a target value space that affects the model accuracy is selected, and sampling is performed within the target value space to obtain a second sampling result, so as to optimize the evaluation index prediction model according to the second sampling result.

8. The method according to claim 7, characterized in that The evaluation index prediction model is optimized by the following methods, including: Performing a simulation test on the target autonomous driving system according to the second sampling result to obtain a second simulation test result; The evaluation index prediction model is optimized using the second sampling result and the second simulation test result.

9. The method according to claim 1, characterized in that: The sampling in the uniform distribution space corresponding to the current logical scene includes: In the uniform distribution space, sampling is performed using a Latin hypercube sampling method based on maximizing minimum distance optimization.

10. An automatic driving system failure probability estimation device, characterized in that: include: An initial scenario construction module is used to perform sampling in a uniform distribution space corresponding to the current logical scenario to obtain an initial sampling result, so as to construct multiple initial test scenarios according to the initial sampling result, wherein the uniform distribution space is a parameter value space in which scenario parameters obey a uniform distribution; A prediction model building module, used to perform simulation tests on the target autonomous driving system in the multiple initial test scenarios, obtain initial simulation test results, and build an evaluation index prediction model based on the initial simulation test results and the initial sampling results; An optimization solution module, used to determine a solution target for obtaining reference failure samples based on the evaluation index prediction model, and solve the solution target using an optimization algorithm to obtain a reference failure sample set, wherein the reference failure sample refers to a sample with the largest probability density in the failure scenario determined based on the evaluation index prediction model; A failure probability estimation module is used to use the reference failure sample set to perform sampling in the proposed distribution space to obtain a final sampling result, so as to determine a final failure probability of the target autonomous driving system according to the final sampling result.

Images