Statistical power evaluation methods, devices, equipment and storage media

Abstract

This invention relates to the field of autonomous driving control, and discloses a statistical power evaluation method, apparatus, device, and storage medium. The method includes: acquiring driving data and takeover parameters corresponding to tests of new and old versions of an autonomous driving algorithm; evaluating the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data according to the takeover parameters using a preset resampling algorithm, and obtaining evaluation results; statistically verifying the distribution of takeover behavior on the evaluation results, obtaining verification results, and calculating the statistical power corresponding to the takeover behavior evaluation based on the verification results. This application achieves efficient evaluation of the statistical power of the autonomous driving algorithm evaluation process.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving control, and in particular to a statistical power evaluation method, apparatus, device, and storage medium. Background Technology

[0002] With the rapid development of the automotive industry, automakers are continuously increasing their research investment in active safety technologies and intelligent technologies, namely autonomous driving technologies, in order to realize autonomous driving in the near future. To achieve autonomous driving functionality, how to efficiently test and evaluate the rationality of new and old versions of autonomous driving algorithms and the stability of autonomous driving systems has become a major research direction in the industry. Regarding testing methods for autonomous driving algorithms, simulation testing technology, compared to real-road fully autonomous driving testing, has advantages such as high efficiency, repeatability, no safety risks, low cost, and the ability to simulate several scenarios, and is therefore widely used in autonomous driving algorithm testing.

[0003] Currently, the evaluation of the statistical effectiveness of various autonomous driving algorithm evaluation processes involves using a large number of test scenarios to statistically evaluate the number of take-offs for the new and old versions of autonomous driving algorithms over a long period of time. Then, the take-off numbers are used to analyze the evaluation effectiveness of the evaluation method for the new and old versions of autonomous driving algorithms. As a result, the analysis of the statistical effectiveness of autonomous driving take-off numbers is time-consuming and inefficient. In other words, the existing statistical effectiveness analysis for autonomous driving algorithm evaluation processes suffers from low analysis efficiency. Summary of the Invention

[0004] The main objective of this invention is to address the problem of low analytical efficiency in existing statistical power analysis methods for evaluating autonomous driving algorithms.

[0005] The first aspect of the present invention provides a statistical efficacy evaluation method, the statistical efficacy evaluation method comprising: acquiring driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms; evaluating the takeover behavior of the new and old versions of autonomous driving algorithms on the driving data according to the takeover parameters using a preset resampling algorithm, and obtaining an evaluation result; performing statistical verification of the takeover behavior distribution on the evaluation result, obtaining a verification result, and calculating the statistical efficacy corresponding to the takeover behavior evaluation based on the verification result.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the takeover parameters include a first takeover parameter and a second takeover parameter. The step of evaluating the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data according to the takeover parameters using a preset resampling algorithm to obtain evaluation results includes: sampling the driving data according to preset sampling conditions to obtain a corresponding sample set, wherein the sample set includes multiple simulated sample sets and road test sample sets corresponding to multiple driving scenarios; evaluating the takeover behavior of the new and old versions of the autonomous driving algorithm during simulated test driving on the simulated sample set according to the first takeover parameter to obtain a first evaluation result; evaluating the takeover behavior of the new version of the autonomous driving algorithm during road test driving on the road test sample set according to the second takeover parameter to obtain a second evaluation result; randomly setting the takeover parameters and jumping to execute the step of sampling the driving data according to preset sampling conditions to obtain a corresponding sample set, until the step is executed a preset number of times, resulting in multiple first evaluation results and multiple second evaluation results, wherein the evaluation results include each first evaluation result and each second evaluation result.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of evaluating the takeover behavior of the new and old versions of the autonomous driving algorithm in simulated test driving based on the first takeover parameter to obtain a first evaluation result includes: counting the number of first divergences in the simulated sample set where the new and old versions of the autonomous driving algorithm have different takeover behaviors in various driving scenarios; calculating the first simulated takeover number of the new and old versions of the autonomous driving algorithm in simulated test driving based on the first divergence number, and evaluating the takeover behavior of the simulated sample set to obtain a second simulated takeover number; adjusting the first and second simulated takeover numbers according to the first takeover parameter and a preset random algorithm, and using the adjusted first and second simulated takeover numbers as the first evaluation result.

[0008] Optionally, in a third implementation of the first aspect of the present invention, adjusting the first and second simulated takeover numbers according to the first takeover parameters and a preset random algorithm includes: extracting the first takeover repair probability, takeover correlation, and takeover misjudgment probability from the first takeover parameters; generating a first random number based on the first takeover repair probability and the takeover correlation; and adjusting the first and second simulated takeover numbers based on the first random number and the takeover misjudgment probability.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of evaluating the takeover behavior of the new version of the autonomous driving algorithm during road testing based on the second takeover parameter to obtain a second evaluation result includes: counting the first road test takeover number of the new version of the autonomous driving algorithm in each driving scenario in the road test sample set; determining the second takeover repair probability based on the second takeover parameter, and generating a second random number based on the second takeover repair probability; adjusting the first road test takeover number based on the second random number to obtain a second road test takeover number, and using the first road test takeover number and the second road test takeover number as the second evaluation result.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of statistically verifying the takeover behavior distribution of the evaluation results to obtain a verification result includes: respectively statistically analyzing the first takeover behavior distribution information of the first simulated takeover number and the second takeover behavior distribution information of the second simulated takeover number; comparing the first takeover behavior distribution information and the second takeover behavior distribution information, and obtaining a first verification result based on the comparison result; calculating a first average distance of the takeover behavior distribution based on the first drive-test takeover number and calculating a second average distance of the takeover behavior distribution based on the second drive-test takeover number; comparing the first average distance and the second average distance, and obtaining a second verification result based on the comparison result, wherein the verification result includes the first verification result and the second verification result.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of calculating the statistical efficacy corresponding to the takeover behavior assessment based on the verification results includes: according to a preset takeover reference number, counting the effective takeover behaviors in the first verification results, and calculating a first proportion of the effective takeover behaviors in the first verification results to all takeover behaviors; according to a preset takeover reference distance, counting the effective takeover behaviors in the second verification results, and calculating a second proportion of the effective takeover behaviors in the second verification results to all takeover behaviors; using the first proportion as the statistical efficacy corresponding to the takeover behavior assessment of the new and old versions of the autonomous driving algorithm in simulated test driving, and using the second proportion as the statistical efficacy corresponding to the takeover behavior assessment of the new and old versions of the autonomous driving algorithm in road test driving.

[0012] A second aspect of the present invention provides a statistical efficacy evaluation device, comprising: a data acquisition module for acquiring driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms; a behavior evaluation module for evaluating the takeover behavior of the new and old versions of autonomous driving algorithms on the driving data according to the takeover parameters and using a preset resampling algorithm, and obtaining an evaluation result; and an efficacy calculation module for statistically verifying the distribution of takeover behavior on the evaluation result, obtaining a verification result, and calculating the statistical efficacy corresponding to the takeover behavior evaluation based on the verification result.

[0013] Optionally, in a first implementation of the second aspect of the present invention, the behavior evaluation module includes: a sample sampling unit, configured to sample the driving data according to preset sampling conditions to obtain a corresponding sample set, wherein the sample set includes multiple simulated sample sets and road test sample sets corresponding to multiple driving scenarios; a first evaluation unit, configured to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm during simulated test driving based on the first takeover parameter, and obtain a first evaluation result; a second evaluation unit, configured to evaluate the takeover behavior of the new version of the autonomous driving algorithm during road test driving based on the second takeover parameter, and obtain a second evaluation result; and a resampling evaluation unit, configured to randomly set the takeover parameter and jump to execute the step of sampling the driving data according to preset sampling conditions to obtain a corresponding sample set, until the step is executed a preset number of times, thereby obtaining multiple first evaluation results and multiple second evaluation results, wherein the evaluation results include each first evaluation result and each second evaluation result.

[0014] Optionally, in a second implementation of the second aspect of the present invention, the first evaluation unit includes: counting the number of first divergences in the simulated sample set where the new and old versions of the autonomous driving algorithm exhibit different takeover behaviors in various driving scenarios; calculating the first simulated takeover number of the new and old versions of the autonomous driving algorithm during simulated test driving based on the first divergence number, and evaluating the takeover behavior of the simulated sample set to obtain a second simulated takeover number; adjusting the first and second simulated takeover numbers according to the first takeover parameter and a preset random algorithm, and using the adjusted first and second simulated takeover numbers as the first evaluation result.

[0015] Optionally, in a third implementation of the second aspect of the present invention, the first evaluation unit further includes: extracting the first takeover repair probability, takeover correlation and takeover misjudgment probability from the first takeover parameters; generating a first random number based on the first takeover repair probability and the takeover correlation; and adjusting the first and second simulated takeover numbers based on the first random number and the takeover misjudgment probability.

[0016] Optionally, in a fourth implementation of the second aspect of the present invention, the second evaluation unit includes: counting the number of first road test takeovers of the new version of the autonomous driving algorithm in various driving scenarios in the road test sample set; determining the second takeover repair probability based on the second takeover parameter, and generating a second random number based on the second takeover repair probability; adjusting the first road test takeover number based on the second random number to obtain the second road test takeover number, and using the first road test takeover number and the second road test takeover number as the second evaluation result.

[0017] Optionally, in a fifth implementation of the second aspect of the present invention, the efficacy calculation module includes: a distribution statistics unit, used to respectively count the first takeover behavior distribution information of the first simulated takeover number and the second takeover behavior distribution information of the second simulated takeover number; a distribution comparison unit, used to compare the first takeover behavior distribution information and the second takeover behavior distribution information, and obtain a first verification result based on the comparison result; a distance calculation unit, used to calculate a first average distance of the takeover behavior distribution based on the first drive-test takeover number and a second average distance of the takeover behavior distribution based on the second drive-test takeover number; and a distance comparison unit, used to compare the first average distance and the second average distance, and obtain a second verification result based on the comparison result, wherein the verification result includes the first verification result and the second verification result.

[0018] Optionally, in a sixth implementation of the second aspect of the present invention, the efficacy calculation module further includes: a first proportion calculation unit, configured to count the effective takeover behaviors in the first verification result according to a preset takeover reference number, and calculate a first proportion of the effective takeover behaviors in the first verification result to all takeover behaviors; a second proportion calculation unit, configured to count the effective takeover behaviors in the second verification result according to a preset takeover reference distance, and calculate a second proportion of the effective takeover behaviors in the second verification result to all takeover behaviors; and a proportion evaluation unit, configured to use the first proportion as the statistical efficacy corresponding to the takeover behavior evaluation of the new and old versions of the autonomous driving algorithm in simulated test driving, and to use the second proportion as the statistical efficacy corresponding to the takeover behavior evaluation of the new and old versions of the autonomous driving algorithm in road test driving.

[0019] A third aspect of the present invention provides a statistical efficacy evaluation device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the statistical efficacy evaluation device to perform the various steps of the above-described statistical efficacy evaluation method.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the statistical power evaluation method described above.

[0021] The technical solution provided by this invention involves acquiring driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms; evaluating the takeover behavior of the new and old versions of autonomous driving algorithms using a preset resampling algorithm according to the takeover parameters, and obtaining evaluation results; statistically verifying the distribution of takeover behavior in the evaluation results to obtain verification results; and calculating the statistical power corresponding to the takeover behavior evaluation based on the verification results. Compared with the prior art, this application evaluates the takeover behavior by resampling from the acquired driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms using simulated driving and actual driving, then calculates the proportion of effective takeover behavior in each resampling evaluation result, and then compares the results of the proportion calculation to obtain the evaluation of the statistical power of simulated driving. By using resampling to evaluate the statistical power of simulated driving, an efficient evaluation of the statistical power of the autonomous driving algorithm evaluation process is achieved. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the first embodiment of the statistical power evaluation method in this invention;

[0023] Figure 2 This is a schematic diagram of the second embodiment of the statistical efficacy evaluation method in this invention;

[0024] Figure 3 This is a schematic diagram of the third embodiment of the statistical power evaluation method in this invention;

[0025] Figure 4 This is a schematic diagram of one embodiment of the statistical efficacy evaluation device in this invention;

[0026] Figure 5 This is a schematic diagram of another embodiment of the statistical efficacy evaluation device in this invention;

[0027] Figure 6 This is a schematic diagram of one embodiment of the statistical efficacy evaluation device in this invention. Detailed Implementation

[0028] This invention provides a statistical efficacy evaluation method, apparatus, device, and storage medium. The method includes: acquiring driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms; evaluating the takeover behavior of the new and old versions of autonomous driving algorithms on the driving data according to the takeover parameters using a preset resampling algorithm, and obtaining evaluation results; statistically verifying the distribution of takeover behavior on the evaluation results, obtaining verification results, and calculating the statistical efficacy corresponding to the takeover behavior evaluation based on the verification results. This application achieves efficient evaluation of the statistical efficacy of the autonomous driving algorithm evaluation process.

[0029] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the statistical power evaluation method in this invention includes:

[0031] 101. Obtain driving data and takeover parameters corresponding to the testing of the new and old versions of the autonomous driving algorithm;

[0032] It is understood that the executing entity of this invention can be a statistical efficacy evaluation device, a terminal, or a server; no specific limitation is made here. This embodiment of the invention will be described using a server as an example.

[0033] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0034] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0035] In this embodiment, the old and new versions of the autonomous driving algorithm refer to different versions of the autonomous driving algorithm before and after the improvement; the driving data refers to various driving data used to test the autonomous driving algorithm, such as driving scenario data, driving path data, and historical driving test data; the takeover parameters refer to the relevant parameters used to evaluate the experimental and control groups when resampling. Here, resampling is a statistical method of statistical distribution, which means assuming that the distribution of the old data observed in the future conforms to the empirical distribution of historical or already observed data, and then generating the future dataset through sampling simulation, calculating the corresponding indicators and statistically analyzing their distribution.

[0036] In practical applications, by acquiring the latest version of the current autonomous driving algorithm and its corresponding older version, and then searching through the relevant autonomous driving database, driving data corresponding to the testing of both versions of the autonomous driving algorithm is obtained, along with pre-set takeover parameters for resampling. By acquiring the new version of the autonomous driving algorithm, the corresponding driving data, and the takeover parameters, the data preparation required for resampling is constructed, enabling effective statistical power evaluation of different testing methods under various test data factors.

[0037] 102. Based on the takeover parameters, use the preset resampling algorithm to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data, and obtain the evaluation results.

[0038] In this embodiment, the resampling algorithm refers to sampling driving scenarios multiple times and then constructing two test datasets (an experimental group and a control group as examples) using takeover parameters. This allows for the evaluation of the two repeatedly sampled datasets. The takeover behavior refers to the number of times the autonomous driving system takes over during simulated or road-test driving of the corresponding test scenarios. The takeover behavior evaluation refers to the statistical analysis of the number of takeover behaviors performed on the sampled scenario set using simulated or road-test driving. By constructing different scenario sets using resampling and using takeover parameters to construct two sets of takeover test data corresponding to the sampling, and by using simulated driving and road testing to evaluate takeover behavior, takeover evaluation is achieved for different scenario combinations and varying numbers of takeovers, ensuring the scientific validity of a large amount of test data and evaluation results.

[0039] In practical applications, the takeover parameters here include first takeover parameters and second takeover parameters. Driving data is sampled according to preset sampling conditions to obtain a corresponding sample set, which includes simulated sample sets and road test sample sets corresponding to multiple driving scenarios. Then, based on the first takeover parameters, the takeover behavior of the new and old versions of the autonomous driving algorithm is evaluated during simulated test driving on the simulated sample set, yielding a first evaluation result. Based on the second takeover parameters, the takeover behavior of the new version of the autonomous driving algorithm is evaluated during road test driving on the road test sample set, yielding a second evaluation result. By randomly setting the takeover parameters and jumping to execute the steps of sampling driving data according to preset sampling conditions to obtain a corresponding sample set, and evaluating takeover behavior during simulated driving and road test driving, the process continues until a preset number of executions is reached, resulting in multiple first evaluation results and multiple second evaluation results. The evaluation results include each first evaluation result and each second evaluation result.

[0040] 103. Perform statistical verification of the distribution of takeover behavior on the assessment results, obtain the verification results, and calculate the statistical power corresponding to the takeover behavior assessment based on the verification results.

[0041] In this embodiment, statistical verification refers to the statistical analysis of the number of interventions in each of the two road test methods' two large sampling groups, followed by the creation of corresponding verification charts to calculate the probability that the number of interventions and non-interventions in the test group reach a preset ratio. Statistical efficacy refers to the statistical analysis of the autonomous driving algorithm's efficacy in terms of intervention counts and intervention mileage based on the aforementioned statistical efficacy results, thereby evaluating the statistical efficacy of the corresponding road test method. By statistically verifying the two intervention results and calculating statistical efficacy, and utilizing the corresponding verification charts and intervention count and intervention mileage calculation indicators, statistical analysis of the road test evaluation efficacy of the simulation platform or other test methods is achieved.

[0042] In practical applications, the distribution information of the first takeover behavior of the first simulated takeover number and the distribution information of the second takeover behavior of the second simulated takeover number are statistically analyzed and compared. A first verification result is obtained based on the comparison. A first average distance of the takeover behavior distribution is calculated based on the first road test takeover number, and a second average distance of the takeover behavior distribution is calculated based on the second road test takeover number. The first and second average distances are then compared, and a second verification result is obtained based on the comparison. The verification result includes both the first and second verification results. Then, according to a preset takeover reference number, the effective takeover behaviors in the first verification result are statistically analyzed, and a first proportion of effective takeover behaviors in the first verification result relative to all takeover behaviors is calculated. According to a preset takeover reference distance, the effective takeover behaviors in the second verification result are statistically analyzed, and a second proportion of effective takeover behaviors in the second verification result relative to all takeover behaviors is calculated. The first proportion is used as the statistical efficacy of the new and old versions of the autonomous driving algorithm in simulated test driving takeover behavior evaluation, and the second proportion is used as the statistical efficacy of the new and old versions of the autonomous driving algorithm in road test driving takeover behavior evaluation.

[0043] In this embodiment of the invention, driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms are acquired. Based on the takeover parameters, a preset resampling algorithm is used to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data, obtaining evaluation results. The evaluation results are then statistically validated based on the distribution of takeover behavior, obtaining validation results. Based on the validation results, the statistical power corresponding to the takeover behavior evaluation is calculated. Compared to existing technologies, this application evaluates the takeover behavior by resampling from the acquired driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms using simulated driving and actual driving. The proportion of effective takeover behavior is calculated from each resampling evaluation result, and the statistical power of simulated driving is evaluated by comparing the calculated proportions. By using resampling to evaluate the statistical power of simulated driving, an efficient evaluation of the statistical power of the autonomous driving algorithm evaluation process is achieved.

[0044] Please see Figure 2 The second embodiment of the statistical power evaluation method in this invention includes:

[0045] 201. Obtain driving data and takeover parameters corresponding to the testing of the new and old versions of the autonomous driving algorithm;

[0046] 202. Sampling is performed on the driving data according to the preset sampling conditions to obtain the corresponding sample set, which includes the simulation sample set and the road test sample set corresponding to multiple driving scenarios.

[0047] In this embodiment, the sampling conditions include two aspects: simulated driving sampling and road test driving sampling. The simulated driving sampling condition involves sampling with replacement at the bag level from the acquired driving data, ensuring that the number of sampled scenarios is the same as the original number of scenarios. Here, a bag refers to the data storage unit for simulated driving; each bag contains information such as the main vehicle, obstacles, and map at the time of data collection, and the duration covered by a bag varies from several seconds to several minutes. The road test driving sampling condition involves sampling with replacement from the road test dataset at the driving session level until the total sampled mileage is greater than or equal to the original dataset.

[0048] In practical applications, based on the driving data obtained above, and according to the preset sampling conditions, the scenarios in the simulated driving scenario set are sampled with replacement at the bag level, and the number of scenarios after sampling is the same as the number of scenarios in the original set, to obtain the simulated sample set; in road test driving, the road test scenario dataset is sampled with replacement at the driving session level until the total sampling mileage is greater than or equal to the original dataset, to obtain the road test sample set.

[0049] 203. Count the number of first divergences in the simulation sample set where the new and old versions of the autonomous driving algorithm have different takeover behaviors in various driving scenarios;

[0050] In this embodiment, the discrepancy refers to the fact that under the test of various driving scenarios, the autonomous driving algorithm designed to take over is sometimes designed to take over and sometimes not, and the different test results lead to the discrepancy.

[0051] In practical applications, based on the simulated sample set obtained from the above processing, the takeover behavior of each sample in the simulated sample set is compared using simulated driving. Based on the evaluation results of the takeover behavior, the number of first divergences in the simulated sample set where the new and old versions of the autonomous driving algorithm have different takeover behaviors in various driving scenarios is counted.

[0052] 204. Based on the first number of divergences, calculate the first simulated takeover number of the new and old versions of the autonomous driving algorithm during simulated test driving, and evaluate the takeover behavior of the simulated sample set to obtain the second simulated takeover number.

[0053] In this embodiment, based on the first number of divergences obtained from the above processing, the first simulated takeover number is obtained by using the takeover number corresponding to each divergence scenario in the simulated driving test of the new and old versions of the autonomous driving algorithm, and the second simulated takeover number is obtained by evaluating the takeover behavior of each scenario set corresponding to each sample in the simulated sample set.

[0054] 205. Based on the first takeover parameters, adjust the first and second simulated takeover numbers according to a preset random algorithm, and use the adjusted first and second simulated takeover numbers as the first evaluation result;

[0055] In this embodiment, the first takeover parameter includes four parameters: P_fix: the degree to which the simulated new autonomous driving version is better than the observed old autonomous driving version. In the simulated generation of the new autonomous driving version, each takeover has a probability of being repaired with a P_fix value; Fp: the probability that a non-takeover (diverge) is misjudged as a takeover in the simulated road test; Fn: the probability that a takeover is misjudged as a non-takeover in the simulated road test; Rho: the takeover correlation between different autonomous driving versions in the simulated road test. The higher the correlation, the greater the probability that both autonomous driving versions will takeover in the same simulation scenario. The random algorithm here refers to the algorithm for generating relevant random numbers. A preferred implementation method is to generate the corresponding random numbers using multivariate random numbers, that is, by extracting the first takeover repair probability, takeover correlation, and takeover misjudgment probability from the first takeover parameter; generating the first random number based on the first takeover repair probability and takeover correlation; and adjusting the first and second simulated takeover numbers based on the first random number and the takeover misjudgment probability.

[0056] In practical applications, based on the first takeover parameter obtained from the above processing, and following a preset random algorithm, the first takeover parameter is first used as the control group data for simulated driving. Then, the first takeover parameter is adjusted using the takeover parameter P_fix, multiplied by (1-P_fix), and used as the experimental group data for simulated driving. The correlation Rho of the takeover parameter is used as the correlation between the takeover numbers in the experimental group and the control group. Multinomial distribution random numbers are used to generate two sets of correlated binomial distribution random numbers. Then, for the resampled experimental and control groups, human error is simulated and added to adjust the two data sets. For each takeover, there is a probability Fn of becoming non-takeover, and for each non-takeover divergence, there is a probability Fp of becoming takeover. The adjusted first and second simulated takeover numbers are then used as the first evaluation result. By using the first takeover parameter and multinomial distribution random numbers to construct two sets of simulated driving evaluation data, two sets of data can be constructed under corresponding conditions, reducing unnecessary data correlation and improving the scientific validity of the sample evaluation data, ultimately ensuring effective comparison of statistical power.

[0057] 206. Count the number of first-pass takeovers of the new version of the autonomous driving algorithm in various driving scenarios in the road test sample set;

[0058] In this embodiment, based on the results of the above sampling process, the mileage and number of take-offs of the new version of the autonomous driving algorithm in each driving scenario in the above road test sample set are counted as the road test data of the sampling control group to obtain the first road test take-off count.

[0059] 207. Determine the repair probability of the second connector based on the second connector parameters, and generate a second random number based on the repair probability of the second connector;

[0060] In this embodiment, the second takeover parameter refers to P_fix, which is the degree to which the simulated new autonomous driving algorithm is better than the observed old autonomous driving algorithm.

[0061] In practical applications, based on the second takeover parameters, the probability of second takeover repair corresponding to the degree to which the simulated new autonomous driving algorithm is better than the observed old autonomous driving algorithm is determined. Based on the second takeover repair probability, the mileage and number of takeovers on the sampling set are counted as sampling control group data. Then, a second random number is generated using a random number generation algorithm.

[0062] 208. Adjust the number of test connections for the first route based on the second random number to obtain the number of test connections for the second route, and use the number of test connections for the first route and the number of test connections for the second route as the second evaluation result;

[0063] In this embodiment, based on the aforementioned second random number, the probability of each of the first road test nozzles being repaired (p_fix) is simulated using the second random number and used as the road test data of the sampling experimental group to obtain the second road test nozzle count. The first and second road test nozzle counts are then used as the second evaluation result. By constructing two sets of road test sample sets, different evaluation result data under corresponding nozzle repair probabilities can be achieved, thereby diversifying the statistical data under different factors.

[0064] 209. Randomly set the takeover parameters and jump to execute the step of sampling driving data according to the preset sampling conditions to obtain the corresponding sample set until the preset number of executions is stopped, and multiple first evaluation results and multiple second evaluation results are obtained. The evaluation results include each first evaluation result and each second evaluation result.

[0065] In this embodiment, by setting the first and second takeover parameters at any time, the system jumps to execute steps to sample the driving data according to preset sampling conditions, obtain corresponding sample sets, and then construct two sets of datasets from the samples. This process continues until a preset number of iterations are reached, achieving resampling and obtaining multiple first evaluation results and multiple second evaluation results. The evaluation results include each first evaluation result and each second evaluation result. By utilizing the resampling method and random parameters to resample the driving data, evaluation results corresponding to various scenario sets are obtained, making the evaluation data more randomized and ensuring the effectiveness and scientific validity of the final statistical power.

[0066] 210. Perform statistical verification of the distribution of takeover behavior on the assessment results, obtain the verification results, and calculate the statistical power corresponding to the takeover behavior assessment based on the verification results.

[0067] In this embodiment of the invention, the number of first divergences where the new and old versions of the autonomous driving algorithm exhibit different takeover behaviors in various driving scenarios is statistically analyzed in the simulated sample set. Based on the first divergence number, the first simulated takeover number of the new and old versions of the autonomous driving algorithm during simulated test driving is calculated, and the takeover behavior of the simulated sample set is evaluated to obtain a second simulated takeover number. Based on the first takeover parameter, the first and second simulated takeover numbers are adjusted according to a preset random algorithm, and the adjusted first and second simulated takeover numbers are used as the first evaluation result. The first road test takeover number of the new version of the autonomous driving algorithm in various driving scenarios is statistically analyzed in the road test sample set. Based on the second takeover parameter, the second takeover repair probability is determined, and a second random number is generated based on the second takeover repair probability. Based on the second random number, the first road test takeover number is adjusted to obtain the second road test takeover number, and the first and second road test takeover numbers are used as the second evaluation result. Compared to existing technologies, this application utilizes resampling methods and takeover parameters to construct multiple first evaluation results and multiple second evaluation results for simulated driving and road test driving based on driving data. This enables scenario evaluation combinations of various random combinations of driving data, allowing the final efficacy evaluation to comprehensively and fully calculate the situation under multiple factors, so that the final efficacy statistics are more in line with experimental scientificity and evaluation needs.

[0068] Please see Figure 3 The third embodiment of the statistical power evaluation method in this invention includes:

[0069] 301. Obtain driving data and takeover parameters corresponding to the testing of the new and old versions of the autonomous driving algorithm;

[0070] 302. Based on the takeover parameters, use the preset resampling algorithm to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data, and obtain the evaluation results.

[0071] 303. Calculate the distribution information of the first takeover behavior of the first simulated takeover number and the distribution information of the second takeover behavior of the second simulated takeover number respectively;

[0072] In this embodiment, the distribution information refers to the normal distribution plot drawn by performing z-verification on the evaluation results, thereby obtaining the data distribution information of all evaluation results.

[0073] In practical applications, the z-verification method is used to plot a normal distribution map of the first takeover behavior of the first simulated takeover number obtained from the above processing, and then the plotted normal distribution map is analyzed to obtain the distribution information of the first takeover behavior. Similarly, the z-verification method is used to plot a normal distribution map of the second takeover behavior of the second simulated takeover number obtained from the above processing, and then the plotted normal distribution map is analyzed to obtain the distribution information of the second takeover behavior.

[0074] 304. Compare the distribution information of the first takeover behavior and the distribution information of the second takeover behavior, and obtain the first verification result based on the comparison results;

[0075] In this embodiment, based on the first and second takeover behavior distribution information obtained through the above processing, the distribution interval probabilities of the two distribution information are compared according to a preset normal distribution probability. Then, based on the comparison result, a first verification result is obtained. A preferred implementation method here is to statistically analyze the data that satisfy the effective takeover behavior within the 95% interval of the normal distribution, compare the data sizes of the two, and obtain the first verification result. By using a normal distribution graph to assist in the analysis and evaluation results, the probability of satisfying the takeover behavior can be obtained, providing a more scientific probability data analysis result for the statistical effectiveness of the corresponding evaluation method.

[0076] 305. Calculate the first average distance of the distribution of takeover behavior based on the number of takeovers in the first route test, and calculate the second average distance of the distribution of takeover behavior based on the number of takeovers in the second route test;

[0077] In this embodiment, the average distance refers to the average number of miles between the takeover of the autonomous driving algorithm in simulated driving or actual driving.

[0078] In practical applications, the first number of takeovers in simulated driving is obtained based on the above processing, the takeover mileage corresponding to the distribution of takeover behaviors is calculated, and the takeover mileages corresponding to each takeover behavior distribution are added together to obtain the first average distance; and the second number of takeovers in simulated driving is obtained based on the above processing, the takeover mileage corresponding to the distribution of takeover behaviors is calculated, and the takeover mileages corresponding to each takeover behavior distribution are added together to obtain the second average distance.

[0079] 306. Compare the first average distance and the second average distance, and obtain the second verification result based on the comparison result. The verification result includes the first verification result and the second verification result.

[0080] In this embodiment, the calculated first average distance and second average distance are compared to determine the autonomous driving mileage distance, and a second verification result is obtained based on the comparison result. Furthermore, the verification result includes both the first and second verification results.

[0081] 307. According to the preset takeover reference number, count the effective takeover behaviors in the first verification result, and calculate the first proportion of effective takeover behaviors in the first verification result to all takeover behaviors;

[0082] In this embodiment, the takeover reference number refers to the takeover behavior that meets the autonomous driving conditions within the probability interval corresponding to the takeover behavior in the autonomous driving test. By using the takeover reference number to extract the valid takeover behavior data from the first verification result that meets the preset conditions, and then processing the takeover behavior proportion, the effectiveness evaluation of the autonomous driving of the autonomous driving algorithm can be obtained.

[0083] In practical applications, the number of effective takeover behaviors that meet the autonomous driving conditions within the corresponding range is counted according to the preset takeover reference number. Then, based on the number of effective takeover behaviors obtained from the statistics, the proportion of the effective takeover behaviors to all takeover behaviors in the first verification result is calculated to obtain the first proportion.

[0084] 308. According to the preset takeover reference distance, count the effective takeover behaviors in the second verification results, and calculate the second proportion of effective takeover behaviors in the second verification results to all takeover behaviors;

[0085] In this embodiment, the takeover reference distance refers to the mileage of a vehicle's autonomous driving process that meets the corresponding level of automated driving. By analyzing the second verification result using the takeover reference distance, the number of effective takeover actions within the mileage required for the autonomous driving algorithm to meet the corresponding level of automated driving processing can be obtained, thus enabling an assessment of the automation level of the autonomous driving algorithm.

[0086] In practical applications, based on the preset takeover reference distance, the effective takeover behaviors that meet the corresponding takeover reference distance in the second verification results are counted, and the proportion of the effective takeover behaviors to all takeover behaviors in the second verification results is calculated based on the number of effective takeover behaviors obtained from the statistics, thus obtaining the second proportion.

[0087] 309. The first ratio is used as the statistical efficacy of the new and old versions of the autonomous driving algorithm in the evaluation of takeover behavior in simulated test driving, and the second ratio is used as the statistical efficacy of the new and old versions of the autonomous driving algorithm in the evaluation of takeover behavior in road test driving.

[0088] In this embodiment, based on the above processing results, the first ratio is used as the statistical efficacy of the new and old versions of the autonomous driving algorithm in evaluating takeover behavior during simulated test driving, and the second ratio is used as the statistical efficacy of the new and old versions of the autonomous driving algorithm in evaluating takeover behavior during road test driving. This achieves the statistical evaluation of the efficacy of the new and old versions of the autonomous driving algorithm in different scenario sets using different road testing methods.

[0089] In this embodiment of the invention, a first verification result is obtained by statistically analyzing and comparing the first takeover behavior distribution information of the first simulated takeover number and the second takeover behavior distribution information of the second simulated takeover number. Then, based on the first road test takeover number, a first average distance of the takeover behavior distribution is calculated, and based on the second road test takeover number, a second average distance of the takeover behavior distribution is calculated and compared to obtain a second verification result. Finally, the proportion of effective takeover behaviors in the two verification results is statistically analyzed, and the statistical efficacy of the corresponding road test method is obtained by evaluating the proportion. Compared to existing technologies, this application verifies the first and second evaluation results and calculates effective takeover behaviors, enabling the statistical evaluation of the evaluation efficacy of new and old versions of autonomous driving algorithms in different scenario sets using different road test methods, and providing a more efficient evaluation of the efficacy of road test methods under different scenario datasets.

[0090] The statistical power evaluation method in the embodiments of the present invention has been described above. The statistical power evaluation apparatus in the embodiments of the present invention will be described below. Please refer to [link / reference]. Figure 4 One embodiment of the statistical efficacy evaluation device in this invention includes:

[0091] The data acquisition module 401 is used to acquire driving data and takeover parameters corresponding to the testing of new and old versions of autonomous driving algorithms.

[0092] The behavior evaluation module 402 is used to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data according to the takeover parameters and using a preset resampling algorithm, and obtain the evaluation result.

[0093] The efficacy calculation module 403 is used to perform statistical verification of the distribution of takeover behavior on the evaluation results, obtain the verification results, and calculate the statistical efficacy corresponding to the takeover behavior evaluation based on the verification results.

[0094] In this embodiment of the invention, driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms are acquired. Based on the takeover parameters, a preset resampling algorithm is used to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data, obtaining evaluation results. The evaluation results are then statistically validated based on the distribution of takeover behavior, obtaining validation results. Based on the validation results, the statistical power corresponding to the takeover behavior evaluation is calculated. Compared to existing technologies, this application evaluates the takeover behavior by resampling from the acquired driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms using simulated driving and actual driving. The proportion of effective takeover behavior is calculated from each resampling evaluation result, and the statistical power of simulated driving is evaluated by comparing the calculated proportions. By using resampling to evaluate the statistical power of simulated driving, an efficient evaluation of the statistical power of the autonomous driving algorithm evaluation process is achieved.

[0095] Please see Figure 5 Another embodiment of the statistical efficacy evaluation device in this invention includes:

[0096] The data acquisition module 401 is used to acquire driving data and takeover parameters corresponding to the testing of new and old versions of autonomous driving algorithms.

[0097] The behavior evaluation module 402 is used to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data according to the takeover parameters and using a preset resampling algorithm, and obtain the evaluation result.

[0098] The efficacy calculation module 403 is used to perform statistical verification of the distribution of takeover behavior on the evaluation results, obtain the verification results, and calculate the statistical efficacy corresponding to the takeover behavior evaluation based on the verification results.

[0099] Furthermore, the behavior assessment module 402 includes:

[0100] The sample sampling unit 4021 is used to sample the driving data according to preset sampling conditions to obtain a corresponding sample set, wherein the sample set includes multiple simulated sample sets and road test sample sets corresponding to multiple driving scenarios; the first evaluation unit 4022 is used to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm during simulated test driving based on the first takeover parameters to obtain a first evaluation result; the second evaluation unit 4023 is used to evaluate the takeover behavior of the new version of the autonomous driving algorithm during road test driving based on the second takeover parameters to obtain a second evaluation result; the resampling evaluation unit 4024 is used to randomly set the takeover parameters and jump to execute the step of sampling the driving data according to preset sampling conditions to obtain a corresponding sample set until the step is executed a preset number of times, thereby obtaining multiple first evaluation results and multiple second evaluation results, wherein the evaluation results include each first evaluation result and each second evaluation result.

[0101] Furthermore, the first evaluation unit 4022 includes:

[0102] The number of first divergences in the simulated sample set where the new and old versions of the autonomous driving algorithm exhibit different takeover behaviors in various driving scenarios is counted. Based on the first divergence number, the first simulated takeover number of the new and old versions of the autonomous driving algorithm during simulated test driving is calculated, and the takeover behavior of the simulated sample set is evaluated to obtain the second simulated takeover number. Based on the first takeover parameter, the first and second simulated takeover numbers are adjusted according to a preset random algorithm, and the adjusted first and second simulated takeover numbers are used as the first evaluation result.

[0103] Furthermore, the first evaluation unit 4022 also includes:

[0104] Extract the first takeover repair probability, takeover correlation, and takeover misjudgment probability from the first takeover parameters; generate a first random number based on the first takeover repair probability and the takeover correlation, and adjust the first and second simulated takeover numbers based on the first random number and the takeover misjudgment probability.

[0105] Furthermore, the second evaluation unit 4023 includes:

[0106] The number of first road test takeovers of the new version of the autonomous driving algorithm in each driving scenario is counted in the road test sample set; the second takeover repair probability is determined according to the second takeover parameter, and a second random number is generated based on the second takeover repair probability; the first road test takeover number is adjusted according to the second random number to obtain the second road test takeover number, and the first road test takeover number and the second road test takeover number are used as the second evaluation result.

[0107] Furthermore, the efficacy calculation module 403 includes:

[0108] The distribution statistics unit 4031 is used to respectively count the first takeover behavior distribution information of the first simulated takeover number and the second takeover behavior distribution information of the second simulated takeover number; the distribution comparison unit 4032 is used to compare the first takeover behavior distribution information and the second takeover behavior distribution information, and obtain a first verification result based on the comparison result; the distance calculation unit 6033 is used to calculate a first average distance of the takeover behavior distribution based on the first drive-test takeover number and a second average distance of the takeover behavior distribution based on the second drive-test takeover number; the distance comparison unit 6034 is used to compare the first average distance and the second average distance, and obtain a second verification result based on the comparison result, wherein the verification result includes the first verification result and the second verification result.

[0109] Furthermore, the efficacy calculation module 403 also includes:

[0110] The first proportion calculation unit 4035 is used to count the effective takeover behaviors in the first verification result according to a preset takeover reference number, and calculate the first proportion of the effective takeover behaviors in the first verification result to all takeover behaviors; the second proportion calculation unit 4036 is used to count the effective takeover behaviors in the second verification result according to a preset takeover reference distance, and calculate the second proportion of the effective takeover behaviors in the second verification result to all takeover behaviors; the proportion evaluation unit 4037 is used to use the first proportion as the statistical efficacy of the new and old versions of the autonomous driving algorithm in simulated test driving, and to use the second proportion as the statistical efficacy of the new and old versions of the autonomous driving algorithm in road test driving.

[0111] In this embodiment of the invention, driving data and takeover parameters corresponding to tests of new and old versions of autonomous driving algorithms are acquired; a first evaluation of takeover behavior in simulated driving is conducted using four takeover parameters through resampling; and a second evaluation of takeover behavior in road-tested driving is conducted using one takeover parameter through resampling. The proportion of effective takeover behavior is then calculated based on the results of the first and second evaluations, and the calculated takeover proportions are compared to obtain the statistical efficacy of simulated driving and actual driving. Compared to existing technologies, this application achieves highly efficient efficacy evaluation of the simulation platform by using resampling and preset takeover parameters to evaluate the efficacy of various takeover behaviors in simulated driving.

[0112] above Figure 4 and Figure 5The statistical efficacy evaluation device in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The statistical efficacy evaluation device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0113] Figure 6 This is a schematic diagram of the structure of a statistical efficacy evaluation device 600 provided in an embodiment of the present invention. The statistical efficacy evaluation device 600 can vary considerably due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 610 (e.g., one or more processors) and a memory 620, and one or more storage media 630 (e.g., one or more mass storage devices) for storing application programs 633 or data 632. The memory 620 and storage media 630 can be temporary or persistent storage. The program stored in the storage media 630 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the statistical efficacy evaluation device 600. Furthermore, the processor 610 may be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the statistical efficacy evaluation device 600.

[0114] The statistical efficacy evaluation device 600 may also include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input / output interfaces 660, and / or one or more operating systems 631, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 6 The illustrated structure of the statistical power evaluation device does not constitute a limitation on the statistical power evaluation device. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0115] The present invention also provides a statistical power evaluation device, wherein the computer device includes a memory and a processor, the memory storing computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor performs each step of the statistical power evaluation method in the above embodiments.

[0116] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the various steps of the statistical power evaluation method.

[0117] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0118] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0120] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A statistical power evaluation method, characterized in that, The statistical power evaluation method includes: Obtain driving data and takeover parameters corresponding to the testing of new and old versions of autonomous driving algorithms; According to the takeover parameters, the takeover behavior of the new and old versions of the autonomous driving algorithm is evaluated on the driving data using a preset resampling algorithm to obtain the evaluation results. The assessment results are statistically validated based on the distribution of takeover behavior to obtain validation results. Based on the validation results, the statistical power corresponding to the takeover behavior assessment is calculated. The takeover parameters include first takeover parameters and second takeover parameters. The step of evaluating the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data according to the takeover parameters and using a preset resampling algorithm to obtain evaluation results includes: sampling the driving data according to preset sampling conditions to obtain corresponding sample sets, wherein the sample sets include multiple simulated sample sets and road test sample sets corresponding to multiple driving scenarios; evaluating the takeover behavior of the new and old versions of the autonomous driving algorithm during simulated test driving on the simulated sample sets according to the first takeover parameters to obtain a first evaluation result; evaluating the takeover behavior of the new version of the autonomous driving algorithm during road test driving on the road test sample sets according to the second takeover parameters to obtain a second evaluation result; randomly setting the takeover parameters and jumping to execute the step of sampling the driving data according to preset sampling conditions to obtain corresponding sample sets, until the step is executed a preset number of times, resulting in multiple first evaluation results and multiple second evaluation results, wherein the evaluation results include each first evaluation result and each second evaluation result.

2. The statistical power evaluation method according to claim 1, characterized in that, The step of evaluating the takeover behavior of the new and old versions of the autonomous driving algorithm during simulated test driving based on the first takeover parameter to obtain a first evaluation result includes: Count the number of first divergences in the simulated sample set where the new and old versions of the autonomous driving algorithm exhibit different takeover behaviors in various driving scenarios; Based on the first number of divergences, the first simulated takeover number of the new and old versions of the autonomous driving algorithm during simulated test driving is calculated, and the takeover behavior is evaluated on the simulated sample set to obtain the second simulated takeover number. Based on the first takeover parameters, the first and second simulated takeover numbers are adjusted according to a preset random algorithm, and the adjusted first and second simulated takeover numbers are used as the first evaluation result.

3. The statistical power evaluation method according to claim 2, characterized in that, The step of adjusting the number of the first and second simulated connections according to the first connection parameters and a preset random algorithm includes: Extract the first takeover repair probability, takeover correlation, and takeover misjudgment probability from the first takeover parameters; A first random number is generated based on the first takeover repair probability and the takeover correlation, and the first and second simulated takeover numbers are adjusted based on the first random number and the takeover misjudgment probability.

4. The statistical power evaluation method according to claim 2, characterized in that, The step of evaluating the takeover behavior of the new version of the autonomous driving algorithm during road testing based on the second takeover parameter, and obtaining a second evaluation result, includes: Count the number of first-pass takeovers of the new version of the autonomous driving algorithm in each driving scenario in the road test sample set; Based on the second pipe connection parameters, determine the second pipe connection repair probability, and generate a second random number based on the second pipe connection repair probability; Based on the second random number, the first road test take-off number is adjusted to obtain the second road test take-off number, and the first road test take-off number and the second road test take-off number are used as the second evaluation result.

5. The statistical power evaluation method according to claim 4, characterized in that, The statistical verification of the takeover behavior distribution on the evaluation results yields the verification results, including: The distribution information of the first takeover behavior of the first simulated takeover number and the distribution information of the second takeover behavior of the second simulated takeover number are statistically analyzed respectively. The first takeover behavior distribution information and the second takeover behavior distribution information are compared, and a first verification result is obtained based on the comparison result; Based on the first number of road test takeovers, calculate the first average distance of the takeover behavior distribution, and based on the second number of road test takeovers, calculate the second average distance of the takeover behavior distribution; The first average distance and the second average distance are compared, and a second verification result is obtained based on the comparison result, wherein the verification result includes the first verification result and the second verification result.

6. The statistical power evaluation method according to claim 5, characterized in that, The step of calculating the statistical power corresponding to the takeover behavior assessment based on the verification results includes: According to the preset takeover reference number, count the effective takeover behaviors in the first verification result, and calculate the first proportion of the effective takeover behaviors in the first verification result to all takeover behaviors; According to the preset takeover reference distance, count the effective takeover behaviors in the second verification results, and calculate the second proportion of effective takeover behaviors in the second verification results to all takeover behaviors; The first ratio is used as the statistical power of the new and old versions of the autonomous driving algorithm in evaluating the takeover behavior in simulated test driving, and the second ratio is used as the statistical power of the new and old versions of the autonomous driving algorithm in evaluating the takeover behavior in road test driving.

7. A statistical power evaluation device, characterized in that, The statistical power evaluation device includes: The data acquisition module is used to acquire driving data and takeover parameters corresponding to the testing of new and old versions of autonomous driving algorithms; The behavior evaluation module is used to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm on the driving data according to the takeover parameters and using a preset resampling algorithm, and obtain the evaluation result. The efficacy calculation module is used to perform statistical verification of the distribution of takeover behavior on the evaluation results, obtain the verification results, and calculate the statistical efficacy corresponding to the takeover behavior evaluation based on the verification results. The takeover parameters include a first takeover parameter and a second takeover parameter. The behavior evaluation module includes: a sample sampling unit, used to sample the driving data according to preset sampling conditions to obtain a corresponding sample set, wherein the sample set includes a simulation sample set and a road test sample set corresponding to multiple driving scenarios; a first evaluation unit, used to evaluate the takeover behavior of the new and old versions of the autonomous driving algorithm in simulated test driving based on the first takeover parameter, and obtain a first evaluation result; a second evaluation unit, used to evaluate the takeover behavior of the new version of the autonomous driving algorithm in road test driving based on the second takeover parameter, and obtain a second evaluation result; and a resampling evaluation unit, used to randomly set the takeover parameters and jump to execute the step of sampling the driving data according to preset sampling conditions to obtain a corresponding sample set, until the step is executed a preset number of times, and obtain multiple first evaluation results and multiple second evaluation results, wherein the evaluation results include each first evaluation result and each second evaluation result.

8. A statistical power evaluation device, characterized in that, The statistical efficacy evaluation device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the statistical power evaluation device to perform the steps of the statistical power evaluation method as described in any one of claims 1-6.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the statistical power evaluation method as described in any one of claims 1-6.

Images