Testing method and device for autonomous driving vehicle

Through the testing of preset scenario database and multiple site scenarios, the problem that the existing technology is difficult to comprehensively evaluate the perception and decision-making behavior of autonomous vehicles is solved, and a multi-dimensional evaluation of autonomous vehicles is realized, providing more comprehensive and accurate testing is provided. result.

CN114354219BActive Publication Date: 2024-11-15SUZHOU ZHITU TECH CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202210014605.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-07
Publication Date
2024-11-15
Estimated Expiration
2042-01-07

AI Technical Summary

Technical Problem

Existing autonomous driving vehicle testing methods are difficult to evaluate perception technology, plan decision-making behaviors and participate as traffic participants from multiple dimensions and angles. Especially in high-level autonomous driving systems, existing testing methods cannot fully cover people, vehicles, and The interaction of the environment.

Method used

Provide an autonomous vehicle testing method, extract the corresponding test scenarios of multiple sites through a preset scene library, determine the test scenarios required by the vehicle to be tested based on the test tasks, conduct tests and evaluate its autonomous driving performance, including driving autonomous data , learning evolution data and traffic coordination data, and comprehensive evaluation using quantitative and qualitative evaluation indicators.

Benefits of technology

Achieve multi-dimensional and multi-angle evaluation of autonomous driving vehicles, can meet the testing needs of different types of vehicles, provide a stable data foundation, and have a more comprehensive coverage. It is suitable for traditional ADAS and high-level autonomous driving systems, improving the accuracy of testing. Sex and universality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114354219B_ABST
    Figure CN114354219B_ABST
Patent Text Reader

Abstract

The present invention provides a method and device for testing an autonomous driving vehicle, wherein the method includes determining the test scenario required for the vehicle to be tested according to the test task; extracting test cases of the test scenario required for the vehicle to be tested from a preset scenario library, wherein the scenario library contains test cases of test scenarios corresponding to a variety of venues; testing the vehicle to be tested based on the test cases of the test scenario required for the vehicle to be tested, and obtaining the test results of the vehicle to be tested; and evaluating the autonomous driving performance of the vehicle to be tested based on the test results. In this method, the test scenario is derived from data from a variety of venues, and the test scenario is selected according to the test task, so that the autonomous driving vehicle can be tested efficiently, accurately, and economically. The rich data source can ensure the distribution and coverage of the test scenario, and it has strong versatility. The test scenario covers the interactive objects in the traffic world, so that multi-dimensional and multi-angle evaluation can be achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle autonomous driving testing, and in particular to a testing method and device for an autonomous driving vehicle. Background Art

[0002] In the initial stages of design and development of smart driving products, functional safety and expected functional safety are receiving more and more attention. From component design, software and hardware development, system integration to final vehicle integration, each round of development must consider safety factors and conduct continuous testing, iteration and optimization. Therefore, vehicle testing accompanies the entire product development process. The core goal of the test is to verify the function based on known scenario data and with the help of a variety of test environments, and to reduce the number of these risk scenarios or control risks through a variety of testing methods, including algorithm optimization, functional safety, and expected functional safety analysis, that is, to use all means to minimize the unknown and dangerous range. The testing technology of the self-driving system has also changed from function-based to scenario-based testing.

[0003] The testing and evaluation methods for autonomous driving vehicles proposed in related technologies are mainly based on simulation-based tests or real-vehicle VIL (Vehicle in the loop) tests, which are somewhat different from real-site tests. In addition, existing testing methods are mainly based on the ADAS (Advanced Driver Assistance System) evaluation system. However, as part of the traffic environment, autonomous driving vehicles have different interactions with traffic participants, such as road vehicles, vulnerable groups on the road, traffic signs, temporary traffic obstacles, etc. Therefore, for high-level autonomous driving, the evaluation should be the coupling between people, vehicles, and the environment, not just the evaluation of the automatic control behavior of the vehicle itself. Therefore, the methods in related technologies are difficult to evaluate the perception technology, planning and decision-making behavior, and participation behavior of autonomous driving as a traffic participant from multiple dimensions and angles. Summary of the invention

[0004] The purpose of the present invention is to provide a testing method and device for an autonomous driving vehicle, which can evaluate the perception technology, planning and decision-making behavior of autonomous driving, and the participation behavior as a traffic participant from multiple dimensions and angles.

[0005] In a first aspect, the present invention provides a testing method for an autonomous driving vehicle, the method comprising: determining a test scenario required for the vehicle to be tested according to a test task; extracting test cases for the test scenario required for the vehicle to be tested from a preset scenario library; wherein the scenario library contains test cases for test scenarios corresponding to a variety of venues; testing the vehicle to be tested based on the test cases for the test scenario required for the vehicle to be tested, and obtaining test results for the vehicle to be tested; and evaluating the autonomous driving performance of the vehicle to be tested based on the test results.

[0006] In an optional implementation, the test scenarios corresponding to the various venues include standard regulatory scenarios, natural driving scenarios, dangerous accident scenarios, scenario metaset construction scenarios and simulation scenarios; the above-mentioned scenario library is determined in the following manner: for each test scenario, a test case for the current test scenario is constructed according to the scenario feature parameters corresponding to the current test scenario.

[0007] In an optional implementation, the test cases of the current test scenario are determined in the following manner: based on a preset development file, the scenario elements and scenario characteristic parameters are determined; according to the interaction between the scenario elements and the preset goals, the current test scenario is generated; in the current test scenario, the preset examination elements are highlighted to obtain a logical test scenario; and the scenario characteristic parameters in the logical test scenario are refined to obtain test cases of the current test scenario.

[0008] In an optional embodiment, the above-mentioned step of constructing a test case for the current test scenario based on the scene characteristic parameters corresponding to the current test field includes: verifying the current test scenario based on simulation testing, field testing and road testing; adjusting the current test scenario according to the verification results to obtain the final current test scenario.

[0009] In an optional embodiment, before the step of determining the test scenarios required for the vehicle to be tested according to the test tasks, the method further includes: determining examination factors according to the development documents of the vehicle to be tested; and determining the test tasks based on the examination factors.

[0010] In an optional embodiment, the above step of determining the test scenario required for the vehicle to be tested based on the test task includes: determining the operating scenario of the vehicle to be tested based on the test task; determining the scenario elements based on the operating scenario; and determining the test scenario required for the vehicle to be tested based on the scenario elements.

[0011] In an optional embodiment, the test results include: driving autonomy data of the vehicle to be tested, learning evolution data of the vehicle to be tested in the same test scenario, and traffic coordination data of the vehicle to be tested; the step of evaluating the autonomous driving performance of the vehicle to be tested based on the test results includes: determining the evaluation result of the intelligent dimension autonomous driving performance of the vehicle to be tested based on the driving autonomy data of the vehicle to be tested; wherein the driving autonomy data includes basic performance data, driving quality data and vehicle safety performance data of the vehicle to be tested; determining the evaluation result of the time dimension autonomous driving performance of the vehicle to be tested based on the learning evolution data of the vehicle to be tested in the same test scenario; wherein the learning evolution data includes scene avoidance data, environmental cognition data and decision planning data; determining the evaluation result of the spatial dimension autonomous driving performance of the vehicle to be tested based on the traffic coordination data of the vehicle to be tested; wherein the traffic coordination data includes data collected from the perspective of internal occupants, data collected from the perspective of external traffic participants, and data collected from a global perspective.

[0012] In an optional embodiment, the above-mentioned step of evaluating the autonomous driving performance of the vehicle to be tested based on the test results includes: analyzing the test results according to preset quantitative evaluation indicators to obtain evaluation values ​​corresponding to the quantitative evaluation indicators; wherein the quantitative evaluation indicators include speed control accuracy, position control accuracy, acceleration and deceleration, collision time TTC, headway THW, lateral acceleration and lane change time indicators; analyzing the test results according to preset qualitative evaluation indicators to obtain evaluation values ​​corresponding to the qualitative evaluation indicators; wherein the qualitative evaluation indicators include braking deceleration range and steering wheel adjustment cycle indicators during lane change; evaluating the evaluation values ​​corresponding to the quantitative evaluation indicators and the evaluation values ​​corresponding to the qualitative evaluation indicators through a preset fuzzy comprehensive evaluation method to obtain the final evaluation results.

[0013] In an optional embodiment, the scenario library contains the number of test cases for each test scenario; after the step of evaluating the autonomous driving performance of the vehicle to be tested based on the test results, the method further includes: saving the test case number corresponding to the test scenario required for the vehicle to be tested, the test results, and the evaluation results of the autonomous driving performance of the vehicle to be tested based on the test results.

[0014] In a second aspect, the present invention provides a testing device for an autonomous driving vehicle, the device comprising: a scenario determination module, for determining a test scenario required for the vehicle to be tested according to a test task; a scenario extraction module, for extracting test cases for the test scenario required for the vehicle to be tested from a preset scenario library; wherein the scenario library contains test cases for test scenarios corresponding to a variety of venues; a test monitoring module, for monitoring test data during the test of the vehicle to be tested, and obtaining test results of the vehicle to be tested; and a test evaluation module, for evaluating the autonomous driving performance of the vehicle to be tested based on the test results.

[0015] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned autonomous driving vehicle testing method.

[0016] In a fourth aspect, the present invention provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned autonomous driving vehicle testing method.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] The present invention provides a method and device for testing an autonomous vehicle, the method comprising: determining the test scenario required for the vehicle to be tested according to the test task; extracting the test case of the test scenario required for the vehicle to be tested from a preset scenario library; wherein the scenario library contains test cases of the test scenario corresponding to multiple venues; testing the vehicle to be tested based on the test case of the test scenario required for the vehicle to be tested, and obtaining the test result of the vehicle to be tested; and evaluating the autonomous driving performance of the vehicle to be tested based on the test result. This method can select a suitable test scenario according to the test task, and then can fully test the autonomous driving vehicle according to the test requirements. Secondly, the test scenario for testing the autonomous driving vehicle of the present application is information and data collected from the test scenarios corresponding to multiple venues, and is constructed and generated according to the development file of the autonomous driving vehicle. Therefore, the test scenario can meet the multiple test requirements of the autonomous driving vehicle, and then can fully provide the test results. In addition, the information and data in the test scenarios corresponding to multiple venues are used as the basis for constructing the test scenario, which can make the distribution and coverage of the test scenario more comprehensive, and can be constructed for different types of autonomous driving vehicles. The test scenario data has high repeatability, which can provide a stable data foundation for the field of autonomous driving vehicle testing.

[0019] In addition, the test scenario of this application covers the interactive objects in the traffic world, that is, the scenario participants are designed, which can replace the main participants in the real traffic world to test the interactive performance of the vehicle to be tested, and thus can realize the test of the automatic control adaptability of the autonomous driving vehicle when facing people, vehicles and the environment. It is not limited to evaluating the automatic control ability of the autonomous driving vehicle itself, and can evaluate the autonomous driving vehicle from multiple dimensions and angles. This test method is not only applicable to traditional ADAS testing, but also to high-level autonomous driving system field testing. The test scenario, test case, and test evaluation method are also commonly used in road testing of autonomous driving vehicles, and have strong versatility.

[0020] Other features and advantages of the present invention will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present invention.

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are specifically listed below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A flowchart of a method for testing an autonomous driving vehicle provided by an embodiment of the present invention;

[0024] Figure 2 A flowchart of another method for testing an autonomous driving vehicle provided by an embodiment of the present invention;

[0025] Figure 3 A flowchart of another method for testing an autonomous driving vehicle provided by an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of the structure of a testing device for an autonomous driving vehicle provided by an embodiment of the present invention;

[0027] Figure 5 A scene verification and data collection flow chart provided by an embodiment of the present invention;

[0028] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0030] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Autonomous driving technology is developing rapidly. Currently, more and more autonomous driving products have begun commercial operation. From L0 at the beginning to the current L3 and L4 autonomous driving products in trial operation, and even the future L5 autonomous driving products, safety is always the first concern that needs to be solved. How to make autonomous driving products safe and reliable before commercialization is still a difficult problem. The testing technology of self-driving systems has also changed from function-based to scenario-based testing. The difficulty of testing autonomous driving systems above L3 lies in the endless scenarios. Unlike ADAS and traditional vehicle testing, the performance of traditional test vehicles is generally monotonous and predictable, while the autonomous driving system is a coupling between people, vehicles and the environment, and the performance is unstable and not monotonous. At the same time, the test of the autonomous driving system is not only to evaluate the automatic control behavior of the vehicle itself, but also to evaluate its perception technology, planning and decision-making behavior as an intelligent carrier, and its participation behavior as a traffic participant.

[0032] There are corresponding testing methods at different stages of product development. The development stage pursues development efficiency. For rapid iteration, simulation testing technology is generally used. In the mass production stage of the whole vehicle, field testing technology is often used to verify the autonomous driving system. Real vehicle field testing can solve the problem that simulation testing cannot fully reflect the test conditions of real vehicles in real traffic environments. ADAS field testing currently has corresponding standards and regulations for reference, forming a unified testing method in the industry, but for autonomous driving products above level L3, there are fewer current standards and regulations. Each company has its own development and testing team and there is no unified testing method. How to make full use of test site resources to carry out efficient, economical and accurate testing and evaluate autonomous driving vehicles from multiple dimensions and angles based on the results, including autonomous driving perception technology, planning and decision-making behavior, and participation behavior as a traffic participant is an urgent problem to be solved in field testing.

[0033] Based on the above problems, an embodiment of the present invention provides a method and device for testing an autonomous driving vehicle. The method can be applied to the scenario of testing an autonomous driving vehicle, especially the test of the autonomous driving vehicle operation scenario. To facilitate the understanding of this embodiment, a method for testing an autonomous driving vehicle disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following specific steps:

[0034] Step S102: determining the test scenario required for the vehicle to be tested according to the test task.

[0035] The above-mentioned test task usually refers to the corresponding test plan set by the tester according to his own test needs. The test task carries the scenario triggering conditions required for testing the vehicle to be tested. Specifically, in this embodiment, the scenario triggering conditions may be scenario elements that need to be examined for the autonomous driving vehicle. In this embodiment, upon receiving the test task (i.e., the scenario triggering conditions), the test task (i.e., the scenario triggering conditions) will be automatically analyzed to determine the test scenario that meets the scenario triggering conditions. For example, when the test task is to test the interaction capability between the vehicle to be tested and the scenario participants, the test scenario that can test the interaction capability between the vehicle to be tested and the scenario participants will be automatically determined as the required test scenario; when the test task is to examine the impact of rainfall on the vehicle to be tested, the test scenario that can test the impact of rainfall on the vehicle to be tested will be automatically determined as the required test scenario.

[0036] Step S104, extracting test cases of test scenarios required by the vehicle to be tested from a preset scenario library; wherein the scenario library contains test cases of test scenarios corresponding to various venues.

[0037] In this embodiment, the preset scenario library can be used by the data acquisition and recording system to collect information and data for constructing test scenarios from test scenarios corresponding to various venues. The information and data have characteristic parameters of the test scenarios, and the information and data can also be used as evaluation indicators for test cases for constructing test scenarios; then the collected information and data are used to construct test scenarios in the simulation software, and these test scenarios are stored in the scenario library, and can be directly used for real vehicle testing and simulation testing. Among them, when constructing the test scenario, the scenario construction will also be carried out according to the development file of the vehicle to be tested, and at this time, a test case that can be used to test the vehicle to be tested can be obtained. In some embodiments, the data source in the preset scenario library is also provided by the target object transportation system, and the target object transportation system can participate in the construction of scenario participants. The scenario participants can replace the main participants in the real traffic world, such as traffic signs, temporary traffic obstacles, real road vehicles and real road vulnerable groups (such as pedestrians, bicycles and electric scooters, etc.). The vehicle to be tested can interact with the scenario participants to test the traffic coordination of the vehicle to be tested, wherein the vehicle to be tested can collide with the above-mentioned scenario target participants.

[0038] In some embodiments, it is necessary to refer to the development files and test tasks (i.e., scenario triggering conditions) of the vehicle to be tested at the same time. At this time, the test cases that meet the test tasks (i.e., scenario triggering conditions) and can be used for the vehicle to be tested can be directly and automatically determined and extracted.

[0039] The scenario library contains the test case number of each test scenario; the number can facilitate data processing and reproduction of problem scenarios.

[0040] Step S106 , testing the vehicle to be tested based on the test case of the test scenario required by the vehicle to be tested, and obtaining a test result of the vehicle to be tested.

[0041] After extracting the test case of the test scene required by the vehicle to be tested, the vehicle to be tested is made to enter the test case for testing. Among them, the test case has a parameter space that meets the test task and / or a scene participant that meets the test task. When used specifically, the scene participant may not exist in the test scene, or it may exist simultaneously with the parameter space. According to the actual test requirements, the movement trajectory of the above parameter space and scene participants needs to be set in advance by the test operator, which can be used for direct testing, so as to facilitate the fast and efficient execution of field tests. When the vehicle to be tested arrives at the test case and the vehicle to be tested reaches the set parameter space, the test instruction that meets the test task will be triggered, and the instruction will specify the specific content of testing the vehicle to be tested to complete the test. In addition, when the interactive ability of the vehicle to be tested needs to be tested in the test task, the test case will also have a scene participant constructed by the target object transport system. When the vehicle to be tested reaches the set conditions in the test case, the scene participant will also automatically move according to the movement trajectory set in advance, without the need for manual operation. During the test, the vehicle under test is monitored in real time. After the test, the test result of the vehicle under test can be obtained based on the test situation of the vehicle under test in the test case and / or the interaction between the vehicle under test and the scenario participants.

[0042] Step S108, evaluating the autonomous driving performance of the vehicle to be tested based on the test results.

[0043] In the specific implementation, after the test of the vehicle to be tested is completed and the test results are obtained, the value of the test indicator can be determined according to the test results. The value of the test indicator is the evaluation result. According to the evaluation result, the autonomous driving performance of the vehicle to be tested can be evaluated. In addition, after obtaining the evaluation result, the test case number, test results and evaluation results of the autonomous driving performance of the vehicle to be tested based on the test results corresponding to the test scenario required for the vehicle to be tested will also be saved, which can facilitate data processing and reproduction of problem scenarios.

[0044] The above-mentioned test method for autonomous driving vehicles can select appropriate test scenarios according to the test tasks, and then can fully test the autonomous driving vehicles according to the test requirements. Secondly, the test scenarios for testing autonomous driving vehicles in this application are information and data collected from test scenarios corresponding to multiple venues, and are constructed and generated according to the development files of the autonomous driving vehicle. Therefore, the test scenarios can meet the various test requirements of the autonomous driving vehicle, and then can fully provide test results. In addition, the information and data in the test scenarios corresponding to multiple venues are used as the basis for the construction of the test scenarios, which can make the distribution and coverage of the test scenarios more comprehensive, and can be constructed for different types of autonomous driving vehicles. The test scenario data has high repeatability and can provide a stable data foundation for the field of autonomous driving vehicle testing; in addition, the test scenarios of this application cover the interactive objects in the traffic world, that is, the scene participants are designed, and the scene participants can replace the main participants in the real traffic world to test the interactive performance of the vehicle to be tested, and then can realize the test of the autonomous driving vehicle's own automatic control adaptability when facing people, cars and the environment, which is not limited to evaluating the automatic control ability of the autonomous driving vehicle itself, and then can evaluate the autonomous driving vehicle from multiple dimensions and angles. This test method is not only applicable to traditional ADAS testing, but also to field testing of high-level autonomous driving systems. The test scenarios, test cases, and test evaluation methods are also commonly used in road testing of autonomous driving vehicles, and are highly versatile.

[0045] The embodiment of the present invention also provides another method for testing an autonomous driving vehicle, which is implemented on the basis of the method of the above embodiment; the method focuses on describing the specific process of determining the test scene required for the vehicle to be tested according to the test task (implemented by the following steps S206-S210), and the specific process of evaluating the autonomous driving performance of the vehicle to be tested based on the test results (implemented by the following steps S216-S220); Figure 2 As shown, the method comprises the following specific steps:

[0046] Step S202, determining the inspection factors according to the development documents of the vehicle to be tested.

[0047] After the autonomous driving vehicle is produced, it will come with its own vehicle development file. In this embodiment, the design scheme operational domain (ODD) and design functions in the development file are mainly based on ODD, which refers to the operational standard of the unmanned driving system function setting, including but not limited to the natural environment, natural geography and time period, traffic volume and road characteristics, etc., to determine the inspection factors used to test the autonomous driving vehicle. Inspection factors refer to the main influencing factors used for reference when testing the autonomous driving vehicle. Inspection factors can be scene participants, weather environment, or road characteristics. Specifically, when the ODD of the autonomous driving vehicle is that it can travel on a road with a speed range of [30,80] and a bumpy road surface, the inspection factor can be determined to be the road characteristic of the speed range of [30,80] and a bumpy road surface; when the ODD of the autonomous driving vehicle is that it can automatically travel in rainy and snowy weather, the inspection factor can be determined to be the weather environment of rainy and snowy weather.

[0048] Step S204: determine the test task based on the above-mentioned examination factors.

[0049] Once the assessment factors are determined, the test tasks that can test the autonomous driving vehicle can be determined based on the assessment factors. For example, when the assessment factors are a speed range of [30,80] and a bumpy road surface, the test task can be to test the autonomous driving behavior of the autonomous driving vehicle in a scenario where the speed range meets the speed range of [30,80] and the road surface is bumpy. When the assessment factors are rainy or snowy weather, the test task can be to test the autonomous driving performance of the autonomous driving vehicle in a scenario where rainy or snowy weather may occur.

[0050] Step S206: Based on the above test task, determine the operation scenario of the vehicle to be tested.

[0051] After the test task is finally determined, the test scenario in which the vehicle to be tested can drive normally and automatically will be determined according to the test task. The test scenario in which the vehicle to be tested can drive normally and automatically is the operation scenario.

[0052] Step S208, determining scene elements according to the above-mentioned operation scene.

[0053] After the operation scenario is determined, the scene elements in the operation scenario will be determined through the characteristic parameters of the operation scenario. In specific implementation, the scene elements include inherent elements (such as road topology, traffic signs, temporary traffic facilities), slowly changing elements (such as weather environment (rain, snow, fog and light intensity, visibility, etc.)), and transient elements (such as traffic participants).

[0054] Step S210: Based on the above scenario elements, determine the test scenario required for the vehicle to be tested.

[0055] After the scenario elements in the operating scenario are determined, the test scenario required for the vehicle to be tested is determined from the operating scenario based on the extent to which the scenario elements can satisfy the test task.

[0056] Step S212, extracting test cases of test scenarios required by the vehicle to be tested from a preset scenario library; wherein the scenario library contains test cases of test scenarios corresponding to a variety of venues.

[0057] Specifically, the test scenarios corresponding to the above-mentioned multiple venues include standard regulatory scenarios, natural driving scenarios, dangerous accident scenarios, scenario meta-set construction scenarios and simulation scenarios; standard regulatory scenarios are basic test scenarios that must be met during the research and development and certification stages of autonomous driving functions, and are mainly derived from existing standards, evaluation procedures, etc. For example, ISO, NHTSA, E-NCAP, C-NCAP and other standards and evaluation procedures have made test provisions for existing autonomous driving functions. The data of natural driving scenarios comes from the real natural driving state of the car, which is the most basic data source. This scenario can provide multi-dimensional information such as vehicle data, driver behavior, road environment, etc., and is a sufficient test scenario to prove the effectiveness of autonomous driving. Dangerous accident scenarios mainly cover three categories of scenarios: severe weather environment, complex road traffic and typical traffic accidents. It is a necessary test scenario to prove the effectiveness of autonomous driving. Its data mainly comes from the traffic accident database, which is the key to verifying the safety and reliability of autonomous driving control strategies. The scene meta-set constructs the scene by performing different permutations and combinations of static elements, dynamic elements, and driver behavior elements, and traversing the values, expanding the parameters to reorganize the scene boundaries, and is a test scene that can effectively supplement unknown working conditions. Its data comes from the existing scene database resources. By parameterizing the existing simulation scenes, the corresponding types of scenes are randomly generated or automatically reorganized. In this embodiment, the scene meta-set constructs the scene and can be formed by the VIL system according to the collected scene information and data according to the test task, and the scenes formed by these combinations are new test scenes that have not been encountered before, such as dangerous scenes and complex traffic flow scenes that have not been encountered. Simulation scenes refer to simulation scenes directly constructed by simulation software, in which the data format in the simulation scene is an openX format data set. This application collects information and data of test scenes corresponding to various venues based on the data acquisition and recording system. The above information and data include vehicle bus data, inertial navigation and positioning information, lane line position relationship information, target vehicle information and other CAN data, human-computer interaction HMI audio and video data, scene video data, and laser radar Ethernet data in the test, which can also be used for test result evaluation.

[0058] The data in the scenario library collects information and data from the test scenarios corresponding to the above-mentioned various venues, and can collect different scenario parameter information and scenario factors. For example, it can collect high-speed road scene parameter information with a speed range of [60, 120] from natural driving scenarios, and it can also collect parameter information of urban complex scenes with a speed range of [30, 80] from dangerous accident scenarios, ensuring that the test scenarios can be distributed comprehensively, and then can be constructed for different types of autonomous driving vehicles. At the same time, the above-mentioned collected data can also be used as a reference for extreme case data, thereby supporting the solution of the long-tail effect.

[0059] In a specific implementation, the above scenario library can be determined in the following manner: for each test scenario, a test case of the current test scenario is constructed according to the scenario characteristic parameters corresponding to the current test scenario.

[0060] Since a large number of test scenarios are stored in the scenario library, but not all test scenarios are applicable to the autonomous driving vehicle to be tested, it is necessary to select test scenarios that are applicable to the autonomous driving vehicle to be tested from the scenario library. When selecting the test scenario, it is also necessary to refer to the development files of the vehicle to be tested. Then, based on the scenario feature parameters of the selected test scenario, a test case for the current test scenario is constructed. The test case refers to a test scenario that can meet the test task.

[0061] In order to ensure the feasibility of the scenario, the test case for constructing the current test scenario according to the scenario characteristic parameters corresponding to the current test scene can also be determined by the following steps 20-21:

[0062] Step 20, verifying the current test scenario based on simulation testing, field testing and road testing.

[0063] Simulation testing (SIL (software in-the-loop), MIL (Model in the Loop), DIL (Drive in the Loop)) refers to the use of simulation software to build simulation scenarios against the real world and simulate the real driving environment. It is generally composed of a traffic scenario module, a sensor module, a vehicle dynamics module and a test management module. Field testing includes VIL testing and real vehicle testing. VIL testing refers to testing in an open field using scenarios implemented in simulation software. In field testing, only the autonomous driving vehicle to be tested is real, and its perception information of the scene used for testing is input by the simulation software. Dangerous scenes and complex traffic flow scenes are also implemented in the scene simulation software. Real vehicle testing refers to testing of autonomous driving vehicles in closed areas. In this embodiment, the real vehicle test has artificially constructed scene participants, so the field test in this embodiment has a controllable test scene. Road testing refers to the testing method of autonomous driving vehicles on real roads. When driving on real roads, this embodiment can collect scene parameter information and data of real roads, provide a rich data source for the database, and also provide a basis for gradually enriching the test data set. Each test method is different, and the focus of the test task of the autonomous driving vehicle is also different, which makes the selected test cases also different.

[0064] In specific implementation, this method first conducts simulation test verification on the constructed scenario, then conducts field test on the verified scenario, and then conducts road test on the autonomous driving vehicle. Figure 5 As shown, model development refers to the process of building a scene based on the collected data and information, simulation testing refers to SIL and MIL testing, and VIL testing refers to the process of putting the scene built by the simulation software into an open field through the VIL system. In the process of each of the above-mentioned test methods, the above-mentioned collected scene data and information are evaluation indicators, and the scene characteristic parameters of the corresponding test scene can be verified based on the collected scene data and information to reduce the difference between the test scene and the real scene and ensure the authenticity of the current test scene. In specific implementation, the test scene is verified through the data acquisition and recording system, and when the simulation test is verified, the openX format data set required for the simulation test is verified.

[0065] Step 21, adjusting the current test scenario according to the verification result to obtain the final current test scenario.

[0066] After verifying the scene characteristic parameters of the corresponding current test scene based on the collected scene data and information, a verification result can be obtained. According to the deviation of each scene parameter in the verification result, the scene parameters of the corresponding test scene are adjusted to update the final current test scene.

[0067] Specifically, the test cases of the current test scenario can be determined by the following steps 30-33:

[0068] Step 30, based on the preset development file, determine the scene elements and scene feature parameters.

[0069] The development documents of an autonomous driving vehicle can provide the conditions for the vehicle to drive normally. When the autonomous driving performance of an autonomous driving vehicle needs to be tested, referring to its development documents can determine which scene elements and scene feature parameters are required in the test scenario.

[0070] Step 31, generating a current test scenario based on the interaction between scenario elements and preset targets.

[0071] After determining the scenario elements and scenario characteristic parameters that meet the development document, the test scenario can be determined, and the test scenario that can be used for testing can be extracted by combining the autonomous driving behavior of the autonomous driving vehicle model (i.e., the preset target) in the test scenario and the interaction ability of the scenario elements. This test scenario can also be called a functional scenario, which is used for project definition, hazard analysis, and risk assessment;

[0072] Among them, when the scene elements are road vehicles of scene participants, the data such as the horizontal and vertical relative distances, collision time TTC (Time-To-Collision), vehicle speed (such as speed, acceleration), angle and headway THW (Time-Head-Way) obtained by the interaction between the autonomous driving vehicle and the road vehicle are monitored, and then it is determined whether the interaction meets the test task. If it does, it is generated as a usable test scene. In this embodiment, the interaction data between the vehicle to be tested and the road vehicles of the scene participants are monitored by a high-precision inertial navigation system, and the system provides the index of the above data for evaluating the interaction performance of the autonomous driving vehicle. The high-precision inertial navigation system has high measurement accuracy and can be used as a true value evaluation.

[0073] Step 32, highlighting the preset examination elements in the current test scenario to obtain a logical test scenario.

[0074] After generating the test scenario, it is necessary to highlight the assessment factors that meet the test task, further describe the requirements of the test scenario in detail, and then further extract test scenarios from the above test scenarios to obtain a logical test scenario that can meet the test requirements.

[0075] Step 33, refine the scenario characteristic parameters in the logic test scenario to obtain the test case of the current test scenario.

[0076] After obtaining the logic test scenario, the scenario characteristic parameters in the logic test scenario are refined according to the test task, such as determining the specific value of each scenario characteristic parameter in the logic test scenario to clearly describe the relationship between entities. Among them, the expected behavior of the object under test (i.e. the autonomous driving vehicle under test) and the scenario characteristic parameters (such as the trigger conditions of the weather environment and the movement trajectory of the scene participants, etc.) also need to be set in advance by the test operator. When the logic test scenario has detailed data, any number of specific scenarios can be generated, and the specific scenario is the test case of the current test scenario.

[0077] Step S214: testing the vehicle to be tested based on the test cases of the test scenarios required by the vehicle to be tested, and obtaining a test result of the vehicle to be tested.

[0078] After extracting the test case of the test scene required by the vehicle to be tested, the vehicle to be tested enters the test case for testing. The test case contains parameter spaces that meet the test task and / or scene participants that meet the test task. When the vehicle to be tested arrives at the test case and reaches the set parameter space, the test instruction that meets the test task will be triggered, and the instruction specifies the specific content of testing the vehicle to be tested to complete the test. In addition, when the test task requires testing the interactive ability of the vehicle to be tested, the test case will also contain scene participants constructed by the target object transport system. When the vehicle to be tested meets the set conditions in the test case, the scene participants will also automatically move according to the motion trajectory set in advance without manual operation.

[0079] During the test, the vehicle under test is monitored in real time. After the test, the test result of the vehicle under test can be obtained based on the test situation of the vehicle under test in the test case and / or the interaction between the vehicle under test and the scenario participants.

[0080] Step S216: Analyze the test results according to the preset quantitative evaluation index to obtain the evaluation value corresponding to the quantitative evaluation index.

[0081] Among them, the quantitative evaluation indicators include speed control accuracy, position control accuracy, acceleration and deceleration, collision time TTC, headway THW, lateral acceleration and lane change duration indicators; in specific implementation, the test data is processed according to the developed data processing script to obtain quantitative evaluation indicators, among which the speed control accuracy, position control accuracy, acceleration and deceleration, collision time TTC, headway THW, lateral acceleration and lane change duration data required for evaluation in the quantitative evaluation indicators can also be provided by the above-mentioned internal occupant perspective (i.e., driver model), external traffic participant perspective and global perspective. The specific data of the quantitative evaluation indicators are provided below, as shown in Table 1:

[0082]

[0083] Table 1

[0084] Step S218: analyzing the test results according to the preset qualitative evaluation indicators to obtain evaluation values ​​corresponding to the qualitative evaluation indicators.

[0085] Among them, the qualitative evaluation indicators include the braking deceleration range and the steering wheel adjustment cycle indicators during the lane change process; in specific implementation, the data for the above-mentioned reference qualitative evaluation indicators can also be provided through the above-mentioned internal occupant perspective (i.e., driver model), external traffic participant perspective, and global perspective. For example, when conducting road tests in natural driving scenarios, a driver driving model is built, and test data that passes the qualitative evaluation is obtained through the driver model. The test data includes the braking deceleration range and the steering wheel adjustment cycle during the lane change process, etc.

[0086] Step S220, evaluating the evaluation values ​​corresponding to the quantitative evaluation indicators and the evaluation values ​​corresponding to the qualitative evaluation indicators through a preset fuzzy comprehensive evaluation method to obtain a final evaluation result.

[0087] After obtaining the evaluation values ​​obtained through qualitative evaluation indicators and the evaluation values ​​obtained through quantitative evaluation indicators, it is necessary to first determine the set of assessment factors according to the test task, and then determine the above evaluation value set according to the prominence of the factors of the test task (that is, the importance of the factors), and rank the above evaluation value set. After that, the evaluation values ​​of the qualitative evaluation are quantified, and the fuzziness and subjective judgment are eliminated to obtain the final evaluation results and output the evaluation report. Through this method, information data with multiple evaluation factors can be better processed.

[0088] In the above-mentioned testing method for autonomous driving vehicles, the data of the scenario library can be extracted from real roads, and can also be freely constructed based on the extracted information. The scene sources are rich, which can avoid the problem of low coverage of the scenario library due to the single test scene type, so it can be constructed for different types of autonomous driving vehicles. In addition, this method determines the specific test cases that can be used for testing based on the scenes in the scenario library and the development documents of the autonomous driving vehicle. In addition, this method emphasizes the inspection factors during testing to extract more specific test cases for testing, thereby conducting more accurate, efficient and economical tests. In addition, after obtaining the test results, this method comprehensively considers the many factors that affect the performance of autonomous driving, and through fuzziness and subjective judgment of the test results, it can better draw evaluation conclusions.

[0089] At the same time, the collected data can also verify the simulation scene and the construction scene to ensure the authenticity and feasibility of the test scene. In addition, the scene test of this method includes simulation test, field test, and road test. The application scene of the autonomous driving vehicle can be fully verified from the test data set, and the data closed loop of the scene library is well formed. In addition, during the test process, scene data can be continuously collected, and these data and verified new scenes can further enrich the scene library.

[0090] The embodiment of the present invention also provides another method for testing an autonomous driving vehicle, which is implemented on the basis of the method in the above embodiment; the method focuses on describing a specific process of evaluating the autonomous driving performance of the vehicle to be tested based on the test results (implemented by the following steps S306-S312), such as Figure 3 As shown, the method comprises the following specific steps:

[0091] Step S302: Determine the test scenario required for the vehicle to be tested according to the test task.

[0092] Step S304: extracting test cases of the test scenarios required by the vehicle to be tested from a preset scenario library.

[0093] Step S306, based on the test cases of the test scenarios required by the vehicle to be tested, the vehicle to be tested is tested to obtain a test result of the vehicle to be tested; wherein the test result includes driving autonomy data of the vehicle to be tested, learning evolution data of the vehicle to be tested in the same test scenario, and traffic coordination data of the vehicle to be tested.

[0094] Among them, driving autonomy data includes basic performance data, driving quality data and vehicle safety performance data of the vehicle to be tested; among them, learning evolution data includes scene avoidance data, environmental cognition data and decision planning data; among them, traffic coordination data includes data collected from the perspective of internal occupants, data collected from the perspective of external traffic participants and data collected from a global perspective.

[0095] In specific implementation, the basic performance data of the vehicle to be tested refers to comfort, maneuverability, economy, and ride comfort. The comfort of the autonomous driving vehicle is judged by the deceleration change rate and yaw rate change rate of the autonomous driving vehicle during the test, and the maneuverability of the autonomous driving vehicle is evaluated by whether the acceleration, braking, steering and other data of the autonomous driving vehicle during the test can meet the normal driving of the vehicle. The economy of the autonomous driving vehicle is evaluated by whether the autonomous driving vehicle can complete the unit transportation work with the least fuel consumption under certain use conditions during the test. The ride comfort of the autonomous driving vehicle is evaluated by whether the vibration and impact generated by the vehicle during the test within the general driving speed range can make people feel uncomfortable, tired, or even damage their health, or cause damage to the goods. In addition, since the autonomous driving vehicle also needs to ensure that when the vehicle itself has problems or external factors, the autonomous driving vehicle can automatically control the direction, brakes, throttle, and various technical and condition configurations of the vehicle to ensure smooth operation and avoid traffic accidents, so when the basic performance data is obtained, the driving quality data and vehicle safety performance data can also be obtained at the same time. In addition, the comfort data and smoothness data in the basic performance data can also provide a basis for the driving quality data.

[0096] Learning evolution data means that after the autonomous driving vehicle to be tested is tested in a test case and the evaluation result is obtained, the application will also make the vehicle enter the same test case again to perform the same test task for testing, and the learning evolution data of the autonomous driving vehicle can be obtained through the performance of the whole vehicle. For example, if the autonomous driving vehicle has decision ambiguity in the test scenario, such as changing lanes back and forth at the high-speed gate in the test scenario, or frequent acceleration and deceleration hesitation at the intersection without traffic lights in the test scenario, etc., resulting in manual takeover of the autonomous driving system (or autonomous driving vehicle), the same test case (i.e. the above test scenario) is repeatedly constructed after a period of time for testing, and the frequency of manual takeover of the autonomous driving vehicle is reduced to test whether the autonomous driving vehicle has successfully learned how to avoid. In addition, the learning evolution data of the autonomous driving vehicle can be obtained through driver-like properties. At this time, the driver-like properties are the driver model, which is constructed by extracting the driver's behavioral characteristics and parameter space based on a large amount of real driver data statistical modeling. The driver model can also be used to monitor whether the autonomous driving vehicle can drive smoothly. The learning ability of the autonomous driving vehicle can also be evaluated based on the operations of the driver model, such as the frequency of manual takeover, the timing of takeover, the quality of reminders, etc. The operations of the driver model belong to the HMI (Human-Machine Interaction) evaluation. In addition, the comfort, maneuverability, and smoothness data in the driving autonomy data of the above-mentioned vehicles to be tested can also be evaluated through driver-like performance.

[0097] In specific implementation, the data collected from the internal occupant's perspective is the data collected by the above-mentioned driver model; the data collected from the perspective of external traffic participants is the data collected from the scene participants in the scene feature parameters, and is mainly evaluated based on the interaction between the autonomous driving vehicle and the scene participants, such as whether the decision of the autonomous driving vehicle is reasonable, the smoothness of control, whether it affects the scene participants, etc. The data collected from the global perspective, such as the decision-making, control accuracy, safety and impact on traffic flow of the autonomous driving vehicle, are collected through global monitoring for evaluation.

[0098] Step S308: Determine the evaluation result of the intelligent dimension automatic driving performance of the vehicle to be tested based on the driving autonomy data of the vehicle to be tested.

[0099] In a specific implementation, after obtaining the above-mentioned driving autonomy data, the value of the test index can be determined through the driving autonomy data, and then the test index is compared with the preset index threshold to obtain the evaluation result of the intelligent dimension autonomous driving performance of the vehicle to be tested.

[0100] Step S310, determining an evaluation result of the time dimension autonomous driving performance of the vehicle to be tested based on the learning evolution data of the vehicle to be tested in the same test scenario.

[0101] In a specific implementation, after obtaining the above-mentioned learning evolution data, the value of the test index can be determined by the learning evolution data, and then the test index is compared with the preset index threshold to obtain the evaluation result of the autonomous driving performance of the vehicle to be tested in the time dimension.

[0102] Step S312: determining an evaluation result of the spatial dimension autonomous driving performance of the vehicle to be tested based on the traffic coordination data of the vehicle to be tested.

[0103] In a specific implementation, after obtaining the above-mentioned traffic coordination data, the value of the test index can be determined through the traffic coordination data, and then the test index is compared with the preset index threshold to obtain the evaluation result of the spatial dimension autonomous driving performance of the vehicle to be tested.

[0104] The above-mentioned autonomous driving vehicle testing method can simultaneously evaluate the test results through multiple dimensions and multiple aspects, can draw evaluation conclusions more quickly and accurately, and can also more comprehensively evaluate the autonomous driving vehicle under the test scenario.

[0105] Corresponding to the above method embodiment, the embodiment of the present invention also provides a testing device for an autonomous driving vehicle; Figure 4 As shown, the device comprises:

[0106] A scenario determination module 40 is used to determine the test scenario required for the vehicle to be tested according to the test task;

[0107] A scene extraction module 41 is used to extract test cases of the test scenes required by the vehicle to be tested from a preset scene library;

[0108] Among them, the scenario library contains test cases for test scenarios corresponding to various venues;

[0109] The test monitoring module 42 is used to test the vehicle to be tested, monitor the test of the vehicle to be tested, and obtain the test result of the vehicle to be tested;

[0110] The test evaluation module 43 is used to evaluate the autonomous driving performance of the vehicle to be tested based on the test results.

[0111] The test device for the above-mentioned autonomous driving vehicle can select a suitable test scene according to the test task, and then can fully test the autonomous driving vehicle according to the test requirements. Secondly, the test scene for testing the autonomous driving vehicle of the present application is the information and data collected from the test scenes corresponding to various venues, and is constructed and generated according to the development files of the autonomous driving vehicle. Therefore, the test scene can meet the various test requirements of the autonomous driving vehicle, and then can fully provide the test results. In addition, the information and data in the test scenes corresponding to various venues are used as the basis for the construction of the test scene, which can make the distribution and coverage of the test scene more comprehensive, and can be constructed for different types of autonomous driving vehicles. The test scene data has high repeatability and can provide a stable data foundation for the field of autonomous driving vehicle testing; in addition, the test scene of the present application covers the interactive objects in the traffic world, that is, the scene participants are designed, and the scene participants can replace the main participants in the real traffic world to test the interactive performance of the vehicle to be tested, and then can realize the test of the autonomous driving vehicle's own automatic control adaptability when facing people, cars and the environment, which is not limited to evaluating the automatic control ability of the autonomous driving vehicle itself, and then can evaluate the autonomous driving vehicle from multiple dimensions and angles. This test method is not only applicable to traditional ADAS testing, but also to field testing of high-level autonomous driving systems. The test scenarios, test cases, and test evaluation methods are also commonly used in road testing of autonomous driving vehicles, and are highly versatile.

[0112] Furthermore, the scenario determination module 40 is also used to construct a test case for the current test scenario according to the scenario characteristic parameters corresponding to the current test scenario for each test scenario. Specifically, the test scenarios corresponding to the various sites include standard regulatory scenarios, natural driving scenarios, dangerous accident scenarios, scenario meta-set construction scenarios, and simulation scenarios.

[0113] In a specific implementation, the above-mentioned scenario determination module 40 also includes a scenario selection module, which is used to determine scenario elements and scenario feature parameters based on a preset development file; generate a current test scenario according to the interaction between the scenario elements and preset goals; highlight the preset examination elements in the current test scenario to obtain a logical test scenario; and refine the scenario feature parameters in the logical test scenario to obtain a test case for the current test scenario.

[0114] In some embodiments, the above-mentioned scenario determination module 40 also includes a scenario verification module, which is used to verify the current test scenario based on simulation testing, field testing and road testing; adjust the current test scenario according to the verification results to obtain the final current test scenario.

[0115] In some embodiments, the scenario determination module 40 is further used to determine the examination factors according to the development documents of the vehicle to be tested; and determine the test tasks based on the examination factors.

[0116] In some embodiments, the scenario determination module 40 is further used to determine the operating scenario of the vehicle to be tested based on the test task; determine the scenario elements according to the operating scenario; and determine the test scenario required for the vehicle to be tested based on the scenario elements.

[0117] Furthermore, the above-mentioned test evaluation module 43 is also used to determine the evaluation result of the intelligent dimension autonomous driving performance of the vehicle to be tested based on the driving autonomy data of the vehicle to be tested; wherein the driving autonomy data includes the basic performance data, driving quality data and vehicle safety performance data of the vehicle to be tested; based on the learning evolution data of the vehicle to be tested in the same test scene, determine the evaluation result of the time dimension autonomous driving performance of the vehicle to be tested; wherein the learning evolution data includes scene avoidance data, environmental cognition data and decision planning data; based on the traffic coordination data of the vehicle to be tested, determine the evaluation result of the spatial dimension autonomous driving performance of the vehicle to be tested; wherein the traffic coordination data includes data collected from the perspective of internal occupants, data collected from the perspective of external traffic participants and data collected from a global perspective.

[0118] In some embodiments, the test evaluation module 43 is also used to analyze the test results according to preset quantitative evaluation indicators to obtain evaluation values ​​corresponding to the quantitative evaluation indicators; wherein the quantitative evaluation indicators include speed control accuracy, position control accuracy, acceleration and deceleration, collision time TTC, headway THW, lateral acceleration and lane change duration indicators; according to preset qualitative evaluation indicators, the test results are analyzed to obtain evaluation values ​​corresponding to the qualitative evaluation indicators; wherein the qualitative evaluation indicators include braking deceleration range and steering wheel adjustment cycle indicators during lane change; the evaluation values ​​corresponding to the quantitative evaluation indicators and the evaluation values ​​corresponding to the qualitative evaluation indicators are evaluated by a preset fuzzy comprehensive evaluation method to obtain the final evaluation results.

[0119] In a specific implementation, the above-mentioned device also includes a result storage module, which is used to save the test case number, test results and evaluation results of the autonomous driving performance of the vehicle to be tested based on the test results corresponding to the test scenario required by the vehicle to be tested.

[0120] The autonomous driving vehicle testing device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0121] The embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, the electronic device includes a processor 101 and a memory 100, wherein the memory 100 stores machine executable instructions that can be executed by the processor 101, and the processor 101 executes the machine executable instructions to implement the above-mentioned autonomous driving vehicle testing method.

[0122] The memory 100 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0123] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 101 or the instruction in the form of software. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100 and completes the steps of the method of the above embodiment in combination with its hardware.

[0124] An embodiment of the present invention also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement an autonomous driving vehicle testing method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0125] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0126] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for testing an autonomous driving vehicle, characterized in that: The method comprises: Determine the test scenarios required for the vehicle to be tested based on the test tasks; Extracting test cases of the test scenarios required by the vehicle to be tested from a preset scenario library; wherein the scenario library contains test cases of test scenarios corresponding to multiple venues; Based on the test cases of the test scenarios required by the vehicle to be tested, the vehicle to be tested is tested to obtain a test result of the vehicle to be tested; Evaluating the autonomous driving performance of the vehicle to be tested based on the test results; The test scenarios corresponding to the various venues include standard regulatory scenarios, natural driving scenarios, dangerous accident scenarios, scenario meta-set construction scenarios and simulation scenarios; the scenario library is determined in the following manner: For each test scenario, construct a test case for the current test scenario according to the scenario characteristic parameters corresponding to the current test scenario; The test cases of the current test scenario are determined in the following manner: Based on the preset development file, the scene elements and scene characteristic parameters are determined; the scene elements include inherent elements, slowly changing elements and transient elements; the inherent elements include road topology, traffic signs and temporary traffic facilities, the slowly changing elements include weather environment, and the transient elements include traffic participants; Generate the current test scenario according to the interaction between the scenario elements and the preset targets; Highlighting preset examination elements in the current test scenario to obtain a logical test scenario; Refining the scenario characteristic parameters in the logic test scenario to obtain a test case for the current test scenario; Before the step of determining the test scenario required for the vehicle to be tested according to the test task, the method further includes: Determine the inspection factors according to the development documents of the vehicle to be tested; Based on the inspection factors, determining the test tasks; The step of evaluating the autonomous driving performance of the vehicle to be tested based on the test result comprises: According to the preset quantitative evaluation indicators, the test results are analyzed to obtain the evaluation values ​​corresponding to the quantitative evaluation indicators; wherein the quantitative evaluation indicators include speed control accuracy, position control accuracy, acceleration and deceleration, collision time TTC, headway THW, lateral acceleration and lane change duration indicators; Analyzing the test results according to preset qualitative evaluation indicators to obtain evaluation values ​​corresponding to the qualitative evaluation indicators; wherein the qualitative evaluation indicators include braking deceleration interval and steering wheel adjustment cycle indicators during lane change; The evaluation values ​​corresponding to the quantitative evaluation indicators and the evaluation values ​​corresponding to the qualitative evaluation indicators are evaluated by a preset fuzzy comprehensive evaluation method to obtain a final evaluation result.

2. The method according to claim 1, characterized in that The step of constructing a test case for the current test scenario according to the scenario characteristic parameters corresponding to the current test scenario includes: Verifying the current test scenario based on simulation testing, field testing and road testing; The current test scenario is adjusted according to the verification result to obtain a final current test scenario.

3. The method according to any one of claims 1 to 2, characterized in that: The step of determining the test scenario required for the vehicle to be tested according to the test task includes: Based on the test task, determining the operation scenario of the vehicle to be tested; Determining scenario elements according to the operating scenario; Based on the scenario elements, a test scenario required for the vehicle to be tested is determined.

4. The method according to claim 1, characterized in that The test result includes: driving autonomy data of the vehicle to be tested, learning evolution data of the vehicle to be tested in the same test scenario, and traffic coordination data of the vehicle to be tested; the step of evaluating the automatic driving performance of the vehicle to be tested based on the test result includes: Based on the driving autonomy data of the vehicle to be tested, determining an evaluation result of the intelligent dimension autonomous driving performance of the vehicle to be tested; wherein the driving autonomy data includes basic performance data, driving quality data and vehicle safety performance data of the vehicle to be tested; Determine an evaluation result of the time dimension autonomous driving performance of the vehicle under test based on the learning evolution data of the vehicle under test in the same test scenario; wherein the learning evolution data includes scenario avoidance data, environmental cognition data and decision planning data; Based on the traffic coordination data of the vehicle to be tested, an evaluation result of the autonomous driving performance of the vehicle to be tested in a spatial dimension is determined; wherein the traffic coordination data includes data collected from the perspective of internal occupants, data collected from the perspective of external traffic participants, and data collected from a global perspective.

5. The method according to claim 1, characterized in that The scenario library contains the number of the test case of each test scenario; after the step of evaluating the automatic driving performance of the vehicle to be tested based on the test results, the method further includes: Save the test case number corresponding to the test scenario required for the vehicle to be tested, the test result, and the evaluation result of the automatic driving performance of the vehicle to be tested based on the test result.

6. A testing device for an autonomous driving vehicle, characterized in that: The device comprises: A scenario determination module is used to determine the test scenario required for the vehicle to be tested according to the test task; A scenario extraction module is used to extract test cases of the test scenarios required by the vehicle to be tested from a preset scenario library; wherein the scenario library contains test cases of test scenarios corresponding to multiple venues; A test monitoring module, used to monitor the test data of the vehicle under test during the test, and obtain the test result of the vehicle under test; A test evaluation module, used to evaluate the autonomous driving performance of the vehicle to be tested based on the test results; The test scenarios corresponding to the multiple sites include standard regulatory scenarios, natural driving scenarios, dangerous accident scenarios, scenario meta-set construction scenarios and simulation scenarios; the scenario determination module is further used to construct a test case for the current test scenario according to the scenario characteristic parameters corresponding to the current test scenario for each test scenario; The scenario determination module also includes a scenario selection module, which is used to determine scenario elements and scenario feature parameters based on a preset development file; the scenario elements include inherent elements, slowly changing elements and transient elements; the inherent elements include road topology, traffic signs and temporary traffic facilities, the slowly changing elements include weather environment, and the transient elements include traffic participants; the current test scenario is generated according to the interaction between the scenario elements and the preset targets; the preset inspection elements are highlighted in the current test scenario to obtain a logic test scenario; the scenario feature parameters in the logic test scenario are refined to obtain a test case for the current test scenario; The scenario determination module is further used to determine the inspection factors according to the development documents of the vehicle to be tested; and determine the test task based on the inspection factors; The test evaluation module is also used to analyze the test results according to preset quantitative evaluation indicators to obtain evaluation values ​​corresponding to the quantitative evaluation indicators; wherein the quantitative evaluation indicators include speed control accuracy, position control accuracy, acceleration and deceleration, collision time TTC, headway THW, lateral acceleration and lane change duration indicators; according to preset qualitative evaluation indicators, the test results are analyzed to obtain evaluation values ​​corresponding to the qualitative evaluation indicators; wherein the qualitative evaluation indicators include braking deceleration range and steering wheel adjustment cycle indicators during lane change; the evaluation values ​​corresponding to the quantitative evaluation indicators and the evaluation values ​​corresponding to the qualitative evaluation indicators are evaluated by a preset fuzzy comprehensive evaluation method to obtain a final evaluation result.

Citation Information

Patent Citations

  • Testing method and device for driverless automobiles

    CN105954040A

  • Driving performance testing method and device for autonomous vehicle

    CN109520744A

  • Automatic driving test case generation method based on scenes and tasks

    CN110597711A

  • Vehicle test scene determination method and device, equipment and storage medium

    CN111579251A

  • Multi-dimensional comprehensive evaluation method and device for automatic driving automobile

    CN112465395A