A performance testing method for an autonomous driving system

Abstract

A vehicle testing method includes: acquiring a set of test parameter items corresponding to the function to be tested of the vehicle under test (200, 610) and the range of values ​​to be tested for each test parameter item in the set of test parameter items (S101); determining multiple values ​​for each test parameter item within the range of values ​​to be tested (S102); generating multiple test instructions based on the set of test parameter items and the multiple values ​​for each test parameter item (S103); sending the test instructions to a test device in a test scenario (S104); acquiring the driving state of the vehicle under test (200, 610) when implementing the function to be tested (S105); and generating test results based on the driving state and the test instructions (S106). This method can improve testing accuracy.

Description

Technical Field

[0001] This application relates to the field of autonomous driving testing, and in particular to a performance testing method for an autonomous driving system. Background Technology

[0002] Autonomous driving refers to a system where a vehicle operates automatically without driver intervention, relying on sensors to collect environmental information and drive accordingly. The research and development of autonomous vehicles involves conducting safety tests to determine their performance limits and, based on these limits, whether optimization of the autonomous driving system is necessary. Summary of the Invention

[0003] Based on this, the embodiments of this application provide a performance testing method for an autonomous driving system. Specifically, it improves a vehicle testing method, device, system, and storage medium, aiming to improve the testing accuracy of the performance boundaries of autonomous vehicles.

[0004] In a first aspect, embodiments of this application provide a vehicle testing method, including:

[0005] Obtain the set of test parameter items corresponding to the function to be tested of the vehicle under test and the range of test values ​​for each test parameter item in the set of test parameter items, wherein the vehicle under test can implement the function to be tested in the test scenario based on the observation data of traffic elements in the test scenario and the preset autonomous driving algorithm.

[0006] Multiple values ​​are determined for each of the test parameter items within the range of values ​​to be measured;

[0007] Multiple test instructions are generated based on the set of test parameters and the values, wherein each test instruction includes a value of each test parameter in the set of test parameters, and some or all of the values ​​of the same test parameter in any two test instructions are different.

[0008] The test instruction is sent to the test device in the test scenario, wherein the test instruction is used to instruct the test device to perform a corresponding operation in the test scenario according to the test parameter item, and the operation is used to adjust the state parameters of the traffic element in the test scenario;

[0009] Obtain the driving status of the vehicle under test when implementing the function to be tested;

[0010] Test results are generated based on the driving state and the test instructions. The test results are used to indicate the actual value range of the test parameter item corresponding to the function to be tested.

[0011] Secondly, embodiments of this application also provide an autonomous driving testing device, including one or more processors, which work individually or jointly to perform the steps of the vehicle testing method described above.

[0012] Thirdly, embodiments of this application also provide an autonomous driving testing system, including:

[0013] The vehicle under test is used to implement the functions to be tested in the test scenario based on the observation data of traffic elements in the test scenario and the preset autonomous driving algorithm.

[0014] The autonomous driving test device described above is used to generate test instructions;

[0015] The testing equipment is used to perform corresponding operations in the test scenario according to the test instructions, the operations being used to adjust the state parameters of the traffic elements in the test scenario.

[0016] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the vehicle testing method described above.

[0017] This application provides a performance testing method for an autonomous driving system. Specifically, it provides a vehicle testing method by generating multiple test instructions, each including a value of each test parameter in a set of test parameter items corresponding to the function to be tested. At least one test parameter in any two test instructions has a different value. This allows the testing equipment to perform corresponding operations in a test scenario based on different test instructions, adjusting the state parameters of traffic elements in the test scenario. This enables the vehicle under test to perform the test of the function to be tested in the test scenario based on observation data of traffic elements and a preset autonomous driving algorithm. Finally, based on the driving state of the vehicle under test when performing the function to be tested and the multiple test instructions, the actual value range of the test parameter items corresponding to the function to be tested can be determined, thus determining the performance boundary of the vehicle under test and greatly improving the accuracy of performance boundary testing for autonomous vehicles.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of a scenario for implementing the vehicle testing method provided in the embodiments of this application;

[0021] Figure 2 This is a schematic diagram of another scenario for implementing the vehicle testing method provided in the embodiments of this application;

[0022] Figure 3 This is a schematic flowchart illustrating the steps of a vehicle testing method provided in an embodiment of this application;

[0023] Figure 4 This is a flowchart illustrating the vehicle testing method in an embodiment of this application;

[0024] Figure 5 This is a schematic block diagram of the structure of an autonomous driving test device provided in an embodiment of this application;

[0025] Figure 6 This is a schematic block diagram of the structure of an autonomous driving test system provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0028] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0029] Currently, the main approach involves simulating autonomous vehicles driving according to predefined driving actions in a set driving scenario to obtain simulated values ​​for corresponding performance indicators. Based on these simulated and expected values, the performance boundaries of the autonomous vehicle are then determined. However, this method suffers from the inability to realistically simulate vehicle driving, leading to inaccurate determinations of the autonomous vehicle's performance boundaries.

[0030] To address the aforementioned issues, this application provides a performance testing method for an autonomous driving system. Specifically, it provides a vehicle testing method that generates multiple test instructions, each including a value of each test parameter in a set of test parameter items corresponding to the function to be tested. At least one test parameter in any two test instructions has a different value. This allows the testing equipment to perform corresponding operations in a test scenario based on different test instructions, adjusting the state parameters of traffic elements in the test scenario. This enables the vehicle under test to perform the test of the function to be tested in the test scenario based on observation data of traffic elements and a preset autonomous driving algorithm. Finally, based on the vehicle's driving state when performing the test function and the multiple test instructions, the actual value range of the test parameter items corresponding to the function to be tested can be determined, thus defining the performance boundary of the vehicle under test and greatly improving the accuracy of performance boundary testing for autonomous vehicles.

[0031] Please see Figure 1 , Figure 1 This is a schematic diagram of a scenario for implementing the vehicle testing method provided in the embodiments of this application. Figure 1 As shown, the scenario includes an autonomous driving test device 100, a vehicle under test 200, and a test vehicle 300. The autonomous driving test device 100 generates multiple test commands for testing the functions of the vehicle under test 200 and sends these commands to the test vehicle 300. The vehicle under test 200 implements the function under test in the test scenario based on observation data of traffic elements and a preset autonomous driving algorithm. The test vehicle 300 performs corresponding operations in the test scenario according to the test parameters and their values ​​in the test commands, thereby adjusting the state parameters of the traffic elements in the test scenario.

[0032] For example, the test vehicle 300 is communicatively connected to the autonomous driving test device 100 and to the vehicle under test 200. It can be understood that the test vehicle 300 can serve as a traffic element in this test scenario. Based on test commands issued by the autonomous driving test device 100, the test vehicle 300 adjusts its position, speed, and / or trajectory, enabling the vehicle under test 200 to autonomously drive within the test scenario based on observation data of the test vehicle 300 and a pre-set autonomous driving algorithm, thereby achieving the function to be tested.

[0033] In one embodiment, the test vehicle 300 includes a vehicle platform and a traffic participant model mounted on the vehicle platform. The traffic participant model moves with the test vehicle 300 and can serve as a traffic element in the test scenario. The traffic participant model can include pedestrian models, vehicle models, bicycle models, motorcycle models, etc.

[0034] The test vehicle 300 adjusts its position, speed and / or trajectory based on the test instructions issued by the autonomous driving test device 100, thereby adjusting the position, speed and / or trajectory of the traffic participant model mounted on the vehicle platform. This enables the test vehicle 200 to drive autonomously in the test scenario based on the observation data of the traffic participant model and the preset autonomous driving algorithm, so as to realize the function to be tested.

[0035] Please see Figure 2 , Figure 2 This is a schematic diagram of another scenario for implementing the vehicle testing method provided in the embodiments of this application. For example... Figure 2 As shown, the scenario includes an autonomous driving test device 100, a vehicle under test 200, and a drone 400. The drone 400 is communicatively connected to the autonomous driving test device 100. The drone 400 includes a body 410 and a power system 420, which is mounted on the body 410 and provides flight propulsion for the drone 400. The drone 400 also carries a traffic participant model, which moves with the drone 400 and serves as a traffic element in the test scenario.

[0036] In one embodiment, the drone 400 adjusts its position, speed, and / or trajectory based on test instructions issued by the autonomous driving test device 100, thereby adjusting the position, speed, and / or trajectory of the traffic participant model mounted on the drone 400, so that the vehicle under test 200 can drive autonomously in the test scenario based on the observation data of the traffic participant model and the preset autonomous driving algorithm to achieve the function to be tested.

[0037] The following will combine Figure 1 or Figure 2 The scenario described herein provides a detailed introduction to the vehicle testing method provided by the embodiments of this application. It should be noted that... Figure 1 or Figure 2 The scenarios described are only used to explain the vehicle testing method provided in the embodiments of this application, but do not constitute a limitation on the application scenarios of the vehicle testing method provided in the embodiments of this application.

[0038] Please see Figure 3 , Figure 3 This is a schematic flowchart illustrating the steps of a vehicle testing method provided in an embodiment of this application.

[0039] like Figure 3 As shown, the vehicle testing method may include steps S101 to S106.

[0040] Step S101: Obtain the set of test parameter items corresponding to the function to be tested of the vehicle under test and the range of test values ​​for each test parameter item in the set of test parameter items.

[0041] The test vehicle can perform the function to be tested in the test scenario based on the observation data of traffic elements and the preset autonomous driving algorithm. The traffic elements in the test scenario are matched with the function to be tested. Traffic elements may include other traffic participants and / or static scene elements in the test scenario. Static scene elements include one or more of the following: lane lines, road signs, and indicator lights. Other traffic participants include one or more of the following: people, vehicles, motorcycles, and bicycles. Other traffic participants include real traffic participants or traffic participant models. For example, traffic participant models may include pedestrian models, vehicle models, bicycle models, motorcycle models, etc.

[0042] The functions to be tested on the vehicle under test may include lane keeping, driving based on lane speed limits, driving based on indicator light status, avoiding decelerating vehicles ahead in the same lane, avoiding vehicles cutting into the same lane, and braking suddenly before colliding with other road users. For example, in testing the lane keeping function, the vehicle under test performs lane keeping based on observation data of lane lines in the test scenario and a preset autonomous driving algorithm.

[0043] For example, in tests involving driving based on indicator light status, the vehicle under test navigates the test scenario using observation data of the indicator lights and a pre-set autonomous driving algorithm. As another example, in tests involving emergency braking before colliding with other road users, the vehicle under test performs emergency braking before colliding with other road users in the test scenario using observation data of other road users and a pre-set autonomous driving algorithm.

[0044] In one embodiment, before obtaining the set of test parameter items corresponding to the function to be tested, it is necessary to first determine the function to be tested of the vehicle under test. The function to be tested of the vehicle under test is determined based on the user's operation on the human-computer interaction page. For example, a function to be tested selection page for the vehicle under test is displayed; the function to be tested of the vehicle under test is determined based on the user's operation on the function to be tested selection page.

[0045] The "Test Function Selection" page includes a list of test functions for the vehicle under test. This list is related to the level, design requirements, and / or operational design domain of the autonomous driving algorithm within the vehicle. For example, based on the user's selection in the test function list, the test functions for the system under test are determined from the list.

[0046] In one embodiment, the mapping relationship between pre-stored function identifier information and a set of test parameter items is obtained; based on the function identifier information and mapping relationship of the function to be tested of the vehicle under test, the set of test parameter items corresponding to the function to be tested of the vehicle under test is determined; and the range of values ​​to be tested for each test parameter item in the set of test parameter items is selected from multiple preset value ranges. The mapping relationship between the pre-stored function identifier information and the set of test parameter items is determined based on the level and design requirements of the autonomous driving algorithm, and the multiple preset value ranges are determined based on user operations on the human-machine interface, or the multiple preset value ranges are set by the user based on existing traffic data and the operational design domain of the autonomous driving algorithm.

[0047] For example, the method for selecting the range of values ​​to be measured for each test parameter item from a set of multiple preset value ranges can be as follows: Select any preset value range from the set of multiple preset value ranges as the range of values ​​to be measured for each test parameter item in the set of test parameter items. Alternatively, select the largest preset value range from the set of multiple preset value ranges as the range of values ​​to be measured for each test parameter item in the set of test parameter items.

[0048] In one embodiment, a target test parameter item template corresponding to the function to be tested of the vehicle under test is obtained from a plurality of preset test parameter item templates; a set of test parameter items corresponding to the function to be tested and the range of values ​​to be tested for each test parameter item in the set of test parameter items are obtained from the target test parameter item templates. By presetting a plurality of test parameter item templates, the test parameter item template corresponding to the function to be tested can be determined conveniently and quickly, thereby facilitating the determination of the set of test parameter items and the range of values ​​to be tested for each test parameter item in the set of test parameter items.

[0049] The preset test parameter item templates are determined based on the level of the autonomous driving algorithm, design requirements, and operational design domain. Different functions to be tested correspond to different test parameter item templates. The preset test parameter item template includes a set of test parameter items corresponding to a function to be tested and the range of values ​​to be tested for each test parameter item in the set of test parameter items.

[0050] In one embodiment, a template editing page is displayed, which shows a target test parameter item template and a confirmation button. Based on the user's editing operation on the target test parameter item template, the value range of the test parameter items in the target test parameter item template is edited. In response to the user's triggering operation on the confirmation button, the set of test parameter items corresponding to the function to be tested and the value range to be tested for each test parameter item in the set of test parameter items are obtained from the edited target test parameter item template. By providing a test parameter item template, users can easily edit it and conveniently determine the set of test parameter items and the value range to be tested for each of the test parameter items in the set of test parameter items.

[0051] Step S102: Determine multiple values ​​for each test parameter within the range of values ​​to be measured.

[0052] For example, the value of each test parameter is discretized within the range of values ​​to be measured, resulting in multiple values ​​for each test parameter. By discretizing the value of each test parameter, multiple values ​​for each test parameter can be obtained quickly. The entire process can obtain the data required for testing without user intervention, thus improving testing efficiency.

[0053] For example, within the range of values ​​to be tested, the values ​​of each test parameter are discretized using equal-step and / or variable-step methods. The discretization steps for each test parameter may be different or the same. Discretizing the test parameter values ​​using the equal-step method ensures that the interval between adjacent values ​​of a test parameter is the same, resulting in smoother generated test commands. This allows for smoother adjustment of the state parameters of traffic elements in the test scenario, improving test performance. Discretizing the test parameter values ​​using the variable-step method ensures that the interval between adjacent values ​​of a test parameter is different, allowing the generated test commands to cover more scenarios. This, in turn, enables better adjustment of the state parameters of traffic elements in the test scenario, further improving test performance.

[0054] Step S103: Generate multiple test instructions based on the set of test parameter items and multiple values ​​for each test parameter item.

[0055] Each test instruction includes one value for each test parameter in the set of test parameter items. Any two test instructions may have some or all of the same test parameter item with different values. Alternatively, any two test instructions may have some of the same test parameter item with the same values.

[0056] For example, when generating test instructions, the values ​​of some test parameters in the test parameter item set are kept unchanged, while the values ​​of another part of the test parameter item set are changed to generate multiple test instructions, which are then rotated. This can generate more test instructions, allowing the generated test instructions to cover more situations.

[0057] For example, if the set of test parameters includes test parameter A, test parameter B, and test parameter C, and the values ​​of test parameter A include A1 and A2, the values ​​of test parameter B include B1 and B2, and the values ​​of test parameter C include C1 and C2, then the combinations of values ​​for test parameter A, test parameter B, and test parameter C are [A1 B1C1], [A1 B1 C2], [A1 B2 C1], [A1 B2 C2], [A2 B1 C1], [A2 B1 C2], [A2 B2 C1], and [A2 B2C2], which can generate 8 test instructions.

[0058] Step S104: Send the test command to the test device in the test scenario;

[0059] The testing equipment performs corresponding operations within the test scenario based on each test parameter item and its value in the test instruction. These operations adjust the state parameters of traffic elements within the test scenario. Adjusting the state parameters of traffic elements in the test scenario may include: adjusting the position information, speed information, and trajectory of other traffic participants in the test scenario. Alternatively, adjusting the state parameters of traffic elements in the test scenario may include: adjusting the degree of lane line absence, speed limit information of road signs, and / or the indication status of indicator lights in the test scenario.

[0060] In one embodiment, the testing equipment is further used to modify the operating parameters of the vehicle under test. These operating parameters include hardware and / or software operating parameters of the vehicle under test, with the software operating parameters including the operating parameters of the autonomous driving algorithm. The autonomous driving algorithm includes software modules such as a recognition module, a perception module, a decision-making and planning module, and a control module. The operating parameters of the autonomous driving algorithm can include the operating parameters of the recognition module, the perception module, the decision-making and planning module, and the control module.

[0061] For example, the hardware of the vehicle under test may include a vision sensor, a radar device, a control chip, etc. The vision sensor may include at least one of the following: a binocular camera, a wide-angle camera, an infrared camera, etc. The hardware operating parameters of the vision sensor may include shooting orientation, field of view, resolution, exposure parameters, etc. The radar device may include at least one of the following: lidar, ultrasonic radar, millimeter-wave radar, etc. The hardware operating parameters of the radar device may include detection range, detection direction, distance detection accuracy, etc.

[0062] Step S105: Obtain the driving status of the vehicle under test when implementing the function to be tested.

[0063] For example, the driving status of the vehicle under test while implementing the function to be tested can be obtained by communicating with the vehicle under test. Alternatively, the driving status of the vehicle under test while implementing the function to be tested can be obtained by communicating with the test equipment. Or, the driving status of the vehicle under test while implementing the function to be tested can be obtained by communicating with the roadside equipment.

[0064] The test vehicle, testing equipment, and / or roadside equipment include environmental sensors. These sensors acquire observational data of the test vehicle while performing the function under test, and the driving state of the test vehicle while performing the function is determined based on this data. For example, environmental sensors may include radar and / or vision sensors. Radar may include at least one of the following: lidar, ultrasonic radar, millimeter-wave radar, etc., and vision sensors may include at least one of the following: binocular camera, wide-angle camera, infrared camera, etc.

[0065] In one embodiment, the multiple test instructions include a first test instruction and a second test instruction. The value of each test parameter item in the second test instruction is determined based on the range of values ​​to be measured for each test parameter item and the target driving state of the vehicle under test. The target driving state of the vehicle under test is determined after the test equipment executes the operation corresponding to the first test instruction. The first test instruction is sent to the test equipment before the second test instruction.

[0066] By determining the value of each test parameter item by observing the vehicle's driving state while performing the function under test and the range of values ​​to be measured for each test parameter item, and generating test instructions based on the redefined values ​​of each test parameter item, the actual value range of the test parameter items corresponding to the function under test can be obtained more quickly, reducing the number of tests and thus improving testing efficiency.

[0067] For example, after obtaining the driving state of the vehicle under test when implementing the function to be tested, multiple values ​​of each test parameter item are redefined based on the obtained driving state and the range of values ​​to be tested for each test parameter item, and multiple test instructions are generated according to the set of test parameter items and the multiple values ​​of each redefined test parameter item.

[0068] For example, if the obtained driving state does not meet the preset driving state conditions, the deviation length of the test parameter item's value is reduced, and according to the reduced deviation length, the value of each test parameter item is discretized within the range of values ​​to be measured, resulting in multiple redefined values ​​for each test parameter item. The deviation length of the test parameter item's value can be reduced using a bisection method, or other methods can be used; this embodiment does not specifically limit this approach.

[0069] For example, if the obtained driving state meets the preset driving state conditions, the dispersion length of the test parameter item is increased, and the value of each test parameter item is discretized within the range of values ​​to be measured according to the increased dispersion length, resulting in multiple values ​​for each redefined test parameter item. The dispersion length of the test parameter item value can be increased using a bisection method, or other methods can be used; this embodiment does not specifically limit this approach.

[0070] Step S106: Generate test results based on driving status and test instructions.

[0071] The test result indicates the actual value range of each test parameter in the set of test parameter items corresponding to the function under test. This actual value range represents the value boundary of the test parameter item corresponding to the function under test when the vehicle under test implements the function under test. An example is shown below.

[0072] For example, if the set of test parameters corresponding to the function to be tested includes one test parameter, the actual value range can be represented as a line segment; if the set of test parameters corresponding to the function to be tested includes two test parameter items, the actual value range can be represented as a plane; and if the set of test parameters corresponding to the function to be tested includes three test parameter items, the actual value range can be represented as a sphere.

[0073] In one embodiment, test instructions corresponding to a driving state that meets preset driving state conditions are obtained; based on the value of each test parameter item in the obtained multiple test instructions, the actual value range of each test parameter item in the test parameter items corresponding to the function to be tested is determined. The driving state of the vehicle under test when implementing the function to be tested includes a first driving state and a second driving state. The first driving state meets the preset driving state conditions, while the second driving state does not meet the preset driving state conditions. The preset driving state conditions are related to the function to be tested of the vehicle under test, and the actual value range includes the value of each test parameter item in the test instruction corresponding to the first driving state.

[0074] In one embodiment, test evaluation standard information for the function to be tested is obtained; based on the test evaluation standard information and the range of values ​​to be measured for each test parameter in the set of test parameter items corresponding to the function to be tested of the vehicle under test, the expected evaluation index of the function to be tested is determined. The test evaluation standard information can be determined based on a pre-stored mapping relationship between function identifiers and test evaluation standard information, as well as the function identifier of the function to be tested. The mapping relationship between function identifiers and test evaluation standard information can be set based on actual circumstances, and this embodiment does not specifically limit this.

[0075] For example, the test evaluation criteria information includes the standard value range of the test parameter items corresponding to the function to be tested. The expected evaluation index of the function to be tested can be determined by: determining the deviation between the value range of the function to be tested and the standard value range, and determining the expected evaluation index of the function to be tested based on the deviation. A higher expected evaluation index indicates a better function to be tested, and a lower expected evaluation index indicates a worse function to be tested. The expected evaluation index is negatively correlated with the deviation; that is, a larger deviation indicates a smaller expected evaluation index, and a smaller deviation indicates a larger expected evaluation index.

[0076] In one embodiment, the expected evaluation index of the vehicle under test is determined based on the expected evaluation indices of all the functions to be tested. For example, the expected evaluation indices of all the functions to be tested are summed to obtain the expected evaluation index of the vehicle under test. A higher expected evaluation index indicates better performance of the vehicle under test, while a lower expected evaluation index indicates worse performance.

[0077] Please see Figure 4 , Figure 4 This is a flowchart illustrating a vehicle testing method according to an embodiment of this application. Figure 4As shown, the user inputs the autonomous driving algorithm level, design requirements, and operational design domain of the vehicle under test. Based on these parameters, a list of functions to be tested is generated, from which the user selects a function. After selection, the set of test parameters and the range of values ​​for each parameter are determined. Multiple test instructions are then generated based on the set of test parameters and the range of values ​​for each parameter. These values ​​are determined from the range of values ​​to be tested. The test instructions are then sent to the testing equipment, which executes them. Finally, the driving status of the vehicle under test is obtained, and the test results are output based on the driving status and the test instructions.

[0078] The vehicle testing method provided in the above embodiments generates multiple test instructions, each including a value of each test parameter in the set of test parameter items corresponding to the function to be tested. At least one test parameter in any two test instructions has a different value. This allows the testing equipment to perform corresponding operations in the test scenario based on different test instructions, adjusting the state parameters of traffic elements in the test scenario. This enables the vehicle under test to perform the test of the function to be tested in the test scenario based on observation data of traffic elements and a preset autonomous driving algorithm. Finally, based on the driving state of the vehicle under test when performing the function to be tested and the multiple test instructions, the actual value range of the test parameter items corresponding to the function to be tested can be determined, thus determining the performance boundary of the vehicle under test and greatly improving the accuracy of performance boundary testing for autonomous vehicles.

[0079] Please see Figure 5 , Figure 5 This is a schematic block diagram of the structure of an autonomous driving test device provided in an embodiment of this application.

[0080] like Figure 5 As shown, the autonomous driving test device 500 may include one or more processors 510, which work individually or together to perform the aforementioned vehicle test method.

[0081] For example, the autonomous driving test device 500 also includes a memory 520.

[0082] For example, processor 510 and memory 520 are connected via bus 530, such as an I2C (Inter-integrated Circuit) bus.

[0083] Specifically, the processor 510 can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0084] Specifically, the memory 520 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc.

[0085] The processor 510 is used to run a computer program stored in the memory 520, and performs the following steps when executing the computer program:

[0086] Obtain the set of test parameter items corresponding to the function to be tested of the vehicle under test and the range of test values ​​for each test parameter item in the set of test parameter items, wherein the vehicle under test can implement the function to be tested in the test scenario based on the observation data of traffic elements in the test scenario and the preset autonomous driving algorithm.

[0087] Multiple values ​​are determined for each of the test parameter items within the range of values ​​to be measured;

[0088] Multiple test instructions are generated based on the set of test parameters and the values, wherein each test instruction includes a value of each test parameter in the set of test parameters, and some or all of the values ​​of the same test parameter in any two test instructions are different.

[0089] The test instruction is sent to the test device in the test scenario, wherein the test instruction is used to instruct the test device to perform a corresponding operation in the test scenario according to the test parameter item, and the operation is used to adjust the state parameters of the traffic element in the test scenario;

[0090] Obtain the driving status of the vehicle under test when implementing the function to be tested;

[0091] Test results are generated based on the driving state and the test instructions. The test results are used to indicate the actual value range of the test parameter item corresponding to the function to be tested.

[0092] Optionally, the traffic elements include other traffic participants in the test scenario, which include one or more of the following: people, vehicles, motorcycles, and bicycles.

[0093] Optionally, the other traffic participants include real traffic participants or traffic participant models.

[0094] Optionally, adjusting the state parameters of the traffic elements in the test scenario includes adjusting the position information, speed information, and trajectory of other traffic participants in the test scenario.

[0095] Optionally, the traffic elements include static scene elements, which include one or more of the following: lane lines, road signs, and indicator lights.

[0096] Optionally, adjusting the state parameters of the traffic elements in the test scenario includes: adjusting the degree of missing lane lines, the speed limit information of the road signs, and / or the indication status of the indicator lights in the test scenario.

[0097] Optionally, the testing equipment can also be used to modify the operating parameters of the vehicle under test.

[0098] Optionally, the operating parameters include the hardware operating parameters and / or software operating parameters of the vehicle under test.

[0099] Optionally, the software operating parameters include the operating parameters of the autonomous driving algorithm.

[0100] Optionally, the plurality of test instructions include a first test instruction and a second test instruction, wherein the value of each test parameter item in the second test instruction is determined based on the range of values ​​to be tested and the target driving state of the vehicle under test, and the target driving state of the vehicle under test is determined after the test equipment executes the operation corresponding to the first test instruction.

[0101] Optionally, when the processor 510 determines multiple values ​​for each of the test parameter items within the range of values ​​to be measured, it is configured to:

[0102] Discretize the value of each test parameter within the range of values ​​to be measured to obtain multiple values ​​for each test parameter.

[0103] Optionally, when the processor 510 discretizes the value of each test parameter within the range of values ​​to be measured, it is configured to:

[0104] Within the range of values ​​to be measured, the values ​​of each test parameter are discretized with equal and / or variable step sizes.

[0105] Optionally, the step lengths of each of the test parameter items may be different or the same.

[0106] Optionally, the function to be tested of the vehicle under test is determined based on the user's operation on the human-computer interaction page.

[0107] Optionally, the processor 510 is further configured to perform the following steps:

[0108] The functions to be tested for the vehicle under test are determined based on the user's actions on the displayed function selection page.

[0109] Optionally, the test function selection page includes a list of test functions. When the processor 510 determines the test functions of the vehicle under test based on the user's operation on the displayed test function selection page, it is configured to:

[0110] Based on the user's selection operation in the list of functions to be tested, the functions to be tested for the vehicle under test are determined from the list of functions to be tested.

[0111] Optionally, the list of functions to be tested is related to the level, design requirements information, and / or operating design domain of the autonomous driving algorithm.

[0112] Optionally, when the processor 510 acquires the set of test parameter items corresponding to the test function of the vehicle under test and the range of test values ​​for each test parameter item in the set of test parameter items, it is configured to:

[0113] Obtain the mapping relationship between pre-stored function identifier information and test parameter item set;

[0114] Based on the function identification information of the function to be tested of the vehicle under test and the mapping relationship, determine the set of test parameter items corresponding to the function to be tested of the vehicle under test.

[0115] Select the range of values ​​to be tested for each test parameter item from the set of test parameter items, which is a plurality of preset ranges of values.

[0116] Optionally, the mapping relationship between the pre-stored function identification information and the test parameter item set is determined based on the level and design requirements information of the autonomous driving algorithm.

[0117] Optionally, multiple preset value ranges are determined based on the user's actions on the human-computer interaction page.

[0118] Optionally, multiple preset value ranges are determined based on the operational design domain of the autonomous driving algorithm.

[0119] Optionally, when the processor 510 acquires the set of test parameter items corresponding to the test function of the vehicle under test and the range of test values ​​for each test parameter item in the set of test parameter items, it is configured to:

[0120] Obtain the target test parameter item template corresponding to the function to be tested of the vehicle under test from a set of preset test parameter item templates;

[0121] Obtain the set of test parameter items corresponding to the function to be tested and the range of test values ​​for each test parameter item in the set of test parameter items from the target test parameter item template.

[0122] Optionally, the preset templates for multiple test parameter items are determined based on the autonomous driving algorithm's level, design requirements information, and operational design domain.

[0123] Optionally, the processor 510 is further configured to perform the following steps:

[0124] Based on the user's editing operation on the target test parameter item template in the displayed template editing page, the value range of the test parameter item in the target test parameter item template is edited;

[0125] In response to the user's confirmation button on the template editing page, the set of test parameter items corresponding to the function to be tested and the range of test values ​​for each test parameter item in the set of test parameter items are obtained from the edited target test parameter item template.

[0126] Optionally, when the processor 510 generates test results based on the driving state and the test instructions, it is configured to:

[0127] Obtain the test command corresponding to the driving state that satisfies the preset driving state conditions;

[0128] Based on the value of each test parameter item in the multiple test instructions obtained, determine the actual value range of each test parameter item in the test parameter items corresponding to the function to be tested.

[0129] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the autonomous driving test device described above can be referred to the corresponding process in the aforementioned vehicle test method embodiments, and will not be repeated here.

[0130] Please see Figure 6 , Figure 6 This is a schematic block diagram of the structure of an autonomous driving test system provided in an embodiment of this application.

[0131] like Figure 6 As shown, the autonomous driving test system 600 may include:

[0132] The test vehicle 610 is used to implement the test function of the test vehicle in the test scenario based on the observation data of traffic elements in the test scenario and the preset autonomous driving algorithm.

[0133] The autonomous driving test device 620 is used to generate test instructions;

[0134] The testing device 630 is used to perform corresponding operations in the test scenario according to the test instructions, the operations being used to adjust the state parameters of the traffic elements in the test scenario.

[0135] The autonomous driving test device 620 is communicatively connected to the test equipment 630. For example, the autonomous driving test device 620 is communicatively connected to the vehicle under test 610.

[0136] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the autonomous driving test system described above can be referred to the corresponding process in the aforementioned vehicle test method embodiments, and will not be repeated here.

[0137] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, and a processor executing the program instructions to implement the steps of the vehicle testing method provided in the above embodiments.

[0138] The computer-readable storage medium can be an internal storage unit of the control terminal or unmanned aerial vehicle described in any of the foregoing embodiments, such as the hard disk or memory of the control terminal or unmanned aerial vehicle. The computer-readable storage medium can also be an external storage device of the control terminal or unmanned aerial vehicle, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the control terminal or unmanned aerial vehicle.

[0139] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0140] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0141] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A vehicle testing method, characterized in that, include: Obtain the set of test parameter items corresponding to the function to be tested of the vehicle under test and the range of test values ​​for each test parameter item in the set of test parameter items, wherein the vehicle under test can implement the function to be tested in the test scenario based on the observation data of traffic elements in the test scenario and the preset autonomous driving algorithm. Within the range of values ​​to be measured, the value of each test parameter is discretized with equal and / or variable step sizes to obtain multiple values ​​for each test parameter. Multiple test instructions are generated based on the set of test parameters and the values, wherein each test instruction includes a value of each test parameter in the set of test parameters, and some or all of the values ​​of the same test parameter in any two test instructions are different. The test instruction is sent to the test device in the test scenario, wherein the test instruction is used to instruct the test device to perform a corresponding operation in the test scenario according to the test parameter item, and the operation is used to adjust the state parameters of the traffic element in the test scenario; Obtain the driving status of the vehicle under test when implementing the function to be tested; Based on the driving state and the test command, test results are generated, and the test results are used to indicate the actual value range of the test parameter item corresponding to the function to be tested. The multiple test instructions include a first test instruction and a second test instruction. The value of each test parameter item in the second test instruction is determined based on the range of values ​​to be tested and the target driving state of the vehicle under test. The target driving state of the vehicle under test is determined after the test equipment executes the operation corresponding to the first test instruction. If the target driving state does not meet the preset driving state conditions, the current trip length is reduced; if the target driving state meets the preset driving state conditions, the current trip length is increased.

2. The vehicle testing method according to claim 1, characterized in that, The traffic elements include other traffic participants in the test scenario, which include one or more of the following: people, vehicles, motorcycles, and bicycles.

3. The vehicle testing method according to claim 2, characterized in that, The other traffic participants include real traffic participants or traffic participant models.

4. The vehicle testing method according to claim 2, characterized in that, The adjustment of the state parameters of the traffic elements in the test scenario includes: adjusting the position information, speed information, and trajectory of other traffic participants in the test scenario.

5. The vehicle testing method according to claim 1, characterized in that, The traffic elements include static scene elements, which include one or more of the following: lane lines, road signs, and indicator lights.

6. The vehicle testing method according to claim 5, characterized in that, The adjustment of the state parameters of the traffic elements in the test scenario includes: adjusting the degree of missing lane lines, the speed limit information of the road signs, and / or the indication status of the indicator lights in the test scenario.

7. The vehicle testing method according to claim 1, characterized in that, The testing equipment is also used to modify the operating parameters of the vehicle under test.

8. The vehicle testing method according to claim 7, characterized in that, The operating parameters include the hardware operating parameters and / or software operating parameters of the vehicle under test.

9. The vehicle testing method according to claim 8, characterized in that, The software operating parameters include the operating parameters of the autonomous driving algorithm.

10. The vehicle testing method according to claim 1, characterized in that, The distances of each of the test parameters may be different or the same.

11. The vehicle testing method according to any one of claims 1-9, characterized in that, The functions to be tested on the vehicle under test are determined based on the user's operations on the human-computer interaction page.

12. The vehicle testing method according to claim 11, characterized in that, The method further includes: Displays the test function selection page for the vehicle under test; The functions to be tested for the vehicle under test are determined based on the user's actions on the function selection page.

13. The vehicle testing method according to claim 12, characterized in that, The function selection page includes a list of functions to be tested. Determining the functions to be tested for the vehicle under test based on the user's actions on the function selection page includes: Based on the user's selection operation in the list of functions to be tested, the functions to be tested for the vehicle under test are determined from the list of functions to be tested.

14. The vehicle testing method according to claim 13, characterized in that, The list of functions to be tested is based on the level, design requirements, and / or operational design domain of the autonomous driving algorithm.

15. The vehicle testing method according to any one of claims 1-9, characterized in that, The acquisition of the set of test parameter items corresponding to the test function of the vehicle under test and the range of test values ​​for each test parameter item in the set of test parameter items includes: Obtain the mapping relationship between pre-stored function identifier information and test parameter item set; Based on the function identification information of the function to be tested of the vehicle under test and the mapping relationship, determine the set of test parameter items corresponding to the function to be tested of the vehicle under test. Select the range of values ​​to be tested for each test parameter item from the set of test parameter items, which is a plurality of preset ranges of values.

16. The vehicle testing method according to claim 15, characterized in that, The mapping relationship between the pre-stored functional identification information and the set of test parameter items is determined based on the level and design requirements information of the autonomous driving algorithm.

17. The vehicle testing method according to claim 15, characterized in that, The multiple preset value ranges are determined based on the user's actions on the human-computer interaction page.

18. The vehicle testing method according to claim 15, characterized in that, The multiple preset value ranges are determined based on the operational design domain of the autonomous driving algorithm.

19. The vehicle testing method according to any one of claims 1-9, characterized in that, The acquisition of the set of test parameter items corresponding to the test function of the vehicle under test and the range of test values ​​for each test parameter item in the set of test parameter items includes: Obtain the target test parameter item template corresponding to the function to be tested of the vehicle under test from a set of preset test parameter item templates; Obtain the set of test parameter items corresponding to the function to be tested and the range of test values ​​for each test parameter item in the set of test parameter items from the target test parameter item template.

20. The vehicle testing method according to claim 19, characterized in that, The preset test parameter templates are determined based on the autonomous driving algorithm's level, design requirements, and operational design domain.

21. The vehicle testing method according to claim 19, characterized in that, The method further includes: The template editing page is displayed, which shows the target test parameter template and a confirmation button; Based on the user's editing operation on the target test parameter item template, the value range of the test parameter items in the target test parameter item template is edited; In response to the user's confirmation button, the set of test parameter items corresponding to the function to be tested and the range of test values ​​for each test parameter item in the set of test parameter items are obtained from the edited target test parameter item template.

22. The vehicle testing method according to claim 1, characterized in that, The generation of test results based on the driving state and the test command includes: Obtain the test command corresponding to the driving state that satisfies the preset driving state conditions; Based on the value of each test parameter item in the multiple test instructions obtained, determine the actual value range of each test parameter item in the test parameter items corresponding to the function to be tested.

23. An autonomous driving testing device, characterized in that, It includes one or more processors, which work individually or together, for performing the steps of the vehicle testing method as described in any one of claims 1-22.

24. An autonomous driving testing system, characterized in that, include: The vehicle under test is used to implement the functions to be tested in the test scenario based on the observation data of traffic elements in the test scenario and the preset autonomous driving algorithm. The autonomous driving test apparatus as described in claim 23 is used to generate test instructions; The testing equipment is used to perform corresponding operations in the test scenario according to the test instructions, the operations being used to adjust the state parameters of the traffic elements in the test scenario.

25. The autonomous driving test system according to claim 24, characterized in that, The testing equipment includes a test vehicle or a drone, the drone carrying a traffic participant model, which moves with the drone in the test scenario.

26. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the steps of the vehicle testing method as described in any one of claims 1-22.

Images