Production method and device of test case, terminal and computer readable storage medium

Abstract

The invention provides a production method of a test case, and aims to provide a motion state of a simulation test participating object capable of combining a single mode or multiple modes of a real traffic flow, a real vehicle, a simulator object, a virtual simulation object and the like by means of characteristics and advantages of virtual-real combined simulation data. And a fully fused high-quality case making method is carried out, so that the problems of virtual-real combination test case quality verification and compliance case generation are solved. The parameter verification link in the case making process can optimize the quality of the whole case, the made case is low in cost, high in value and repeatable, the test object and the opponent object can be well guided to perform action execution according to requirements in the test task execution process, the effectiveness of test execution is improved, and the research performance of the test result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent transportation, and particularly to a test case production method, device, terminal, and computer-readable storage medium. Background Art

[0002] Virtual-real combined simulation testing is a new and important testing technology developed for both autonomous driving testing and vehicle-road cooperation scenario testing. The key of this technology is to fully integrate the traffic data with natural motion characteristics collected from the real physical world with the virtual data generated by the simulation system, so that the fused scenario data can meet partial detail customization intervention according to requirements while closely approaching the scenario characteristics of the real world as much as possible. The data with such characteristics is used as the scenario data of the test case to support autonomous driving or vehicle-road cooperation scenario testing.

[0003] In the current test case formulation technology for autonomous driving or vehicle-road cooperation scenario testing, first, due to the immaturity of the traffic holographic perception technology in the industry, the scenario data of the macroscopic background traffic flow in the test case is generally generated by pure simulation software. The generated pure simulation traffic flow strictly follows the fixed logic preset by the simulation system and has no mapping connection relationship with the traffic flow data of real events. The formulated test cases cannot closely approach the scenario characteristics of the real world to the greatest extent, so that the test object in the simulation scenario cannot reflect its state in the real world to the greatest extent. Second, there are few current mature solutions for the test case formulation method based on the virtual-real combination. For the test cases formulated in the case of multi-modal test objects participating, due to the weakness of the technology in the case parameter verification link, the quality and compliance of the case parameters cannot be effectively verified. The quality of the virtual-real combined test cases formulated in this way is often not high and cannot well guide the efficient execution of the test task. Summary of the Invention

[0004] The purpose of the present invention is to propose a test case production method to solve the problem of the quality of current virtual-real combined test cases.

[0005] The present invention proposes a test case production method, which includes: Obtain background traffic flow and inject the background traffic flow into the simulation test system; wherein, the background traffic flow is obtained by twin of real traffic data, and the road characteristics of the road section of the actual road corresponding to the background traffic flow are similar to the road characteristics of the case road of the preset test case. Based on the virtual background traffic flow sample data, add the participating objects of each simulation test case; among them, the triggering conditions of each simulation participating object are preset in advance, and the triggering conditions include the birth point, movement trajectory, speed and heading; among them, the simulation test participating objects include one or several of real vehicles, simulator objects, and virtual simulation objects; After each simulation test participating object is triggered, a test case is obtained.

[0006] In one optional embodiment, obtaining the background traffic flow includes: Obtain complete traffic flow data within a local range of the real world based on roadside perception devices; the local range of the real world is selected based on the use case road of the test case; Use the traffic data stream as the background traffic flow, where the traffic flow data includes the basic attributes and motion state attribute information of each traffic participant.

[0007] In one optional embodiment, after each simulation test participating object is triggered, obtaining a test case includes: Generate a dynamic policy based on the combination of a fixed rule algorithm and an AI intelligent algorithm, optimize the fused use case data after the combination of virtual and real, and obtain a test case.

[0008] In one optional embodiment, after each simulation test participating object is triggered, obtaining a test case includes: Simulate and optimize the microscopic behavior of each use case participating object to ensure that the running behavior of each participating object in the scenario conforms to the preset, and obtain a test case.

[0009] In one optional embodiment, after each simulation test participating object is triggered, obtaining a test case includes: Determine that each simulation test participating object meets the formulated use case quality evaluation criteria, and the use case quality evaluation criteria include: restricting one or more behaviors such as no collision, no penetration, and no sudden change in speed, heading, and position among the simulation test participating objects.

[0010] In one optional embodiment, the method further includes: Based on the discrimination of the formulated use case quality evaluation criteria, perform use case parameter verification and optimization, and iteratively generate the most reasonable use case parameters to obtain a test case.

[0011] In one optional embodiment, the method further includes: After the simulation test system accesses the background traffic flow, all simulation use case participating objects, and use case parameter data, perform spatio-temporal calibration.

[0012] The present invention also proposes a test case production device, including: A background traffic flow editing module is used to obtain the background traffic flow and inject the background traffic flow into the simulation test system; wherein, the background traffic flow is obtained by twinning based on real traffic data, and the road characteristics of the road segments of the actual road corresponding to the background traffic flow are similar to the road characteristics of the case road in the preset test case. Overall use case scenario editing is used to add each simulation test participant object to the test case based on the virtual background traffic flow sample data; wherein, the triggering conditions of each simulation participant object are preset in advance, and the triggering conditions include the birth point, movement trajectory, speed and heading; wherein, the simulation test participant objects include one or several of real vehicles, simulator objects, and virtual simulation objects. A use case generation module is used to obtain test cases after each simulation test participant object is triggered.

[0013] The present invention also proposes a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the method proposed in any embodiment of the present invention.

[0014] Compared with the prior art, the present invention has the following advantages and technical effects: A test case production method proposed by the present invention intends to invent a high-quality use case formulation method that can fully integrate the motion states of simulation test participant objects such as real traffic flow, real vehicles, simulator objects, and virtual simulation objects by virtue of the characteristics and advantages of virtual-real combined simulation data, so as to solve the problems of quality verification of virtual-real combined test cases and generation of compliant use cases. The parameter verification link in the use case formulation process can optimize the quality of the entire use case. The formulated use cases are low-cost, high-value, and repeatable, and can well guide the test object and the opponent object to perform actions as required during the test task execution, improving the effectiveness of test execution and the feasibility of test results. Description of the Drawings

[0015] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 It is a flowchart of the test case production method in an embodiment of the present invention; Figure 2 It is a schematic diagram of the background traffic flow editing interface in an embodiment of the present invention; Figure 3 It is a schematic diagram of the overall use case scenario editing interface in an embodiment of the present invention; Figure 4It is a schematic structural diagram of a production device for test cases in an embodiment of the present invention; Figure 5 It is a schematic structural diagram of a terminal in an embodiment of the present invention. Detailed implementation manners

[0016] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0017] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0018] The present invention proposes a method for producing test cases, which is applied to a simulation test system. The simulation test system has a display device, and the user configures the test cases through the user interface displayed on the display device. Implementing the test case configuration method of the present invention, the display interface is as Figure 1 - Figure 2 shown, and includes a test site map, a background traffic flow editing interface, and an overall use case scenario editing interface. The test case addition of the present invention can include the basic information of the test case and the test case scenario information. The test case configuration method of the present application includes: Step S101: Obtain the background traffic flow and inject the background traffic flow into the simulation test system; wherein, the background traffic flow is obtained by twin of real traffic data, and the road characteristics of the road sections of the actual road corresponding to the background traffic flow are similar to the road characteristics of the use case road of the preset test case. Specifically, the system can determine the loaded background traffic flow by calling the stored "background traffic flow file selection" to select the "background traffic flow time interval".

[0019] In this embodiment, WEBGL map is used as an example. The WEBGL map supports operations such as zooming in, zooming out, and rotating the map view through interface operations, and clicks on the relevant interactive components in the interface for operations. The background traffic flow of the present application can obtain the complete traffic flow data within a local range of the real world based on roadside perception devices; the local range of the real world is selected based on the use case road of the test case; the traffic data stream is used as the background traffic flow, wherein the traffic flow data includes the basic attributes and motion state attribute information of each traffic participant.

[0020] See Figure 2 , which is the background traffic flow editing interface. This interface can perform the process of obtaining the background traffic flow, specifically as follows: For the "background traffic flow time interval", it represents intercepting the time interval range of the historical traffic flow data of a certain section of XX road. Click the "Configuration" button to start editing the time interval.

[0021] Background traffic flow time interval: The time interval range can be manually selected; click "Background traffic flow preview - Start" in the interface, and the intercepted background traffic flow data can be previewed on the WEBGL-based map. The presented range of the map can be adjusted through the view slide rail. During the preview process, the number of frames before and after can be finely adjusted, and pausing and continuing the preview are supported. When the time interval obtained through fine adjustment can meet the expectation, in the paused state of the preview, click "Set as start time" or "Set as end time", and the current historical data time will be automatically filled into the "background traffic flow time interval" and saved to obtain the background traffic flow in this embodiment.

[0022] Step S102: Based on the virtual background traffic flow sample data, add each simulation test participant object of the test case; among them, the trigger condition of each simulation participant object is preset, and the trigger condition includes the birth point, movement trajectory, and speed and heading; among them, the simulation test participant objects include one or several of real vehicles, simulator objects, and virtual simulation objects.

[0023] See Figure 3 , which is the overall use case scenario editing interface. This interface is for "overall use case scenario editing", representing editing the overall use case parameters of the test case scenario data. Click the "Edit trajectory and birth point" button to start editing the scenario parameters. The following editing operations can be specifically performed: Edit the trajectory and birth point in this interface: The type and quantity of simulator objects are matched according to the configuration of "Test main vehicle / opponent vehicle / pedestrian / quantity"; Edit object: Click one type of simulator object, such as "opponent vehicle", and several groups of "Edit object" and the corresponding legends on the map will be displayed according to the configured quantity. The relevant parameters of each object are edited independently; Edit birth time: It represents the time interval when the simulator is relative to the intercepted background traffic flow, that is, the generation time of the start time of the historical data; Edit position: Edit through the map, which is used to specify the initial position where the simulator is generated in the background traffic flow; Edit trajectory: Edit through the map, which is used to specify the driving trajectory of the simulator in the background traffic flow. The driving trajectory can change lanes. The driving trajectory is used to guide the actual driving operation of the simulator during the execution of the test task. If it does not drive according to the regulations, it is determined that the test fails; Edit average speed: It is used to specify the average driving speed of the simulator in the background traffic flow. The average speed is used to guide the actual driving operation of the simulator during the execution of the test task; Editing initial speed: It is used to specify the initial speed of the simulator in the background traffic flow. If the initial speed does not meet the use case conditions, it is determined that the test fails; Step S103: After each simulation test participant is triggered, test cases are obtained.

[0024] Specifically, dynamic policies can be generated based on the combination of fixed rule algorithms and AI intelligent algorithms to optimize the fused use case data after the combination of virtual and real, and test cases are obtained. For example: Simulate and optimize the microscopic behaviors of each use case participant to ensure that the running behaviors of each participant in the scenario meet the preset requirements, and test cases are obtained. It is necessary to pre-determine that each simulation test participant meets the formulated use case quality evaluation criteria, and the use case quality evaluation criteria include: restricting one or several behaviors such as no collision, no penetration, and no sudden change in speed, heading, and position among the simulation test participants. Based on the discrimination of the formulated use case quality evaluation criteria, perform use case parameter verification and optimization, and iteratively generate the most reasonable use case parameters to obtain test cases.

[0025] A test case production method proposed by the present invention intends to invent a high-quality use case formulation method that can fully integrate the motion states of simulation test participants such as real traffic flow, real vehicles, simulator objects, and virtual simulation objects in a single or multimodal combination by virtue of the characteristics and advantages of virtual-real combined simulation data, and solve the problems of quality verification of virtual-real combined test cases and generation of compliant use cases. The parameter verification link in the use case formulation process can optimize the quality of the entire use case. The formulated use cases are low-cost, high-value, and repeatable, and can well guide the test object and the opponent object to perform actions as required during the test task execution, improving the effectiveness of test execution and the feasibility of test results.

[0026] In this application, in order to more accurately process the data in the system and avoid the increase in object control complexity caused by spatio-temporal misalignment during the object processing process, after the simulation test system accesses the background traffic flow, all simulation use case participants, and use case parameter data, spatio-temporal calibration is performed.

[0027] Please refer to Figure 4 , Figure 4 is a schematic structural diagram of a test case production device 10 provided by an embodiment of the present invention. In this embodiment, each module included in the test case production device is used to execute each step in the corresponding embodiment of the present invention. Specifically, please refer to the relevant descriptions in the corresponding embodiments. For the sake of convenience of description, only the parts related to this embodiment are shown. See Figure 4, the production device 10 of test cases may include: a background traffic flow processing module 100, configured to obtain background traffic flow and inject the background traffic flow into a simulation test system; wherein, the background traffic flow is obtained by twinning based on real traffic data, and the road characteristics of the sections of the actual road corresponding to the background traffic flow are similar to the road characteristics of the use case road of the preset test case; an overall use case scenario editor 200, configured to add each simulation test participant object based on virtual background traffic flow sample data; wherein, the trigger condition of each simulation participant object is preset, and the trigger condition includes a birth point, a movement trajectory, and a speed and heading; wherein, the simulation test participant objects include one or more of a real vehicle, a simulator object, and a virtual simulation object; A use case generation module 300, configured to obtain test cases after each simulation test participant object is triggered.

[0028] In one optional embodiment, the background traffic flow processing module 100 is configured to obtain complete traffic flow data within a local range of the real world based on roadside perception devices; the local range of the real world is selected based on the use case road of the test case; and use the traffic data stream as the background traffic flow, wherein the traffic flow data includes the basic attributes and motion state attribute information of each traffic participant.

[0029] In one optional embodiment, the use case generation module 300 is configured to generate a dynamic strategy based on a combination of a fixed rule algorithm and an AI intelligent algorithm, optimize the fused use case data after the combination of virtual and real, and obtain test cases.

[0030] In one optional embodiment, the use case generation module 300 is configured to perform simulation tuning on the microscopic behaviors of each use case participant object to ensure that the running behaviors of each participant object in the scenario meet the preset requirements, and obtain test cases.

[0031] In one optional embodiment, the use case generation module 300 is configured to determine that each simulation test participant object meets the formulated use case quality evaluation criteria, and the use case quality evaluation criteria include: restricting one or more behaviors such as no collision, no penetration, and no sudden change in speed, heading, and position among the simulation test participant objects.

[0032] In one optional embodiment, the use case generation module 300 is configured to perform use case parameter verification and optimization based on the discrimination of the formulated use case quality evaluation criteria, iteratively generate the most reasonable use case parameters, and obtain test cases.

[0033] In one optional embodiment, the use case generation module 300 is configured to perform spatio-temporal calibration after the simulation test system accesses the background traffic flow, all simulation use case participant objects, and use case parameter data.

[0034] Figure 5 This is a schematic structural diagram of a terminal device provided by an embodiment of the present invention. As Figure 5 shown, the terminal device 700 of this embodiment includes: a processor 710, a memory 720, and a computer program 730 stored in the memory 720 and executable on the processor 710, such as a program for a point cloud rendering method. When the processor 710 executes the computer program 730, it implements the steps in each of the production methods of the above-mentioned various test cases, such as Figure 1 S101 to S103 shown. Alternatively, when the processor 710 executes the computer program 730, it implements the functions of each module in the above embodiment. For example, Figure 5 the functions of each module shown, for specific details, please refer to the relevant descriptions in the corresponding embodiments.

[0035] Exemplarily, the computer program 730 can be divided into one or more modules. One or more modules are stored in the memory 720 and executed by the processor 710 to implement the production method of the test cases provided by the embodiments of the present invention. One or more modules can be a series of computer program instruction segments capable of completing specific functions, and these instruction segments are used to describe the execution process of the computer program 730 in the terminal device 700. For example, the computer program 730 can implement the production method of the test cases provided by the embodiments of the present invention.

[0036] The terminal device 700 may include, but is not limited to, a processor 710 and a memory 720. Those skilled in the art can understand that Figure 5 this is only an example of the terminal device 700 and does not constitute a limitation on the terminal device 700. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the terminal device may further include input / output devices, network access devices, buses, etc.

[0037] The so-called processor 710 may be a central processing unit, or may also be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0038] The memory 720 may be an internal storage unit of the terminal device 700, such as the hard disk or memory of the terminal device 700. The memory 720 may also be an external storage device of the terminal device 700, such as a plug-in hard disk, a smart memory card, a flash memory card, etc. equipped on the terminal device 700. Further, the memory 720 may also include both the internal storage unit and the external storage device of the terminal device 700.

[0039] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the test case production method in each of the above embodiments.

[0040] An embodiment of the present invention provides a computer program product. When the computer program product runs on a terminal device, the terminal device is caused to execute the test case production method in each of the above embodiments.

[0041] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

[0042] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A production method of test cases, characterized in that, The method includes: Obtain the background traffic flow and inject the background traffic flow into the simulation test system; wherein, the background traffic flow is obtained by twinning based on real traffic data, and the road characteristics of the section of the actual road corresponding to the background traffic flow are similar to the road characteristics of the test road in the preset test case; Based on the virtual background traffic flow sample data, add each simulation test participant object of the test case; wherein, the trigger condition of each simulation participant object is preset, and the trigger condition includes the birth point, movement trajectory, speed and heading; wherein, the simulation test participant objects include one or several of real vehicles, simulator objects, and virtual simulation objects; After each simulation test participant object is triggered, obtain the test case.

2. The method according to claim 1, wherein Obtaining the background traffic flow includes: Obtain complete traffic flow data within a local range of the real world based on roadside perception devices; the local range of the real world is selected based on the test road of the test case; Use the traffic data stream as the background traffic flow, wherein the traffic flow data includes the basic attributes and movement state attribute information of each traffic participant.

3. The method according to claim 1, characterized in that After each simulation test participant object is triggered, obtaining the test case includes: Generate a dynamic strategy based on the combination of a fixed rule algorithm and an AI intelligent algorithm, optimize the fused use case data after combining virtual and real, and obtain the test case.

4. The method according to claim 1, wherein After each simulation test participant object is triggered, obtaining the test case includes: Perform simulation optimization on the microscopic behavior of each use case participant object to ensure that the running behavior of each participant object in the scenario conforms to the preset, and obtain the test case.

5. The method according to any one of claims 1-4, characterized in that, After each simulation test participant object is triggered, obtaining the test case includes: Determine that each simulation test participant object meets the formulated use case quality evaluation criteria, and the use case quality evaluation criteria include: restricting one or several behaviors such as no collision, no penetration, and no sudden change in speed, heading, and position among the simulation test participant objects.

6. The method according to claim 5, characterized in that, The method further includes: Based on the discrimination of the formulated use case quality evaluation criteria, perform use case parameter verification and optimization, and iteratively generate the most reasonable use case parameters to obtain the test case.

7. The method according to any one of claims 1 to 4, characterized in that The method further includes: After the simulation test system accesses the background traffic flow, all simulation use case participant objects, and use case parameter data, perform spatio-temporal calibration.

8. A production device for test cases, characterized in that, It includes: A background traffic flow editing module for obtaining the background traffic flow and injecting the background traffic flow into the simulation test system; wherein, the background traffic flow is obtained by twinning based on real traffic data, and the road characteristics of the section of the actual road corresponding to the background traffic flow are similar to the road characteristics of the test road in the preset test case; Overall use case scenario editing for adding each simulation test participant object of the test case based on virtual background traffic flow sample data; wherein, the trigger condition of each simulation participant object is preset, and the trigger condition includes the birth point, movement trajectory, speed and heading; wherein, the simulation test participant objects include one or several of real vehicles, simulator objects, and virtual simulation objects; A use case generation module for obtaining the test case after each simulation test participant object is triggered.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 7.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images