Perception program testing system and method, computer equipment and storage medium
By recording data packets in the test site and testing the control output results of the perceptual program using the ADAS module in the laboratory, the problem of complex and inefficient existing perceptual program testing methods is solved, and efficient perceptual program testing is achieved.
Patent Information
- Application Number
- CN202311596748.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-28
- Publication Date
- 2025-05-30
AI Technical Summary
The existing perceptual program testing methods require on-site testing, and the testing process is complex and inefficient, resulting in tedious on-site testing required for each version upgrade.
By recording multiple data packets at the test site, each data packet including identification information, video information and body signal, the Advanced Assisted Driving System (ADAS) module is used to run the sensing program to be tested in the laboratory to generate control output results, and then test the performance of the sensing program.
Multiple perceptual program tests can be performed through one packet recording, which reduces the test complexity and time and improves the efficiency of perceptual program testing.
Smart Images
Figure CN120066937A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of autonomous driving technology. Specifically, it relates to a perception program testing system, method, computer device, and storage medium. Background Art
[0002] During the road driving process of an autonomous vehicle, the cameras installed on the vehicle can capture images of the surrounding environment of the vehicle. Then, in the perception link, the perception algorithm (or called the perception program) can perform target recognition based on the environmental images to obtain the target recognition result; in the planning and control link, the planning and control algorithm can perform control output based on the target recognition result to control the vehicle to drive. It can be seen from this that the safe driving of autonomous vehicles is closely related to the perception accuracy of their perception programs. Therefore, when the perception program is upgraded or updated, it is necessary to test the perception program, and it can only be officially put into use when it meets the requirements.
[0003] Known perception program testing methods, such as Figure 1 As shown, when a new version needs to be tested, on-site testing is usually required. That is to say, a test vehicle equipped with the new version of the perception program needs to drive a certain distance at the test site. During this period, corresponding data is collected, such as the output result of the perception program, the output result of the planning and control algorithm, and the body signals of the vehicle, etc. Then, problem analysis can be carried out based on the collected data to determine the performance of the perception program. However, every time there is a version upgrade that needs to be tested, on-site testing is required. The testing process is complex and the efficiency is low. Therefore, how to reduce the complexity of perception program testing and improve the testing efficiency has become an urgent technical problem to be solved. Summary of the Invention
[0004] This application provides a perception program testing system, method, computer device, and storage medium to improve the testing efficiency of the perception program. The specific technical solutions are as follows.
[0005] In a first aspect, an embodiment of this application provides a perception program testing system. The system at least includes: a data acquisition device and a micro control unit; the micro control unit at least includes: an Advanced Driver Assistance System (ADAS) module;
[0006] The data acquisition device is used to obtain data packets recorded from a test site. Each data packet at least includes identification information, video information, and corresponding body signals; run the perception program to be tested, and generate perception data corresponding to each identification information based on the video information included in each data packet; and send the perception data corresponding to each identification information and the body signals to the micro control unit;
[0007] The ADAS module is configured to receive the perception data and the vehicle body signals corresponding to each of the identification information, and generate control output results corresponding to each of the identification information of the to-be-tested perception program based on the perception data and the vehicle body signals corresponding to each of the identification information.
[0008] In the embodiments of the present application, multiple data packets can be pre-recorded from a test site. Each data packet includes identification information, video information, and corresponding vehicle body signals, that is, all the data required for the perception algorithm and the planning and control algorithm are recorded. Thus, when testing the to-be-tested perception program, the to-be-tested perception program can be run based on the recorded video information to obtain the perception data corresponding to each data packet. Further, an ADAS (Advanced Driving Assistance System) module can output control output results corresponding to each identification information based on each perception data and the corresponding vehicle body signals. The accuracy of the control output results can indicate the perception accuracy of the perception program. It can be seen that in the embodiments of the present application, multiple tests of the perception program can be performed through one data packet recording. After the perception program is upgraded, it only needs to be tested in the laboratory based on the previously recorded data packets. The test process is simple, thereby improving the test efficiency of the perception program.
[0009] Optionally, the system further includes: a system-on-chip;
[0010] The data acquisition device is specifically configured to send the perception data and the vehicle body signals corresponding to each of the identification information to the system-on-chip;
[0011] The system-on-chip is configured to receive the perception data and the vehicle body signals corresponding to each of the identification information, and send the perception data and the vehicle body signals corresponding to each of the identification information to the ADAS module.
[0012] Optionally, the data acquisition device at least includes: a first format conversion module; the system-on-chip at least includes: a second format conversion module;
[0013] The first format conversion module is configured to convert the perception data corresponding to each of the identification information into a first preset format and then send it to the second format conversion module;
[0014] The second format conversion module is configured to receive the perception data corresponding to each of the identification information in the first preset format, and convert the perception data corresponding to each of the identification information into a second preset format and then send it to the ADAS module.
[0015] Optionally, the system-on-chip further includes: a vehicle simulation module;
[0016] The vehicle simulation module is configured to receive the vehicle body signals corresponding to the respective identification information sent by the data acquisition device, and send the vehicle body signals corresponding to the respective identification information to the ADAS module.
[0017] Optionally, the ADAS module is further configured to send the control output results corresponding to the respective identification information of the to-be-tested perception program to the vehicle simulation module;
[0018] The vehicle simulation module is further configured to receive and output the control output results corresponding to the respective identification information of the to-be-tested perception program.
[0019] Optionally, each of the data packets corresponds to a test scenario, and the test scenario includes: automatic emergency braking, forward collision warning, rear traffic crossing reminder, or forward traffic crossing reminder.
[0020] In a second aspect, an embodiment of the present application provides a method for testing a perception program, and the method includes:
[0021] Obtain data packets recorded from a test site, where each of the data packets includes at least identification information, video information, and corresponding vehicle body signals;
[0022] Run the to-be-tested perception program, and generate perception data corresponding to the respective identification information based on the video information included in the respective data packets;
[0023] Generate control output results corresponding to the respective identification information of the to-be-tested perception program based on the perception data and the vehicle body signals corresponding to the respective identification information.
[0024] Optionally, the method further includes:
[0025] Obtain historical control output results corresponding to the respective identification information of at least one other version of the perception program;
[0026] For any version of the perception program, compare the historical control output results corresponding to the respective identification information thereof with the control output results corresponding to the respective identification information of the to-be-tested perception program to determine the perception accuracy of the to-be-tested perception program.
[0027] In a third aspect, an embodiment of the present application provides a computer device, including: a memory and a processor, and the memory is coupled to the processor;
[0028] The memory is used to store one or more computer instructions;
[0029] The processor is used to execute the one or more computer instructions to implement the method for testing a perception program as described in the second aspect.
[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which one or more computer instructions are stored, and the instructions are executed by a processor to implement the perception program testing method described in the second aspect above.
[0031] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the perception program testing method described in the second aspect. Description of the Drawings
[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0033] Figure 1 It is a schematic diagram of a known perception program testing process;
[0034] Figure 2 It is a schematic diagram of the architecture of a known in-vehicle perception program testing system;
[0035] Figure 3 It is a schematic diagram of the architecture of the perception program testing system provided by the embodiment of the present application;
[0036] Figure 4 It is a schematic flowchart of a perception program testing method provided by the embodiment of the present application;
[0037] Figure 5 It is a schematic diagram of the structure of a computer device provided by the embodiment of the present application. Detailed Embodiments
[0038] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.
[0039] It should be noted that the terms "include" and "have" and any variations thereof in the embodiments of the present application and the accompanying drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products, or devices.
[0040] The embodiments of the present application disclose a perception program testing system, method, computer device, and storage medium, which can improve the efficiency of perception program testing. The embodiments of the present application will be described in detail below.
[0041] In the embodiments of the present application, in order to improve the efficiency of perception program testing and avoid the need for on-site testing every time the perception program is upgraded, a simulation test bench is specifically built, which can also be called a perception program testing system. That is to say, the perception program can be tested in the laboratory based on this bench, and on-site testing is no longer required.
[0042] In one implementation, the above simulation test bench may include an E-NCAP (Euro New Car Assessment Programme) bench. E-NCAP is an important automotive safety assessment agency dedicated to evaluating the safety performance of newly launched cars. Specifically, it can evaluate a vehicle in various aspects such as collision tests, active safety systems, and passive safety technologies. That is to say, for autonomous vehicles, the built bench can be used to evaluate the content to be evaluated pointed out in E-NCAP. It can be understood that the above simulation test bench can also be built according to other evaluation criteria, which is acceptable, and the embodiments of the present application do not limit this.
[0043] Figure 2 Shows the real vehicle test system architecture diagram, as Figure 2 As shown, when the perception program is tested based on a real vehicle in a real test scenario, the perception module on the SOC (System on Chip) side of the vehicle end acquires real-time data of the test scenario and obtains a perception result; the format of this perception result is an mfr message. Then, a format conversion module converts the perception result from the mfr message to an ap message and provides it to the ADAS module on the MCU (Microcontroller Unit) side. The ADAS module obtains a control output result based on the perception result, the vehicle body signals acquired from the signal input end on the MCU side, and other detections (such as vehicle end gear information, door information, hood information, etc.). Finally, the test result can be judged according to the control output of the ADAS module.
[0044] In the embodiments of the present application, in order to simulate a real vehicle test system, the structural schematic diagram of the perception program test system built is as follows Figure 3 shown. As Figure 3 shown, the perception program test system provided by the embodiments of the present application may include: an IPC (Industrial Personal Computer, data acquisition device), a system-on-chip (SOC), and a microcontroller unit (MCU).
[0045] Among them, the IPC, which can also be referred to as a data acquisition device, is used to obtain data packets (bags) recorded from a test site. Each of the data packets includes at least identification information (bag name), video information, and corresponding vehicle body signals; run the perception program to be tested, generate perception data corresponding to each identification information based on the video information included in each data packet, and send the perception data and vehicle body signals corresponding to each identification information to the microcontroller unit MCU.
[0046] In the embodiments of the present application, multiple data packets can be pre-recorded from a test site. Among them, each data packet can correspond to a test scenario. The above test scenarios can include, for example: automatic emergency braking, forward collision warning, rear traffic crossing reminder, or forward traffic crossing reminder, etc. That is to say, data recording can be performed on all scenarios that need to be tested, and then the perception program can be comprehensively tested; to ensure that after the perception program is officially running on an autonomous vehicle, the vehicle can handle various emergency situations, thereby ensuring the safety of the autonomous vehicle during driving.
[0047] Specifically, an expert can drive an autonomous vehicle to record data at a test site. Each data packet can include the video information corresponding to the test scenario and the corresponding vehicle body signals. Moreover, in order to distinguish different data packets, each data packet can be numbered as its identification information. Through the identification information of each data packet, the connection relationship between the video information and the vehicle body signals in each data packet can be established, so that subsequent control output can be performed for each test scenario. Among them, the above video information can include the video information collected by all video acquisition devices installed in the autonomous vehicle; the above vehicle body signals can include, for example: the speed, acceleration, heading angle, etc. of the autonomous vehicle at the current moment.
[0048] After obtaining the data packets, run the perception program to be tested. Based on the video information included in each data packet, the perception data corresponding to each identification information can be generated. Specifically, the data acquisition device IPC can realize continuous video playback through a script, thereby saving video playback time and improving the test efficiency of the perception program.
[0049] After generating the perception data corresponding to each identification information, the perception data corresponding to each identification information and the vehicle body signals can be sent to the micro control unit MCU; the ADAS module in the micro control unit MCU can receive the perception data corresponding to each identification information and the vehicle body signals, and generate the control output results corresponding to each identification information of the perception program to be tested based on the perception data corresponding to each identification information and the vehicle body signals. That is to say, one data packet corresponds to one control output result, and this control output result can identify the perception accuracy of the perception program.
[0050] In one implementation manner, after obtaining the control output result, at least one historical control output result corresponding to each identification information of other version perception programs can be obtained, and then for any version of the perception program, the historical control output result corresponding to each identification information of it is compared with the control output result corresponding to each identification information of the perception program to be tested to determine the perception accuracy of the perception program to be tested. For example, parameters such as braking time and braking distance in the control output result can be compared to determine the superiority and inferiority of the perception performance of the perception program to be tested and the perception performance of other version perception programs.
[0051] The above-mentioned system-on-chip SOC is used to forward the perception data generated in the data acquisition device IPC and the vehicle body signals included in the data packet. It can be understood that the system-on-chip SOC is an optional device. When the system does not include the system-on-chip SOC, the data acquisition device IPC can directly send the perception data and the vehicle body signals to the micro control unit MCU, which is all acceptable.
[0052] In an optional implementation manner, in order to match the existing vehicle-end system and ensure that the environment for testing the perception program is maximally consistent with the vehicle end, the above-mentioned perception program testing system can include a system-on-chip SOC. In this case, the data acquisition device IPC is specifically used to send the perception data corresponding to each identification information and the vehicle body signals to the system-on-chip SOC; the system-on-chip SOC is used to receive the perception data corresponding to each identification information and the vehicle body signals, and send the perception data corresponding to each identification information and the vehicle body signals to the ADAS module.
[0053] Moreover, in the embodiments of the present application, the sensing data generated by the sensing program to be tested in the data acquisition device IPC is a ros message, while in a real autonomous driving vehicle, the sensing data generated by the sensing program is an mfr message, and the ap message can be recognized by the ADAS module. Therefore, a first format conversion module can be set in the data acquisition device IPC; a second format conversion module can be set in the system-on-chip SOC. The first format conversion module is used to convert the sensing data corresponding to each identification information from a ros message to an mfr message and send it to the second format conversion module; the second format conversion module is used to receive the sensing data corresponding to each identification information of the mfr message, convert the sensing data corresponding to each identification information to an ap message, and then send it to the ADAS module. Specifically, the second format conversion module can send the sensing data to the ADAS module through the someip protocol.
[0054] In one implementation, the above-mentioned system-on-chip SOC may further include: a vehicle simulation module. On the one hand, the vehicle simulation module is used to receive the body signals corresponding to each identification information sent by the data acquisition device IPC and send the body signals corresponding to each identification information to the ADAS module; on the other hand, it is also used to receive and output the control output results corresponding to each identification information of the sensing program to be tested returned by the ADAS module, so as to judge the test results.
[0055] At the same time, on the system-on-chip SOC side and the microcontroller unit MCU side, the original modules need to be adapted. Specifically, from Figure 3 It can be seen that the original sensing module in the system-on-chip SOC and the signal input module in the microcontroller unit MCU are represented by a dotted line box, which indicates that on the system-on-chip SOC side, the output of the sensing module needs to be ignored, and the sensing data sent from the data acquisition device IPC side is used as the sensing result; on the microcontroller unit MCU side, the real vehicle signal state needs to be ignored, and the body signals sent by the vehicle simulation module are used as the body signal input.
[0056] In the embodiments of the present application, multiple data packets can be pre-recorded from a test site. Each data packet includes identification information, video information, and corresponding vehicle body signals, that is, all the data required by the perception algorithm and the motion planning and control algorithm are recorded. Therefore, when testing the perception program to be tested, the perception program to be tested can be run based on the recorded video information to obtain the perception data corresponding to each data packet. Further, the ADAS (Advanced Driving Assistance System) module can output the control output results corresponding to each identification information based on the perception data and the corresponding vehicle body signals, and the accuracy of the control output results can indicate the perception accuracy of the perception program. It can be seen that in the embodiments of the present application, multiple tests of the perception program can be performed through one recording of data packets. After the perception program is upgraded, it only needs to be tested in the laboratory based on the previously recorded data packets, and the test process is simple, thus improving the test efficiency of the perception program.
[0057] Corresponding to the system embodiment, Figure 4 FIG. 5 is a schematic flowchart of a method for testing a perception program provided by an embodiment of the present application. This method can be applied to the above-mentioned perception program test system, and the method includes the following steps:
[0058] S410: Obtain the data packets recorded from the test site, where each data packet includes at least identification information, video information, and corresponding vehicle body signals.
[0059] S420: Run the perception program to be tested, and generate the perception data corresponding to each identification information based on the video information included in each data packet.
[0060] S430: Generate the control output results corresponding to each identification information of the perception program to be tested based on the perception data and the vehicle body signals corresponding to each identification information.
[0061] In one implementation, after generating the perception data corresponding to each identification information, the format of the perception data can be converted into a preset format.
[0062] In one implementation, the above method further includes the following steps: obtaining the historical control output results corresponding to each identification information of at least one other version of the perception program; for any version of the perception program, comparing the historical control output results corresponding to each identification information of it with the control output results corresponding to each identification information of the perception program to be tested, and determining the perception accuracy of the perception program to be tested.
[0063] In the embodiments of the present application, multiple data packets can be pre-recorded from a test site. Each data packet includes identification information, video information, and corresponding vehicle body signals, that is, all the data required for the perception algorithm and the motion planning and control algorithm are recorded. Therefore, when testing the perception program to be tested, the perception program to be tested can be run based on the recorded video information to obtain the perception data corresponding to each data packet. Further, the ADAS (Advanced Driving Assistance System) module can output the control output results corresponding to each identification information based on the perception data and the corresponding vehicle body signals. The accuracy of the control output results can indicate the perception accuracy of the perception program. It can be seen that in the embodiments of the present application, multiple tests of the perception program can be performed through one-time data packet recording. After the perception program is upgraded, it only needs to be tested in the laboratory based on the previously recorded data packets. The test process is simple, thus improving the test efficiency of the perception program.
[0064] Next, a computer device provided by the embodiments of the present application will be introduced. Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of the computer device provided by the embodiments of the present application. The computer device includes:
[0065] One or more processors 40;
[0066] The processor 40 is coupled to the storage device 41. The storage device 41 is used to store one or more programs.
[0067] When the one or more programs are executed by the one or more processors 40, the electronic device realizes the technical solution of Figure 4 a perception program testing method as described above.
[0068] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it realizes the technical solution of Figure 4 a perception program testing method as described above.
[0069] The present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it realizes the technical solution of Figure 4 a perception program testing method as described above.
[0070] Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of an embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present application.
[0071] Those of ordinary skill in the art can understand that the modules in the devices in the embodiments can be distributed in the devices described in the embodiments, or can be correspondingly changed to be located in one or more devices different from the present embodiments. The modules in the above embodiments can be combined into one module, or can be further split into multiple sub-modules.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A perception program testing system, characterized in that, the system at least includes: a data acquisition device and a microcontroller unit; the microcontroller unit at least includes: an Advanced Driver Assistance System (ADAS) module; the data acquisition device is used to obtain data packets recorded from a test site, each data packet at least includes identification information, video information, and corresponding vehicle body signals; run the perception program to be tested, and generate perception data corresponding to each identification information based on the video information included in each data packet; and send the perception data corresponding to each identification information and the vehicle body signals to the microcontroller unit; the ADAS module is used to receive the perception data and the vehicle body signals corresponding to each identification information, and generate control output results corresponding to each identification information of the perception program to be tested based on the perception data and the vehicle body signals corresponding to each identification information.
2. The system according to claim 1, characterized in that, the system further includes: a system-on-chip; the data acquisition device is specifically used to send the perception data and the vehicle body signals corresponding to each identification information to the system-on-chip; the system-on-chip is used to receive the perception data and the vehicle body signals corresponding to each identification information, and send the perception data and the vehicle body signals corresponding to each identification information to the ADAS module.
3. The system according to claim 2, characterized in that, the data acquisition device at least includes: a first format conversion module; the system-on-chip at least includes: a second format conversion module; the first format conversion module is used to convert the perception data corresponding to each identification information into a first preset format and then send it to the second format conversion module; the second format conversion module is used to receive the perception data corresponding to each identification information in the first preset format, and convert the perception data corresponding to each identification information into a second preset format and then send it to the ADAS module.
4. The system according to claim 2, characterized in that, the system-on-chip further includes: a vehicle simulation module; the vehicle simulation module is used to receive the vehicle body signals corresponding to each identification information sent by the data acquisition device, and send the vehicle body signals corresponding to each identification information to the ADAS module.
5. The system according to claim 4, characterized in that, the ADAS module is further used to send the control output results corresponding to each identification information of the perception program to be tested to the vehicle simulation module; the vehicle simulation module is further used to receive and output the control output results corresponding to each identification information of the perception program to be tested.
6. The system according to any one of claims 1-5, characterized in that, each data packet corresponds to a test scenario, and the test scenario includes: automatic emergency braking, forward collision warning, rear traffic cross warning, or forward traffic cross warning.
7. A perception program testing method, characterized in that, the method includes: Obtain data packets recorded from the test site, where each of the data packets includes at least identification information, video information, and corresponding vehicle body signals; Run the perception program to be tested, and generate perception data corresponding to each of the identification information based on the video information included in each of the data packets; Generate control output results corresponding to each of the identification information of the perception program to be tested based on the perception data corresponding to each of the identification information and the vehicle body signals.
8. The method according to claim 7, wherein, the method further includes: Obtain historical control output results corresponding to each of the identification information of at least one other version of the perception program; For any version of the perception program, compare the historical control output results corresponding to each of the identification information thereof with the control output results corresponding to each of the identification information of the perception program to be tested, and determine the perception accuracy of the perception program to be tested.
9. A computer device, wherein, it includes: A memory and a processor, the memory and the processor are coupled; The memory is used to store one or more computer instructions; The processor is used to execute the one or more computer instructions to implement the perception program test method as claimed in claim 7 or 8.
10. A readable storage medium, on which one or more computer instructions are stored, wherein, the instructions are executed by a processor to implement the perception program test method as claimed in claim 7 or 8.