Hardware-in-the-loop simulation test method and system for autonomous driving

Abstract

The present invention provides a hardware-in-the-loop simulation test method for autonomous driving, which includes: receiving sensor data, wherein the sensor data includes data packets of multiple sensors collected in the same time period, and each data packet is marked with a timestamp indicating data generation; subtracting a preset delay time from the timestamp of the data packet of a preset type of sensor to obtain a corrected timestamp; caching the data packets of the multiple sensors in a cache area; reading the data packets of the multiple sensors in the cache area in the order of the corrected timestamps, and writing them into a specified memory area; instantly reading the data packets of the multiple sensors in the specified memory area, and converting the data packets of the multiple sensors into electrical signals and transmitting them to a test device for testing. In addition, the present invention also provides a hardware-in-the-loop test system.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a hardware-in-the-loop simulation test method and system for autonomous driving. Background Art

[0002] With the development of intelligent automobiles, Advanced Driving Assistance System (ADAS) auxiliary driving functions have gradually become standard. The hardware-in-loop test (HIL) system of ADAS sensors has become the key to R&D efficiency.

[0003] However, traditional HIL often provides image data directly through the display, that is, it is implemented based on the HDMI interface conversion method. Since the amount of image data is much larger than data in other formats, the clock cycle of the image data cannot be accurately controlled when providing image data in the above method, resulting in the inability to synchronize data with other sensors. Summary of the invention

[0004] In view of this, it is necessary to provide a hardware-in-the-loop test that can achieve synchronization between image data and other data.

[0005] In a first aspect, the present invention provides a hardware-in-the-loop simulation test method for autonomous driving, the hardware-in-the-loop simulation test method for autonomous driving comprising:

[0006] The computer device receives sensor data, wherein the sensor data includes data packets of multiple sensors collected in the same time period, and each data packet is marked with a timestamp indicating data generation;

[0007] The computer device subtracts a preset delay time from a data packet timestamp of a preset type of sensor to obtain a corrected timestamp;

[0008] The computer device sends the data packets of the plurality of sensors to the simulation device in sequence according to the sequence of the corrected timestamps;

[0009] The simulation device caches the data packets of the plurality of sensors in a cache area;

[0010] The simulation device reads the data packets of the multiple sensors in the buffer area in the order of the correction timestamps and writes them into the designated memory area;

[0011] The simulation device instantly reads the data packets of the multiple sensors in the designated memory area, converts the data packets of the multiple sensors into electrical signals, and transmits them to the testing device for testing.

[0012] In a second aspect, the present invention further provides a hardware-in-the-loop simulation test method for autonomous driving, the hardware-in-the-loop simulation test method for autonomous driving comprising:

[0013] Receive sensor data, wherein the sensor data includes data packets of multiple sensors collected in the same time period, each data packet is marked with a timestamp indicating data generation;

[0014] Subtract the preset delay time from the data packet timestamp of the preset type of sensor to obtain a corrected timestamp;

[0015] Buffering the data packets of the plurality of sensors in a buffer area;

[0016] Reading data packets of multiple sensors in the buffer area in the order of the correction timestamps and writing them into the designated memory area;

[0017] The data packets of the multiple sensors in the designated memory area are read immediately, and the data packets of the multiple sensors are converted into electrical signals and then transmitted to the testing device for testing.

[0018] In a third aspect, the present invention further provides a hardware-in-the-loop simulation test system for autonomous driving, the system comprising a computer device, including a first storage medium and a first processor, wherein the first storage medium stores first hardware-in-the-loop simulation test program instructions;

[0019] The simulation device comprises a second storage medium and a second processor, wherein the second storage medium stores second hardware-in-the-loop simulation test program instructions;

[0020] The first processor and the second processor respectively execute the first program instructions and the second program instructions to jointly implement the above-mentioned hardware-in-the-loop simulation test method for autonomous driving.

[0021] The above-mentioned hardware-in-the-loop simulation test method and system can advance the timestamp of the image sensor data packet by subtracting the delay time from the original timestamp of the image sensor data packet, thereby eliminating the delay of the image sensor data packet due to excessive data volume, and realize the synchronous transmission of the image sensor data packet and the data packets of other sensors whose original timestamp is the same as the original timestamp of the image sensor data packet to the test device. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0023] Figure 1 Schematic diagram of the autonomous driving hardware-in-the-loop simulation test system provided in the embodiment.

[0024] Figure 2 A flowchart of the autonomous driving hardware-in-the-loop simulation test method provided for the first embodiment.

[0025] Figure 3 A schematic diagram of a sub-process of an autonomous driving hardware-in-the-loop simulation test method provided in an embodiment.

[0026] Figure 4 A flowchart of the autonomous driving hardware-in-the-loop simulation test method provided for the second embodiment.

[0027] Figure 5 A schematic diagram of the functional modules of the simulation device in the autonomous driving hardware-in-the-loop simulation test system provided in the embodiment.

[0028] Figure 6 A schematic diagram of data transmission during the autonomous driving hardware-in-the-loop simulation test provided in an embodiment.

[0029] Figure 7 A schematic diagram of the computer equipment structure framework in the autonomous driving hardware-in-the-loop simulation test system provided in the embodiment.

[0030] Figure 8 A schematic diagram of the structural framework of the simulation device in the autonomous driving hardware-in-the-loop simulation test system provided in the embodiment.

[0031] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The drawings in the specification illustrate examples of embodiments of the present invention. It is understood that the proportions shown in the drawings in the specification are not the proportions of the actual implementation of the present invention. They are only for the purpose of illustrative explanation and are not drawn according to the original size. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0033] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate. In other words, the described embodiments are implemented according to a sequence other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations may also include other content. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to only those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] It should be noted that the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0035] The present invention provides a hardware-in-the-loop simulation test method, which can ensure that the data packets of different sensors are time-series synchronized during the test process and achieve a better restoration effect by correcting the timestamps of the data packets of different sensors and storing or transmitting them according to the corrected timestamps.

[0036] Please see Figure 1, which is a schematic diagram of the operating environment of the hardware-in-the-loop simulation test method for autonomous driving. The hardware-in-the-loop simulation test method for autonomous driving is run by a hardware-in-the-loop simulation test system 100 for autonomous driving. The hardware-in-the-loop simulation test system 100 includes a computer device 1, a simulation test device 2, and a test device 3. Among them, the computer device 1 needs to write the data of the hardware-in-the-loop simulation test into the simulation device 2, and send it to the test device 3 after restoration through the simulation device 2. Among them, the computer device 1 can also be called an upper computer. The simulation test device 2 can also be called a lower computer. In this embodiment, the computer device 1 is connected to the simulation test device 2 and the test device 3 in communication. The simulation test device 2 is also connected to the test device 3 in communication. The hardware-in-the-loop simulation test system 100 also provides a control platform 5 for controlling the startup of the computer device 1, the simulation test device 2, and the test device 3. The control platform 5 can provide a user interface 51 for the user to input various operation instructions, including but not limited to startup instructions. At the same time, the user interface 51 also displays the status information, operation information, etc. of the simulation test device 2 and the test device 3.

[0037] In this embodiment, the hardware-in-the-loop simulation test method for autonomous driving includes the following steps.

[0038] Please see Figure 2 , step S101, the computer device 1 receives sensor data, the sensor data includes data packets of multiple sensors collected in the same time period, and each data packet is marked with a timestamp indicating the generation of the data. Specifically, the sensor data can be data sensed by different sensors set on the vehicle 400 or data sensed by sensors set on the roadside. Among them, the multiple sensors 401 include but are not limited to image sensors 401a, inertial measurement units (IMU) 401b, radar sensors 401c, laser radar sensors 401d, etc. The multiple sensors 401 may include multiple sensors of the same type, for example, multiple laser radar sensors, multiple identical image sensors, etc. The image sensor may include image sensors of different models or types. It can be understood that the data packet formats of different sensors are different, and the sizes of the data packets are also different. For example, the data sensed by the image sensor is image data, and the formats of the image data of image sensors of different models may also be different. The inertial measurement unit (IMU), GPS, radar sensor, and laser radar sensor may be text data. In some feasible embodiments, the sensor data may also be collected through autonomous driving simulation software.

[0039] In step S103, the computer device 1 subtracts the preset delay time from the timestamp of the data packet of the preset type of sensor to obtain a corrected timestamp. It can be understood that the data packet of image data is relatively large, and the data packet of text data is relatively small. When the computer device 1 sends data to the test device 3, if it is sent to the test device 3 according to the original data packet timestamp, since the image data packet is relatively large, there will be a delay when it is sent to the test device 3, that is, the image data and text data cannot be consistent with the actual timing when they are sent to the test device 3. Therefore, it is necessary to correct the timestamp of the data packet. Among them, the data packet of the preset type of sensor can be all types of data packets of the sensor, or it can be part of the types of data packets of the sensor.

[0040] In this embodiment, the data packet of the preset type of sensor is only the data packet of the image sensor. Since the data packet of the image sensor is relatively large, the timestamp of the data packet of the image sensor can be processed in advance by subtracting the delay time from the original timestamp of the data packet. The preset delay time can be obtained through multiple experiments. Specifically, the image data sensed by the image sensor can be simulated and sent to the test device 3 multiple times and the corresponding timestamps can be recorded, so as to deduce the delay time generated by the corresponding data packet, and use the delay time as the delay time of each image data.

[0041] Since the data packet of the text data is relatively small, the delay time can be ignored when writing into the simulation device 2, so there is no need to correct the timestamp of the data packet of the text data. However, in some feasible embodiments, the timestamp of the data packet of the text data can also be corrected in the same manner as the data packet of the image data, wherein the preset delay time of the data packet of the text data is 0. It can be understood that the data packet of the preset type of sensor can be the data packet of some types of sensors or the data packet of all types of sensors.

[0042] Step S105: the computer device 1 sends the data packets of the multiple sensors to the simulation device 2 in sequence according to the order of the modified timestamps. Figure 5 As shown, in this embodiment, the simulation device 2 is provided with a Field Programmable Gate Array (FPGA) module 206, an ARM module 202, and a storage medium 201. The storage medium 201 is a memory. It can be understood that the hardware-in-the-loop simulation test method for autonomous driving also sorts each data packet according to the corrected timestamp before sending the data packets of the multiple sensors. For example, the data packets with earlier timestamps are sorted first, and the data packets with later timestamps are sorted later.

[0043] Step S107, the simulation device 2 caches the data packets of the multiple sensors in a cache area. Specifically, the ARM module caches the data packets of the multiple sensors in a cache area R1 of the memory.

[0044] Step S109 , the simulation device reads the data packets of the multiple sensors in the buffer area according to the sequence of the correction timestamps, and writes them into the designated memory area R2 .

[0045] Please see Figure 3 , step S109 specifically includes the following steps.

[0046] S1090, the simulation device 2 uses multiple threads to read the data packets of the multiple sensors from the cache area in a one-to-one correspondence according to the order of the corrected timestamps, wherein each thread corresponds to a data packet of a sensor. For example, each sensor in the image sensor 401a, the inertial measurement unit (IMU) 401b, the radar sensor 401c, and the laser radar sensor 401d corresponds to a thread, and each thread is responsible for reading the corresponding sensor data packet from the cache area R2.

[0047] S1092, the simulation device 2 uses multiple threads to write the data packets of multiple sensors into the designated memory area R2 in a one-to-one correspondence according to the order of timestamps. Figure 5 As shown, the simulation device 2 writes multiple sensor data packets into the designated memory area R2 using the ARM module 202. It can be understood that the simulation device 2 uses multiple threads for processing to ensure that data from different sensors can be processed in parallel, so that data packets between different sensors can be processed according to the original time sequence.

[0048] Please continue to see Figure 2 , step S111, the simulation device 2 instantly reads the data packets of the sensors in the designated memory area, and converts the data packets of the multiple sensors into electrical signals and transmits them to the test device 3 for testing. Specifically, the simulation device 2 reads the data packets of the sensors in the designated memory area through the FPGA module 206. That is, whenever a data packet is written into the designated memory area R2, the FPGA module 206 promptly converts the data packet and transmits it to the test device 3. The time taken by the FPGA module 206 to transmit the electrical signal to the test device 3 is almost negligible. Therefore, the timing of the test device 3 receiving the data packets of the multiple sensors is substantially consistent with the timing of the ARM module 202 writing the data packets of the multiple sensors.

[0049] Because the computer device 1 has corrected the data packet of the image sensor before sending the data packets of multiple sensors, that is, the data packet with a relatively large data volume is sent in advance, thereby avoiding delays in the transmission of large data packets to the test device 3, so that the timing of the data of multiple sensors obtained by the test device 3 is more realistic.

[0050] In this embodiment, since the data packets generated by image sensors of different types or models have different data formats and slightly different sizes, in order to make the sent timing more realistic, before step S103, the computer 1 converts the different image formats in the sensor data into the same image format, so that the corrected timestamp can more accurately restore the real timing.

[0051] The above-mentioned hardware-in-the-loop simulation test method and system can advance the timestamp of the image sensor data packet by subtracting the delay time from the original timestamp of the image sensor data packet, thereby eliminating the delay of the image sensor data packet due to excessive data volume, and realize the synchronous transmission of the image sensor data packet and the data packets of other sensors whose original timestamp is the same as the original timestamp of the image sensor data packet to the test device.

[0052] Please see Figure 4 , which is a hardware-in-the-loop simulation test method provided by the second embodiment. The difference between the second embodiment and the first embodiment is that the hardware-in-the-loop simulation method of the second embodiment describes the computer device 1 and the simulation device 2 as a whole. Specifically, the hardware-in-the-loop simulation test method of the second embodiment includes the following steps.

[0053] Step S201, receiving sensor data, wherein the sensor data includes data packets of multiple sensors collected in the same time period, and each data packet is marked with a timestamp indicating data generation. Specifically, the sensor data may be data sensed by different sensors arranged on the vehicle or data sensed by sensors arranged on the roadside. Among them, the multiple sensors 401 include but are not limited to image sensors 401a, inertial measurement units (IMU) 401b, radar sensors 401c, laser radar sensors 401d, etc. The multiple sensors 401 may include multiple sensors of the same type, for example, multiple laser radar sensors, multiple identical image sensors, etc. The image sensor may include image sensors of different models or types. It can be understood that the data packet formats of different sensors are different, and the sizes of the data packets are also different. For example, the data format sensed by the image sensor is image data, and the formats of image data of image sensors of different models may also be different. The inertial measurement unit (IMU), GPS, radar sensor, and laser radar sensor may be text data. In some feasible embodiments, the sensor data may also be collected through autonomous driving simulation software.

[0054] Step S203, subtract the preset delay time from the timestamp of the data packet of the preset type sensor to obtain a corrected timestamp. It can be understood that the data packet of the image data is relatively large, and the data packet of the text data is relatively small. When the computer device 1 sends data to the test device 3, if it is sent to the test device 3 according to the original data packet timestamp, since the image data packet is relatively large, there will be a delay when it is sent to the test device 3, that is, the image data and text data cannot be consistent with the actual timing when they are sent to the test device 3. Therefore, it is necessary to correct the timestamp of the data packet. Among them, the data packet of the preset type sensor can be all types of data packets of the sensor, or it can be part of the types of data packets of the sensor.

[0055] In this embodiment, the data packet of the preset type of sensor is only the data packet of the image sensor. Since the data packet of the image sensor is relatively large, the timestamp of the data packet of the image sensor can be advanced by subtracting the delay time from the original timestamp of the data packet. The preset delay time can be obtained through multiple experiments. Specifically, the image data sensed by the image sensor can be simulated and sent to the test device 3 multiple times and the corresponding timestamps can be recorded, so as to deduce the delay time generated by the corresponding data packet, and use the delay time as the delay time of each image data.

[0056] Since the data packet of the text data is relatively small, the delay time can be ignored when writing into the simulation device 2, so there is no need to correct the timestamp of the data packet of the text data. However, in some feasible embodiments, the timestamp of the data packet of the text data can also be corrected in the same manner as the data packet of the image data, wherein the preset delay time of the data packet of the text data is 0. It can be understood that the data packet of the preset type of sensor can be the data packet of some types of sensors or the data packet of all types of sensors.

[0057] Step S205 , buffering the data packets of the multiple sensors in a buffer area. Specifically, the ARM module 202 buffers the data packets of the multiple sensors in a buffer area R1 of the memory.

[0058] Step S207, read the data packets of multiple sensors in the buffer area according to the order of the modified timestamps, and write them into the designated memory area R2. Specifically, first, the simulation device 2 uses multiple threads to read the data packets of the multiple sensors from the buffer area one by one according to the order of the modified timestamps, wherein each thread corresponds to a data packet of a sensor. Figure 6As shown, the radar sensor 401c, the inertial measurement unit (IMU) 401b, and the image sensor 401a correspond to threads L1, L2, and L3, respectively, and each thread is responsible for reading the corresponding sensor data packet from the cache area R2. Then, the simulation device 2 uses multiple threads to write the data packets of multiple sensors one by one into the designated memory area R2 in the order of timestamps. The simulation device 2 uses multiple threads to write the multiple sensor data packets into the designated memory area R2. It can be understood that the simulation device 2 uses multiple threads for processing, which can ensure that the data of different sensors can be processed in parallel, so that the data packets between different sensors can be processed according to the original timing.

[0059] Step S209, instantly read the data packets of the sensors in the designated memory area, and convert the data packets of the multiple sensors into electrical signals and transmit them to the test device 3 for testing. Specifically, the simulation device 2 reads the data packets of the sensors in the designated memory area through the FPGA module 206. That is, whenever a data packet is written into the designated memory R2, the FPGA module 206 timely converts the data packet and transmits it to the test device 3. The time taken by the FPGA module 206 to transmit the electrical signal to the test device 3 is almost negligible. Therefore, the timing of the test device 3 receiving the data packets of the multiple sensors is basically consistent with the timing of the ARM module 202 writing the data packets of the multiple sensors.

[0060] Please refer to Figure 7 , which is a schematic diagram of the internal structure of the computer device 1 provided in the first embodiment. The computer device 1 includes a first storage medium 101, a first processor 102 and a bus 103. Among them, the first storage medium 101 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type storage medium (for example, an SD or DX storage medium, etc.), a magnetic storage medium, a disk, an optical disk, etc. In some embodiments, the first storage medium 101 can be an internal storage unit of the computer device 1, such as a hard disk of the computer device 1. In other embodiments, the first storage medium 101 can also be an external storage device of the computer device 1, such as a plug-in hard disk equipped on the computer device 1, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Further, the first storage medium 101 can also include both an internal storage unit of the computer device 1 and an external storage device. The first storage medium 101 can be used not only to store application software and various data installed in the computer device 1, such as the first hardware-in-the-loop simulation test program instructions, but also to temporarily store data that has been output or is to be output.

[0061] The bus 103 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0062] Furthermore, the computer device 1 may also include a display component 104. The display component 104 may be an LED (Light Emitting Diode) display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display component 104 may also be appropriately referred to as a display device or a display unit, which is used to display information processed in the computer device 1 and to display a visual user interface.

[0063] Furthermore, the computer device 1 may also include a communication component 105, which may optionally include a wired communication component and / or a wireless communication component (such as a WI-FI communication component, a Bluetooth communication component, etc.), which is generally used to establish a communication connection between the computer device 1 and other computer devices.

[0064] In some embodiments, the first processor 102 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, and is used to run the program code or process data stored in the first storage medium 101. Specifically, the processor 102 executes the first hardware-in-the-loop simulation test program code to control the computer device 1 to implement the corresponding hardware-in-the-loop simulation test method.

[0065] Figure 7 Only the computer device 1 having components 101-105 and implementing the autonomous driving software development program instructions is shown. It can be understood by those skilled in the art that Figure 7 The structure shown does not constitute a limitation on the computer device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0066] Please refer to Figure 8 , which is a schematic diagram of the internal structure of the simulation device 2 provided in the first embodiment. The simulation device 2 includes a second storage medium 201, a second processor 202, an FPGA module 206, and a bus 203.

[0067] Among them, the second storage medium 101 includes at least one type of readable storage medium, which includes flash memory, hard disk, multimedia card, card-type storage medium (for example, SD or DX storage medium, etc.), magnetic storage medium, disk, optical disk, etc. In some embodiments, the storage medium 101 can be an internal storage unit of the simulation device 2, such as a hard disk of the computer device 1. In other embodiments, the storage medium 101 can also be an external computer device 20100 storage device, such as a plug-in hard disk equipped on the simulation device 2, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. Further, the second storage medium 101 can also include both an internal storage unit of the simulation device 2 and an external storage device. The second storage medium 101 can not only be used to store application software and various types of data installed in the simulation device 2, such as implementing automatic driving software development program instructions, etc., but also can be used to temporarily store data that has been output or is to be output.

[0068] In this embodiment, the second processor 202 is an ARM module 202. Specifically, the ARM module 202 executes the second hardware-in-the-loop simulation test program code to control the simulation device 2 to implement the corresponding hardware-in-the-loop simulation test method.

[0069] The bus 203 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0070] Figure 8 Only components 201-205 are shown, and those skilled in the art will appreciate that Figure 8 The structure shown does not constitute a limitation on the simulation device 2, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0071] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A hardware-in-the-loop simulation test method for autonomous driving, characterized in that: The hardware-in-the-loop simulation test method for autonomous driving includes: The computer device receives sensor data, wherein the sensor data includes data packets of multiple sensors collected in the same time period, and each data packet is marked with a timestamp indicating data generation; The computer device subtracts a preset delay time from a data packet timestamp of a preset type of sensor to obtain a corrected timestamp; The computer device sends the data packets of the plurality of sensors to the simulation device in sequence according to the sequence of the corrected timestamps; The simulation device caches the data packets of the plurality of sensors in a cache area; The simulation device reads the data packets of the multiple sensors in the buffer area in the order of the correction timestamps and writes them into the designated memory area; The simulation device instantly reads the data packets of the multiple sensors in the specified memory area, converts the data packets of the multiple sensors into electrical signals and transmits them to the testing device for testing; wherein the data packets of the preset type of sensor and the data packets of other sensors whose original timestamps are the same as the original timestamps of the data packets of the preset type of sensor are synchronously transmitted to the testing device, the data volume of the data packets of the preset type of sensor is greater than the data volume of the data packets of the other sensors, and the preset type of sensor includes an image sensor.

2. The hardware-in-the-loop simulation test method for autonomous driving according to claim 1, characterized in that: The hardware-in-the-loop simulation test method for autonomous driving specifically includes: The simulation device uses a plurality of threads to read the data packets of the plurality of sensors from the cache area in a one-to-one correspondence according to the sequence of the corrected timestamps, wherein each thread corresponds to one sensor data; and The simulation device uses multiple threads to write each sensor data into the designated memory area in a one-to-one correspondence according to the time stamp sequence.

3. The hardware-in-the-loop simulation test method for autonomous driving according to claim 1, characterized in that: After the computer device subtracts the preset delay time from the data packet timestamp of the preset type of sensor to obtain a corrected timestamp, the hardware-in-the-loop simulation test method for autonomous driving further includes: The computer device sorts the data packets of the plurality of sensors according to the corrected timestamps.

4. The hardware-in-the-loop simulation test method for autonomous driving according to claim 1, characterized in that: Before the computer device subtracts the preset delay time from the data packet timestamp of the preset type of sensor to obtain the corrected timestamp, the hardware-in-the-loop simulation test method further includes: The computer converts different image formats in the sensor data into the same image format.

5. The hardware-in-the-loop simulation test method for autonomous driving according to claim 1, characterized in that: The simulation device uses FPGA to read the data packets of the multiple sensors in the specified memory area, and converts the data packets of the multiple sensors into electrical signals and transmits them to the test device for testing.

6. A hardware-in-the-loop simulation test method for autonomous driving, characterized in that: The hardware-in-the-loop simulation test method for autonomous driving includes: Receive sensor data, wherein the sensor data includes data packets of multiple sensors collected in the same time period, each data packet is marked with a timestamp indicating data generation; Subtract a preset delay time from a data packet timestamp of a preset type of sensor to obtain a corrected timestamp; Buffering the data packets of the plurality of sensors in a buffer area; Reading data packets of a plurality of sensors in the buffer area in the order of the correction timestamps and writing them into a specified memory area; The data packets of the multiple sensors in the designated memory area are read immediately, and the data packets of the multiple sensors are converted into electrical signals and then transmitted to the testing device for testing; wherein the data packets of the preset type of sensor and the data packets of other sensors whose original timestamps are the same as the original timestamps of the data packets of the preset type of sensor are synchronously transmitted to the testing device, the data volume of the data packets of the preset type of sensor is greater than the data volume of the data packets of the other sensors, and the preset type of sensor includes an image sensor.

7. The hardware-in-the-loop simulation test method for autonomous driving according to claim 6, characterized in that: The hardware-in-the-loop simulation test method for autonomous driving specifically includes: Using multiple threads to read the data packets of the multiple sensors from the cache area in a one-to-one correspondence according to the sequence of the corrected timestamps, wherein each thread corresponds to one sensor data; and Using multiple threads, each sensor data is written into the specified memory area in a one-to-one correspondence according to the timestamp sequence.

8. The hardware-in-the-loop simulation test method for autonomous driving according to claim 6, characterized in that: After subtracting the preset delay time from the data packet timestamp of the preset type of sensor to obtain a corrected timestamp, the hardware-in-the-loop simulation test method for autonomous driving further includes: The data packets of the multiple sensors are sorted according to the revised timestamps.

9. The hardware-in-the-loop simulation test method for autonomous driving according to claim 6, characterized in that: Before subtracting the preset delay time from the data packet timestamp of the preset type of sensor to obtain the corrected timestamp, the hardware-in-the-loop simulation test method further includes: Convert different image formats in sensor data into the same image format.

10. A hardware-in-the-loop simulation test system for autonomous driving, characterized in that: The simulation test system comprises: A computer device, comprising a first storage medium and a first processor, wherein the first storage medium stores first hardware-in-the-loop simulation test program instructions; The simulation device comprises a second storage medium and a second processor, wherein the second storage medium stores second hardware-in-the-loop simulation test program instructions; The first processor and the second processor respectively execute the first hardware-in-the-loop simulation test program instructions and the second hardware-in-the-loop simulation test program instructions to jointly implement the hardware-in-the-loop simulation test method for autonomous driving as described in any one of claims 6 to 9.

Images