Simulation data processing method and device and storage medium

Abstract

The invention provides a simulation data processing method and device and a storage medium, and relates to the technical field of image processing, in particular to the technical fields of automatic driving, auxiliary driving, automatic parking, intelligent traffic and the like. According to the specific implementation scheme, N0 paths of video compression data used for simulation testing are split into N1 groups, wherein the N0 paths of video compression data are used for simulating environment images captured by N0 collection devices in a vehicle from different collection view angles in the same time period; the ith group in the N1 groups is sent to the ith decompression module in the N1 decompression modules, so that N0 simulation images used for simulating the environment picture of the vehicle at the same time point after decompression are obtained through the N1 decompression modules; and the N0 simulation images belonging to the same time point are sent to the N2 board cards through the N1 decompression modules, so that the N2 board cards are utilized to send the simulation images belonging to the same time point to the vehicle for simulation testing.

Description

[0001] This application is a divisional application of a Chinese case with the invention name “Simulation data processing method, device and storage medium”, application number “202410999063.6” and application date of July 24, 2024. Technical Field

[0002] The present disclosure relates to the field of image processing technology, and in particular to technical fields such as autonomous driving, assisted driving, automatic parking, and smart transportation. Background Art

[0003] In the hardware-in-the-loop (HIL) simulation system of autonomous driving, the simulation of the camera, such as the simulation of the global motion suppression logic (GMSL) camera, is an indispensable part. Through the simulation of the camera, the entire video processing link in the autonomous driving vehicle control system can be effectively tested. Therefore, how to efficiently simulate the camera is the current research focus. Summary of the invention

[0004] The present disclosure provides a method, device and storage medium for processing simulation data.

[0005] According to one aspect of the present disclosure, a simulation data processing method is provided, comprising:

[0006] Split N0 channels of video compression data used for simulation testing into N1 groups, wherein the N0 channels of video compression data are used to simulate environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives in the same period of time; N0 and N1 are both positive integers greater than or equal to 2;

[0007] Sending the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules, so as to obtain N0 simulation images after decompression for simulating the environment scene of the vehicle at the same time point by using the N1 decompression modules;

[0008] Through N1 decompression modules, N0 simulation images belonging to the same time point are sent to N2 boards, so that the simulation images belonging to the same time point are sent to the vehicle for simulation testing using N2 boards; wherein N2 is a positive integer greater than or equal to 1.

[0009] According to another aspect of the present disclosure, there is provided a simulation device, comprising:

[0010] A data preprocessing unit, used for splitting N0 channels of video compression data for simulation testing into N1 groups, wherein the N0 channels of video compression data are used to simulate environmental images captured by N0 acquisition devices in a vehicle from different acquisition perspectives in the same period of time; N0 and N1 are both positive integers greater than or equal to 2;

[0011] A simulation test unit is used to send the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules, so as to use the N1 decompression modules to obtain N0 decompressed simulation images for simulating the environmental scenes of the vehicle at the same time point; send the N0 simulation images belonging to the same time point to N2 boards through the N1 decompression modules, so as to use the N2 boards to send the simulation images belonging to the same time point to the vehicle for simulation testing; wherein N2 is a positive integer greater than or equal to 1.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0018] In this way, the disclosed scheme can utilize multiple decompression modules to decompress video compressed data in parallel, and after successful decompression, multiple simulation images belonging to the same time point can be directly sent to the board card, so that the board card can be used to send multiple simulation images belonging to the same time point to the vehicle for simulation testing. In this way, the time cost required for the decompression process is effectively reduced, and the decompression efficiency is improved, laying the foundation for realizing efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), and also laying the foundation for improving the simulation test efficiency of vehicles.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0021] Figure 1 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 1 ;

[0022] Figure 2 This is a schematic diagram of a scenario in which a simulation device processes N0 channels of video compression data according to an embodiment of the present application. Figure 1 ;

[0023] Figure 3 This is a schematic diagram of a scenario in which a simulation device processes N0 channels of video compression data according to an embodiment of the present application. Figure 2 ;

[0024] Figure 4 This is a schematic diagram of a simulation scenario corresponding to the total compressed data according to an embodiment of the present application. Figure 1 ;

[0025] Figure 5 This is a schematic diagram of a simulation scenario corresponding to the total compressed data according to an embodiment of the present application. Figure 2 ;

[0026] Figure 6 This is a schematic diagram of a scenario in which a simulation device processes total compressed data according to another embodiment of the present application. Figure 1 ;

[0027] Figure 7 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 2 ;

[0028] Figure 8 This is a schematic diagram of a scenario in which a simulation device processes total compressed data according to another embodiment of the present application. Figure 2 ;

[0029] Fig. 9 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 3 ;

[0030] Fig.10 is an illustrative schematic diagram of splitting each channel of video compression data into multiple data blocks according to an embodiment of the present application;

[0031] Fig.11 is a schematic diagram of the structure of a simulation device according to an embodiment of the present application;

[0032] Fig.12 It is a block diagram of an electronic device used to implement the simulation data processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0034] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0035] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0036] The following is a description of the related technologies of the embodiments of the present disclosure. The following related technologies are optional solutions that can be arbitrarily combined with the technical solutions of the embodiments of the present disclosure, and they all belong to the protection scope of the embodiments of the present disclosure.

[0037] In the hardware-in-the-loop (HIL) simulation system of autonomous driving, the simulation of the camera, such as the simulation of the GMSL camera, is an essential part. Through the simulation of the camera, the entire video processing link in the autonomous driving vehicle control system can be effectively tested. For example, both the simulation of the video content and the testing of the video link timing can be realized.

[0038] For example, in a simulation test scenario, for the video data of road test, it can be first decompressed using a decompression module (such as a graphics processor GPU), and the decompressed video data can be transmitted to a simulation device (such as a server) to be stored in the memory of the simulation device. The decompressed video data stored in the memory of the simulation device is then transmitted to a board (such as a video injection board) through a video link. Finally, the board converts the data format and outputs it synchronously, thereby realizing the simulation of the camera. However, as the bandwidth required for camera data for autonomous driving becomes larger and larger, simulation devices (such as servers) are increasingly unable to meet the requirements in terms of memory and bandwidth, which has become a bottleneck for camera data transmission. As a result, in the simulation environment, the performance (such as pixel resolution, acquisition frequency, etc.) and number of cameras that can be modeled are limited.

[0039] Based on this, the disclosed solution proposes a simulation data processing method to improve the efficiency of camera data transmission and achieve efficient simulation of high-pixel, high-frequency cameras.

[0040] Specifically, Figure 1 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0041] Further, the method includes at least part of the following contents. Figure 1 As shown, the simulation data processing method is applied to a simulation device (e.g., an industrial computer), comprising:

[0042] Step S101: The simulation device splits N0 channels of video compression data for simulation testing into N1 groups.

[0043] Here, the N0 channels of video compression data are used to simulate the environmental images captured by N0 (for example, a positive integer greater than or equal to 2) acquisition devices in the vehicle from different acquisition perspectives in the same period of time. For example, the vehicle is simulated on its driving road, and the environmental images captured by N0 acquisition devices from different acquisition perspectives in the same period of time (for example, within a unit time).

[0044] Furthermore, in one example, different acquisition devices in the vehicle have different acquisition perspectives, so that it is convenient to capture the environmental images around the vehicle through different acquisition devices, providing support for driving decisions.

[0045] Furthermore, in one example, each of the N0 channels of video compression data contains a plurality of frame compression data. For example, in one specific example, one channel of video compression data may at least contain compression data of a plurality of video frames collected by a single acquisition device within a unit time (e.g., within 1 second) (here, the compression data of the video frames may be referred to as frame compression data for short), for example, compression data of 10 frames of images collected by a single acquisition device within 1 second.

[0046] Here, the number of video frames collected per unit time is related to relevant parameters of the acquisition device to be simulated, such as the frame rate, and the disclosed solution does not impose any limitation on this.

[0047] Furthermore, N1 is a positive integer greater than or equal to 2; in one example, the value of N1 is related to the number of decompression modules used for decompression, for example, the value of N1 is equal to the number of decompression modules used for decompression. That is to say, in one example, N0 channels of video compression data can be split into N1 groups according to the number of decompression modules (for example, N1 decompression modules), so that different groups can be sent to different decompression modules for decompression, and N1 decompression modules can be used to parallelly decompress N0 channels of video compression data, thereby providing support for efficient simulation of high-pixel and high-frequency cameras.

[0048] Here, in an example, different channels of video compression data belong to different groups, and the number of channels included in different groups is the same or different, and the disclosed solution does not limit this. For example, the i-th group in the N1 groups contains ni channels of video compression data, and the j-th group in the N1 groups contains nj channels of video compression data. The values ​​of i and j are different, and accordingly, the value of ni is the same as or different from the value of nj.

[0049] Step S102: The simulation device sends the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules for decompression by the i-th decompression module, and then uses the N1 decompression modules to obtain N0 simulation images after decompression for simulating the environmental scenes of the vehicle at the same time point.

[0050] Here, i is a positive integer ranging from 1 to N1. In this way, N1 decompression modules can be used to decompress and obtain N0 simulation images for simulating the environment of the vehicle at the same time point.

[0051] For instance, in one example, there are 4 channels of video compression data that need to be simulated and tested. At this time, the 4 channels of video compression data can be split into 2 groups, wherein the first group of the 2 groups obtained can include the first channel of video compression data and the second channel of video compression data, and the second group includes the third channel of video compression data and the fourth channel of video compression data. Furthermore, the first group is sent to the first decompression module, and the second group is sent to the second decompression module.

[0052] Step S103: The simulation device sends N0 simulation images belonging to the same time point to N2 (a positive integer greater than or equal to 1) boards through N1 decompression modules, so as to use N2 boards to send the simulation images belonging to the same time point to the vehicle for simulation testing.

[0053] For example, after the simulation device determines that all frame compressed data belonging to the same time point are decompressed successfully, that is, after obtaining N0 simulation images belonging to the same time point, it directly sends the N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules, so that all simulation images belonging to the same time point can be sent to the vehicle for simulation testing using N2 boards.

[0054] In this way, the disclosed scheme can utilize multiple decompression modules to decompress video compressed data in parallel, and after successful decompression, multiple simulation images belonging to the same time point can be directly sent to the board card, so that the board card can be used to send multiple simulation images belonging to the same time point to the vehicle for simulation testing. In this way, the time cost required for the decompression process is effectively reduced, and the decompression efficiency is improved, laying the foundation for realizing efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), and also laying the foundation for improving the simulation test efficiency of vehicles.

[0055] Furthermore, compared to the existing decompression method, the disclosed solution can send multiple simulation images belonging to the same time point directly to the board through multiple decompression modules without forwarding through the simulation device. Therefore, compared to the solution of caching a large amount of decompressed video data in the simulation device, it effectively reduces the storage pressure of the simulation device and improves the transmission link bandwidth of the simulation image. Based on this, the disclosed solution can be applicable to larger-scale data sets and can further improve the reliability and accuracy of simulation tests.

[0056] Here, it should be noted that the vehicle described in the present disclosure may specifically be an unmanned vehicle or an assisted driving vehicle, and the present disclosure does not impose any specific restrictions on this.

[0057] Furthermore, it should be noted that, in one example, the decompression module described in the disclosed solution may be specifically a graphics processor GPU, or may be other devices capable of performing graphics processing, and the disclosed solution does not impose any specific limitation on this.

[0058] In a specific example of the disclosed solution, in order to ensure the time consistency of the simulation image received by the vehicle, time alignment may also be performed in the following manner.

[0059] For example, in one example, the following method may be used to perform the first time alignment using N1 decompression modules.

[0060] Specifically, in one example, before N0 simulation images belonging to the same time point are sent to N2 boards through N1 decompression modules, the method further includes:

[0061] The N0 simulation images belonging to the same time point are first time-aligned, so that the N0 simulation images belonging to the same time point can be synchronously sent to the N2 boards.

[0062] Accordingly, the above-mentioned sending of N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules may specifically include:

[0063] After the first time alignment, N0 simulation images belonging to the same time point are synchronously sent to N2 boards through N1 decompression modules.

[0064] It should be noted that in actual scenarios, the reception time of each simulated image received by the N2 board cards from the simulation device may be different. Therefore, in order to ensure that all simulated images belonging to the same time point can be sent synchronously and the vehicle can receive the simulated images synchronously (that is, to ensure the time consistency of the simulated images received by the vehicle), so as to successfully complete the test of the simulation link timing and / or the test of the simulated video content, in this example, the simulation device can also first perform a first time alignment on the N0 simulated images belonging to the same time point, and send them synchronously to the N2 board cards after the alignment. In this way, the time consistency of the simulated images received by the board cards is ensured, which provides strong support for reducing the delay of the simulated images received by the vehicle.

[0065] It can be understood that the above-mentioned time alignment operation (that is, the first time alignment operation) may not change the timestamp of the simulation image, but only aligns the N0 simulation images obtained after decompression at different times, so that all simulation images obtained after decompression at different times but belonging to the same time point can be sent out at the same time (that is, sent synchronously).

[0066] In this way, the disclosed solution can first time-align multiple simulated images belonging to the same time point obtained after decompression, and then send them out synchronously after alignment. In this way, the time synchronization of the decompressed image data in transmission is effectively improved, which provides strong support for reducing the delay of the simulated images received by the vehicle. At the same time, it also provides strong support for realizing accurate simulation of high-pixel and high-frequency cameras.

[0067] Further, in a specific example, the N0 simulation images belonging to the same time point may be first time aligned in the following two ways, specifically including:

[0068] Method 1: Using N1 decompression modules after clock synchronization, N0 simulation images belonging to the same time point are first time-aligned. Here, the N1 decompression modules are clock-synchronized using the central processing unit CPU of the simulation device.

[0069] That is to say, in the first method, each decompression module can be directly used to time-align N0 simulation images belonging to the same time point, so as to ensure the time synchronization of the image data. Here, it should be noted that a preset clock synchronization strategy can be adopted, and the CPU of the simulation device can be used to synchronize the clocks of the N1 decompression modules set in the simulation device, and then when the clocks of the N1 decompression modules are synchronized, each decompression module can be used to time-align N0 simulation images belonging to the same time point.

[0070] Method 2: Use the time synchronization device in the simulation device to send a time signal (such as a pulse signal) to the N1 decompression modules, so that after receiving the time signal, the N1 decompression modules will perform a first time alignment on the decompressed N0 simulation images belonging to the same time point.

[0071] It should be noted that, in one example, the N1 decompression modules in the second method are also decompression modules after clock synchronization with the central processing unit CPU of the simulation device.

[0072] Furthermore, it should be noted that, since each decompression module has an internal clock generator, even if the CPU is used to achieve clock synchronization between all decompression modules in the simulation device, the clocks between the decompression modules may no longer be synchronized due to various reasons during the decoding process or other program running process. Based on this, the second method can also be adopted, such as using the pulse signal sent by the time synchronization device to make the N1 decompression modules perform the first time alignment on the N0 simulation images belonging to the same time point after decompression. In this way, the time synchronization of the image data is further improved, and the delay in the transmission of the image data is greatly reduced.

[0073] For example, if Figure 2 As shown, the simulation device, such as an industrial computer, is integrated with three decompression modules (such as three GPUs) and a time synchronization device (such as sync2 (Synchronization II) card), at this time, the industrial computer can split N0 channels of video compression data into 3 groups according to the number of GPUs, and send the first group obtained by splitting to the first GPU for decompression, the second group to the second GPU for decompression, and the third group to the third GPU for decompression, so as to obtain N0 simulation images obtained after decompression by the three GPUs, which are used to simulate the environmental scenes around the unmanned vehicle or the assisted driving vehicle at the same time point; during the GPU decompression process, the sync2 card in the simulation device sends a pulse signal to the three GPUs, for example, the pulse signal is sent to the three GPUs at a fixed time, so that after the three GPUs receive the pulse signal, the N0 simulation images belonging to the same time point after successful decompression are first time-aligned, and after the first time alignment, the N0 simulation images belonging to the same time point are synchronously sent to the N2 boards through the three decompression modules, thus laying a foundation for further improving the time synchronization accuracy of the image data.

[0074] In this way, the disclosed scheme can utilize the time synchronization device in the simulation device to enable each decompression module to time align N0 simulation images belonging to the same time point after decompression. In this way, the time consistency between the decompression modules is further ensured. At the same time, the time synchronization of the image data is further improved, which provides strong support for ensuring the accuracy of data transmission and the normal operation of subsequent simulation processes.

[0075] It should be noted that, in a specific example, for the above two methods of the first time alignment, the alignment accuracy of the N0 simulation images performed for the first time alignment using the N1 decompression modules after clock synchronization is lower than the alignment accuracy of the N0 simulation images performed for the first time alignment using the time signal.

[0076] It should be noted that, in this example, multiple simulated images belonging to the same time point can use the decompression module of the simulation device to perform time alignment with low time precision (such as millisecond level) (also called "coarse" granularity time alignment), or, in order to improve the alignment accuracy, a time synchronization device can be used to perform time alignment with high time precision (such as microsecond level) (also called "fine" granularity time alignment), and then send them to the board card synchronously after alignment, so as to be sent to the vehicle through the board card for simulation testing. In this way, the time synchronization of the image data is effectively ensured, and the transmission delay of the image data is greatly reduced, which lays the foundation for the accurate simulation of high-pixel and high-frequency cameras. At the same time, it also lays the foundation for the autonomous driving system to respond to environmental changes more quickly and make accurate decisions.

[0077] In a specific example of the disclosed solution, in order to ensure the time consistency of the simulation image received by the vehicle, time alignment may be performed again in the following manner.

[0078] For example, in one example, the second time alignment may be performed using N2 boards in the following manner.

[0079] Specifically, in one example, before using N2 boards to send the simulation images belonging to the same time point to the vehicle for simulation testing, the method further includes:

[0080] The simulation images belonging to the same time point are aligned for a second time using N2 boards, so that after the second time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using the N2 boards.

[0081] That is to say, the disclosed solution can adopt a two-level time alignment strategy. For example, N1 decompression modules are first used to perform the first time alignment (that is, the first time alignment described above), and then N2 boards are used to perform the second time alignment (that is, the second time alignment described above). In this way, the time synchronization of the decompressed image data in transmission is effectively improved, and the synchronization accuracy is effectively improved, thereby providing strong support for reducing the delay of the simulated image received by the vehicle. At the same time, it also provides strong support for efficiently simulating acquisition devices with high pixel resolution or high acquisition frequency, increasing the number of acquisition devices that can be used in simulation scenarios, and thereby enriching the scope of application of the disclosed solution.

[0082] It should be noted that in actual scenarios, the reception time of each simulated image received by the N2 board cards from the simulation device may be different. Therefore, in order to ensure that the N2 board cards can synchronously send out all simulated images belonging to the same time point, and the vehicle can synchronously receive the simulated images (i.e., ensure the time consistency of the simulated images received by the vehicle), so as to successfully complete the test of the simulation link timing and / or the test of the simulated video content, in this example, the simulation device can also use the N2 board cards to perform a second time alignment on multiple simulated images belonging to the same time point, and after the second time alignment, use the N2 board cards to synchronously send all simulated images belonging to the same time point to the vehicle, thereby ensuring the time consistency of the simulated images received by the vehicle.

[0083] It is understandable that the above-mentioned second time alignment operation may not change the timestamp of the simulation image, but only align the simulation images received at different times, so that all simulation images received at different times (also called different time points) but belonging to the same time point can be sent out at the same time (that is, sent synchronously).

[0084] Further, in a specific example, the above-mentioned use of N2 boards to perform a second time alignment on the simulation images belonging to the same time point, so that after the second time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using the N2 boards, may specifically include:

[0085] The simulation device determines whether all simulation images belonging to the same time point are stored in the total cache unit corresponding to the N2 boards; when it is determined that all simulation images belonging to the same time point are stored, the simulation device uses the N2 boards to perform a second time alignment on all simulation images belonging to the same time point, and after the second time alignment, uses the N2 boards to synchronously send all simulation images belonging to the same time point to the vehicle for simulation testing.

[0086] Here, it should be noted that the total cache units corresponding to the N2 boards may represent the set of cache units of all the boards in the N2 boards, that is, the sum of the cache units of all the boards in the N2 boards.

[0087] Furthermore, the maximum capacity of the total cache units corresponding to the N2 boards can be determined based on the total number of video compression data paths of the simulation test; for example, in one example, the maximum capacity of the total cache units corresponding to the N2 boards = the total number of video compression data paths, further, the maximum capacity of the total cache units of the N2 boards is also equal to the number of acquisition devices required for simulation, and further, is also equal to the number of simulated images belonging to the same time point.

[0088] Further, for example, in one example, the simulation device may integrate a board card (that is, N2 takes a value of 1). At this time, after sending (such as synchronously sending) each simulation image belonging to the same time point to the board card, the simulation images belonging to the same time point may be stored in the cache unit of the board card; accordingly, it may be determined whether all simulation images belonging to the same time point are stored in the cache unit of the board card.

[0089] Alternatively, in another example, the simulation device may integrate two or more boards (that is, N2 is a natural number greater than or equal to 2). In this case, each simulation image belonging to the same time point may be sent (for example, synchronously) to multiple boards. Accordingly, each simulation image belonging to the same time point may be stored in the cache units of different boards; accordingly, it may be determined whether all simulation images belonging to the same time point are stored in the total cache unit of all boards.

[0090] For example, take the case where there are 6 acquisition devices (i.e., 6 channels of video compression data) that need to be simulated and tested, and the simulation device is connected to 2 boards. In this case, in one example, each board can be used to receive 3 channels of decompressed simulation images, and store the received simulation images in their respective cache units; accordingly, the simulation device can determine whether the total cache unit of the two boards stores 6 simulation images belonging to the same time point. Further, after determining that the total cache unit of the two boards stores 6 simulation images belonging to the same time point, the two boards can be used to perform a second time alignment on the 6 simulation images at the same time point, for example, the three simulation images in the respective cache units are used to perform a second time alignment on each board. At this time, it can be considered that all simulation images belonging to the same time point have completed the second time alignment, or the two boards can perform a second time alignment on the 6 simulation images in the total cache unit through a time synchronization signal, and after the second time alignment, the 6 simulation images are synchronously sent to the vehicle through the two boards for simulation testing.

[0091] It should be noted that, in one example, in order to store simulation images at different time points, multiple cache units may be provided in the board. At this time, simulation images at different time points may be cached in different cache units, so that only images belonging to the same time point are cached in the same cache unit. This provides effective support for efficiently and synchronously sending all simulation images belonging to the same time point.

[0092] In this way, the disclosed solution can utilize the board to synchronously send all simulated images belonging to the same time point to the vehicle, thereby further improving the time synchronization of the decompressed image data in transmission, thereby further providing strong support for effectively reducing the delay of the simulated images received by the vehicle. At the same time, it also provides strong support for efficiently simulating acquisition devices with high pixel resolution or high acquisition frequency, increasing the number of acquisition devices that can be used in simulation scenarios, and thereby enriching the scope of application of the disclosed solution.

[0093] Further, in a specific example, the above-mentioned determination of whether all simulation images belonging to the same time point are stored in the total cache units corresponding to the N2 boards may specifically include:

[0094] When it is determined that the current occupied capacity of the cache unit of each of the N2 boards reaches the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the total cache unit corresponding to the N2 boards.

[0095] Here, the maximum capacity of the cache unit of each board can be determined based on the number of simulation images belonging to the same time point that the board needs to receive; further, it is related to the total number of video compression data for the simulation test. For example, when the simulation device integrates multiple boards, the maximum capacity of the cache unit of each board can be determined specifically based on the total number of video compression data for the simulation test and the total number of boards.

[0096] For example, if there are 6 acquisition devices for the simulation test, there are 6 simulation images belonging to the same time point. At this time, the maximum capacity of the total cache unit corresponding to the N2 boards can be set to accommodate 6 simulation images. In other words, the sum of the maximum capacity of the cache unit of each board in the N2 boards (that is, the total maximum capacity of the set of cache units of each board) can accommodate the 6 simulation images. Further, if the simulation device integrates 2 boards (that is, N2 is 2), at this time, in an example, the maximum capacity of the cache unit of each board can accommodate 3 simulation images; further, if it is determined that the current occupied capacity of the cache unit of each board has reached the maximum capacity, it can be considered that all simulation images belonging to the same time point have been stored in the total cache unit. In this way, all simulation images belonging to the same time point can be synchronized and sent out after the second time alignment.

[0097] For example, in one example, if there are 6 acquisition devices for simulation test, and the simulation device integrates 3 decompression modules, each decompression module is connected to a board, that is, there are 3 boards, then the board needs to receive (or send out synchronously) 2 simulation images at the same time point; further, in one example, the maximum capacity of the cache unit of each board can be set to accommodate 2 simulation images. Further, if it is determined that the current occupied capacity of the cache unit of each board has reached the maximum capacity, it can be considered that all simulation images belonging to the same time point have been stored in the total cache unit, so that all simulation images belonging to the same time point can be sent out synchronously after the second time alignment.

[0098] It should be noted that the above is only an exemplary description. In actual applications, the maximum capacity of the cache units in different boards may be the same or different, and the disclosed solution does not limit this.

[0099] In this way, the disclosed solution provides a specific solution for quickly determining whether the board has received all the simulated images belonging to the same time point. This provides strong support for ensuring that the vehicle can synchronously receive all the simulated images belonging to the same time point. At the same time, it also provides strong support for further improving the accuracy of time synchronization, thereby laying the foundation for the subsequent smooth completion of the test of the simulation link timing and / or the test of the simulation video content.

[0100] Further, in a specific example, the time alignment accuracy that can be achieved by performing the first time alignment for the N0 simulation images is lower than the time alignment accuracy that can be achieved by performing the second time alignment for the N0 simulation images.

[0101] That is to say, in this example, multiple simulated images belonging to the same time point can be time-aligned once using the decompression module of the simulation device (for example, a relatively low-precision or relatively high-precision time alignment can be performed, and here, for lower precision, it can also be called "coarse"-grained time alignment), and then the board will perform a high-time-precision (for example, microsecond-level) time alignment (also called "fine"-grained time alignment) on the multiple simulated images after the "coarse"-grained time alignment. After the "fine"-grained time alignment, all simulated images belonging to the same time point are synchronously sent to the vehicle for simulation testing. In this way, the time synchronization of the image data is further ensured, and the transmission delay of the image data is greatly reduced, laying the foundation for the accurate simulation of high-pixel and high-frequency cameras. At the same time, it also lays the foundation for the autonomous driving system to respond to environmental changes more quickly and make accurate decisions.

[0102] Further, in a specific example, the number N2 of the boards is related to at least one of the following:

[0103] It is related to the number N0 of acquisition devices in the simulation test;

[0104] It is related to the number of video output interfaces of the decompression module. Here, the number of video output interfaces of the decompression module is related to the number of channels N0 of video compression data;

[0105] Related to the number N1 of decompression modules.

[0106] For example, in one example, the number of the boards is the same as the number of decompression modules (eg, GPUs). Figure 2 Take the scenario shown as an example, Figure 3 As shown, each of the three GPUs is connected to a board (such as a video injection board), that is, the first GPU is connected to the first video injection board to send the simulation image decompressed by the first GPU to the vehicle through the first video injection board for simulation testing; similarly, the second GPU is connected to the second video injection board to send the simulation image decompressed by the second GPU to the vehicle through the second video injection board for simulation testing; the third GPU is connected to the third video injection board to send the simulation image decompressed by the third GPU to the vehicle through the third video injection board for simulation testing.

[0107] Furthermore, it should be pointed out that, in an example, in a scenario where the decompression module and the board are connected one-to-one, the number of output interfaces (also referred to as video output interfaces) in the decompression module is related to the number of video receiving interfaces of the board. For example, the two are the same (for example, both are 4), that is, there is a video output interface in the decompression module, and there is a corresponding video channel in the board. In other words, the number of video interfaces of the two is the same.

[0108] In this way, the disclosed solution can use the board to time align N0 simulated images that are decompressed and belong to the same time point, thereby effectively reducing the delay of video data. Moreover, in the scenario where multiple boards are used, more simulated camera scenes can be supported. At the same time, it can also support efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), thereby simulating more complex driving environments and scenarios. In addition, the use of multiple boards improves the parallel capability of data injection, thereby improving the efficiency of data transmission.

[0109] Here, in one example, the board mentioned above can be specifically a video injection board; further, in one example, the board is one of the following: a field programmable gate array FPGA or a chip capable of implementing video injection requirements. In other words, the disclosed solution is compatible with existing boards, thus improving the scope of application of the disclosed solution, making the disclosed solution both practical and applicable.

[0110] Here, it should be noted that, in one example, the bandwidth of the data receiving link of the board is greater than the bandwidth of the data sending link of the board. This further lays the foundation for reducing the transmission delay of image data and thus achieving accurate simulation of high-pixel and high-frequency cameras.

[0111] Furthermore, in a specific example, before performing a second time alignment on the simulation images belonging to the same time point using the N2 boards, the method further includes:

[0112] The data formats of N0 simulation images belonging to the same time point are converted into a preset format using N2 boards.

[0113] That is to say, in one example, after each board receives multiple simulated images belonging to the same time point, each board can first convert the data format of the multiple simulated images into a preset format, and then perform a second time alignment on the multiple simulated images belonging to the same time point in the preset format, and then send them out synchronously after the second time alignment. In this way, the required data format can be effectively simulated, and the data compatibility is stronger, thereby ensuring that the receiving end, such as the vehicle end, can process the received image data accordingly, thereby laying the foundation for realizing efficient simulation of high-pixel and high-frequency cameras.

[0114] Furthermore, in one example, the preset format is the data format output by the acquisition device of the vehicle under simulated test. For example, the preset format is the data format output by the acquisition device of the GMSL format of the vehicle under simulated test, that is, the GMSL format. In this way, accurate and efficient simulation of the GMSL camera is achieved, laying the foundation for subsequent improvement of the performance stability of the entire autonomous driving system.

[0115] In a specific example, N0 channels of video compression data may be obtained in the following manner; specifically, before the simulation device splits the N0 channels of video compression data for simulation testing into N1 groups (for example, before step S101 described above), the step of obtaining the N0 channels of video compression data is also included, specifically including:

[0116] Step S100 - 1 : The simulation device obtains total compressed data for simulation testing.

[0117] Here, the total compressed data is used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives within a preset time period.

[0118] Furthermore, in one example, the preset time period includes a plurality of unit times.

[0119] Step S100-2: based on at least N0 acquisition devices required for simulation test, split the total compressed data to obtain N0 channels of video compression data.

[0120] For example, in one example, the total compressed data is split based on at least N0 acquisition devices required for simulation test to obtain N0 channels of video compressed data (step S100-2), which specifically includes:

[0121] Step S100-2-1: based on the multiple unit times included in the preset time period, split the total compressed data into multiple groups of data to be decompressed.

[0122] Here, the number of groups of data to be decompressed is the same as the number of unit times contained in the preset time period. Further, each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N0 acquisition devices within a unit time, that is, includes all frame compressed data that can be collected by the N0 acquisition devices within a unit time.

[0123] That is to say, in this step, the simulation device can split the total compressed data containing multiple unit times according to unit time to obtain multiple groups of data to be decompressed, so that each group of data to be decompressed contains all the frame compressed data that can be collected by the N0 acquisition devices within a unit time.

[0124] Step S100-2-2: Select a group of data to be decompressed that meets the time requirement from the multiple groups of data to be decompressed.

[0125] Step S100-2-3: Based on the N0 acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N0 channels of video compression data.

[0126] For instance, in one example, multiple groups of data to be decompressed are sorted in chronological order, the earliest group of data to be decompressed is selected from the sorting results, and the earliest group of data to be decompressed is further split according to multiple acquisition devices to be simulated to obtain the N0 channels of video compression data.

[0127] For example, in one example, Figure 4As shown, the vehicle to be simulated and tested is equipped with acquisition devices at 6 different positions, a total of 6, which can be respectively recorded as acquisition device 1, acquisition device 2, acquisition device 3, acquisition device 4, acquisition device 5 and acquisition device 6; at this time, the total compressed data can include 6 channels of video compression data, and each channel of video compression data is used to simulate the environmental picture captured by a collection device in the vehicle. For example, the first channel of video compression data is used to simulate the environmental picture captured by acquisition device 1, the second channel of video compression data is used to simulate the environmental picture captured by acquisition device 2,..., the sixth channel of video compression data is used to simulate the environmental picture captured by acquisition device 6.

[0128] Furthermore, in one example, the total compressed data is used to simulate the environmental images captured by the N0 acquisition devices from different acquisition perspectives within a unit time (e.g., within 1 second). Accordingly, each channel of video compression data obtained by directly splitting the total compressed data can directly simulate the environmental images captured by a capture device within a unit time (e.g., within 1 second) and at its own acquisition perspective.

[0129] Or, in another example, Figure 5As shown, the total compressed data can be specifically used to simulate the environmental pictures captured by the N0 acquisition devices from different acquisition perspectives within a preset time period (for example, the preset time period includes L unit times, in this case, they can be respectively recorded as the first time unit, the second time unit, ..., the Lth time unit in the preset time period, and L is a positive integer greater than or equal to 2). In this scenario, the total compressed data can be first split into L groups of data to be decompressed according to unit time, which can be respectively recorded as the first group of data to be decompressed corresponding to the first time unit in the preset time period (for example, the first group of data to be decompressed corresponding to the first second in the preset time period), the second group of data to be decompressed corresponding to the second time unit in the preset time period (the second group of data to be decompressed corresponding to the second second in the preset time period), ..., the Lth group of data to be decompressed corresponding to the Lth time unit in the preset time period (the data to be decompressed corresponding to the last second in the preset time period), where each group of data to be decompressed is used to simulate the environmental pictures captured by the N0 acquisition devices from different acquisition perspectives within a unit time; further, based on the N0 acquisition A collection device further splits a group of data to be decompressed into N0 channels of video compressed data. For example, the first group of data to be decompressed corresponding to the first second in the preset time period can be split into N0 channels of video compressed data. At this time, they can be respectively recorded as the first channel of video compressed data corresponding to the first second in the preset time period, the second channel of video compressed data corresponding to the first second in the preset time period, …, the N0th channel of video compressed data corresponding to the first second in the preset time period. At this time, each channel of video compressed data is used to simulate the environmental picture captured by an acquisition device from its own acquisition perspective within a unit time. In this way, it provides strong support for the subsequent rapid use of multiple decompression modules (for example, GPU) to decompress and obtain simulated images in parallel.

[0130] It should be noted that in actual applications, the total compressed data obtained can be split on the device dimension based on N0 acquisition devices to obtain multiple groups of data to be decompressed. At this time, each group of data to be decompressed is used to simulate the environmental picture captured by an acquisition device from its own acquisition perspective within a preset time period (that is, the multiple unit times mentioned above), and then the multiple groups of data to be decompressed are processed from the time dimension according to the unit time to obtain the N0 channels of video compression data mentioned above.

[0131] It should be noted that in practical applications, Figure 6As shown, a simulation device, such as an industrial computer, is integrated with N1 GPUs for decompression. Furthermore, the industrial computer obtains total compressed data for simulation testing from a data center, splits the total compressed data to obtain N0 channels of video compression data, and splits the N0 channels of video compression data into N1 groups according to the number of GPUs, so as to allocate the N1 groups to N1 GPUs for decompression, and then directly sends multiple simulation images belonging to the same time point obtained after decompression by the N1 GPUs to the board card, so as to be sent to the vehicle through the board card for simulation testing.

[0132] In this way, the disclosed solution provides a specific solution for obtaining N0 channels of video compression data. The solution is simple and efficient, and can quickly split the total compressed data into N0 channels of video compression data in a time sequence. This provides strong support for the subsequent rapid use of the decompression module to decompress and obtain simulated images in parallel.

[0133] Figure 7 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figures 1 to 6 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0134] Further, the method includes at least part of the following contents. Figure 7 As shown, including:

[0135] Step S701: The simulation device splits N0 channels of video compression data used for simulation testing into N1 groups.

[0136] Here, the N0 channels of compressed video data are used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives during the same period of time.

[0137] Here, the specific scheme for obtaining N0 channels of video compression data and the specific scheme for obtaining N1 groups can be found in the above description and will not be repeated here.

[0138] Step S702: the simulation device sends the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules.

[0139] Step S703: The simulation device uses the i-th decompression module among the N1 decompression modules to determine ni frames of compressed data belonging to the same time point from the ni (ni is a positive integer greater than or equal to 1 and less than N0) channels of video compressed data contained in the i-th group among the N1 groups.

[0140] Here, each of the ni channels of video compression data contains a plurality of frame compression data.

[0141] Step S704: The simulation device uses the i-th decompression module to decompress ni frame compressed data belonging to the same time point, and decompresses to obtain ni simulation images (the ni simulation images are used to simulate the environmental scenes captured by ni acquisition devices of the vehicle at the same time point), so as to obtain N0 simulation images decompressed by N1 decompression modules and used to simulate the environmental scenes of the vehicle at the same time point.

[0142] For example, in one example, the i-th decompression module may be used to serially decompress ni frames of compressed data belonging to the same time point.

[0143] That is to say, in one example, for the i-th group among the N1 groups obtained by splitting, the i-th decompression module in the simulation device is used to first determine the ni frames of compressed data belonging to the same time point in the ni channels of video compression data contained in the i-th group, and then decompress the ni frames of compressed data belonging to the same time point (for example, serial decompression), and obtain the decompressed ni simulation images belonging to the same time point to simulate the environmental scenes captured by the ni acquisition devices of the vehicle at the same time point. Similarly, the simulation images obtained after decompression by the other decompression modules can be obtained. In this way, the N0 simulation images belonging to the same time point after decompression can be obtained by using the N1 decompression modules. In this way, the high decompression performance of the decompression module is effectively utilized, the time cost required for the decompression process is reduced, and the efficiency of decompression is improved.

[0144] Here, in one example, after the decompression module decompresses a simulation image, it can first store the simulation image in its own cache area, and when the first time alignment is required, the simulation images cached in the cache area belonging to the same time point can be aligned for the first time without being sent directly to the board card via the simulation device. In this way, the storage pressure of the simulation device is effectively reduced.

[0145] Step S705: The simulation device sends N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules, so as to use the N2 boards to send the simulation images belonging to the same time point to the vehicle for simulation testing.

[0146] For instance, in one example, when the simulation device determines that N0 simulation images belonging to the same time point have completed the first time alignment, the simulation device synchronously sends the N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules, so that the simulation images belonging to the same time point can be aligned for the second time using the N2 boards, and then they can be synchronously sent to the vehicle for simulation testing after the second time alignment.

[0147] In this way, the disclosed scheme provides a specific scheme for decompressing multi-channel video compressed data using multiple decompression modules. In this scheme, N0 simulated images belonging to the same time point can be efficiently decompressed and sent directly to the board by N1 decompression modules, so that the board can use the N0 simulated images to be synchronously sent to the vehicle. In this way, the time cost required for the decompression process is effectively reduced, and the decompression efficiency is improved, which lays the foundation for realizing efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), and also lays the foundation for improving the simulation test efficiency of vehicles.

[0148] For example, Figure 8 A schematic diagram of a data link used for simulating a test of an acquisition device of an autonomous driving HIL system is provided, wherein a simulation device (such as an industrial computer) is integrated with multiple GPUs and multiple video injection boards, and the video injection board (such as an FPGA) is plugged into a peripheral component interconnect express (PCIE) card slot of the industrial computer.

[0149] Specifically, Figure 8 As shown, the simulation test process may specifically include:

[0150] First, a simulation device, such as an industrial computer, obtains the total compressed data for simulation testing from a data center. Here, the total compressed data may specifically be video compression data stored in the form of Record in a road test disk in the data center. The simulation device splits the obtained total compressed data into multiple channels of video compression data according to the number of acquisition devices (such as GMSL cameras) to be simulated, that is, the N0 channels of video compression data mentioned above are obtained, for example, the video streams of Advanced Video Coding (H.264) / High Efficiency Video Coding (H.265) are split; according to the number of GPUs, the N0 channels of video compression data are split into N1 groups, and different groups are sent to different GPUs, for example, the compressed video stream is transmitted to the GPU via a PCIE link.

[0151] Secondly, for each GPU, for example, for the i-th (i ranges from 1 to N1) GPU among N1 GPUs, multiple frame compressed data belonging to the same time point can be determined based on the received i-th group, and then the multiple frame compressed data belonging to the same time point can be decompressed serially; here, in one example, the decompressed simulation image can be stored in the cache area of ​​the GPU in YUV (Y represents the brightness (Luma) component, U and V represent the chrominance (Chroma) components) format.

[0152] Again, after receiving the time signal (also called synchronization signal) sent by the sync2 card, each GPU (for example, the 1st GPU to the N1th GPU) can time-align all simulation images belonging to the same time point, and then send them to the video injection board connected to itself via the video interface (such as the display interface (DisplayPort, DP) or the high-definition multimedia interface (High-Definition Multimedia Interface, HDMI)) after time alignment (that is, at the pace of the synchronization signal). For example, for the i-th GPU, it is sent to the i-th video injection board via the video interface after time alignment.

[0153] Finally, each video injection board, for example, for the i-th video injection board, can convert the data format of the received simulation image (for example, YUV format) into a matching format according to the actual interface protocol of the camera to be simulated (for example, GMSL interface protocol, or the interface protocol of a liquid crystal display or a projector). For example, for simulating a real GMSL camera, the data format of the received simulation image (for example, YUV format) can be converted into GMSL format, and then the simulation image in GMSL format is cached in its own cache unit; further, when each of the N1 video injection boards determines that the current occupied capacity of its own cache unit is the maximum capacity, it is considered that each video injection board has obtained all the simulation images belonging to the same time point, and the data format of all the simulation images belonging to the same time point is in GMSL format. Then, each video injection board will time-align all the simulation images at the same time point in GMSL format in its own cache unit and send them to the vehicle (for example, an unmanned vehicle or an assisted driving vehicle) to facilitate the simulation test of the vehicle.

[0154] In this way, multiple GPUs are effectively utilized for parallel decompression. Compared with the existing solutions, the time cost required for the decompression process is effectively reduced, the decompression efficiency is improved, and efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras) is achieved. For example, accurate data format and timing simulation is achieved. At the same time, precise control of GMSL camera content is also achieved, laying the foundation for improving the efficiency of vehicle simulation testing.

[0155] Fig. 9 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 3 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figures 1 to 8 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0156] Further, the method includes at least part of the following contents. Fig. 9 As shown, including:

[0157] Step S901: The simulation device splits N0 channels of video compression data used for simulation testing into N1 groups.

[0158] Here, the N0 channels of compressed video data are used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives during the same period of time.

[0159] Step S902: the simulation device sends the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules.

[0160] Here, N1 decompression modules can be used to decompress and obtain N0 simulation images for simulating the environment around the vehicle at the same time point.

[0161] Step S903: The simulation device uses the i-th decompression module to split the ni-channel video compression data included in the i-th group into Mi data blocks.

[0162] Here, each data block contains ni frames of compressed data belonging to the same time point, and the frame compressed data contained in different data blocks correspond to different time points.

[0163] Further, in one example, the value of Mi is related to the number of frame compression data contained in the video compression data. For example, the value of Mi is less than or equal to the number of video frames collected per unit time. Further, in one example, the value of Mi is equal to the number of video frames collected per unit time.

[0164] For example, each channel of video compression data included in the i-th group includes compressed data of multiple frames of images (also referred to as multiple video frames) collected by a collection device in a unit time. Here, the unit time may include multiple time points. For example, for a scene where 10 frames of images can be collected in 1 second, the unit time includes 10 different time points. Specifically, Fig.10 As shown, the i-th group includes 3 channels of video compression data, wherein each channel of video compression data includes 10 frames of compressed images collected within 1 second. At this time, the frame compressed images belonging to the same time point can be divided into a data block according to the time points of different frame compressed images in each channel of video compression data. For example, the 11th frame compressed image belonging to time point 1 in the 1st channel of video compression data, the 21st frame compressed image belonging to time point 1 in the 2nd channel of video compression data, and the 31st frame compressed image belonging to time point 1 in the 3rd channel of video compression data are divided into the 1st data block; the 12th frame compressed image belonging to time point 2 in the 1st channel of video compression data, the 22nd frame compressed image belonging to time point 2 in the 2nd channel of video compression data, and the 32nd frame compressed image belonging to time point 2 in the 3rd channel of video compression data are divided into the 2nd data block; similarly, the 3rd data block, ..., the 9th data block and the 10th data block can be obtained. In this way, the i-th decompression module can decompress the compressed data at the same time point, thereby further ensuring time synchronization and providing strong support for the simulation of video link timing. Moreover, it can also greatly reduce the transmission delay of image data, thereby laying the foundation for the accurate simulation of high-pixel and high-frequency cameras. At the same time, it also lays the foundation for the autonomous driving system to respond to environmental changes faster and make accurate decisions.

[0165] Step S904: The simulation device uses the i-th decompression module to select the j-th data block from the Mi data blocks based on a timing rule.

[0166] Here, the jth data block is at least one of the currently undecompressed data blocks in the Mi data blocks. Further, the jth data block may be the earliest data block in time among all the currently undecompressed data blocks in the Mi data blocks.

[0167] Step S905: The simulation device uses the i-th decompression module to serially decompress the ni frame compressed data in the j-th data block, and uses the i-th decompression module to serially decompress all data blocks in the Mi data blocks, and then uses N1 decompression modules to obtain N0 simulation images belonging to the same time point.

[0168] At this point, N0 decompressed simulation images for simulating the environment around the vehicle at the same time point can be obtained by using N1 decompression modules.

[0169] Step S906: The simulation device sends N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules, so as to use the N2 boards to synchronously send the simulation images belonging to the same time point to the vehicle for simulation testing.

[0170] For instance, in one example, when the simulation device determines that N0 simulation images belonging to the same time point have completed the first time alignment, the simulation device sends the N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules, so that the simulation images belonging to the same time point can be aligned for the second time using the N2 boards, and then the images can be synchronously sent to the vehicle for simulation testing after the second time alignment.

[0171] In this way, the disclosed scheme can utilize the i-th decompression module to split the compressed video data of each channel contained in the i-th group according to the time point to obtain Mi data blocks belonging to different time points, thereby facilitating the i-th decompression module to quickly decompress the compressed data belonging to the same time point, thereby quickly obtaining all simulation images belonging to the same time point to simulate the environmental scene around the vehicle. In this way, multiple decompression modules can be used to quickly and orderly decompress the compressed data at different time points, thereby laying the foundation for realizing efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), and also laying the foundation for improving the simulation test efficiency of vehicles.

[0172] Further, in a specific example, the j-th data block may be decompressed serially in the following manner; specifically, the above-mentioned simulation device uses the i-th decompression module to serially decompress ni frames of compressed data in the j-th data block (for example, the above-mentioned step S905), which specifically includes:

[0173] Step S905 - 1 : The simulation device uses the i-th decompression module to decompress the p-th frame compressed data in the j-th data block.

[0174] Here, the p-th frame compressed data is one of the uncompressed frame compressed data in the selected j-th data block.

[0175] Step S905-2: after decompressing to obtain the simulated image of the p-th frame compressed data, store the simulated image of the p-th frame compressed data in the cache area of ​​the ith decompression module, and decompress the p+1-th frame compressed data.

[0176] Here, the p+1th frame compressed data is one of the uncompressed frame compressed data in the Mith data block.

[0177] That is, the i-th decompression module decompresses the currently undecompressed frame compressed data in the j-th data block, and after the decompression is completed, the i-th decompression module decompresses the next undecompressed frame compressed data, and thus, serial decompression is performed to quickly obtain a simulated image. Fig.10 For example, for the first data block, the i-th decompression module can first decompress the 11th frame compressed image, and after decompression, obtain the simulated image of the 11th frame compressed image and store it in the cache area of ​​the i-th decompression module. Then, the i-th decompression module continues to decompress the 21st frame compressed image in the first data block, and so on, until the last frame (i.e., the 31st frame) compressed image in the first data block is decompressed.

[0178] In this way, the decompression module can be used in the disclosed solution to serially decompress compressed data of different frames in the same data block, which greatly saves the resource usage of the decompression module during decompression and improves the decompression processing efficiency of the decompression module, thereby ensuring the quality of the simulated image obtained after decompression, laying the foundation for the subsequent efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras).

[0179] In a specific example, after the i-th decompression module completes the decompression of the j-th data block, the next data block may be decompressed continuously, so that the decompression process of all data blocks is completed serially; for example, after the simulation device uses the i-th decompression module to serially decompress ni frames of compressed data in the j-th data block, the following may be specifically included:

[0180] After determining that the decompression of the frame compressed data of the j-th data block is completed, the i-th decompression module is used to serially decompress the ni frame compressed data in the j+1-th data block, and this cycle is repeated to serially decompress each data block until the frame compressed data of all data blocks are successfully decompressed.

[0181] Here, the j+1th data block is at least one of the currently undecompressed data blocks in the Mi data blocks. Further, the j+1th data block may be the earliest data block in time among all the currently undecompressed data blocks in the Mi data blocks.

[0182] Continue with Fig.10For example, when three simulation images corresponding to the first data block are obtained after decompression, the second data block can be decompressed, for example, the i-th GPU decompresses the 12th frame compressed image in the second data block; when three simulation images corresponding to the second data block are obtained after decompression, the third data block will be decompressed, for example, the i-th GPU decompresses the 13th frame compressed image in the third data block, and so on. The cycle is repeated to complete the decompression of all data blocks.

[0183] In this way, the disclosed solution utilizes the i-th decompression module to decompress different data blocks in a serial manner in sequence, thereby reducing the pressure of data transmission and effectively improving the processing efficiency of the decompression module, thereby giving full play to the high decompression performance of the decompression module, thereby ensuring the quality of the simulated image obtained after decompression, and laying the foundation for the subsequent efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras).

[0184] The disclosed solution also provides a simulation device, such as Fig.11 As shown, including:

[0185] The data preprocessing unit 1101 is used to split N0 channels of video compression data for simulation testing into N1 groups, wherein the N0 channels of video compression data are used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives in the same period of time; N0 and N1 are both positive integers greater than or equal to 2;

[0186] The simulation test unit 1102 is used to send the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules, so as to use the N1 decompression modules to obtain N0 decompressed simulation images for simulating the environmental scenes of the vehicle at the same time point; send the N0 simulation images belonging to the same time point to N2 boards through the N1 decompression modules, so as to use the N2 boards to send the simulation images belonging to the same time point to the vehicle for simulation testing; wherein N2 is a positive integer greater than or equal to 1.

[0187] In a specific example of the disclosed solution, the simulation test unit is further used to:

[0188] Performing a first time alignment on N0 simulation images belonging to the same time point;

[0189] After the first time alignment, N0 simulation images belonging to the same time point are synchronously sent to N2 boards through N1 decompression modules.

[0190] In a specific example of the disclosed solution, the simulation test unit is specifically used to:

[0191] Using N1 decompression modules after clock synchronization, N0 simulation images belonging to the same time point are first time-aligned, wherein the N1 decompression modules are clock-synchronized using a central processing unit CPU of the simulation device;

[0192] or,

[0193] A time synchronization device is used to send a time signal to the N1 decompression modules, so that the N1 decompression modules perform a first time alignment on the decompressed N0 simulation images belonging to the same time point after receiving the time signal.

[0194] In a specific example of the disclosed scheme, the alignment accuracy of the first time alignment of the N0 simulation images using N1 decompression modules after clock synchronization is lower than the alignment accuracy of the first time alignment of the N0 simulation images using the time signal.

[0195] In a specific example of the disclosed solution, the simulation test unit is further used to:

[0196] The simulation images belonging to the same time point are aligned for a second time using N2 boards, so that after the second time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using the N2 boards.

[0197] In a specific example of the disclosed solution, the simulation test unit is specifically used to:

[0198] Determine whether all simulation images belonging to the same time point are stored in the total cache units corresponding to the N2 boards; wherein the total cache units corresponding to the N2 boards represent the set of cache units of all boards in the N2 boards;

[0199] When it is determined that all simulation images belonging to the same time point are stored, N2 boards are used to perform a second time alignment on all simulation images belonging to the same time point, and after the second time alignment, all simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing.

[0200] In a specific example of the disclosed solution, the simulation test unit is specifically used to:

[0201] When it is determined that the current occupied capacity of the cache unit of each of the N2 boards has reached the maximum capacity, it is determined that the total cache unit corresponding to the N2 boards stores all the simulation images belonging to the same time point, wherein the maximum capacity of the cache unit of the board is determined based on the number of simulation images belonging to the same time point that the board needs to receive, and is related to the total number of channels of video compression data for the simulation test.

[0202] In a specific example of the disclosed solution, the time alignment accuracy that can be achieved by performing the first time alignment for the N0 simulation images is lower than the time alignment accuracy that can be achieved by performing the second time alignment for the N0 simulation images.

[0203] In a specific example of the disclosed solution, the simulation test unit is further used to:

[0204] Before using N2 boards to perform a second time alignment on the simulation images at the same time point, using N2 boards to convert the data formats of N0 simulation images at the same time point into a preset format.

[0205] In a specific example of the disclosed solution, the preset format is the data format output by the acquisition device of the vehicle under simulation test.

[0206] In a specific example of the disclosed solution, the data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

[0207] In a specific example of the disclosed solution, the board is one of the following: a field programmable gate array FPGA or a chip capable of realizing video injection requirements.

[0208] In a specific example of the disclosed solution, the number N2 of the boards is related to at least one of the following:

[0209] It is related to the number N0 of acquisition devices in the simulation test;

[0210] It is related to the number of video output interfaces of the decompression module;

[0211] Related to the number N1 of decompression modules.

[0212] In a specific example of the disclosed solution, the simulation test unit is further used to:

[0213] Using the i-th decompression module, determine ni frames of compressed data belonging to the same time point from the ni channels of video compressed data included in the i-th group, wherein each channel of video compressed data in the ni channels of video compressed data contains a plurality of frames of compressed data;

[0214] The i-th decompression module is used to decompress ni frames of compressed data belonging to the same time point to obtain ni simulated images.

[0215] In a specific example of the disclosed solution, the simulation test unit is further used to:

[0216] Using the i-th decompression module, the ni channels of video compression data contained in the i-th group are split into Mi data blocks, wherein each data block contains ni frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of Mi is related to the number of frame compression data contained in the video compression data.

[0217] In a specific example of the disclosed solution, the simulation test unit is specifically used to:

[0218] Using the i-th decompression module, based on the timing rule, select the j-th data block from the Mi data blocks; wherein the j-th data block is at least one of the currently undecompressed data blocks in the Mi data blocks;

[0219] The i-th decompression module is used to serially decompress ni frames of compressed data in the j-th data block.

[0220] In a specific example of the disclosed solution, the simulation test unit is specifically used to:

[0221] Decompressing the p-th frame compressed data in the j-th data block by using the i-th decompression module; wherein the p-th frame compressed data is one of the undecompressed frame compressed data in the selected j-th data block;

[0222] After decompressing to obtain the simulated image of the p-th frame compressed data, the simulated image of the p-th frame compressed data is stored in the cache area of ​​the ith decompression module, and the p+1-th frame compressed data is decompressed; the p+1-th frame compressed data is one of the undecompressed frame compressed data in the Mi-th data block.

[0223] In a specific example of the disclosed solution, the simulation test unit is further used to:

[0224] After determining that the decompression of the frame compressed data of the j-th data block is completed, the i-th decompression module is used to serially decompress the ni frame compressed data in the j+1-th data block, and each data block is decompressed serially; wherein the j+1-th data block is at least one of the currently uncompressed data blocks in the Mi data blocks.

[0225] In a specific example of the disclosed solution, the data preprocessing unit is further used to:

[0226] Obtaining total compressed data for simulation testing; wherein the total compressed data is used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives within a preset period of time;

[0227] The total compressed data is split based on at least N0 acquisition devices required for simulation test to obtain the N0 channels of video compression data.

[0228] In a specific example of the disclosed solution, the data preprocessing unit is specifically used to:

[0229] Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N0 collection devices within the unit time;

[0230] Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data;

[0231] Based on the N0 acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N0 channels of video compression data.

[0232] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0233] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0234] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0235] Fig.12 A schematic block diagram of an example electronic device 1200 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0236] like Fig.12As shown, the device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0237] A number of components in the device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0238] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1201 performs the various methods and processes described above, such as the simulation data processing method. For example, in some embodiments, the simulation data processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the simulation data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to execute the simulation data processing method in any other appropriate manner (eg, by means of firmware).

[0239] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0240] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0241] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0242] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0243] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0244] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0245] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0246] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A simulation data processing method, applied to a simulation device, comprising: Using N1 decompression modules to obtain N0 decompressed simulation images for simulating the environment of the vehicle at the same time point; wherein the N0 simulation images are used to simulate the environment captured from different acquisition perspectives at the same time point; N0 and N1 are both positive integers greater than or equal to 2; Performing a first time alignment on N0 simulation images belonging to the same time point; After the first time alignment, N0 simulation images belonging to the same time point are synchronously sent to N2 boards through N1 decompression modules, so that the simulation images belonging to the same time point are sent to the vehicle for simulation testing using N2 boards; N2 is a positive integer greater than or equal to 1.

2. The method according to claim 1, wherein: The first time alignment of N0 simulation images belonging to the same time point includes: Using N1 decompression modules after clock synchronization, N0 simulation images belonging to the same time point are first time-aligned, wherein the N1 decompression modules are clock-synchronized using a central processing unit CPU of the simulation device; or, A time synchronization device in the simulation device is used to send a time signal to the N1 decompression modules, so that the N1 decompression modules perform a first time alignment on the decompressed N0 simulation images belonging to the same time point after receiving the time signal.

3. The method according to claim 2, wherein: The alignment accuracy of performing a first time alignment on the N0 simulation images using the N1 decompression modules after clock synchronization is lower than the alignment accuracy of performing a first time alignment on the N0 simulation images using the time signal.

4. The method according to any one of claims 1 to 3, wherein: Before using N2 boards to send the simulation images belonging to the same time point to the vehicle for simulation testing, the method further includes: The simulation images belonging to the same time point are aligned for a second time using N2 boards, so that after the second time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using the N2 boards.

5. The method according to claim 4, wherein: The method of performing a second time alignment on the simulation images at the same time point by using N2 boards, and after the second time alignment, using the N2 boards to synchronously send the simulation images at the same time point to the vehicle for simulation testing includes: Determine whether the total cache units corresponding to the N2 boards store all the simulation images belonging to the same time point; wherein the total cache units corresponding to the N2 boards represent the set of cache units of all the boards in the N2 boards; When it is determined that all simulation images belonging to the same time point are stored, N2 boards are used to perform a second time alignment on all simulation images belonging to the same time point, and after the second time alignment, all simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing.

6. The method according to claim 5, wherein: The determining whether all simulation images belonging to the same time point are stored in the total cache units corresponding to the N2 boards includes: When it is determined that the current occupied capacity of the cache unit of each of the N2 boards has reached the maximum capacity, it is determined that the total cache unit corresponding to the N2 boards stores all the simulation images belonging to the same time point, wherein the maximum capacity of the cache unit of the board is determined based on the number of simulation images belonging to the same time point that the board needs to receive, and is related to the total number of channels of video compression data for the simulation test.

7. The method according to claim 4, wherein: The time alignment accuracy that can be achieved by performing the first time alignment for the N0 simulation images is lower than the time alignment accuracy that can be achieved by performing the second time alignment for the N0 simulation images.

8. The method according to claim 4, wherein: Before using N2 boards to perform a second time alignment on the simulation images belonging to the same time point, the method further includes: The data formats of N0 simulation images belonging to the same time point are converted into a preset format using N2 boards.

9. The method according to claim 8, wherein: The preset format is the data format output by the acquisition device of the vehicle under simulation test.

10. The method according to claim 9, wherein: The data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

11. The method according to claim 4, wherein: The board is one of the following: a field programmable gate array FPGA or a chip capable of realizing video injection requirements.

12. The method according to claim 4, wherein: The number N2 of the boards is related to at least one of the following: It is related to the number N0 of acquisition devices in the simulation test; It is related to the number of video output interfaces of the decompression module; Related to the number N1 of decompression modules.

13. The method according to claim 1, wherein: The method of using N1 decompression modules to obtain N0 decompressed simulation images for simulating the environment of the vehicle at the same time point includes: The i-th group among N1 groups is sent to the i-th decompression module among N1 decompression modules, so as to use the N1 decompression modules to obtain N0 decompressed simulation images for simulating the environmental scenes of the vehicle at the same time point; wherein, the N1 groups are obtained by splitting the N0 channels of video compression data used for simulation testing.

14. The method according to claim 13, wherein: After sending the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules, the method further includes: Using the i-th decompression module, determine ni frames of compressed data belonging to the same time point from the ni channels of video compressed data included in the i-th group, wherein each channel of video compressed data in the ni channels of video compressed data contains a plurality of frames of compressed data; The i-th decompression module is used to decompress ni frames of compressed data belonging to the same time point to obtain ni simulated images.

15. The method according to claim 14, further comprising: Using the i-th decompression module, the ni channels of video compression data contained in the i-th group are split into Mi data blocks, wherein each data block contains ni frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of Mi is related to the number of frame compression data contained in the video compression data.

16. The method according to claim 15, wherein: The step of using the i-th decompression module to determine ni frames of compressed data belonging to the same time point from the ni channels of video compressed data included in the i-th group, and using the i-th decompression module to decompress the ni frames of compressed data belonging to the same time point comprises: Using the i-th decompression module, based on the timing rule, select the j-th data block from the Mi data blocks; wherein the j-th data block is at least one of the currently undecompressed data blocks in the Mi data blocks; The i-th decompression module is used to serially decompress ni frames of compressed data in the j-th data block.

17. The method according to claim 16, wherein: The method of using the i-th decompression module to serially decompress ni frames of compressed data in the j-th data block comprises: Decompressing the p-th frame compressed data in the j-th data block by using the i-th decompression module; wherein the p-th frame compressed data is one of the undecompressed frame compressed data in the selected j-th data block; After decompressing to obtain the simulated image of the p-th frame compressed data, the simulated image of the p-th frame compressed data is stored in the cache area of ​​the ith decompression module, and the p+1-th frame compressed data is decompressed; the p+1-th frame compressed data is one of the undecompressed frame compressed data in the Mi-th data block.

18. The method according to claim 16, further comprising: After determining that the decompression of the frame compressed data of the j-th data block is completed, the i-th decompression module is used to serially decompress the ni frame compressed data in the j+1-th data block, and each data block is decompressed serially; wherein the j+1-th data block is at least one of the currently uncompressed data blocks in the Mi data blocks.

19. The method according to any one of claims 13 to 18, further comprising: Obtaining total compressed data for simulation testing; wherein the total compressed data is used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives within a preset period of time; The total compressed data is split based on at least N0 acquisition devices required for simulation test to obtain the N0 channels of video compression data.

20. The method according to claim 19, wherein: The method of splitting the total compressed data based on at least N0 acquisition devices required for the simulation test to obtain the N0 channels of video compressed data includes: Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N0 collection devices within the unit time; Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data; Based on the N0 acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N0 channels of video compression data.

21. A simulation device comprising: N1 decompression modules; N2 boards; N2 is a positive integer greater than or equal to 1 A simulation test unit is used to use N1 decompression modules to obtain N0 decompressed simulation images for simulating the environmental scenes of the vehicle at the same time point; wherein the N0 simulation images are used to simulate the environmental scenes captured from different acquisition perspectives at the same time point; N0 and N1 are both positive integers greater than or equal to 2; the N0 simulation images belonging to the same time point are first time aligned; after the first time alignment, the N0 simulation images belonging to the same time point are synchronously sent to N2 boards through N1 decompression modules, so that the simulation images belonging to the same time point are sent to the vehicle for simulation testing using the N2 boards.

22. The apparatus according to claim 21, wherein The simulation test unit is specifically used for: Using N1 decompression modules after clock synchronization, N0 simulation images belonging to the same time point are first time-aligned, wherein the N1 decompression modules are clock-synchronized using a central processing unit CPU of the simulation device; or, A time synchronization device is used to send a time signal to the N1 decompression modules, so that the N1 decompression modules perform a first time alignment on the decompressed N0 simulation images belonging to the same time point after receiving the time signal.

23. The apparatus of claim 22, wherein: The alignment accuracy of performing a first time alignment on the N0 simulation images using the N1 decompression modules after clock synchronization is lower than the alignment accuracy of performing a first time alignment on the N0 simulation images using the time signal.

24. The device according to any one of claims 21 to 23, wherein: The simulation test unit is also used for: The simulation images belonging to the same time point are aligned for a second time using N2 boards, so that after the second time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using the N2 boards.

25. The apparatus of claim 24, wherein: The simulation test unit is specifically used for: Determine whether the total cache units corresponding to the N2 boards store all the simulation images belonging to the same time point; wherein the total cache units corresponding to the N2 boards represent the set of cache units of all the boards in the N2 boards; When it is determined that all simulation images belonging to the same time point are stored, N2 boards are used to perform a second time alignment on all simulation images belonging to the same time point, and after the second time alignment, all simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing.

26. The apparatus of claim 25, wherein: The simulation test unit is specifically used for: When it is determined that the current occupied capacity of the cache unit of each of the N2 boards has reached the maximum capacity, it is determined that the total cache unit corresponding to the N2 boards stores all the simulation images belonging to the same time point, wherein the maximum capacity of the cache unit of the board is determined based on the number of simulation images belonging to the same time point that the board needs to receive, and is related to the total number of channels of video compression data for the simulation test.

27. The apparatus of claim 24, wherein: The time alignment accuracy that can be achieved by performing the first time alignment for the N0 simulation images is lower than the time alignment accuracy that can be achieved by performing the second time alignment for the N0 simulation images.

28. The apparatus of claim 24, wherein: The simulation test unit is also used for: Before using N2 boards to perform a second time alignment on the simulation images at the same time point, using N2 boards to convert the data formats of N0 simulation images at the same time point into a preset format.

29. The apparatus of claim 28, wherein: The preset format is the data format output by the acquisition device of the vehicle under simulation test.

30. The apparatus of claim 29, wherein: The data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

31. The apparatus of claim 24, wherein: The board is one of the following: a field programmable gate array FPGA or a chip capable of realizing video injection requirements.

32. The apparatus of claim 24, wherein: The number N2 of the boards is related to at least one of the following: It is related to the number N0 of acquisition devices in the simulation test; It is related to the number of video output interfaces of the decompression module; Related to the number N1 of decompression modules.

33. The apparatus of claim 21, wherein: The simulation test unit is specifically used for: The i-th group among N1 groups is sent to the i-th decompression module among N1 decompression modules, so as to use the N1 decompression modules to obtain N0 decompressed simulation images for simulating the environmental scenes of the vehicle at the same time point; wherein, the N1 groups are obtained by splitting the N0 channels of video compression data used for simulation testing.

34. The apparatus of claim 33, wherein: The simulation test unit is also used for: Using the i-th decompression module, determine ni frames of compressed data belonging to the same time point from the ni channels of video compressed data included in the i-th group, wherein each channel of video compressed data in the ni channels of video compressed data contains a plurality of frames of compressed data; The i-th decompression module is used to decompress ni frames of compressed data belonging to the same time point to obtain ni simulated images.

35. The apparatus of claim 34, wherein: The simulation test unit is also used for: Using the i-th decompression module, the ni channels of video compression data contained in the i-th group are split into Mi data blocks, wherein each data block contains ni frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of Mi is related to the number of frame compression data contained in the video compression data.

36. The apparatus of claim 35, wherein: The simulation test unit is specifically used for: Using the i-th decompression module, based on the timing rule, select the j-th data block from the Mi data blocks; wherein the j-th data block is at least one of the currently undecompressed data blocks in the Mi data blocks; The i-th decompression module is used to serially decompress ni frames of compressed data in the j-th data block.

37. The apparatus of claim 36, wherein: The simulation test unit is specifically used for: Decompressing the p-th frame compressed data in the j-th data block by using the i-th decompression module; wherein the p-th frame compressed data is one of the undecompressed frame compressed data in the selected j-th data block; After decompressing to obtain the simulated image of the p-th frame compressed data, the simulated image of the p-th frame compressed data is stored in the cache area of ​​the ith decompression module, and the p+1-th frame compressed data is decompressed; the p+1-th frame compressed data is one of the undecompressed frame compressed data in the Mi-th data block.

38. The apparatus of claim 36, wherein: The simulation test unit is also used for: After determining that the decompression of the frame compressed data of the j-th data block is completed, the i-th decompression module is used to serially decompress the ni frame compressed data in the j+1-th data block, and each data block is decompressed serially; wherein the j+1-th data block is at least one of the currently uncompressed data blocks in the Mi data blocks.

39. The apparatus according to any one of claims 33 to 38, wherein: Also includes: Data preprocessing unit; The data preprocessing unit is used for: Obtaining total compressed data for simulation testing; wherein the total compressed data is used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives within a preset period of time; The total compressed data is split based on at least N0 acquisition devices required for simulation test to obtain the N0 channels of video compression data.

40. The apparatus of claim 39, wherein: The data preprocessing unit is specifically used for: Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N0 collection devices within the unit time; Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data; Based on the N0 acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N0 channels of video compression data.

41. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 20.

42. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-20.

43. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 20.

Images