Simulation data processing method and device, and storage medium

Abstract

The disclosure provides a simulation data processing method and device and a storage medium, relates to the technical field of image processing, and in particular to the technical field of automatic driving, auxiliary driving, automatic parking and intelligent transportation. The specific implementation scheme is as follows: N0 pieces of video compression data used for simulation test are split into N1 groups, the N0 pieces of video compression data are used to simulate environment pictures captured by N0 acquisition devices in a vehicle from different acquisition angles in the same time period; the i-th group in the N1 groups is sent to the i-th decompression module in N1 decompression modules, so that N0 simulation images for simulating environment pictures of the vehicle at the same time point are obtained after decompression by using the N1 decompression modules; and the N0 simulation images belonging to the same time point are sent to N2 boards by the N1 decompression modules, so that the simulation images belonging to the same time point are sent to the vehicle for simulation test by using the N2 boards.

Description

[0001] This application is a divisional application of a Chinese invention entitled “Simulation Data Processing Method, Device and Storage Medium”, application number “202410999063.6”, filed on July 24, 2024. Technical Field

[0002] This disclosure relates to the field of image processing technology, and more particularly to the fields of autonomous driving, driver assistance, automatic parking, and intelligent transportation. Background Technology

[0003] In hardware-in-the-loop (HIL) simulation systems for autonomous driving, camera simulation, such as the simulation of Global Motion Suppression Logic (GMSL) cameras, is an essential component. Camera simulation enables effective testing of the entire video processing chain within the autonomous vehicle control system. Therefore, how to efficiently perform camera simulation is a current research focus. Summary of the Invention

[0004] This disclosure provides a method, apparatus, and storage medium for processing simulation data.

[0005] According to one aspect of this disclosure, a simulation data processing method is provided, comprising:

[0006] The N0 channels of compressed video data used for simulation testing are split into N1 groups. The N0 channels of compressed video data are used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition angles at the same time. N0 and N1 are both positive integers greater than or equal to 2.

[0007] The i-th group of the N1 groups is sent to the i-th decompression module of the N1 decompression modules, so as to obtain N0 simulation images of the vehicle's environment at the same time point after decompression using the N1 decompression modules.

[0008] N1 decompression modules are used to send N0 simulation images belonging to the same time point to N2 boards, so that the N2 boards can be used to send the simulation images belonging to the same time point to the vehicle for simulation testing; where N2 is a positive integer greater than or equal to 1.

[0009] According to another aspect of this disclosure, a simulation device is provided, comprising:

[0010] The data preprocessing unit is used to split the N0 channels of compressed video data used for simulation testing into N1 groups, wherein the N0 channels of compressed video data are used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition angles at the same time; N0 and N1 are both positive integers greater than or equal to 2;

[0011] The simulation test unit is used to send the i-th group from the N1 groups to the i-th decompression module from the N1 decompression modules, so as to obtain N0 simulation images of the vehicle's environment at the same time point after decompression using the N1 decompression modules; and to send the N0 simulation images belonging to the same time point to N2 boards through the N1 decompression modules, so as to send the simulation images belonging to the same time point to the vehicle for simulation testing using the N2 boards; where N2 is a positive integer greater than or equal to 1.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] The memory is communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0016] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0017] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0018] In this way, the disclosed solution can use multiple decompression modules to decompress video compressed data in parallel, and after successful decompression, send multiple simulation images belonging to the same time point directly to the board. The board can then send multiple simulation images belonging to the same time point to the vehicle for simulation testing. This effectively reduces the time cost required for the decompression process, improves the efficiency of decompression, and lays the foundation for achieving efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras). It also lays the foundation for improving the efficiency of vehicle simulation testing.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0021] Figure 1 This is an illustrative flowchart of a simulation data processing method according to an embodiment of this application. Figure 1 ;

[0022] Figure 2 This is a schematic diagram of a scenario where a simulation device processes N0 channels of video compression data according to an embodiment of this application. Figure 1 ;

[0023] Figure 3 This is a schematic diagram of a scenario where a simulation device processes N0 channels of video compression data according to an embodiment of this application. Figure 2 ;

[0024] Figure 4 This is a simulation scene diagram corresponding to the total compressed data according to an embodiment of this application. Figure 1 ;

[0025] Figure 5 This is a simulation scene diagram corresponding to the total compressed data according to an embodiment of this application. Figure 2 ;

[0026] Figure 6 This is a schematic diagram of a scenario where a simulation device processes total compressed data according to another embodiment of this application. Figure 1 ;

[0027] Figure 7 This is an illustrative flowchart of a simulation data processing method according to an embodiment of this application. Figure 2 ;

[0028] Figure 8 This is a schematic diagram of a scenario where a simulation device processes total compressed data according to another embodiment of this application. Figure 2 ;

[0029] Figure 9 This is an illustrative flowchart of a simulation data processing method according to an embodiment of this application. Figure 3 ;

[0030] Figure 10 This is an illustrative diagram illustrating how compressed video data is divided into multiple data blocks according to an embodiment of this application;

[0031] Figure 11 This is a schematic diagram of the structure of a simulation device according to an embodiment of this application;

[0032] Figure 12 This is a block diagram of an electronic device used to implement the simulation data processing method of the embodiments of this disclosure. Detailed Implementation

[0033] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0034] In this document, the term "and / or" merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The term "at least one" in this document indicates any combination of at least two of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this document refer to and distinguish between multiple similar technical terms, not to restrict the order or to limit there to only two. For example, "first feature" and "second feature" refer to two categories / two features; the first feature can be one or more, and the second feature can also be one or more.

[0035] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can still be practiced even without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0036] The following describes the related technologies of the embodiments of this disclosure. The following related technologies are optional solutions and can be combined with the technical solutions of the embodiments of this disclosure in any way, and they all fall within the protection scope of the embodiments of this disclosure.

[0037] In hardware-in-the-loop (HIL) simulation systems for autonomous driving, camera simulation, such as simulating a GMSL camera, is an essential component. Camera simulation allows for effective testing of the entire video processing chain within the autonomous vehicle control system; for example, it enables both the simulation of video content and the testing of the video link timing.

[0038] For example, in simulation testing scenarios, for video data from road test disk drops, a decompression module (e.g., a graphics processing unit, GPU) can be used to decompress the data. The decompressed video data is then transferred to the simulation device (e.g., a server) for storage in the simulation device's memory. The decompressed video data stored in the simulation device's memory is then transmitted to a board (e.g., a video injection board) via a video link. Finally, the board performs data format conversion and outputs the data synchronously. This is how camera simulation is achieved. However, as the bandwidth required for autonomous driving camera data increases, simulation devices (e.g., servers) are increasingly unable to meet the requirements in terms of memory and bandwidth. This becomes a bottleneck for camera data transmission, thus limiting the performance (e.g., pixel resolution, acquisition frequency) and the number of cameras that can be modeled in the simulation environment.

[0039] Based on this, the present disclosure proposes a simulation data processing method to improve camera data transmission efficiency and achieve efficient simulation of high-pixel, high-frequency cameras.

[0040] Specifically, Figure 1 This is an illustrative flowchart of a simulation data processing method according to an embodiment of this application. Figure 1 This method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0041] Furthermore, the method includes at least a portion of the following: For example... Figure 1 As shown, the simulation data processing method, applied to a simulation device (e.g., an industrial control computer), includes:

[0042] Step S101: The simulation device splits the N0 channels of video compression data used for simulation testing into N1 groups.

[0043] Here, the N0 video compression data is used to simulate the environmental images captured by N0 (e.g., a positive integer greater than or equal to 2) acquisition devices in a vehicle from different acquisition perspectives within the same time period. For example, it simulates the environmental images captured by N0 acquisition devices from different acquisition perspectives within the same time period (e.g., per unit time) while the vehicle is traveling on its road.

[0044] Furthermore, in one example, different acquisition devices in the vehicle have different acquisition perspectives, which makes it easier to capture images of the environment around the vehicle through different acquisition devices, providing support for driving decisions.

[0045] Furthermore, in one example, each video compression data packet in the N0 video compression data contains multiple frame compression data. For example, in a specific example, for one video compression data, it may contain at least the compressed data of multiple video frames captured by a single acquisition device within a unit time (e.g., within 1 second) (here, the compressed data of video frames can be simply referred to as frame compression data), for example, it may contain the compressed data of 10 frames of images captured by a single acquisition device within 1 second.

[0046] Here, the number of video frames acquired per unit time is related to the relevant parameters of the acquisition device to be simulated, such as the frame rate, and this disclosed solution does not impose any restrictions on this.

[0047] Furthermore, N1 is a positive integer greater than or equal to 2. In one example, the value of N1 is related to the number of decompression modules used for decompression; for example, the value of N1 is equal to the number of decompression modules used for decompression. That is, in one example, N0 channels of video compressed data can be split into N1 groups according to the number of decompression modules (e.g., N1 decompression modules). This facilitates sending different groups to different decompression modules for decompression, realizing parallel decompression of N0 channels of video compressed data using N1 decompression modules, thereby providing support for efficient simulation of high-pixel, high-frequency cameras.

[0048] In one example, different video compression data streams belong to different groups, and the number of streams contained in different groups may be the same or different. This disclosure does not impose any restrictions on this. For example, the i-th group among the N1 groups contains ni video compression data streams, and the j-th group among the N1 groups contains nj video compression data streams. The values ​​of i and j are different, and correspondingly, the value of ni may be the same as or different from the value of nj.

[0049] Step S102: The simulation device sends the i-th group of the N1 groups to the i-th decompression module of the N1 decompression modules, so as to decompress it using the i-th decompression module, and then uses the N1 decompression modules to obtain N0 simulation images of the vehicle's environment at the same time point after decompression.

[0050] Here, i is a positive integer ranging from 1 to N1. Thus, using N1 decompression modules, N0 simulation images can be obtained to simulate the environment surrounding the vehicle at the same point in time.

[0051] For example, in one example, there are 4 channels of video compression data that need to be simulated and tested. In this case, the 4 channels of video compression data can be split into 2 groups. The first group can contain the 1st and 2nd channels of video compression data, and the second group can contain the 3rd and 4th channels of video compression data. Further, the 1st group is sent to the 1st decompression module, and the 2nd group is sent to the 2nd decompression module.

[0052] Step S103: The simulation device sends N0 simulation images belonging to the same time point to N2 (positive integers greater than or equal to 1) boards through N1 decompression modules, so as to use N2 boards to send simulation images belonging to the same time point to the vehicle for simulation testing.

[0053] For example, after the simulation equipment determines that all frame compressed data belonging to the same time point has been successfully decompressed, that is, after obtaining N0 simulation images belonging to the same time point, it directly sends the N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules, so that the N2 boards can send all simulation images belonging to the same time point to the vehicle for simulation testing.

[0054] In this way, the disclosed solution can use multiple decompression modules to decompress video compressed data in parallel, and after successful decompression, send multiple simulation images belonging to the same time point directly to the board. The board can then send multiple simulation images belonging to the same time point to the vehicle for simulation testing. This effectively reduces the time cost required for the decompression process, improves the efficiency of decompression, and lays the foundation for achieving efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras). It also lays the foundation for improving the efficiency of vehicle simulation testing.

[0055] Furthermore, compared to existing decompression methods, the present invention can directly send multiple simulation images belonging to the same time point to the board through multiple decompression modules without forwarding through the simulation device. Therefore, compared to the solution of caching a large amount of decompressed video data in the simulation device, it effectively reduces the storage pressure on the simulation device and improves the transmission link bandwidth of the simulation images. Based on this, the present invention can be applied to larger datasets and can further improve the reliability and accuracy of simulation testing.

[0056] It should be noted that the vehicles described in this disclosure may be driverless vehicles or driver-assisted vehicles, and this disclosure does not impose any specific restrictions on them.

[0057] Furthermore, it should be noted that, in one example, the decompression module described in this disclosure may specifically be a graphics processing unit (GPU), or it may be other devices capable of graphics processing; this disclosure does not impose any specific limitations on this.

[0058] In a specific example of the disclosed solution, in order to ensure the time consistency of the simulation images received by the vehicle, time alignment can also be performed in the following manner.

[0059] For example, in one example, the first time alignment can be performed using N1 decompression modules as follows.

[0060] Specifically, in one example, before sending N0 simulation images belonging to the same time point to N2 boards via N1 decompression modules, the method further includes:

[0061] The N0 simulation images belonging to the same time point are aligned in the first time, so that the N0 simulation images belonging to the same time point can be sent to the N2 boards synchronously.

[0062] Accordingly, the above-described method of sending N0 simulation images belonging to the same time point to N2 boards via N1 decompression modules can specifically include:

[0063] After the initial alignment, N0 simulation images belonging to the same time point are synchronously sent to N2 boards through N1 decompression modules.

[0064] It should be noted that in real-world scenarios, the reception times of the simulated images received by the N2 boards from the simulation device may differ. Therefore, in order to ensure that all simulated images belonging to the same time point can be transmitted synchronously and that the vehicle can receive the simulated images synchronously (i.e., ensuring the time consistency of the simulated images received by the vehicle), so as to successfully complete the testing of the simulation link timing and / or the testing of the simulated video content, in this example, the simulation device can also first align the N0 simulated images belonging to the same time point, and then synchronously send them to the N2 boards after alignment. This ensures the time consistency of the simulated images received by the boards and provides strong support for reducing the latency of the simulated images received by the vehicle.

[0065] It is understandable that the above time alignment operation (i.e., the first time alignment operation) does not change the timestamp of the simulation image, but only aligns the N0 simulation images obtained after decompression at different times, so that all simulation images obtained after decompression at different times but belonging to the same time point can be sent at the same time (i.e., sent synchronously).

[0066] In this way, the present invention can align multiple simulation images obtained after decompression that belong to the same time point in the first time, and then transmit them synchronously after alignment. This effectively improves the time synchronization of the decompressed image data during transmission, provides strong support for reducing the delay of the simulation images received by the vehicle, and also provides strong support for achieving accurate simulation of high-pixel and high-frequency cameras.

[0067] Furthermore, in a specific example, the N0 simulation images belonging to the same time point can be aligned in the first time using the following two methods:

[0068] Method 1: Using N1 decompression modules synchronized by clock, N0 simulation images belonging to the same time point are aligned in the first time. Here, the N1 decompression modules are clock synchronized using the CPU of the simulation device.

[0069] In other words, in this first method, each decompression module can directly perform time alignment on N0 simulated images belonging to the same time point, thus ensuring the time synchronization of the image data. It should be noted that a preset clock synchronization strategy can be adopted, and the CPU of the simulation device can be used to synchronize the clocks of the N1 decompression modules set in the simulation device. Therefore, with the clocks of the N1 decompression modules synchronized, each decompression module can be used to perform time alignment on the N0 simulated images belonging to the same time point.

[0070] Method 2: Use the time synchronization device in the simulation device to send a time signal (such as a pulse signal) to the N1 decompression modules, so that the N1 decompression modules will perform a first time alignment on the N0 simulation images belonging to the same time point after receiving the time signal.

[0071] It should be noted that, in one example, the N1 decompression modules in this method two are also decompression modules that use the CPU of the simulation device for clock synchronization.

[0072] Furthermore, it should be noted that since each decompression module has an internal clock generator, even if clock synchronization between all decompression modules in the simulation device is achieved using the CPU, the clocks between the decompression modules may still become out of sync for various reasons during the decoding process or other program execution. Therefore, method two can also be used, for example, by using pulse signals emitted by a time synchronization device to enable the N1 decompression modules to perform initial time alignment of the N0 simulated images belonging to the same time point after decompression. This further improves the time synchronization of the image data and greatly reduces the latency in image data transmission.

[0073] For example, such as Figure 2 As shown, the simulation device, such as the industrial control computer, integrates three decompression modules (e.g., three GPUs) and one time synchronization device (e.g., sync2). II) Card): At this point, the industrial control computer can split the N0 channels of video compression data into 3 groups according to the number of GPUs. The first group is sent to the first GPU for decompression, the second group to the second GPU for decompression, and the third group to the third GPU for decompression. This results in N0 simulated images obtained by the three GPUs, used to simulate the environment around the autonomous or assisted driving vehicle at the same time point. During the GPU decompression process, the sync2 card in the simulation device sends pulse signals to the three GPUs, for example, periodically. This allows the three GPUs to receive the pulse signals and perform first-time alignment on the N0 simulated images belonging to the same time point after successful decompression. After the first-time alignment, the three decompression modules synchronously send the N0 simulated images belonging to the same time point to the N2 cards. This lays the foundation for further improving the time synchronization accuracy of the image data.

[0074] In this way, the disclosed solution can utilize the time synchronization device in the simulation equipment to enable each decompression module to time-align the N0 simulation images that are decompressed and belong to the same time point. This further ensures the time consistency between the decompression modules and also further improves the time synchronization of the image data. This provides strong support for ensuring the accuracy of data transmission and the normal operation of subsequent simulation processes.

[0075] It should be noted that, in a specific example, for the two methods of first-time alignment mentioned above, the alignment accuracy of using N1 decompression modules synchronized by clock to perform first-time alignment of the N0 simulation images is lower than the alignment accuracy of using the time signal to perform first-time alignment of the N0 simulation images.

[0076] It should be noted that in this example, multiple simulated images belonging to the same point in time can be aligned with low time precision (e.g., millisecond level) using the decompression module of the simulation device (also known as "coarse" granular time alignment). Alternatively, to improve alignment accuracy, a time synchronization device can be used for high time precision (e.g., microsecond level) alignment (also known as "fine" granular time alignment). After alignment, they are synchronously sent to the board, which then sends them to the vehicle for simulation testing. This effectively ensures the time synchronization of image data, greatly reduces the latency of image data transmission, and lays the foundation for accurate simulation of high-pixel and high-frequency cameras. At the same time, it also lays the foundation for the autonomous driving system to respond to environmental changes more quickly and make accurate decisions.

[0077] In a specific example of the disclosed solution, in order to ensure the time consistency of the simulation images received by the vehicle, time alignment can be performed again in the following manner.

[0078] For example, in one example, N2 boards can be used for a second time alignment in the following way.

[0079] Specifically, in one example, before sending simulation images belonging to the same time point to the vehicle for simulation testing using N2 boards, the method further includes:

[0080] The simulation images belonging to the same time point are aligned in a second time using N2 boards. After the second time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using N2 boards.

[0081] In other words, the disclosed solution can adopt a two-level time alignment strategy. For example, the first time alignment is performed using N1 decompression modules (i.e., the first time alignment mentioned above), and the second time alignment is performed using N2 boards (i.e., the second time alignment mentioned above). This effectively improves the time synchronization of the decompressed image data during transmission and also effectively improves the synchronization accuracy, thereby providing strong support for reducing the latency of the simulated images received by the vehicle. At the same time, it also provides strong support for efficiently simulating high-pixel resolution or high-acquisition-frequency acquisition devices, increasing the number of acquisition devices that can be used in the simulation scene, and thus enriching the applicability of the disclosed solution.

[0082] It should be noted that in real-world scenarios, the reception times of the simulated images received by the N2 boards from the simulation device may differ. Therefore, to ensure that the N2 boards can synchronously transmit all simulated images belonging to the same time point, and that the vehicle can synchronously receive the simulated images (i.e., ensuring the time consistency of the simulated images received by the vehicle), in order to successfully complete the testing of the simulation link timing and / or the testing of the simulated video content, in this example, the simulation device can also use the N2 boards to perform a second time alignment on multiple simulated images belonging to the same time point. After the second time alignment, the N2 boards can then synchronously transmit all simulated images belonging to the same time point to the vehicle, thereby ensuring the time consistency of the simulated images received by the vehicle.

[0083] It is understandable that the above-mentioned second time alignment operation can also be performed without changing the timestamp of the simulation image, but only aligning the simulation images received at different times, so as to send out all simulation images received at different times (also known as different time points) but belonging to the same time point at the same time (i.e., synchronously).

[0084] Furthermore, in a specific example, the above-described method of using N2 boards to perform a second time alignment on simulation images belonging to the same time point, so that after the second time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using N2 boards, can specifically include:

[0085] The simulation device determines whether the total cache unit corresponding to the N2 boards contains all simulation images belonging to the same time point. If it is determined that all simulation images belonging to the same time point are stored, the simulation device uses the N2 boards to perform a second time alignment on all simulation images belonging to the same time point. After the second time alignment, the N2 boards are used to synchronously send all simulation images belonging to the same time point to the vehicle for simulation testing.

[0086] It should be noted that the total cache units corresponding to N2 boards can represent the set of cache units of all boards in N2 boards, that is, the sum of cache units of all boards in N2 boards.

[0087] Furthermore, the maximum capacity of the total cache units corresponding to N2 boards can be determined based on the total number of video compression data streams in the simulation test; for example, in one example, the maximum capacity of the total cache units corresponding to N2 boards = the total number of video compression data streams. Furthermore, the maximum capacity of the total cache units of N2 boards is also equal to the number of acquisition devices to be simulated, and further, it is also equal to the number of simulated images belonging to the same time point.

[0088] Furthermore, for example, in one example, the simulation device can integrate a board (i.e., N2 is 1). In this case, after each simulation image belonging to the same time point is sent (e.g., sent synchronously) to the board, each simulation image belonging to the same time point can be stored in the buffer unit of the board. Accordingly, it can be determined whether the buffer unit of the board contains all simulation images belonging to the same time point.

[0089] Alternatively, in another example, the simulation device can integrate two or more boards (i.e., N2 is a natural number greater than or equal to 2). In this case, simulation images belonging to the same time point can be sent (e.g., sent synchronously) to multiple boards. Correspondingly, the simulation images belonging to the same time point can be stored in the cache units of different boards. Accordingly, it can be determined whether the total cache unit of all boards contains all simulation images belonging to the same time point.

[0090] For example, consider a scenario where there are six acquisition devices (i.e., six channels of compressed video data) and two boards connected to the simulation device. In one example, each board can receive three channels of decompressed simulation images and store them in its respective buffer unit. Correspondingly, the simulation device can determine whether the total buffer unit of the two boards contains all six simulation images belonging to the same time point. Further, after determining that the total buffer unit of the two boards contains all six simulation images belonging to the same time point, the two boards can be used to perform a second time alignment on the six simulation images. For example, each board can perform a second time alignment on the three simulation images in its respective buffer unit. At this point, it can be considered that all simulation images belonging to the same time point have completed the second time alignment. Alternatively, the two boards can use a time synchronization signal to perform a second time alignment on the six simulation images in the total buffer unit, and after the second time alignment, the six simulation images are synchronously sent to the vehicle through the two boards for simulation testing.

[0091] It should be noted that, in one example, in order to store simulation images at different time points, the board can also be equipped with multiple cache units. In this case, simulation images at different time points can be cached in different cache units, so that only images belonging to the same time point are cached in the same cache unit. This provides effective support for efficiently synchronizing and sending out all simulation images belonging to the same time point.

[0092] In this way, the present invention can use the board to synchronously send all simulation images belonging to the same point in time to the vehicle, thereby further improving the time synchronization of the decompressed image data during transmission, thus providing strong support for effectively reducing the latency of the simulation images received by the vehicle. At the same time, it also provides strong support for efficiently simulating high pixel resolution or high acquisition frequency acquisition devices, increasing the number of acquisition devices that can be used in the simulation scene, and thus enriching the scope of application of the present invention.

[0093] Furthermore, in a specific example, the above-described method of determining whether the total cache units corresponding to the N2 boards contain all simulation images belonging to the same time point can specifically include:

[0094] Given that the current capacity of the cache units of each of the N2 boards has reached its maximum capacity, it is determined that the total cache units corresponding to the N2 boards contain all simulation images belonging to the same time point.

[0095] Here, the maximum capacity of the buffer unit of each board can be determined based on the number of simulated images belonging to the same time point that the board needs to receive; furthermore, it is related to the total number of video compression data streams in the simulation test. For example, when the simulation device integrates multiple boards, the maximum capacity of the buffer unit of each board can be specifically determined based on the total number of video compression data streams in the simulation test and the total number of boards.

[0096] For example, if there are 6 acquisition devices required for the simulation test, there will be 6 simulation images belonging to the same time point. In this case, the maximum capacity of the total cache unit corresponding to N2 boards can be set to accommodate 6 simulation images. In other words, the sum of the maximum capacity of the cache units of each board in N2 boards (that is, the total maximum capacity of the set of cache units of each board) can accommodate these 6 simulation images. Further, if the simulation device integrates 2 (that is, N2 is 2) boards, in one example, the maximum capacity of the cache unit of each board can accommodate 3 simulation images. Furthermore, if it is determined that the current capacity occupied by the cache unit of each board has reached the maximum capacity, it can be considered that all simulation images belonging to the same time point have been stored in the total cache unit. In this way, all simulation images belonging to the same time point can be synchronized and sent out after a second time alignment.

[0097] For example, in one scenario, if there are six acquisition devices required for the simulation test, and each simulation device integrates three decompression modules, with each decompression module connected to a board (a total of three boards), then the number of simulation images that each board needs to receive (or transmit synchronously) at the same time point can be two. Furthermore, in one example, the maximum capacity of each board's buffer unit can be set to hold two simulation images. Further, if it is determined that the current capacity occupied by each board's buffer unit has reached its maximum capacity, then it can be assumed that all simulation images belonging to the same time point are stored in the total buffer unit. Thus, all simulation images belonging to the same time point can be synchronized after a second time alignment and transmitted synchronously.

[0098] It should be noted that the above is only an illustrative example. In actual applications, the maximum capacity of the cache units in different boards may be the same or different, and this disclosure does not limit this.

[0099] Thus, this disclosed solution provides a specific method for quickly determining whether the board has received all simulation images belonging to the same time point. This provides strong support for ensuring that the vehicle can synchronously receive all simulation images belonging to the same time point, and also provides strong support for further improving the time synchronization accuracy. This lays the foundation for the subsequent successful testing of the simulation link timing and / or the simulation video content.

[0100] Furthermore, in a specific example, the time alignment accuracy achievable by performing the first time alignment for the N0 simulated images is lower than the time alignment accuracy achievable by performing the second time alignment for the N0 simulated images.

[0101] In other words, in this example, multiple simulated images belonging to the same point in time can be time-aligned once using the decompression module of the simulation device (for example, a time alignment with relatively low or relatively high precision; here, the lower precision can also be called "coarse" granular time alignment). Then, the board performs a high-precision (e.g., microsecond-level) time alignment on the multiple simulated images after the "coarse" granular time alignment (also called "fine" granular time alignment). After the "fine" granular time alignment, all simulated images belonging to the same point in time are synchronously sent to the vehicle for simulation testing. This further ensures the time synchronization of image data, greatly reduces the latency of image data transmission, lays the foundation for accurate simulation of high-pixel and high-frequency cameras, and also lays the foundation for the autonomous driving system to respond to environmental changes more quickly and make accurate decisions.

[0102] Furthermore, in a specific example, the number N2 of the boards is related to at least one of the following:

[0103] It is related to the number N0 of data acquisition devices in the simulation test;

[0104] The number of video output interfaces of the decompression module is related to the number of video compression data channels N0.

[0105] It is related to the number of decompression modules, N1.

[0106] For example, in one example, the number of boards is the same as the number of decompression modules (e.g., GPUs). Continuing with, for example... Figure 2 Taking the scenario shown as an example, such as Figure 3 As shown, each of the three GPUs is connected to a board (e.g., a video injection board). Specifically, the first GPU is connected to the first video injection board to send the simulation image decompressed by the first GPU to the vehicle for simulation testing. Similarly, the second GPU is connected to the second video injection board to send the simulation image decompressed by the second GPU to the vehicle for simulation testing. The third GPU is connected to the third video injection board to send the simulation image decompressed by the third GPU to the vehicle for simulation testing.

[0107] Furthermore, it should be noted that in one example, in a scenario where the decompression module and the board are connected one-to-one, the number of output interfaces (also known as video output interfaces) in the decompression module is related to the number of video receiving interfaces on the board. For example, they are the same (e.g., both are 4). That is, if there is one video output interface in the decompression module, there is a corresponding video path in the board. In other words, the number of video interfaces in both is the same.

[0108] In this way, the disclosed solution can use the board to time-align N0 simulated images that are decompressed and belong to the same time point, thus effectively reducing the latency of video data. Moreover, in scenarios using multiple boards, more simulated camera scenarios can be supported. At the same time, it can also support efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), thereby simulating more complex driving environments and scenarios. In addition, the use of multiple boards improves the parallel capability of data injection, thereby improving the efficiency of data transmission.

[0109] Here, in one example, the aforementioned board can specifically be a video injection board; further, in one example, the board is one of the following: a Field Programmable Gate Array (FPGA) or a chip capable of implementing video injection requirements. In other words, the solution disclosed herein is compatible with existing boards, thus expanding the scope of application of the solution and making it both practical and applicable.

[0110] It should be noted that, in one example, the bandwidth of the board's data receiving link is greater than the bandwidth of the board's data sending link. This further lays the foundation for reducing the latency of image data transmission and thus achieving accurate simulation of high-pixel and high-frequency cameras.

[0111] Furthermore, in a specific example, before performing a second time alignment on simulation images belonging to the same time point using N2 boards, the method further includes:

[0112] The data format of N0 simulation images belonging to the same time point is converted into a preset format using N2 boards.

[0113] In other words, in one example, after each board receives multiple simulated images belonging to the same time point, each board can first convert the data format of these multiple simulated images into a preset format, and then perform a second time alignment on the multiple simulated images belonging to the same time point under the preset format, and then send them out synchronously after the second time alignment. In this way, the required data format can be effectively simulated, and the data compatibility is stronger, thereby ensuring that the receiving end, such as the vehicle end, can process the received image data accordingly, thus laying the foundation for achieving efficient simulation of high-pixel, high-frequency cameras.

[0114] Furthermore, in one example, the preset format is the data format output by the acquisition device of the vehicle being simulated and tested. For example, the preset format is the data format output by the acquisition device of the vehicle being simulated and tested in GMSL format, that is, GMSL format. In this way, accurate and efficient simulation of the GMSL camera is achieved, laying the foundation for improving the performance and stability of the entire autonomous driving system in the future.

[0115] In a specific example, N0 channels of compressed video data can be obtained in the following manner; specifically, before the simulation device splits the N0 channels of compressed video data used for simulation testing into N1 groups (for example, before step S101 described above), a step of acquiring N0 channels of compressed video data is also included, specifically including:

[0116] Step S100-1: The simulation device acquires the total compressed data used for simulation testing.

[0117] Here, the total compressed data is used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives within a preset time period.

[0118] Furthermore, in one example, the preset time period includes multiple units of time.

[0119] Step S100-2: Based on at least N0 acquisition devices required for simulation testing, split the total compressed data to obtain the N0 channels of video compressed data.

[0120] For example, in one instance, the total compressed data is split based on at least N0 acquisition devices required for simulation testing to obtain the N0 channels of video compressed data (step S100-2), specifically including:

[0121] Step S100-2-1: Based on the multiple unit times included in the preset time period, the total compressed data is split into multiple groups of data to be decompressed.

[0122] Here, the number of data sets to be decompressed is the same as the number of unit times contained in the preset time period. Furthermore, each set of data to be decompressed contains frame compression data of the video frames that the N0 acquisition devices can acquire within a unit time, that is, it contains all the frame compression data that the N0 acquisition devices can acquire within a unit time.

[0123] In other words, in this step, the simulation device can split the total compressed data containing multiple units of time into multiple sets of data to be decompressed, so that each set of data to be decompressed contains all the frame compressed data that the N0 acquisition devices can collect within one unit of time.

[0124] Step S100-2-2: Select a set of data that meets the time requirement from the multiple sets of data to be decompressed.

[0125] Step S100-2-3: Based on the N0 acquisition devices required for simulation testing, split the set of data to be decompressed that meets the time requirements into the N0 channels of video compressed data.

[0126] For example, in one example, multiple sets of data to be decompressed are sorted according to time sequence, and the earliest set of data to be decompressed is selected from the sorting results. According to the multiple acquisition devices to be simulated, the earliest set of data to be decompressed is further split to obtain the N0 channels of video compressed data.

[0127] For example, in one example, such as Figure 4As shown, the vehicle to be simulated has six acquisition devices installed at six different locations, which can be labeled as acquisition device 1, acquisition device 2, acquisition device 3, acquisition device 4, acquisition device 5, and acquisition device 6. The total compressed data can then contain six channels of video compressed data, with each channel used to simulate the environmental image captured by one of the acquisition devices in the vehicle. For example, the first channel of video compressed data simulates the environmental image captured by acquisition device 1, the second channel simulates the environmental image captured by acquisition device 2, and so on, until the sixth channel simulates the environmental image captured by acquisition device 6.

[0128] Furthermore, in one example, the total compressed data is used to simulate the environmental images captured by the N0 acquisition devices from different acquisition perspectives within a unit of time (e.g., within 1 second). Correspondingly, each video compressed data obtained by directly splitting the total compressed data can directly simulate the environmental images captured by an acquisition device within a unit of time (e.g., within 1 second) and from its own acquisition perspective.

[0129] Or, in another example, such as Figure 5As shown, the total compressed data can be specifically used to simulate the environmental images captured by the N0 acquisition devices from different acquisition perspectives within a preset time period (for example, the preset time period includes L units of time, which can be denoted as the 1st time unit, 2nd time unit, ..., Lth time unit within the preset time period, where L is a positive integer greater than or equal to 2). In this scenario, the total compressed data can first be divided into L groups of data to be decompressed according to unit time, which can be denoted as the 1st group of data to be decompressed corresponding to the 1st time unit within the preset time period (for example, the 1st group of data to be decompressed corresponding to the 1st second within the preset time period), the 2nd group of data to be decompressed corresponding to the 2nd time unit within the preset time period (the 2nd group of data to be decompressed corresponding to the 2nd second within the preset time period), ..., the Lth group of data to be decompressed corresponding to the Lth time unit within the preset time period (the data to be decompressed corresponding to the last second within the preset time period). Here, each group of data to be decompressed is used to simulate the environmental images captured by the N0 acquisition devices from different acquisition perspectives within one unit of time; furthermore, based on the N0 acquisition devices... The acquisition device further splits a set of data to be decompressed into N0 channels of video compressed data. For example, the first set of data to be decompressed corresponding to the first second of a preset time period can be split into N0 channels of video compressed data. These channels can be denoted as the first channel of video compressed data corresponding to the first second of the preset time period, the second channel of video compressed data corresponding to the first second of the preset time period, and so on, until the N0th channel of video compressed data corresponding to the first second of the preset time period. Each channel of video compressed data is used to simulate the environmental scene captured by an acquisition device from its own acquisition perspective within a unit of time. This provides strong support for the subsequent rapid decompression of multiple decompression modules (e.g., GPU) to obtain the simulated image in parallel.

[0130] It should be noted that in practical applications, the total compressed data acquired by N0 acquisition devices can be split into multiple sets of data to be decompressed based on the device dimension. In this case, each set of data to be decompressed is used to simulate the environmental scene captured by an acquisition device from its own acquisition perspective within a preset time period (i.e., the multiple unit time mentioned above). Then, the multiple sets of data to be decompressed are processed according to the time dimension based on the unit time to obtain the N0 channels of video compressed data mentioned above.

[0131] It should be noted that in practical applications, such as Figure 6As shown, the simulation device, such as an industrial control computer, integrates N1 GPUs for decompression. Further, the industrial control computer obtains the total compressed data for simulation testing from the data center, splits the total compressed data to obtain N0 channels of video compressed data, and splits the N0 channels of video compressed data into N1 groups according to the number of GPUs, so as to allocate the N1 groups to N1 GPUs for decompression, and then sends the multiple simulation images belonging to the same time point obtained after decompression by N1 GPUs directly to the board, so as to send them to the vehicle for simulation testing through the board.

[0132] Thus, this disclosed solution provides a specific method for obtaining N0 channels of compressed video data. This method is simple and efficient, and can quickly split the total compressed data into N0 channels of compressed video data according to the time sequence. This provides strong support for the subsequent rapid decompression by the decompression module to obtain the simulation image in parallel.

[0133] Figure 7 This is an illustrative flowchart of a simulation data processing method according to an embodiment of this application. Figure 2 This method can be optionally applied to electronic devices, such as personal computers, servers, and server clusters. It is understood that the above... Figures 1 to 6 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0134] Furthermore, the method includes at least a portion of the following: For example... Figure 7 As shown, it includes:

[0135] Step S701: The simulation equipment splits the N0 channels of video compression data used for simulation testing into N1 groups.

[0136] Here, the N0 video compression data is used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition angles during the same time period.

[0137] The specific schemes for obtaining N0 channels of compressed video data and for obtaining N1 groups can be found in the descriptions above, and will not be repeated here.

[0138] Step S702: The simulation device sends the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules.

[0139] Step S703: The simulation device uses the i-th decompression module among the N1 decompression modules to determine the ni frames of compressed data belonging to the same time point from the ni (ni is a positive integer greater than or equal to 1 and less than N0) of video compressed data contained in the i-th group of the N1 groups.

[0140] Here, each video compression data packet in the ni-channel video compression data contains multiple frames of compressed data.

[0141] Step S704: The simulation device uses the i-th decompression module to decompress the ni frame compressed data belonging to the same time point and decompress them to obtain ni simulation images (these ni simulation images are used to simulate the environmental images captured by the ni acquisition devices of the vehicle at the same time point), so as to obtain N0 simulation images decompressed by N1 decompression modules and used to simulate the environmental images of the vehicle at the same time point.

[0142] For example, in one example, the i-th decompression module can be used to serially decompress the compressed data of frames ni belonging to the same time point.

[0143] In other words, in one example, for the i-th group among the N1 groups obtained from the decompression, the i-th decompression module in the simulation device first identifies the ni frames of compressed video data belonging to the same time point within the ni channels of compressed video data contained in the i-th group. Then, it decompresses these ni frames (e.g., serial decompression) to obtain ni simulated images belonging to the same time point, simulating the environmental images captured by the ni acquisition devices of the vehicle at the same time point. Similarly, the simulated images obtained after decompression by the other decompression modules can be obtained. Thus, using N1 decompression modules, N0 simulated images belonging to the same time point can be obtained. This effectively utilizes the high decompression performance of the decompression modules, reduces the time cost required for the decompression process, and improves the efficiency of decompression.

[0144] In one example, after the decompression module obtains a simulation image, it can first store the simulation image in its own buffer. When the first-time alignment is required, it can perform the first-time alignment of the simulation images cached in the buffer that belong to the same time point, and then send them directly to the board without going through the simulation device. In this way, the storage pressure of the simulation device is effectively reduced.

[0145] Step S705: The simulation equipment sends N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules, so as to use N2 boards to send the simulation images belonging to the same time point to the vehicle for simulation testing.

[0146] For example, in one instance, after determining that N0 simulation images belonging to the same time point have completed the first time alignment, the simulation device uses N1 decompression modules to synchronously send the N0 simulation images belonging to the same time point to N2 boards. The N2 boards are then used to perform a second time alignment of the simulation images belonging to the same time point, and after the second time alignment, they are synchronously sent to the vehicle for simulation testing.

[0147] Thus, this disclosed solution provides a specific scheme for decompressing multi-channel video compressed data using multiple decompression modules. In this scheme, N0 simulated images belonging to the same time point can be efficiently decompressed and sent directly to the board by N1 decompression modules. The board then synchronously sends these N0 simulated images to the vehicle. This effectively reduces the time cost required for the decompression process, improves the efficiency of decompression, and lays the foundation for achieving efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras). It also lays the foundation for improving the efficiency of vehicle simulation testing.

[0148] For example, Figure 8 A schematic diagram of the data link used for simulating and testing the acquisition device of an autonomous driving HIL system is provided. Here, the simulation device (such as an industrial control computer) integrates multiple GPUs and multiple video injection boards, and the video injection boards (such as FPGAs) are plugged into the Peripheral Component Interconnect Express (PCIE) slot of the industrial control computer.

[0149] Specifically, such as Figure 8 As shown, the simulation test process can specifically include:

[0150] First, the simulation equipment, such as an industrial control computer, obtains the total compressed data for simulation testing from the data center. Specifically, the total compressed data can be the video compressed data of road tests stored in the data center in Record format. The simulation equipment splits the obtained total compressed data into multiple video compressed data streams according to the number of acquisition devices to be simulated (e.g., GMSL cameras), that is, it obtains the aforementioned N0-channel video compressed data. For example, it splits the video streams into High-Level Video Coding (H.264) / High-Efficiency Video Coding (H.265). According to the number of GPUs, the N0-channel video compressed data is split into N1 groups, and different groups are sent to different GPUs. For example, the compressed video stream is transmitted to the GPU through a PCIe link.

[0151] Secondly, for each GPU, for example, for the i-th GPU (i takes values ​​from 1 to N1) out of N1 GPUs, multiple frame compressed data belonging to the same time point can be determined based on the received i-th group, and then the multiple frame compressed data belonging to the same time point can be decompressed serially; here, in one example, the decompressed simulation image can be stored in the GPU's cache area in YUV (Y represents the luminance (Luma) component, and U and V represent the chrominance (Chroma) components) format.

[0152] Furthermore, after receiving the time signal (also known as the synchronization signal) from the sync2 card, each GPU (e.g., the 1st to the N1st GPU) can time-align all simulation images belonging to the same time point. Then, after time alignment (i.e., in step with the synchronization signal), it is sent to the video injection board connected to it via a video interface (e.g., DisplayPort, DP, or High-Definition Multimedia Interface, HDMI). For example, for the i-th GPU, after time alignment, it is sent to the i-th video injection board via the video interface.

[0153] Finally, each video injection board, for example, for the i-th video injection board, can convert the received simulation image data format (e.g., YUV format) into a matching format according to the actual interface protocol of the camera to be simulated (e.g., GMSL interface protocol, or the interface protocol of an LCD or projector). For example, for simulating a real GMSL camera, the received simulation image data format (e.g., YUV format) can be converted into GMSL format, and then the GMSL format simulation image is cached in its own cache unit. Furthermore, if each of the N1 video injection boards determines that the current capacity occupied by its own cache unit is the maximum capacity, it is assumed that each video injection board has obtained all simulation images belonging to the same time point, and the data format of all simulation images belonging to the same time point is GMSL format. Then, each video injection board will time-align all simulation images of the same time point in GMSL format in its own cache unit and send them to the vehicle (e.g., an autonomous vehicle or an assisted driving vehicle) so that the vehicle can conduct simulation testing.

[0154] This approach effectively utilizes multiple GPUs for parallel decompression, significantly reducing the time cost of the decompression process compared to existing solutions, improving decompression efficiency, and enabling efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras). For example, it achieves precise simulation of data formats and timing, and also enables precise control of GMSL camera content, thus laying the foundation for improving the efficiency of vehicle simulation testing.

[0155] Figure 9 This is an illustrative flowchart of a simulation data processing method according to an embodiment of this application. Figure 3 This method can be optionally applied to electronic devices, such as personal computers, servers, and server clusters. It is understood that the above... Figures 1 to 8 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0156] Furthermore, the method includes at least a portion of the following: For example... Figure 9 As shown, it includes:

[0157] Step S901: The simulation equipment splits the N0 channels of video compression data used for simulation testing into N1 groups.

[0158] Here, the N0 video compression data is used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition angles during the same time period.

[0159] Step S902: The simulation device sends the i-th group among the N1 groups to the i-th decompression module among the N1 decompression modules.

[0160] Here, N1 decompression modules can be used to decompress and obtain N0 simulation images for simulating the environment around the vehicle at the same point in time.

[0161] Step S903: The simulation device uses the i-th decompression module to split the ni-channel video compressed data contained in the i-th group into Mi data blocks.

[0162] Here, each data block contains ni frames of compressed data belonging to the same time point, and the time points corresponding to the frame compressed data contained in different data blocks are different.

[0163] Furthermore, in one example, the value of Mi is related to the number of frame compressed data contained in the video compressed data. For example, the value of Mi is less than or equal to the number of video frames captured per unit time. In another example, the value of Mi is equal to the number of video frames captured per unit time.

[0164] For example, the compressed video data contained in the i-th group includes compressed data of multiple frames (also called multiple video frames) captured by a single acquisition device within a unit of time. Here, the unit of time can contain multiple time points; for example, in a scenario where 10 frames can be captured per second, this unit of time contains 10 different time points. Specifically, as... Figure 10 As shown, the i-th group contains 3 channels of video compressed data, where each channel of video compressed data includes 10 frames of compressed images captured within 1 second. Based on the time points of different frames in each channel of video compressed data, frames belonging to the same time point can be divided into a data block. For example, the 11th frame of the first channel of video compressed data belonging to time point 1, the 21st frame of the second channel of video compressed data belonging to time point 1, and the 31st frame of the third channel of video compressed data belonging to time point 1 can be divided into the first data block; the 12th frame of the first channel of video compressed data belonging to time point 2, the 22nd frame of the second channel of video compressed data belonging to time point 2, and the 32nd frame of the third channel of video compressed data belonging to time point 2 can be divided into the second data block; similarly, the third, ..., the 9th, and the 10th data blocks can be obtained. This allows the i-th decompression module to decompress compressed data at the same time point, further ensuring time synchronization and providing strong support for simulating video link timing. Moreover, it can greatly reduce the latency of image data transmission, thus laying the foundation for accurate simulation of high-pixel and high-frequency cameras. At the same time, it also lays the foundation for enabling autonomous driving systems to respond to environmental changes more quickly and make accurate decisions.

[0165] Step S904: The simulation device uses the i-th decompression module to select the j-th data block from the Mi data blocks based on timing rules.

[0166] Here, the j-th data block is at least one of the currently undecompressed data blocks among the Mi data blocks. Furthermore, the j-th data block can be the oldest data block among all currently undecompressed data blocks in the Mi data blocks.

[0167] Step S905: The simulation device uses the i-th decompression module to serially decompress the ni frame compressed data in the j-th data block, so as to use the i-th decompression module to serially decompress all data blocks in the Mi data blocks, and then uses N1 decompression modules to obtain N0 simulation images belonging to the same time point.

[0168] Thus, by using N1 decompression modules, N0 simulation images can be obtained after decompression, which are used to simulate the environment around the vehicle at the same point in time.

[0169] Step S906: The simulation equipment sends N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules, so as to use N2 boards to synchronously send the simulation images belonging to the same time point to the vehicle for simulation testing.

[0170] For example, in one instance, after determining that N0 simulation images belonging to the same time point have completed the first time alignment, the simulation device sends the N0 simulation images belonging to the same time point to N2 boards through N1 decompression modules. The N2 boards are then used to perform a second time alignment of the simulation images belonging to the same time point, and after the second time alignment, they are synchronously sent to the vehicle for simulation testing.

[0171] In this way, the disclosed solution can use the i-th decompression module to split the compressed video data contained in the i-th group according to time points to obtain Mi data blocks belonging to different time points. This facilitates the i-th decompression module to quickly decompress the compressed data belonging to the same time point, thereby quickly obtaining all simulation images belonging to the same time point to simulate the environmental scene around the vehicle. In this way, multiple decompression modules can be used quickly and orderly to decompress compressed data at different time points, thus laying the foundation for efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras) and improving the efficiency of vehicle simulation testing.

[0172] Further, in a specific example, the j-th data block can be decompressed serially as follows: Specifically, the simulation device described above utilizes the i-th decompression module to serially decompress the ni-th frame compressed data in the j-th data block (for example, step S905 described above), specifically including:

[0173] Step S905-1: The simulation device uses the i-th decompression module to decompress the p-th frame compressed data in the j-th data block.

[0174] Here, the compressed data of the p-th frame is one of the uncompressed compressed data of the selected j-th data block.

[0175] Step S905-2: After decompressing and obtaining the simulation image of the p-th frame compressed data, store the simulation image of the p-th frame compressed data in the buffer area of ​​the i-th decompression module, and decompress the p+1-th frame compressed data.

[0176] Here, the compressed data of the p+1th frame is one of the uncompressed compressed data of the Mith data block.

[0177] In other words, the i-th decompression module decompresses the currently uncrushed frame compressed data in the j-th data block, and after decompression is complete, the i-th decompression module decompresses the next uncrushed frame compressed data. This serial decompression process quickly yields the simulation image. For example, continuing with... Figure 10 For example, for the first data block, the i-th decompression module can first decompress the 11th frame compressed image. After obtaining the simulation image of the 11th frame compressed image and storing it in the buffer area of ​​the i-th decompression module, the i-th decompression module continues to decompress the 21st frame compressed image in the first data block, and so on, until the last frame (i.e. the 31st frame) compressed image in the first data block is decompressed.

[0178] In this way, the present invention can use the decompression module to serially decompress compressed data of different frames in the same data block. This greatly saves the resource usage of the decompression module during decompression and improves the processing efficiency of the decompression module, thereby ensuring the quality of the simulation image obtained after decompression. This lays the foundation for the subsequent realization of efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras).

[0179] In a specific example, after the i-th decompression module completes the decompression of the j-th data block, it can continue to decompress the next data block, thus serially completing the decompression process of all data blocks; for example, after the simulation device uses the i-th decompression module to serially decompress the ni frames of compressed data in the j-th data block, it may further include:

[0180] After the frame compressed data of the j-th data block is decompressed, the ni frame compressed data in the (j+1)-th data block is serially decompressed using the ith decompression module. This process is repeated until all data blocks are successfully decompressed.

[0181] Here, the (j+1)th data block is at least one of the currently undecompressed data blocks among the Mi data blocks. Furthermore, the (j+1)th data block can be the oldest data block among all currently undecompressed data blocks in the Mi data blocks.

[0182] Continue with Figure 10For example, after decompressing and obtaining the three simulated images corresponding to the first data block, the second data block can be decompressed. For instance, the i-th GPU decompresses the 12th frame compressed image in the second data block. After decompressing and obtaining the three simulated images corresponding to the second data block, the third data block will be decompressed. For instance, the i-th GPU decompresses the 13th frame compressed image in the third data block. This process is repeated until all data blocks are decompressed.

[0183] In this way, the proposed solution utilizes the i-th decompression module to decompress different data blocks sequentially in a serial manner. This reduces the pressure on data transmission and effectively improves the processing efficiency of the decompression module, thereby fully leveraging its high decompression performance. This ensures the quality of the simulation image obtained after decompression and lays the foundation for subsequent efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras).

[0184] This disclosure also provides a simulation device, such as... Figure 11 As shown, it includes:

[0185] The data preprocessing unit 1101 is used to split the N0 channels of video compressed data used for simulation testing into N1 groups, wherein the N0 channels of video compressed data are used to simulate the environmental images captured by N0 acquisition devices in the vehicle from different acquisition angles at the same time; N0 and N1 are both positive integers greater than or equal to 2;

[0186] The simulation test unit 1102 is used to send the i-th group of the N1 groups to the i-th decompression module of the N1 decompression modules, so as to obtain N0 simulation images of the vehicle's environment at the same time point after decompression using the N1 decompression modules; and to send the N0 simulation images belonging to the same time point to N2 boards through the N1 decompression modules, so as to send the simulation images belonging to the same time point to the vehicle for simulation testing using the N2 boards; wherein, N2 is a positive integer greater than or equal to 1.

[0187] In a specific example of the scheme disclosed herein, the simulation test unit is further configured to:

[0188] Align the N0 simulation images belonging to the same time point in the first time.

[0189] After the initial alignment, N0 simulation images belonging to the same time point are synchronously sent to N2 boards through N1 decompression modules.

[0190] In a specific example of the disclosed solution, the simulation test unit is specifically used for:

[0191] Using N1 decompression modules synchronized by clock, N0 simulation images belonging to the same time point are aligned in the first time. The N1 decompression modules are clock synchronized using the CPU of the simulation device.

[0192] or,

[0193] A time synchronization device is used to send a time signal to the N1 decompression modules, so that the N1 decompression modules perform a first time alignment on the N0 simulation images belonging to the same time point after receiving the time signal.

[0194] In a specific example of the scheme disclosed herein, the alignment accuracy of the N1 decompression modules synchronized by clock for first-time alignment of the N0 simulation images is lower than the alignment accuracy of the N0 simulation images for first-time alignment using the time signal.

[0195] In a specific example of the scheme disclosed herein, the simulation test unit is further configured to:

[0196] The simulation images belonging to the same time point are aligned in a second time using N2 boards. After the second time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using N2 boards.

[0197] In a specific example of the disclosed solution, the simulation test unit is specifically used for:

[0198] Determine whether the total cache units corresponding to the N2 boards contain all simulation images belonging to the same time point; wherein, the total cache units corresponding to the N2 boards represent the set of cache units of all boards in the N2 boards;

[0199] Given that all simulation images belonging to the same time point are stored, N2 boards are used to perform a second time alignment on all simulation images belonging to the same time point, and after the second time alignment, all simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing.

[0200] In a specific example of the disclosed solution, the simulation test unit is specifically used for:

[0201] Given that the current capacity of the buffer units of each of the N2 boards has reached its maximum capacity, it is determined that the total buffer units of the N2 boards contain all the simulation images belonging to the same time point. The maximum capacity of the buffer units of the boards is determined based on the number of simulation images belonging to the same time point that the boards need to receive, which is related to the total number of video compression data streams in the simulation test.

[0202] In a specific example of the scheme disclosed herein, the time alignment accuracy that the N0 simulated images can achieve by performing the first time alignment is lower than the time alignment accuracy that the N0 simulated images can achieve by performing the second time alignment.

[0203] In a specific example of the scheme disclosed herein, the simulation test unit is further configured to:

[0204] Before using N2 boards to perform a second time alignment on the simulation images belonging to the same time point, the data format of N0 simulation images belonging to the same time point is converted into a preset format using N2 boards.

[0205] In a specific example of the scheme disclosed herein, the preset format is the data format output by the acquisition device of the vehicle being simulated and tested.

[0206] In a specific example of the scheme disclosed herein, the data format output by the vehicle's acquisition device is Global Motion Blur Suppression Logic (GMSL).

[0207] In one specific example of the scheme disclosed herein, the board is one of the following: a field-programmable gate array (FPGA) or a chip capable of implementing video injection requirements.

[0208] In a specific example of the disclosed scheme, the number N2 of the boards is related to at least one of the following:

[0209] It is related to the number N0 of data acquisition devices in the simulation test;

[0210] It is related to the number of video output interfaces of the decompression module;

[0211] It is related to the number of decompression modules, N1.

[0212] In a specific example of the scheme disclosed herein, the simulation test unit is further configured to:

[0213] Using the i-th decompression module, ni frame compressed data belonging to the same time point are determined from the ni channels of video compressed data contained in the i-th group, wherein each video compressed data packet in the ni channels of video compressed data contains multiple frame compressed data.

[0214] The i-th decompression module is used to decompress the ni frame compressed data belonging to the same time point to obtain ni simulation images.

[0215] In a specific example of the scheme disclosed herein, the simulation test unit is further configured to:

[0216] Using the i-th decompression module, the ni-channel video compressed data contained in the i-th group are split into Mi data blocks. Each data block contains ni frame compressed data belonging to the same time point. The time points corresponding to the frame compressed data contained in different data blocks are different. The value of Mi is related to the number of frame compressed data contained in the video compressed data.

[0217] In a specific example of the disclosed solution, the simulation test unit is specifically used for:

[0218] Using the i-th decompression module, based on timing rules, the j-th data block is selected from the Mi data blocks; wherein the j-th data block is at least one of the currently undecompressed data blocks among the Mi data blocks;

[0219] The i-th decompression module is used to decompress the ni frames of compressed data in the j-th data block in sequence.

[0220] In a specific example of the disclosed solution, the simulation test unit is specifically used for:

[0221] The i-th decompression module is used to decompress the p-th frame compressed data in the j-th data block; wherein the p-th frame compressed data is one of the undecompressed frame compressed data in the j-th data block.

[0222] After decompressing to obtain the simulation image of the p-th frame compressed data, the simulation image of the p-th frame compressed data is stored in the buffer area of ​​the i-th decompression module, and the p+1-th frame compressed data is decompressed; the p+1-th frame compressed data is one of the undecompressed frame compressed data in the Mi-th data block.

[0223] In a specific example of the scheme disclosed herein, the simulation test unit is further configured to:

[0224] After the frame compression data of the j-th data block is decompressed, the ni frame compression data in the (j+1)-th data block are serially decompressed using the ith decompression module, so as to serially decompress each data block; wherein the (j+1)-th data block is at least one of the currently undecompressed data blocks among the Mi data blocks.

[0225] In a specific example of the scheme disclosed herein, the data preprocessing unit is further configured to:

[0226] Obtain total compressed data for simulation testing; wherein, the total compressed data is used to simulate environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives within a preset time period;

[0227] Based on at least N0 acquisition devices required for simulation testing, the total compressed data is split to obtain the N0 channels of video compressed data.

[0228] In a specific example of the disclosed solution, the data preprocessing unit is specifically used for:

[0229] Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed contains frame compressed data of the video frames that can be collected by the N0 acquisition devices within a unit time period.

[0230] Select one set of data that meets the time requirement from the multiple sets of data to be decompressed;

[0231] Based on the N0 acquisition devices required for simulation testing, a set of data to be decompressed that meets the time requirements is split into the N0 channels of video compressed data.

[0232] For a description of the specific functions and examples of each unit of the apparatus in this disclosure embodiment, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, which will not be repeated here.

[0233] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0234] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0235] Figure 12 A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0236] like Figure 12As shown, device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1202 or a computer program loaded from storage unit 1208 into random access memory (RAM) 1203. The RAM 1203 may also store various programs and data required for the operation of device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. Input / output (I / O) interface 1205 is also connected to bus 1204.

[0237] Multiple components in device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0238] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as simulation data processing methods. For example, in some embodiments, the simulation data processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by the computing unit 1201, one or more steps of the simulation data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured to perform a simulation data processing method by any other suitable means (e.g., by means of firmware).

[0239] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0240] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0241] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0242] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0243] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0244] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0245] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0246] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A simulation data processing method, applied to a simulation device, comprising: N1 decompression modules are used to obtain N0 simulation images of the vehicle's environment at the same time point after decompression; wherein, the N0 simulation images are used to simulate the environment captured from different acquisition angles at the same time point; N0 and N1 are both positive integers greater than or equal to 2; Align the N0 simulation images belonging to the same time point in the first time. After initial alignment, N0 simulation images belonging to the same time point are synchronously sent to N2 boards through N1 decompression modules; N2 is a positive integer greater than or equal to 1. Using N2 boards, simulation images belonging to the same time point are second-time aligned, so that after the second-time alignment, the simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing using N2 boards; wherein, the time alignment accuracy that the N0 simulation images can achieve by performing the first time alignment is lower than the time alignment accuracy that the N0 simulation images can achieve by performing the second time alignment.

2. The method according to claim 1, wherein, The step of aligning N0 simulated images belonging to the same time point in the first time includes: Using N1 decompression modules synchronized by clock, N0 simulation images belonging to the same time point are aligned in the first time. The N1 decompression modules are clock synchronized using the CPU of the simulation device. or, The time synchronization device in the simulation device sends a time signal to the N1 decompression modules, so that the N1 decompression modules perform a first time alignment on the N0 simulation images belonging to the same time point after receiving the time signal.

3. The method according to claim 2, wherein, The alignment accuracy of the N0 simulation images being aligned in the first time using N1 decompression modules synchronized by clock is lower than the alignment accuracy of the N0 simulation images being aligned in the first time using the time signal.

4. The method according to any one of claims 1-3, wherein, The process of using N2 boards to perform a second time alignment on simulation images belonging to the same time point, and then using N2 boards to synchronously send the simulation images belonging to the same time point to the vehicle for simulation testing, includes: Determine whether the total cache units corresponding to the N2 boards contain all simulation images belonging to the same time point; wherein, the total cache units corresponding to the N2 boards represent the set of cache units of all boards in the N2 boards; Given that all simulation images belonging to the same time point are stored, N2 boards are used to perform a second time alignment on all simulation images belonging to the same time point, and after the second time alignment, all simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing.

5. The method according to claim 4, wherein, Determining whether the total cache units corresponding to the N2 boards contain all simulation images belonging to the same time point includes: Given that the current capacity of the buffer units of each of the N2 boards has reached its maximum capacity, it is determined that the total buffer units of the N2 boards contain all the simulation images belonging to the same time point. The maximum capacity of the buffer units of the boards is determined based on the number of simulation images belonging to the same time point that the boards need to receive, which is related to the total number of video compression data streams in the simulation test.

6. The method according to any one of claims 1-3, wherein, Before using N2 boards to perform a second time alignment on simulation images belonging to the same time point, the following is also included: The data format of N0 simulation images belonging to the same time point is converted into a preset format using N2 boards.

7. The method according to claim 6, wherein, The preset format is the data format output by the data acquisition device of the vehicle being simulated and tested.

8. The method according to claim 7, wherein, The data output by the vehicle's acquisition device is in the format of Global Motion Blur Suppression Logic (GMSL).

9. The method according to any one of claims 1-3, wherein, The board is one of the following: a field-programmable gate array (FPGA) or a chip capable of implementing video injection requirements.

10. The method according to any one of claims 1-3, wherein, The number N2 of the boards is related to at least one of the following: It is related to the number N0 of data acquisition devices in the simulation test; It is related to the number of video output interfaces of the decompression module; It is related to the number of decompression modules, N1.

11. The method according to claim 1, wherein, The N0 simulation images obtained by using N1 decompression modules to simulate the vehicle's environment at the same point in time include: The i-th group of N1 groups is sent to the i-th decompression module of N1 decompression modules, so as to obtain N0 simulation images of the vehicle's environment at the same time point after decompression using the N1 decompression modules; wherein, the N1 groups are obtained by splitting the N0 channels of video compressed data used for simulation testing.

12. The method according to claim 11, wherein, After sending the i-th group out of N1 groups to the i-th decompression module out of N1 decompression modules, the method further includes: Using the i-th decompression module, ni frame compressed data belonging to the same time point are determined from the ni channels of video compressed data contained in the i-th group, wherein each video compressed data packet in the ni channels of video compressed data contains multiple frame compressed data. The i-th decompression module is used to decompress the ni frame compressed data belonging to the same time point to obtain ni simulation images.

13. The method of claim 12, further comprising: Using the i-th decompression module, the ni-channel video compressed data contained in the i-th group are split into Mi data blocks. Each data block contains ni frame compressed data belonging to the same time point. The time points corresponding to the frame compressed data contained in different data blocks are different. The value of Mi is related to the number of frame compressed data contained in the video compressed data.

14. The method according to claim 13, wherein, The step of using the i-th decompression module to determine the ni frames of compressed data belonging to the same time point from the ni channels of compressed video data contained in the i-th group, and using the i-th decompression module to decompress the ni frames of compressed data belonging to the same time point, includes: Using the i-th decompression module, based on timing rules, the j-th data block is selected from the Mi data blocks; wherein the j-th data block is at least one of the currently undecompressed data blocks among the Mi data blocks; The i-th decompression module is used to decompress the ni frames of compressed data in the j-th data block in sequence.

15. The method according to claim 14, wherein, The step of using the i-th decompression module to serially decompress the ni frames of compressed data in the j-th data block includes: The i-th decompression module is used to decompress the p-th frame compressed data in the j-th data block; wherein the p-th frame compressed data is one of the undecompressed frame compressed data in the j-th data block. After decompressing to obtain the simulation image of the p-th frame compressed data, the simulation image of the p-th frame compressed data is stored in the buffer area of ​​the i-th decompression module, and the p+1-th frame compressed data is decompressed; the p+1-th frame compressed data is one of the undecompressed frame compressed data in the Mi-th data block.

16. The method of claim 14, further comprising: After the frame compression data of the j-th data block is decompressed, the ni frame compression data in the (j+1)-th data block are serially decompressed using the ith decompression module, so as to serially decompress each data block; wherein the (j+1)-th data block is at least one of the currently undecompressed data blocks among the Mi data blocks.

17. The method according to any one of claims 11-16, further comprising: Obtain total compressed data for simulation testing; wherein, the total compressed data is used to simulate environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives within a preset time period; Based on at least N0 acquisition devices required for simulation testing, the total compressed data is split to obtain the N0 channels of video compressed data.

18. The method according to claim 17, wherein, The total compressed data is split into N0 channels of compressed video data based on at least N0 acquisition devices required for simulation testing, including: Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed contains frame compressed data of the video frames that can be collected by the N0 acquisition devices within a unit time period. Select one set of data that meets the time requirement from the multiple sets of data to be decompressed; Based on the N0 acquisition devices required for simulation testing, a set of data to be decompressed that meets the time requirements is split into the N0 channels of video compressed data.

19. A simulation device, comprising: N1 decompression modules; N2 boards; N2 is a positive integer greater than or equal to 1. The simulation testing unit is used to obtain N0 simulated images of the vehicle's environment at the same time point using N1 decompression modules; wherein the N0 simulated images are used to simulate the environment captured from different acquisition angles at the same time point; N0 and N1 are both positive integers greater than or equal to 2; the N0 simulated images belonging to the same time point are first time-aligned; after the first time alignment, the N0 simulated images belonging to the same time point are synchronously sent to N2 boards through the N1 decompression modules; N2 is a positive integer greater than or equal to 1; the N2 boards are used to perform a second time alignment of the simulated images belonging to the same time point, so that after the second time alignment, the simulated images belonging to the same time point are synchronously sent to the vehicle for simulation testing using the N2 boards; wherein the time alignment accuracy achieved by the first time alignment of the N0 simulated images is lower than the time alignment accuracy achieved by the second time alignment of the N0 simulated images.

20. The device according to claim 19, wherein, The simulation test unit is specifically used for: Using N1 decompression modules synchronized by clock, N0 simulation images belonging to the same time point are aligned in the first time. The N1 decompression modules are clock synchronized using the CPU of the simulation device. or, A time synchronization device is used to send a time signal to the N1 decompression modules, so that the N1 decompression modules perform a first time alignment on the N0 simulation images belonging to the same time point after receiving the time signal.

21. The device according to claim 20, wherein, The alignment accuracy of the N0 simulation images being aligned in the first time using N1 decompression modules synchronized by clock is lower than the alignment accuracy of the N0 simulation images being aligned in the first time using the time signal.

22. The device according to any one of claims 19-21, wherein, The simulation test unit is specifically used for: Determine whether the total cache units corresponding to the N2 boards contain all simulation images belonging to the same time point; wherein, the total cache units corresponding to the N2 boards represent the set of cache units of all boards in the N2 boards; Given that all simulation images belonging to the same time point are stored, N2 boards are used to perform a second time alignment on all simulation images belonging to the same time point, and after the second time alignment, all simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing.

23. The device according to claim 22, wherein, The simulation test unit is specifically used for: Given that the current capacity of the buffer units of each of the N2 boards has reached its maximum capacity, it is determined that the total buffer units of the N2 boards contain all the simulation images belonging to the same time point. The maximum capacity of the buffer units of the boards is determined based on the number of simulation images belonging to the same time point that the boards need to receive, which is related to the total number of video compression data streams in the simulation test.

24. The device according to any one of claims 19-21, wherein, The simulation test unit is also used for: Before using N2 boards to perform a second time alignment on the simulation images belonging to the same time point, the data format of N0 simulation images belonging to the same time point is converted into a preset format using N2 boards.

25. The device according to claim 24, wherein, The preset format is the data format output by the data acquisition device of the vehicle being simulated and tested.

26. The device according to claim 25, wherein, The data output by the vehicle's acquisition device is in the format of Global Motion Blur Suppression Logic (GMSL).

27. The device according to any one of claims 19-21, wherein, The board is one of the following: a field-programmable gate array (FPGA) or a chip capable of implementing video injection requirements.

28. The device according to any one of claims 19-21, wherein, The number N2 of the boards is related to at least one of the following: It is related to the number N0 of data acquisition devices in the simulation test; It is related to the number of video output interfaces of the decompression module; It is related to the number of decompression modules, N1.

29. The device according to claim 19, wherein, The simulation test unit is specifically used for: The i-th group of N1 groups is sent to the i-th decompression module of N1 decompression modules, so as to obtain N0 simulation images of the vehicle's environment at the same time point after decompression using the N1 decompression modules; wherein, the N1 groups are obtained by splitting the N0 channels of video compressed data used for simulation testing.

30. The device according to claim 29, wherein, The simulation test unit is also used for: Using the i-th decompression module, ni frame compressed data belonging to the same time point are determined from the ni channels of video compressed data contained in the i-th group, wherein each video compressed data packet in the ni channels of video compressed data contains multiple frame compressed data. The i-th decompression module is used to decompress the ni frame compressed data belonging to the same time point to obtain ni simulation images.

31. The device according to claim 30, wherein, The simulation test unit is also used for: Using the i-th decompression module, the ni-channel video compressed data contained in the i-th group are split into Mi data blocks. Each data block contains ni frame compressed data belonging to the same time point. The time points corresponding to the frame compressed data contained in different data blocks are different. The value of Mi is related to the number of frame compressed data contained in the video compressed data.

32. The device according to claim 31, wherein, The simulation test unit is specifically used for: Using the i-th decompression module, based on timing rules, the j-th data block is selected from the Mi data blocks; wherein the j-th data block is at least one of the currently undecompressed data blocks among the Mi data blocks; The i-th decompression module is used to decompress the ni frames of compressed data in the j-th data block in sequence.

33. The device according to claim 32, wherein, The simulation test unit is specifically used for: The i-th decompression module is used to decompress the p-th frame compressed data in the j-th data block; wherein the p-th frame compressed data is one of the undecompressed frame compressed data in the j-th data block. After decompressing to obtain the simulation image of the p-th frame compressed data, the simulation image of the p-th frame compressed data is stored in the buffer area of ​​the i-th decompression module, and the p+1-th frame compressed data is decompressed; the p+1-th frame compressed data is one of the undecompressed frame compressed data in the Mi-th data block.

34. The device according to claim 32, wherein, The simulation test unit is also used for: After the frame compression data of the j-th data block is decompressed, the ni frame compression data in the (j+1)-th data block are serially decompressed using the ith decompression module, so as to serially decompress each data block; wherein the (j+1)-th data block is at least one of the currently undecompressed data blocks among the Mi data blocks.

35. The device according to any one of claims 29-34, wherein, Also includes: Data preprocessing unit; The data preprocessing unit is used for: Obtain total compressed data for simulation testing; wherein, the total compressed data is used to simulate environmental images captured by N0 acquisition devices in the vehicle from different acquisition perspectives within a preset time period; Based on at least N0 acquisition devices required for simulation testing, the total compressed data is split to obtain the N0 channels of video compressed data.

36. The device according to claim 35, wherein, The data preprocessing unit is specifically used for: Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed contains frame compressed data of the video frames that can be collected by the N0 acquisition devices within a unit time period. Select one set of data that meets the time requirement from the multiple sets of data to be decompressed; Based on the N0 acquisition devices required for simulation testing, a set of data to be decompressed that meets the time requirements is split into the N0 channels of video compressed data.

37. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-18.

38. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-18.

39. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-18.

Images