Simulation data processing method and device and storage medium

Abstract

The invention provides a simulation data processing method and device and a storage medium, and relates to the technical field of image processing, in particular to the technical fields of automatic driving, auxiliary driving, automatic parking, intelligent traffic and the like. According to the specific implementation scheme, N paths of video compression data used for simulation testing are obtained, and the N paths of video compression data are used for simulating environment images captured by N collection devices in a vehicle from different collection view angles in the same time period; each path of video compression data in the N paths of video compression data comprises a plurality of pieces of frame compression data; determining a plurality of frame compression data belonging to the same time point from different paths of video compression data based on the time information of each path of video compression data; and sending the plurality of frame compression data belonging to the same time point to a graphics processing unit (GPU) for decompression to obtain a plurality of simulation images for simulating the environment picture of the vehicle at the same time point.

Description

[0001] This application is a divisional application of a Chinese case with the invention name “Simulation data processing method, device and storage medium”, application number “202411000734.X” and application date of July 24, 2024. Technical Field

[0002] The present disclosure relates to the field of image processing technology, and in particular to technical fields such as autonomous driving, assisted driving, automatic parking, and smart transportation. Background Art

[0003] Camera simulation, such as that for Global Motion Suppression Logic (GMSL) cameras, is essential for autonomous driving hardware-in-the-loop (HIL) simulation systems. This allows for effective testing of the entire video processing chain within autonomous vehicle control systems. Therefore, efficient camera simulation is a key research priority. Summary of the Invention

[0004] The present disclosure provides a simulation data processing method, device and storage medium.

[0005] According to one aspect of the present disclosure, a simulation data processing method is provided, which is applied to a simulation device, comprising:

[0006] Acquire N channels of compressed video data for a simulation test, wherein the N channels of compressed video data are used to simulate environmental images captured by N acquisition devices in a vehicle from different acquisition perspectives during the same period of time; each channel of compressed video data in the N channels of compressed video data contains multiple frames of compressed data;

[0007] Based on the time information of each channel of video compression data, multiple frame compression data belonging to the same time point are determined from different channels of video compression data;

[0008] Multiple frames of compressed data belonging to the same time point are sent to a graphics processing unit (GPU) for decompression to obtain multiple simulation images for simulating the environment of the vehicle at the same time point.

[0009] According to another aspect of the present disclosure, there is provided a simulation device, comprising:

[0010] a data determination unit configured to obtain N channels of compressed video data for simulation testing, wherein the N channels of compressed video data are used to simulate environmental images captured by N acquisition devices in a vehicle from different acquisition perspectives during the same period of time; each channel of compressed video data in the N channels of compressed video data contains a plurality of frames of compressed data;

[0011] The data processing unit is used to determine multiple frames of compressed data belonging to the same time point from different channels of video compressed data based on the time information of each channel of video compressed data; and send the multiple frames of compressed data belonging to the same time point to the graphics processing unit (GPU) for decompression to obtain multiple simulated images for simulating the vehicle's environmental scenes at the same time point.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method in the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements any one of the methods according to the embodiments of the present disclosure.

[0018] In this way, the disclosed solution can send multiple frame compressed data belonging to the same time point in each video compression data to the GPU for decompression, and then obtain multiple simulated images for simulating the environmental scenes around the vehicle at the same time point, effectively utilizing the high decompression performance of the GPU. Compared with the existing decompression method, the disclosed solution effectively reduces the time cost required for the decompression process and improves the decompression efficiency. This lays the foundation for achieving efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), and also lays the foundation for improving the simulation test efficiency of vehicles.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0021] Figure 1This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 1 ;

[0022] Figure 2 is a schematic diagram of a simulation scenario corresponding to the total compressed data according to an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of a simulation scenario corresponding to total compressed data according to another embodiment of the present application;

[0024] Figure 4 This is a schematic diagram of a scenario in which a simulation device processes total compressed data according to an embodiment of the present application. Figure 1 ;

[0025] Figure 5 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 2 ;

[0026] Figure 6 This is a schematic diagram of a scenario in which a simulation device processes total compressed data according to an embodiment of the present application. Figure 2 ;

[0027] Figure 7 This is a schematic diagram of a scenario in which a simulation device processes total compressed data according to an embodiment of the present application. Figure 3 ;

[0028] Figure 8 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 3 ;

[0029] Figure 9 is an illustrative schematic diagram of splitting each channel of video compression data into multiple data blocks according to an embodiment of the present application;

[0030] Figure 10 is a structural diagram of a simulation device according to an embodiment of the present application;

[0031] Figure 11 It is a block diagram of an electronic device used to implement the simulation data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0033] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them, and do not mean to limit the order or to limit to only two. For example, the first feature and the second feature refer to two categories / two features. The first feature can be one or more, and the second feature can also be one or more.

[0034] In addition, numerous specific details are provided in the following detailed description to better illustrate the present disclosure. Those skilled in the art will appreciate that the present disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main points of the present disclosure.

[0035] The following describes the related technologies of the embodiments of the present disclosure. The following related technologies are optional solutions that can be combined with the technical solutions of the embodiments of the present disclosure in any way, and all of them fall within the protection scope of the embodiments of the present disclosure.

[0036] Camera simulation, such as that of a GMSL camera, is an essential component of autonomous driving hardware-in-the-loop (HIL) simulation systems. This allows for effective testing of the entire video processing chain within the autonomous vehicle control system. For example, it can simulate video content and test the timing of the video chain.

[0037] For example, using traditional field-programmable gate array (FPGA) simulations of GMSL cameras can effectively achieve accurate data format and timing simulation. During simulation, compressed video data can be decompressed using the FPGA or the CPU of the simulation device. However, as camera pixels and frame rates increase, the decompression time required by the FPGA or CPU increases, decompression performance degrades, and can even fail to meet the camera's pixel and frame rate requirements, resulting in a severe drop in frame rate. Therefore, efficient simulation of GMSL cameras is a key research topic.

[0038] Based on this, the disclosed solution proposes a simulation data processing method to improve the decompression performance of video compressed data and achieve efficient simulation of high-pixel, high-frequency cameras.

[0039] Specifically, Figure 1 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0040] Furthermore, the method includes at least part of the following contents. Figure 1 As shown, the simulation data processing method is applied to a simulation device (e.g., an industrial computer), comprising:

[0041] Step S101: The simulation device obtains N channels of video compression data for simulation testing.

[0042] Here, the N channels of compressed video data are used to simulate the environmental images captured by N capture devices in a vehicle from different capture perspectives within the same time period. For example, the simulation is used to simulate the environmental images captured by N capture devices from different capture perspectives within the same time period (e.g., per unit time) while the vehicle is traveling on its road.

[0043] Furthermore, in one example, different acquisition devices in the vehicle have different acquisition perspectives, so that the vehicle's environmental images (such as the environmental images around the vehicle) can be captured by different acquisition devices, providing support for driving decisions.

[0044] Furthermore, in one example, each of the N channels of video compression data contains multiple frames of compressed data. For example, in one specific example, one channel of video compression data may include at least compressed data of multiple video frames captured by a single acquisition device within a unit of time (e.g., 1 second) (herein, compressed data of video frames may be referred to as frame compressed data for short), for example, compressed data of 10 frames of images captured by a single acquisition device within 1 second.

[0045] Here, the number of video frames captured per unit time is related to relevant parameters of the capture device to be simulated, such as the frame rate, and the present disclosure does not impose any restrictions on this.

[0046] Step S102: The simulation device determines a plurality of frame compressed data belonging to the same time point from different channels of video compressed data based on the time information of each channel of video compressed data.

[0047] Here, in this example, the time information of the video compression data may specifically include the timestamp of each frame compressed data in the video compression data; further, the "time point" can be specifically understood as the simulated moment. For example, if it is necessary to simulate the environmental scene captured by the vehicle from different acquisition perspectives at moment A, it is necessary to determine the multiple frame compressed data belonging to the moment A based on the timestamps of each channel of video compression data, for example, determine the multiple frame compressed data with a timestamp of time A.

[0048] Step S103: The simulation device sends a plurality of frames of compressed data belonging to the same time point to the graphics processor GPU for decompression, so as to obtain a plurality of simulation images for simulating the environment scene of the vehicle at the same time point.

[0049] In this way, the disclosed solution can send multiple frame compressed data belonging to the same time point in each video compression data to the GPU for decompression, and then obtain multiple simulated images for simulating the environmental scenes around the vehicle at the same time point, effectively utilizing the high decompression performance of the GPU. Compared with the existing decompression method, the disclosed solution effectively reduces the time cost required for the decompression process and improves the decompression efficiency. This lays the foundation for achieving efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), and also lays the foundation for improving the simulation test efficiency of vehicles.

[0050] Furthermore, because the disclosed solution utilizes a GPU for decompression, it can preserve image details to the greatest extent possible during the decompression process, avoiding image information loss due to decompression. This provides strong support for autonomous driving systems to respond more quickly to environmental changes and make accurate decisions. Furthermore, the GPU decompression solution used in the disclosed solution can support larger datasets, further improving the reliability and accuracy of simulation tests.

[0051] Here, it should be noted that the vehicle described in the present disclosure may be specifically an unmanned vehicle or an assisted driving vehicle, and the present disclosure does not impose any specific restrictions on this.

[0052] In a specific example, N channels of compressed video data may be obtained in the following manner. Specifically, the above-mentioned obtaining of N channels of compressed video data for simulation testing (e.g., step S101 described above) specifically includes:

[0053] Step S101 - 1 : The simulation device obtains total compressed data for simulation testing.

[0054] Here, the total compressed data is used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives within a preset time period.

[0055] Furthermore, in one example, the preset time period includes a plurality of unit times.

[0056] Step S101 - 2 : splitting the total compressed data based on at least N acquisition devices required for simulation test to obtain the N channels of video compression data.

[0057] For example, in one example, the above-mentioned splitting of the total compressed data based on at least the N acquisition devices required for the simulation test to obtain the N channels of video compressed data may specifically include:

[0058] Step S101 - 2 - 1 : Based on the multiple unit times included in the preset time period, split the total compressed data into multiple groups of data to be decompressed.

[0059] Here, the number of groups of data to be decompressed is the same as the number of unit times included in the preset time period. Furthermore, each group of data to be decompressed includes frame compressed data of video frames that can be captured by the N capture devices within a unit time, that is, includes all frame compressed data that can be captured by the N capture devices within a unit time.

[0060] That is to say, in this step, the simulation device can split the total compressed data containing multiple unit times according to unit time to obtain multiple groups of data to be decompressed, so that each group of data to be decompressed contains all the frame compressed data that can be collected by the N acquisition devices within a unit time.

[0061] Step S101-2-2: Select a group of data to be decompressed that meets the time requirement from the multiple groups of data to be decompressed.

[0062] Step S101 - 2 - 3 : Based on the N acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N channels of video compression data.

[0063] For example, in one example, multiple groups of data to be decompressed are sorted in chronological order, and the earliest group of data to be decompressed is selected from the sorting results. According to the multiple acquisition devices to be simulated, the earliest group of data to be decompressed is further split to obtain the N channels of video compression data.

[0064] For example, in one example, Figure 2As shown, the unmanned vehicle or assisted driving vehicle to be simulated and tested is equipped with acquisition devices at 6 different positions, a total of 6, which can be respectively recorded as acquisition device 1, acquisition device 2, acquisition device 3, acquisition device 4, acquisition device 5 and acquisition device 6; at this time, the total compressed data can include 6 channels of video compression data, and each channel of video compression data is used to simulate the environmental picture captured by an acquisition device in the vehicle. For example, the first channel of video compression data is used to simulate the environmental picture captured by acquisition device 1, the second channel of video compression data is used to simulate the environmental picture captured by acquisition device 2,..., the sixth channel of video compression data is used to simulate the environmental picture captured by acquisition device 6.

[0065] Furthermore, in one example, the total compressed data is used to simulate the environmental images captured by the N capture devices from different capture perspectives within a unit time (e.g., 1 second). Accordingly, each channel of video compressed data obtained by directly splitting the total compressed data can directly simulate the environmental images captured by a capture device within a unit time (e.g., 1 second) and from its own capture perspective.

[0066] Or, in another example, Figure 3As shown, the total compressed data can be specifically used to simulate the environmental scenes captured by the N acquisition devices from different acquisition perspectives within a preset time period (for example, the preset time period includes L unit times, in this case, they can be respectively recorded as the 1st time unit, the 2nd time unit, ..., the Lth time unit in the preset time period, where L is a positive integer greater than or equal to 2). In this scenario, the total compressed data can be first split into L groups of data to be decompressed according to unit time, which can be respectively recorded as the 1st group of data to be decompressed corresponding to the 1st time unit in the preset time period (for example, the 1st group of data to be decompressed corresponding to the 1st second in the preset time period), the 2nd group of data to be decompressed corresponding to the 2nd time unit in the preset time period (the 2nd group of data to be decompressed corresponding to the 2nd second in the preset time period), ..., the Lth group of data to be decompressed corresponding to the Lth time unit in the preset time period (the data to be decompressed corresponding to the last second in the preset time period). Here, each group of data to be decompressed is used to simulate the environmental scenes captured by the N acquisition devices from different acquisition perspectives within a unit time; further, Based on N acquisition devices, a group of data to be decompressed is further split and divided into N channels of video compression data. For example, for the first group of data to be decompressed corresponding to the first second in the preset time period, it can be split into N channels of video compression data. At this time, they can be recorded as the first channel of video compression data corresponding to the first second in the preset time period, the second channel of video compression data corresponding to the first second in the preset time period, ..., the Nth channel of video compression data corresponding to the first second in the preset time period. At this time, each channel of video compression data is used to simulate the environmental picture captured by an acquisition device from its own acquisition perspective within a unit of time. In this way, it provides strong support for the subsequent rapid and orderly use of GPU decompression to obtain simulated images.

[0067] It should be noted that in actual applications, the total compressed data obtained can be split in the device dimension based on N acquisition devices to obtain multiple groups of data to be decompressed. At this time, each group of data to be decompressed is used to simulate the environmental picture captured by an acquisition device from its own acquisition perspective within a preset time period (that is, the multiple unit times mentioned above), and then the multiple groups of data to be decompressed are processed from the time dimension according to the unit time to obtain the N channels of video compression data mentioned above.

[0068] It should be noted that in actual applications, Figure 4 As shown, a simulation device, such as an industrial computer, is integrated with a GPU. Furthermore, the industrial computer obtains the total compressed data for simulation testing from a data center, and splits the total compressed data to obtain N channels of video compression data, and then sends multiple frame compressed data belonging to the same time point in different channels of video compression data to (for example, serially sent to) the GPU of the industrial computer for decompression.

[0069] In this way, the disclosed solution provides a specific solution for obtaining N-channel video compression data. The solution is simple and efficient, and can quickly split the total compressed data into N-channel video compression data according to the time sequence. This provides strong support for the subsequent rapid and orderly use of GPU decompression to obtain simulated images.

[0070] Figure 5 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters and other electronic devices. It is understood that the above Figures 1 to 4 The relevant contents of the method shown can also be applied to this example, and this example will not elaborate on the relevant contents.

[0071] Furthermore, the method includes at least part of the following contents. Figure 5 Shown, including:

[0072] Step S501: The simulation device obtains N channels of video compression data for simulation testing.

[0073] Here, the N channels of video compression data are used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives during the same period of time; each channel of video compression data in the N channels of video compression data contains multiple frames of compressed data.

[0074] Here, the specific solution for obtaining N channels of video compression data can be found in the above description and will not be repeated here.

[0075] Step S502: The simulation device determines a plurality of frame compressed data belonging to the same time point from different channels of video compressed data based on the time information of each channel of video compressed data.

[0076] Step S503: The simulation device sends a plurality of frames of compressed data belonging to the same time point to the graphics processor GPU for decompression, so as to obtain a plurality of simulation images for simulating the environment scene of the vehicle at the same time point.

[0077] Here, in one example, after decompressing a simulation image, the GPU may first send the simulation image to the simulation device for storage, for example, Figure 6 As shown, the simulation image successfully decompressed by the GPU is cached in the cache area of ​​the simulation device.

[0078] Step S504: when multiple simulation images belonging to the same time point are obtained after decompression, the simulation device sends the multiple simulation images belonging to the same time point to the board.

[0079] Step S505: The simulation device uses the board card to perform a first time alignment on multiple simulation images belonging to the same time point, so as to synchronously send them to the vehicle for simulation testing after the first time alignment.

[0080] For example, in one example, after the simulation device (such as the CPU of the simulation device) determines that all the frame compressed data belonging to the same time point are decompressed successfully, Figure 6 As shown, when the simulation device determines that all simulation images belonging to the same time point are stored in its cache area, it sends all simulation images belonging to the same time point to the board card, and uses the board card to perform a first time alignment on all simulation images belonging to the same time point, and then sends them out synchronously after the first time alignment.

[0081] It should be noted that in actual scenarios, the reception time of each simulated image received by the board from the simulation device may be different. Therefore, in order to ensure that all simulated images belonging to the same time point can be sent synchronously and the vehicle can receive the simulated images synchronously (that is, to ensure the time consistency of the simulated images received by the vehicle), so as to successfully complete the test of the simulation link timing and / or the test of the simulated video content, in this example, the simulation device can also use the board to perform a first time alignment on multiple simulated images belonging to the same time point, and after the first time alignment, use the board to send multiple simulated images belonging to the same time point to the vehicle synchronously, thereby ensuring the time consistency of the simulated images received by the vehicle.

[0082] It can be understood that the above-mentioned time alignment operation (i.e., the first time alignment operation) may not change the timestamp of the simulation image, but only align the simulation images received at different times, so that all simulation images received at different times (also called different time points) but belonging to the same time point can be sent at the same time (i.e., sent synchronously).

[0083] In this way, the disclosed solution uses the board card to perform a first time alignment on multiple simulated images at the same time point after decompression, so that multiple simulated images at the same time point can be sent to the vehicle synchronously, effectively improving the time synchronization of the decompressed image data in transmission, thereby providing strong support for reducing the delay of the simulated images received by the vehicle, and at the same time, providing strong support for achieving accurate simulation of high-pixel and high-frequency cameras.

[0084] Furthermore, in a specific example, the simulation device described above uses a board card to perform a first time alignment on multiple simulation images belonging to the same time point, so as to synchronously send the images to the vehicle for simulation testing after the first time alignment (for example, step S505 described above), which specifically includes:

[0085] Step S505 - 1 : The simulation device determines whether all simulation images belonging to the same time point are stored in the cache unit of the board.

[0086] Here, it should be noted that the board in this example may refer to the general term for the boards integrated in the simulation device. Accordingly, the cache unit of the board in this example may specifically be the total cache unit of all boards integrated in the simulation device.

[0087] For example, in one example, the simulation device may be integrated into a board card. In this case, after the simulation device sends the simulation images belonging to the same time point to the board card, the simulation images belonging to the same time point may be stored in the cache unit of the board card. Accordingly, step S505-1 may be specifically: determining whether all the simulation images belonging to the same time point are stored in the cache unit of the board card.

[0088] Alternatively, in another example, the simulation device may integrate two or more boards. In this case, the simulation device may send each simulation image belonging to the same time point to multiple boards. Accordingly, each simulation image belonging to the same time point may be stored in the cache units of different boards. Accordingly, step S505-1 may be specifically: determining whether all simulation images belonging to the same time point are stored in the total cache unit of all boards.

[0089] For example, take the case where there are 6 acquisition devices that need to be simulated tested (that is, a total of 6 channels of video compression data) and the simulation device is connected to 2 boards. At this time, in one example, each board can be used to receive 3 channels of decompressed simulation images and store the received simulation images in its respective cache unit; accordingly, the simulation device can determine whether the total cache unit of the two boards stores 6 simulation images belonging to the same time point.

[0090] It should be noted that, in one example, in order to store simulation images at different time points, multiple cache units can be set in the board. At this time, simulation images at different time points can be cached in different cache units, so that only images belonging to the same time point are cached in the same cache unit. This provides effective support for efficiently and synchronously sending all simulation images belonging to the same time point.

[0091] Step S505-2: When it is determined that all simulation images belonging to the same time point are stored, the simulation device uses the board card to perform a first time alignment on all simulation images belonging to the same time point, and synchronously sends them to the vehicle for simulation testing after the first time alignment.

[0092] For example, continuing with the example of a simulation test in which there are a total of 6 acquisition devices and the simulation devices are connected to 2 boards, after determining that the total cache unit of the 2 boards stores 6 simulation images belonging to the same time point, the 2 boards can be used to perform a first time alignment on the 6 simulation images at the same time point. For example, each board can be used to perform a first time alignment on the 3 simulation images in its own cache unit. At this time, it can be considered that all simulation images belonging to the same time point have completed the first time alignment. Alternatively, the two boards can use a time synchronization signal to perform a first time alignment on the 6 simulation images in the total cache unit, and after the first time alignment, the 6 simulation images can be synchronously sent to the vehicle through the 2 boards for simulation testing.

[0093] In this way, the disclosed solution can use the board to synchronously send all simulated images belonging to the same time point to the vehicle, thereby effectively improving the time synchronization of the decompressed image data in transmission, thereby providing strong support for effectively reducing the delay of the simulated images received by the vehicle, and at the same time, it also provides strong support for achieving accurate simulation of high-pixel and high-frequency cameras.

[0094] Furthermore, in a specific example, the simulation device described above determines whether all simulation images belonging to the same time point are stored in the cache unit of the board (for example, step S505-1 described above), specifically including:

[0095] When it is determined that the current occupied capacity of the cache unit of the board card reaches the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the cache unit of the board card.

[0096] Here, the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data in the simulation test.

[0097] It should be noted that, in this example, the board can also be a general term for the boards integrated into the simulation device, and accordingly, the cache unit of the board can refer to the total cache unit of all the boards integrated into the simulation device. At this time, "the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data in the simulation test" can be understood as: the maximum capacity of the total cache unit of the board is determined based on the total number of channels of video compression data in the simulation test. For example, in one example, the maximum capacity of the total cache unit of the board = the total number of channels of video compression data, further, the maximum capacity of the total cache unit of the board is also equal to the number of acquisition devices required for simulation, and further, it is also equal to the number of simulated images belonging to the same time point.

[0098] Furthermore, in the case where the simulation device is integrated with multiple boards, the maximum capacity of the cache unit of each board can be determined based on the total number of channels of video compression data in the simulation test and the total number of boards.

[0099] For example, if there are a total of 6 acquisition devices required for simulation testing, then there are a total of 6 simulation images belonging to the same time point. At this time, the maximum capacity of the total cache unit of the board can be set to accommodate 6 simulation images. In other words, the sum of the maximum capacity of the cache units of each board (that is, the total maximum capacity of the set of cache units of each board) can accommodate the 6 simulation images. Furthermore, if the simulation device is integrated with 2 boards, then in one example, the maximum capacity of the cache unit of each board can accommodate 3 simulation images; at this time, if it is determined that the current occupied capacity of the cache unit of each board has reached the maximum capacity, it can be considered that all simulation images belonging to the same time point have been stored in the cache unit of the board. In this way, all simulation images belonging to the same time point can be synchronized and sent out after the first time alignment.

[0100] It should be noted that the above is only an exemplary description. In actual applications, the maximum capacity of the cache units in different boards may be the same or different, and the disclosed solution does not limit this.

[0101] In this way, the disclosed solution provides a specific solution for quickly determining whether the board has received all the simulated images belonging to the same time point. This provides strong support for ensuring that the vehicle can synchronously receive all the simulated images belonging to the same time point. At the same time, it also provides strong support for further improving the time synchronization accuracy, thereby laying the foundation for the subsequent smooth completion of the simulation link timing test and / or the simulation video content test.

[0102] Here, in one example, the aforementioned board can be specifically a video injection board. Furthermore, in one example, the video injection board is one of the following: an FPGA or a chip capable of implementing video injection requirements. In other words, the disclosed solution is compatible with existing video injection boards, thus expanding the scope of application of the disclosed solution and making it both practical and applicable.

[0103] Furthermore, in a specific example, the sending of multiple simulation images belonging to the same time point to the board mentioned above may specifically include:

[0104] The central processing unit (CPU) of the simulation device is used to perform second time alignment on multiple simulation images belonging to the same time point, and the images are synchronously sent to the board after the second time alignment.

[0105] That is to say, before the simulation device sends multiple simulation images belonging to the same time point (for example, all simulation images belonging to the same time point) to the board, the central processing unit CPU of the simulation device can first perform a second time alignment on the multiple simulation images belonging to the same time point, and then send them to the board synchronously after the second time alignment. In this way, the image data belonging to the same time point is guaranteed to be second-time aligned at the CPU level, thereby providing support for ensuring the accuracy of image data synchronization. At the same time, it also lays the foundation for subsequently reducing the time cost required for the board to perform the first time alignment of the image data.

[0106] Here, it should be noted that, in one example, the time alignment accuracy that can be achieved by the central processing unit CPU for the multiple simulation images by performing the second time alignment is lower than the time alignment accuracy that can be achieved by the board card for the multiple simulation images by performing the first time alignment. In this way, in the disclosed scheme, multiple simulation images belonging to the same time point will first use the CPU of the simulation device to perform a low alignment accuracy (such as millisecond level) time alignment (also called "coarse" granularity time alignment), and then the board card will perform a high alignment accuracy (such as microsecond level) time alignment (also called "fine" granularity time alignment) on the multiple simulation images after the "coarse" granularity time alignment. After the "fine" granularity time alignment, all simulation images belonging to the same time point will be synchronously sent to the vehicle for simulation testing. In this way, the time synchronization of the image data is further ensured, and the delay in the transmission of the image data is greatly reduced, laying the foundation for the accurate simulation of high-pixel and high-frequency cameras. At the same time, it also lays the foundation for the autonomous driving system to respond to environmental changes more quickly and make accurate decisions.

[0107] It should be noted here that, in one example, the bandwidth of the board's data receiving link is greater than the bandwidth of the board's data sending link. This further lays the foundation for reducing the delay in image data transmission and thus achieving accurate simulation of high-pixel and high-frequency cameras.

[0108] Furthermore, in a specific example, before using the board to perform a first time alignment on multiple simulation images belonging to the same time point, the method further includes:

[0109] The board is used to convert the data formats of multiple simulation images belonging to the same time point into a preset format.

[0110] In other words, in one example, after receiving multiple simulated images from a simulation device at the same time, the board can first convert the data format of these multiple simulated images into a preset format, then perform a first time alignment on these multiple simulated images at the same time in the preset format, and then synchronously transmit them after the first time alignment. This effectively simulates the required data format and enhances data compatibility, ensuring that the receiving end, such as the vehicle, can process the received image data accordingly, laying the foundation for efficient simulation of high-pixel, high-frequency cameras.

[0111] Furthermore, in one example, the preset format is the data format output by the acquisition device of the vehicle under simulation test. For example, the preset format is the format output by the GMSL format acquisition device of the vehicle under simulation test, that is, the GMSL format. In this way, accurate and efficient simulation of the GMSL camera is achieved, laying the foundation for subsequent improvement of the performance stability of the entire autonomous driving system.

[0112] For example, Figure 7 This paper provides a schematic diagram of the data link used to simulate the data acquisition equipment of an autonomous driving HIL system. Here, the simulation device (such as an industrial computer) integrates a GPU and CPU, and a board (such as an FPGA injection board) is plugged into the peripheral component interconnect express (PCIE) card slot of the industrial computer.

[0113] Specifically, if Figure 7As shown, first, the simulation device, such as an industrial computer, obtains the total compressed data for simulation testing from the data center. For example, the total compressed data can be specifically a compressed video stream obtained by compression using High Efficiency Video Coding (HEVC). The simulation device splits the obtained total compressed data into multiple channels of video compression data according to the number of acquisition devices (such as GMSL cameras) to be simulated, that is, the N channels of video compression data mentioned above are obtained; secondly, the CPU of the industrial computer determines multiple frame compression data belonging to the same time point from different channels of video compression data based on the time information of each channel of video compression data, and sends the multiple frame compression data belonging to the same time point in series to the GPU for decompression using the GPU. Here, the frame compression data sent to the GPU can also be specifically a compressed video stream compressed by HEVC; after the GPU obtains the simulation image after decompression, it is directly sent to the cache area of ​​the industrial computer for temporary storage. Here, the decompressed simulation image can be in YUV (Y represents the luminance (Luma) component, and U and V represent the chrominance (Chroma) components) format. Furthermore, when the industrial computer (such as the CPU) determines that the cache area obtains all the simulated images at the same time point, it performs a second time alignment on all the simulated images belonging to the same time point and sends them to the FPGA injection board; accordingly, the FPGA injection board converts the data format of the received simulated image (such as the YUV format) into the GMSL format to simulate a real GMSL camera, and caches the simulated image in the GMSL format into its own cache unit. Further, when the FPGA injection board determines that the current occupied capacity of the cache unit is the maximum capacity, it is considered that all the simulated images belonging to the same time point have been obtained, and the data format of all the simulated images belonging to the same time point is in the GMSL format, and then all the simulated images at the same time point in the GMSL format in the cache unit are first time aligned, and after the first time alignment, they are synchronously sent to the vehicle to facilitate simulation testing of the vehicle. In this way, the high decompression performance of the GPU is effectively utilized. Compared with existing solutions, the time cost required for the decompression process is effectively reduced, the decompression efficiency is improved, and efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras) is achieved. For example, accurate data format and timing simulation is achieved. At the same time, precise control of GMSL camera content is also achieved, laying the foundation for improving the efficiency of vehicle simulation testing.

[0114] Figure 8 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 3 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters and other electronic devices. It is understood that the above Figures 1 to 7The relevant contents of the method shown can also be applied to this example, and this example will not elaborate on the relevant contents.

[0115] Furthermore, the method includes at least part of the following contents. Figure 8 Shown, including:

[0116] Step S801: The simulation device obtains N channels of video compression data for simulation testing.

[0117] Here, the N channels of video compression data are used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives during the same period of time; each channel of video compression data in the N channels of video compression data contains multiple frames of compressed data.

[0118] Here, the specific solution for obtaining N channels of video compression data can be found in the above description and will not be repeated here.

[0119] Step S802: The simulation device splits the N channels of video compression data into M data blocks based on the time information of each channel of video compression data.

[0120] Here, each data block contains a plurality of frame compressed data belonging to the same time point, and the frame compressed data contained in different data blocks correspond to different time points.

[0121] Furthermore, in one example, the value of M is related to the number of frame compressed data contained in the video compression data. In other words, the value of M is related to the number of video frames collected per unit time. For example, the value of M is less than or equal to the number of video frames collected per unit time. Furthermore, in one example, the value of M is equal to the number of video frames collected per unit time.

[0122] For example, each channel of video compression data includes compressed data of multiple frames of images (also called multiple video frames) collected by an acquisition device within a unit time. Here, the unit time can include multiple time points. For example, for a scene where 10 frames of images can be collected in 1 second, the unit time includes 10 different time points. Specifically, Figure 9As shown, each of the 6 channels of video compression data includes 10 frames of compressed images collected within 1 second. At this time, the frame compressed images belonging to the same time point can be divided into a data block according to the time points of different frame compressed images in each channel of video compression data. For example, the 11th frame compressed image belonging to time point 1 in the 1st channel of video compression data, the 21st frame compressed image belonging to time point 1 in the 2nd channel of video compression data, ..., the 61st frame compressed image belonging to time point 1 in the 6th channel of video compression data are divided into the 1st data block; the 12th frame compressed image belonging to time point 2 in the 1st channel of video compression data, the 22nd frame compressed image belonging to time point 2 in the 2nd channel of video compression data, ..., the 62nd frame compressed image belonging to time point 2 in the 6th channel of video compression data are divided into the 2nd data block; similarly, the 3rd data block, ..., the 9th data block and the 10th data block can be obtained. This allows the GPU to decompress compressed data at the same point in time, further ensuring time synchronization and providing strong support for simulating video link timing. It can also significantly reduce the delay in image data transmission, laying the foundation for accurate simulation of high-pixel and high-frequency cameras. It also lays the foundation for autonomous driving systems to respond more quickly to environmental changes and make accurate decisions.

[0123] Step S803: The simulation device selects the i-th data block from the M data blocks based on a timing rule.

[0124] Here, the i-th data block is at least one of the currently undecompressed data blocks in the M data blocks. Further, the i-th data block may be the earliest data block among all the currently undecompressed data blocks in the M data blocks.

[0125] Step S804: the simulation device sends the multiple frames of compressed data in the i-th data block to the GPU in series for decompression, so as to obtain multiple simulation images for simulating the environment scene of the vehicle at the same time point.

[0126] In this way, the disclosed solution can utilize the time information of each channel of video compression data to split N channels of video compression data according to time points to obtain M data blocks belonging to different time points, and then send multiple frame compressed data in each data block serially to the GPU for decompression. In this way, the GPU memory overflow is effectively avoided during decompression, and at the same time, the foundation is laid for the subsequent rapid and orderly use of the GPU to decompress frame compressed data at different time points.

[0127] Furthermore, in a specific example, multiple frames of compressed data may be sent serially in the following manner; specifically, the simulation device described above serially sends the multiple frames of compressed data in the i-th data block to the GPU for decompression (for example, step S804 described above), specifically including:

[0128] Step S804 - 1 : The simulation device sends the j-th frame compressed data in the i-th data block to the GPU for decompression.

[0129] Here, the j-th frame compressed data is one of the uncompressed frame compressed data in the selected i-th data block.

[0130] Step S804-2: After decompressing to obtain the simulated image of the j-th frame compressed data, for example, after the simulation device receives the response information of the decompression completion from the GPU, or after the simulation device receives the simulated image of the j-th frame compressed data, the j+1-th frame compressed data in the i-th data block is sent.

[0131] Here, the j+1th frame compressed data is one of the uncompressed frame compressed data in the i-th data block.

[0132] That is, the currently undecompressed frame compressed data in the i-th data block is sent to the GPU for decompression, and after the decompression is completed, the next undecompressed frame compressed data is sent to the GPU for decompression, and thus serial decompression is performed to quickly obtain the simulated image. Figure 9 For example, for the first data block, the 11th frame compressed image can be sent to the GPU for decompression first. After obtaining the simulated image of the 11th frame compressed image after decompression, the 21st frame compressed image in the first data block is sent to the GPU for decompression, and so on, until the last frame (i.e., the 61st frame) compressed image in the first data block is sent to the GPU.

[0133] In this way, the disclosed solution sends compressed data of different frames in the same data block serially to the GPU for decompression. This greatly saves the resource usage of the GPU during decompression, while improving the processing efficiency of GPU decompression, thereby ensuring the quality of the simulated image obtained after decompression, and laying the foundation for the subsequent efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras).

[0134] In a specific example, after the decompression of the i-th data block is completed, it can be sent to the GPU for decompression in the following manner. For example, after the above-mentioned step S804, the following may be specifically included:

[0135] Step S805: After determining that the frame compressed data of the i-th data block is decompressed, multiple frame compressed data in the i+1-th data block are sent serially to the GPU for decompression. This cycle is repeated, and each data block is sent serially to the GPU for decompression until the frame compressed data of all data blocks are successfully decompressed.

[0136] Here, the (i+1)th data block is at least one of the currently undecompressed data blocks in the M data blocks. Further, the (i+1)th data block may be the earliest data block among all the currently undecompressed data blocks in the M data blocks.

[0137] Continue with Figure 9 For example, when 6 simulation images corresponding to the first data block are obtained after decompression, the second data block will be decompressed, for example, the 12th frame compressed image in the second data block will be sent to the GPU for decompression; when 6 simulation images corresponding to the second data block are obtained after decompression, the third data block will be decompressed, for example, the 13th frame compressed image in the third data block will be sent to the GPU for decompression, and so on until the last (i.e., the 10th) data block.

[0138] In this way, the disclosed solution sends different data blocks to the GPU in a serial manner for decompression, thereby reducing the pressure of data transmission and effectively improving the processing efficiency of the GPU, thereby giving full play to the high decompression performance of the GPU, thereby ensuring the quality of the simulated image obtained after decompression, and laying the foundation for the subsequent efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras).

[0139] The disclosed solution also provides a simulation device, such as Figure 10 Shown, including:

[0140] The data determination unit 1001 is configured to obtain N channels of compressed video data for a simulation test, wherein the N channels of compressed video data are used to simulate environmental images captured by N acquisition devices in a vehicle from different acquisition perspectives during the same period of time; each channel of the N channels of compressed video data contains multiple frames of compressed data;

[0141] The data processing unit 1002 is used to determine multiple frames of compressed data belonging to the same time point from different channels of video compressed data based on the time information of each channel of video compressed data; and send the multiple frames of compressed data belonging to the same time point to the graphics processor GPU for decompression to obtain multiple simulated images for simulating the environmental scenes of the vehicle at the same time point.

[0142] In a specific example of the present disclosure,

[0143] The data processing unit is also used to send multiple simulation images belonging to the same time point to the board card when multiple simulation images belonging to the same time point are obtained after decompression; use the board card to perform a first time alignment on the multiple simulation images belonging to the same time point, and send them to the vehicle for simulation testing after the first time alignment.

[0144] In a specific example of the disclosed solution, the data processing unit is specifically configured to:

[0145] Determine whether all simulation images belonging to the same time point are stored in the cache unit of the board;

[0146] When it is determined that all simulation images belonging to the same time point are stored, the board is used to perform a first time alignment on all simulation images belonging to the same time point, and after the first time alignment, they are synchronously sent to the vehicle for simulation testing.

[0147] In a specific example of the disclosed solution, the data processing unit is specifically configured to:

[0148] When it is determined that the current occupied capacity of the cache unit of the board has reached the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the cache unit of the board; wherein the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data of the simulation test.

[0149] In a specific example of the disclosed solution, the data processing unit is specifically configured to:

[0150] A central processing unit (CPU) is used to perform second time alignment on multiple simulation images belonging to the same time point, and the images are synchronously sent to the board after the second time alignment.

[0151] In a specific example of the disclosed solution, the time alignment accuracy that the central processing unit CPU can achieve for the multiple simulation images by performing the second time alignment is lower than the time alignment accuracy that the board can achieve for the multiple simulation images by performing the first time alignment.

[0152] In a specific example of the disclosed solution, the data processing unit is further configured to:

[0153] Before performing the first time alignment on the multiple simulation images belonging to the same time point by using the board card, the data formats of the multiple simulation images belonging to the same time point are converted into a preset format by using the board card.

[0154] In a specific example of the disclosed solution, the preset format is the data format output by the acquisition device of the vehicle under simulation test.

[0155] In a specific example of the disclosed solution, the data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

[0156] In a specific example of the disclosed solution, the board is one of the following: a field programmable gate array (FPGA) or a chip capable of achieving video injection requirements.

[0157] In a specific example of the disclosed solution, the data processing unit is further configured to:

[0158] Based on the time information of each channel of video compression data, the N channels of video compression data are split into M data blocks, wherein each data block contains multiple frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of M is related to the number of frame compression data contained in the video compression data.

[0159] In a specific example of the present disclosure, the data processing unit is specifically configured to:

[0160] Based on a time sequence rule, selecting an i-th data block from the M data blocks; wherein the i-th data block is at least one of the M data blocks that is currently not decompressed;

[0161] The multiple frames of compressed data in the i-th data block are serially sent to the GPU for decompression.

[0162] In a specific example of the disclosed solution, the data processing unit is specifically configured to:

[0163] Sending the j-th frame compressed data in the i-th data block to the GPU for decompression; wherein the j-th frame compressed data is one of the undecompressed frame compressed data in the selected i-th data block;

[0164] After obtaining the simulated image of the jth frame compressed data after decompression, the j+1th frame compressed data in the ith data block is sent; the j+1th frame compressed data is one of the uncompressed frame compressed data in the ith data block.

[0165] In a specific example of the disclosed solution, the data processing unit is further configured to:

[0166] After determining that decompression of the frame compressed data of the i-th data block is completed, multiple frame compressed data in the i+1-th data block are sent serially to the GPU for decompression, and each data block is sent serially to the GPU for decompression; wherein the i+1-th data block is at least one of the currently uncompressed data blocks in the M data blocks.

[0167] In a specific example of the disclosed solution, the data determination unit is specifically configured to:

[0168] Obtaining total compressed data for simulation testing; wherein the total compressed data is used to simulate environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives within a preset time period;

[0169] The total compressed data is split based on at least N acquisition devices required for simulation test to obtain the N channels of video compression data.

[0170] In a specific example of the disclosed solution, the data determination unit is specifically configured to:

[0171] Based on the multiple unit times included in the preset time period, the total compressed data is split into multiple groups of to-be-decompressed data, wherein each group of to-be-decompressed data includes frame compressed data of video frames that can be captured by the N capture devices within the unit time;

[0172] Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data;

[0173] Based on the N acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N channels of video compression data.

[0174] For the description of specific functions and examples of each unit of the device in the embodiment of the present disclosure, please refer to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0175] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0176] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0177] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0178] like Figure 11As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. Various programs and data required for the operation of the device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0179] Various components in device 1100 are connected to I / O interface 1105, including an input unit 1106, such as a keyboard and mouse; an output unit 1107, such as various types of displays and speakers; a storage unit 1108, such as a magnetic disk and optical disk; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0180] The computing unit 1101 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the simulation data processing method. For example, in some embodiments, the simulation data processing method can be implemented as a computer software program that is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the simulation data processing method described above can be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the simulation data processing method in any other appropriate manner (eg, by means of firmware).

[0181] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0182] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0183] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0184] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0185] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0186] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0187] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0188] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A simulation data processing method, applied to a simulation device, comprising: Sending multiple frames of compressed data belonging to the same time point to a graphics processor (GPU) for decompression to obtain multiple simulation images for simulating the environment of the vehicle at the same time point; In the case where multiple simulation images belonging to the same time point are obtained after decompression, the multiple simulation images belonging to the same time point are sent to the board; The board is used to perform first time alignment on multiple simulation images belonging to the same time point, so that they are synchronously sent to the vehicle for simulation testing after the first time alignment.

2. The method according to claim 1, wherein: The method of using the board to perform a first time alignment on a plurality of simulation images belonging to the same time point, so as to synchronously send the images to the vehicle for simulation testing after the first time alignment, includes: Determine whether all simulation images belonging to the same time point are stored in the cache unit of the board; When it is determined that all simulation images belonging to the same time point are stored, the board is used to perform a first time alignment on all simulation images belonging to the same time point, and after the first time alignment, they are synchronously sent to the vehicle for simulation testing.

3. The method according to claim 2, wherein: The step of determining whether all simulation images belonging to the same time point are stored in the cache unit of the board includes: When it is determined that the current occupied capacity of the cache unit of the board has reached the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the cache unit of the board; wherein the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data of the simulation test.

4. The method according to claim 1, wherein: The sending of multiple simulation images belonging to the same time point to the board includes: The central processing unit (CPU) of the simulation device is used to perform second time alignment on a plurality of simulation images belonging to the same time point, and the plurality of simulation images are synchronously sent to the board after the second time alignment.

5. The method according to claim 4, wherein: The time alignment accuracy that can be achieved by the central processing unit CPU for the multiple simulation images by performing the second time alignment is lower than the time alignment accuracy that can be achieved by the board card for the multiple simulation images by performing the first time alignment.

6. The method according to any one of claims 1 to 5, wherein: Before using the board to perform the first time alignment on a plurality of simulation images belonging to the same time point, the method further includes: The board is used to convert the data formats of multiple simulation images belonging to the same time point into a preset format.

7. The method according to claim 6, wherein: The preset format is the data format output by the acquisition device of the vehicle under simulation test.

8. The method according to claim 7, wherein: The data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

9. The method according to claim 6, wherein: The board is one of the following: a field programmable gate array FPGA or a chip capable of realizing video injection requirements.

10. The method according to any one of claims 1 to 5, further comprising: A plurality of frame compressed data belonging to the same time point is determined, wherein the plurality of frame compressed data belonging to the same time point is determined from different channels of video compressed data among N channels of video compressed data, and each channel of video compressed data among the N channels of video compressed data contains a plurality of frame compressed data.

11. The method according to claim 10, further comprising: Based on the time information of each channel of video compression data in the N channels of video compression data, the N channels of video compression data are split into M data blocks, wherein each data block contains multiple frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of M is related to the number of frame compression data contained in the video compression data.

12. The method according to claim 11, wherein: The step of determining the plurality of frames of compressed data belonging to the same time point includes: Based on a time sequence rule, selecting an i-th data block from the M data blocks; wherein the i-th data block is at least one of the M data blocks that is not currently decompressed; The step of sending the compressed data of multiple frames belonging to the same time point to a graphics processor (GPU) for decompression includes: The multiple frames of compressed data in the i-th data block are sent serially to the GPU for decompression.

13. The method according to claim 12, wherein: The step of serially sending the plurality of frame compressed data in the i-th data block to the GPU for decompression includes: Sending the j-th frame compressed data in the i-th data block to the GPU for decompression; wherein the j-th frame compressed data is one of the undecompressed frame compressed data in the selected i-th data block; After decompressing the simulated image of the j-th frame compressed data, the j+1-th frame compressed data in the ith data block is sent; the j+1-th frame compressed data is one of the undecompressed frame compressed data in the ith data block.

14. The method according to claim 12, further comprising: After determining that the frame compressed data of the i-th data block has been decompressed, multiple frame compressed data in the i+1-th data block are sent serially to the GPU for decompression, and each data block is sent serially to the GPU for decompression; wherein the i+1-th data block is at least one of the currently uncompressed data blocks in the M data blocks.

15. The method according to claim 10, further comprising: Obtaining total compressed data for simulation testing; wherein the total compressed data is used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives within a preset period of time; The total compressed data is split based on at least N acquisition devices required for simulation test to obtain N channels of video compression data.

16. The method according to claim 15, wherein: The total compressed data is split based on at least N acquisition devices required for simulation test to obtain N channels of video compression data, including: Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N collection devices within the unit time; Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data; Based on the N acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N channels of video compression data.

17. A simulation device comprising: Graphics Processing Unit GPU; Boards; A data processing unit, used for sending a plurality of frame compressed data belonging to the same time point to a graphics processor GPU for decompression, so as to obtain a plurality of simulation images for simulating the environment of the vehicle at the same time point; In the case where multiple simulation images belonging to the same time point are obtained after decompression, the multiple simulation images belonging to the same time point are sent to the board; The board is used to perform first time alignment on multiple simulation images belonging to the same time point, so that they are synchronously sent to the vehicle for simulation testing after the first time alignment.

18. The apparatus according to claim 17, wherein: The data processing unit is specifically used for: Determine whether all simulation images belonging to the same time point are stored in the cache unit of the board; When it is determined that all simulation images belonging to the same time point are stored, the board is used to perform a first time alignment on all simulation images belonging to the same time point, and after the first time alignment, they are synchronously sent to the vehicle for simulation testing.

19. The apparatus according to claim 18, wherein: The data processing unit is specifically used for: When it is determined that the current occupied capacity of the cache unit of the board has reached the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the cache unit of the board; wherein the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data of the simulation test.

20. The apparatus of claim 17, wherein: The data processing unit is specifically used for: The central processing unit (CPU) is used to perform second time alignment on a plurality of simulation images belonging to the same time point, and the images are synchronously sent to the board after the second time alignment.

21. The apparatus according to claim 20, wherein: The time alignment accuracy that can be achieved by the central processing unit CPU for the multiple simulation images by performing the second time alignment is lower than the time alignment accuracy that can be achieved by the board card for the multiple simulation images by performing the first time alignment.

22. The apparatus according to any one of claims 17 to 21, wherein: The data processing unit is further used for: Before the board card is used to perform the first time alignment on the multiple simulation images belonging to the same time point, the board card is used to convert the data formats of the multiple simulation images belonging to the same time point into a preset format.

23. The apparatus of claim 22, wherein: The preset format is the data format output by the acquisition device of the vehicle under simulation test.

24. The apparatus of claim 23, wherein: The data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

25. The apparatus of claim 22, wherein: The board is one of the following: a field programmable gate array FPGA or a chip capable of realizing video injection requirements.

26. The apparatus according to any one of claims 17 to 21, wherein: The data processing unit is further used for: A plurality of frame compressed data belonging to the same time point is determined, wherein the plurality of frame compressed data belonging to the same time point is determined from different channels of video compressed data among N channels of video compressed data, and each channel of video compressed data among the N channels of video compressed data contains a plurality of frame compressed data.

27. The apparatus of claim 26, wherein: The data processing unit is further used for: Based on the time information of each channel of video compression data in the N channels of video compression data, the N channels of video compression data are split into M data blocks, wherein each data block contains multiple frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of M is related to the number of frame compression data contained in the video compression data.

28. The apparatus of claim 27, wherein: The data processing unit is specifically used for: Based on a time sequence rule, selecting an i-th data block from the M data blocks; wherein the i-th data block is at least one of the M data blocks that is not currently decompressed; The multiple frames of compressed data in the i-th data block are sent serially to the GPU for decompression.

29. The apparatus of claim 28, wherein: The data processing unit is specifically used for: Sending the j-th frame compressed data in the i-th data block to the GPU for decompression; wherein the j-th frame compressed data is one of the undecompressed frame compressed data in the selected i-th data block; After decompressing the simulated image of the j-th frame compressed data, the j+1-th frame compressed data in the ith data block is sent; the j+1-th frame compressed data is one of the undecompressed frame compressed data in the ith data block.

30. The apparatus of claim 28, wherein: The data processing unit is further used for: After determining that the frame compressed data of the i-th data block has been decompressed, multiple frame compressed data in the i+1-th data block are sent serially to the GPU for decompression, and each data block is sent serially to the GPU for decompression; wherein the i+1-th data block is at least one of the currently uncompressed data blocks in the M data blocks.

31. The apparatus of claim 26, wherein: Also includes: data determination unit; Among them, the data determination unit is used to obtain the total compressed data for simulation testing; wherein the total compressed data is used to simulate the environmental scenes captured by N acquisition devices in the vehicle from different acquisition perspectives within a preset time period; the total compressed data is split based on at least the N acquisition devices required for the simulation test to obtain N channels of video compression data.

32. The apparatus of claim 31, wherein: The data determination unit is specifically used for: Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N collection devices within the unit time; Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data; Based on the N acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N channels of video compression data.

33. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 16.

34. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-16.

35. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 16.

Images