Simulation data processing method, device and storage medium

Abstract

The present disclosure provides a simulation data processing method, device and storage medium, which relate to the field of image processing technology, especially to the technical fields of autonomous driving, assisted driving, automatic parking and smart transportation. The specific implementation scheme is: obtain N channels of video compression data for simulation testing, wherein the N channels of video compression data are used to simulate the environmental scenes captured by N acquisition devices in the vehicle from different acquisition perspectives in the same period of time; each channel of video compression data in the N channels of video compression data contains multiple frame compression data; based on the time information of each channel of video compression data, multiple frame compression data belonging to the same time point are determined from different channels of video compression data; the multiple frame compression data belonging to the same time point are sent to the graphics processor GPU for decompression to obtain multiple simulation images for simulating the environmental scenes of the vehicle at the same time point.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to technical fields such as autonomous driving, assisted driving, automatic parking, and smart transportation. Background Art

[0002] In the hardware-in-the-loop (HIL) simulation system of autonomous driving, the simulation of the camera, such as the simulation of the Global Motion Suppression Logic (GMSL) camera, is an essential part. Through the simulation of the camera, the entire video processing link in the autonomous driving vehicle control system can be effectively tested. Therefore, how to efficiently simulate the camera is the focus of current research. Summary of the invention

[0003] The present disclosure provides a simulation data processing method, device and storage medium.

[0004] According to one aspect of the present disclosure, a simulation data processing method is provided, which is applied to a simulation device, comprising:

[0005] Acquire N channels of video compression data for simulation testing, wherein the N channels of video compression data are used to simulate environmental images captured by N acquisition devices in a vehicle from different acquisition perspectives during the same period of time; each channel of video compression data in the N channels of video compression data contains a plurality of frame compression data;

[0006] Based on the time information of each channel of video compression data, multiple frame compression data belonging to the same time point are determined from different channels of video compression data;

[0007] The compressed data of multiple frames belonging to the same time point are sent to the graphics processor GPU for decompression to obtain multiple simulation images for simulating the environment scene of the vehicle at the same time point.

[0008] According to another aspect of the present disclosure, there is provided a simulation device, comprising:

[0009] A data determination unit is used to obtain N channels of video compression data for simulation testing, wherein the N channels of video compression data are used to simulate environmental images captured by N acquisition devices in a vehicle from different acquisition perspectives during the same period of time; each channel of video compression data in the N channels of video compression data contains a plurality of frame compression data;

[0010] The data processing unit is used to determine multiple frame compressed data belonging to the same time point from different video compressed data based on the time information of each channel of video compressed data; send the multiple frame compressed data belonging to the same time point to the graphics processor GPU for decompression to obtain multiple simulation images for simulating the environmental scenes of the vehicle at the same time point.

[0011] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0012] at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0015] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0017] In this way, the disclosed scheme can send multiple frame compressed data belonging to the same time point in each channel of video compression data to the GPU for decompression, and then obtain multiple simulated images for simulating the environmental scenes around the vehicle at the same time point, effectively utilizing the high decompression performance of the GPU. Compared with the existing decompression method, the disclosed scheme effectively reduces the time cost required for the decompression process and improves the decompression efficiency. This lays the foundation for realizing efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), and also lays the foundation for improving the simulation test efficiency of vehicles.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 1 ;

[0021] Figure 2 is a schematic diagram of a simulation scenario corresponding to total compressed data according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of a simulation scenario corresponding to total compressed data according to another embodiment of the present application;

[0023] Figure 4 This is a schematic diagram of a scenario in which a simulation device processes total compressed data according to an embodiment of the present application. Figure 1 ;

[0024] Figure 5 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 2 ;

[0025] Figure 6 This is a schematic diagram of a scenario in which a simulation device processes total compressed data according to an embodiment of the present application. Figure 2 ;

[0026] Figure 7 This is a schematic diagram of a scenario in which a simulation device processes total compressed data according to an embodiment of the present application. Figure 3 ;

[0027] Figure 8 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 3 ;

[0028] Fig. 9 is an illustrative schematic diagram of splitting each channel of video compression data into multiple data blocks according to an embodiment of the present application;

[0029] Fig.10 is a schematic diagram of the structure of a simulation device according to an embodiment of the present application;

[0030] Fig.11 It is a block diagram of an electronic device used to implement the simulation data processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0032] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0033] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0034] The following is a description of the related technologies of the embodiments of the present disclosure. The following related technologies are optional solutions that can be arbitrarily combined with the technical solutions of the embodiments of the present disclosure, and they all belong to the protection scope of the embodiments of the present disclosure.

[0035] In the hardware-in-the-loop (HIL) simulation system of autonomous driving, camera simulation, such as GMSL camera simulation, is an essential part. Through camera simulation, the entire video processing link in the autonomous driving vehicle control system can be effectively tested. For example, both the simulation of video content and the testing of video link timing can be realized.

[0036] For example, when simulating GMSL cameras using traditional field programmable gate arrays (FPGAs), accurate data format and timing simulation can be effectively achieved; during the simulation, the compressed video data can be decompressed using the CPU inside the FPGA or the simulation device. However, as the camera pixel and frame rate increase, the decompression time inside the FPGA or the CPU increases, the decompression performance decreases, and even fails to meet the camera pixel and frame rate requirements, resulting in a serious drop in frame rate. Therefore, how to efficiently simulate GMSL cameras is the focus of current research.

[0037] Based on this, the disclosed solution proposes a simulation data processing method to improve the decompression performance of video compressed data and achieve efficient simulation of high-pixel, high-frequency cameras.

[0038] Specifically, Figure 1 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0039] Further, the method includes at least part of the following contents. Figure 1 As shown, the simulation data processing method is applied to a simulation device (e.g., an industrial computer), and includes:

[0040] Step S101: The simulation device obtains N channels of video compression data for simulation testing.

[0041] Here, the N channels of video compression data are used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives in the same period of time. For example, the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives in the same period of time (for example, within a unit of time) are simulated while the vehicle is on its driving road.

[0042] Furthermore, in one example, different acquisition devices in the vehicle have different acquisition perspectives, so that the vehicle's environmental images (such as the environmental images around the vehicle) can be captured through different acquisition devices, providing support for driving decisions.

[0043] Furthermore, in an example, each of the N channels of video compression data contains a plurality of frame compression data. For example, in a specific example, for one channel of video compression data, it may at least include compression data of a plurality of video frames collected by a single acquisition device within a unit time (for example, within 1 second) (here, the compression data of the video frames may be referred to as frame compression data for short), for example, including compression data of 10 frames of images collected by a single acquisition device within 1 second.

[0044] Here, the number of video frames collected per unit time is related to relevant parameters of the acquisition device to be simulated, such as the frame rate, and the disclosed solution does not impose any limitation on this.

[0045] Step S102: the simulation device determines a plurality of frame compressed data belonging to the same time point from different channels of video compressed data based on the time information of each channel of video compressed data.

[0046] Here, in this example, the time information of the video compression data may specifically include the timestamp of each frame of compressed data in the video compression data; further, the "time point" may be specifically understood as the simulated moment. For example, if it is necessary to simulate the environmental scene captured by the vehicle from different acquisition angles at moment A, it is necessary to determine the multiple frame compression data belonging to the moment A based on the timestamps of each channel of video compression data, for example, determine the multiple frame compression data with a timestamp of time A.

[0047] Step S103: The simulation device sends a plurality of frame compressed data belonging to the same time point to the graphics processor GPU for decompression, so as to obtain a plurality of simulation images for simulating the environment scene of the vehicle at the same time point.

[0048] In this way, the disclosed scheme can send multiple frame compressed data belonging to the same time point in each channel of video compression data to the GPU for decompression, and then obtain multiple simulated images for simulating the environmental scenes around the vehicle at the same time point, effectively utilizing the high decompression performance of the GPU. Compared with the existing decompression method, the disclosed scheme effectively reduces the time cost required for the decompression process and improves the decompression efficiency. This lays the foundation for realizing efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras), and also lays the foundation for improving the simulation test efficiency of vehicles.

[0049] Furthermore, since the disclosed solution uses a GPU for decompression, the details in the image can be retained to the greatest extent during the decompression process, avoiding image information loss due to decompression, and providing strong support for the autonomous driving system to respond to environmental changes more quickly and make accurate decisions. Moreover, the GPU decompression solution used in the disclosed solution can also support larger data sets, which can further improve the reliability and accuracy of simulation tests.

[0050] Here, it should be noted that the vehicle described in the present disclosure may specifically be an unmanned vehicle or an assisted driving vehicle, and the present disclosure does not impose any specific restrictions on this.

[0051] In a specific example, N channels of video compression data may be obtained in the following manner; specifically, the above-mentioned obtaining of N channels of video compression data for simulation testing (for example, the above-mentioned step S101) specifically includes:

[0052] Step S101 - 1 : The simulation device obtains total compressed data for simulation testing.

[0053] Here, the total compressed data is used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives within a preset period of time.

[0054] Furthermore, in one example, the preset time period includes a plurality of unit times.

[0055] Step S101 - 2 : based on at least N acquisition devices required for simulation test, the total compressed data is split to obtain the N channels of video compression data.

[0056] For example, in one example, the above-mentioned N acquisition devices at least based on the required simulation test, split the total compressed data to obtain the N channels of video compressed data, which may specifically include:

[0057] Step S101 - 2 - 1 : Based on a plurality of unit times included in the preset time period, split the total compressed data into a plurality of groups of data to be decompressed.

[0058] Here, the number of groups of data to be decompressed is the same as the number of unit times included in the preset time period. Further, each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N acquisition devices within a unit time, that is, includes all frame compressed data that can be collected by the N acquisition devices within a unit time.

[0059] That is to say, in this step, the simulation device can split the total compressed data containing multiple unit times according to unit time to obtain multiple groups of data to be decompressed, so that each group of data to be decompressed contains all the frame compressed data that can be collected by the N acquisition devices within a unit time.

[0060] Step S101 - 2 - 2: Select a group of data to be decompressed that meets the time requirement from the multiple groups of data to be decompressed.

[0061] Step S101 - 2 - 3: Based on the N acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N channels of video compression data.

[0062] For instance, in one example, multiple groups of data to be decompressed are sorted in chronological order, the earliest group of data to be decompressed is selected from the sorting results, and the earliest group of data to be decompressed is further split according to multiple acquisition devices to be simulated to obtain the N channels of video compression data.

[0063] For example, in one example, Figure 2As shown, the unmanned vehicle or assisted driving vehicle to be simulated and tested is equipped with acquisition devices at 6 different positions, a total of 6, which can be respectively recorded as acquisition device 1, acquisition device 2, acquisition device 3, acquisition device 4, acquisition device 5 and acquisition device 6; at this time, the total compressed data can include 6 channels of video compression data, and each channel of video compression data is used to simulate the environmental picture captured by a collection device in the vehicle. For example, the first channel of video compression data is used to simulate the environmental picture captured by collection device 1, the second channel of video compression data is used to simulate the environmental picture captured by collection device 2,..., the sixth channel of video compression data is used to simulate the environmental picture captured by collection device 6.

[0064] Furthermore, in one example, the total compressed data is used to simulate the environmental images captured by the N acquisition devices from different acquisition perspectives within a unit time (e.g., within 1 second). Accordingly, each channel of video compression data obtained by directly splitting the total compressed data can directly simulate the environmental images captured by a capture device within a unit time (e.g., within 1 second) and at its own acquisition perspective.

[0065] Or, in another example, Figure 3As shown, the total compressed data can be specifically used to simulate the environmental scenes captured by the N acquisition devices from different acquisition perspectives within a preset time period (for example, the preset time period includes L unit times, in this case, they can be respectively recorded as the 1st time unit, the 2nd time unit, ..., the Lth time unit in the preset time period, and L is a positive integer greater than or equal to 2). In this scenario, the total compressed data can be first split into L groups of data to be decompressed according to unit time, which can be respectively recorded as the 1st group of data to be decompressed corresponding to the 1st time unit in the preset time period (for example, the 1st group of data to be decompressed corresponding to the 1st second in the preset time period), the 2nd group of data to be decompressed corresponding to the 2nd time unit in the preset time period (the 2nd group of data to be decompressed corresponding to the 2nd second in the preset time period), ..., the Lth group of data to be decompressed corresponding to the Lth time unit in the preset time period (the data to be decompressed corresponding to the last 1 second in the preset time period). Here, each group of data to be decompressed is used to simulate the environmental scenes captured by the N acquisition devices from different acquisition perspectives within a unit time; further, Based on N acquisition devices, a group of data to be decompressed is further split and split into N channels of video compression data. For example, for the first group of data to be decompressed corresponding to the first second in the preset time period, it can be split into N channels of video compression data. At this time, they can be recorded as the first channel of video compression data corresponding to the first second in the preset time period, the second channel of video compression data corresponding to the first second in the preset time period, ..., the Nth channel of video compression data corresponding to the first second in the preset time period. At this time, each channel of video compression data is used to simulate the environmental picture captured by an acquisition device from its own acquisition perspective within a unit time. In this way, it provides strong support for the subsequent rapid and orderly use of GPU decompression to obtain simulated images.

[0066] It should be noted that in actual applications, the total compressed data obtained can be split on the device dimension based on N acquisition devices to obtain multiple groups of data to be decompressed. At this time, each group of data to be decompressed is used to simulate the environmental picture captured by an acquisition device from its own acquisition perspective within a preset time period (that is, the multiple unit times mentioned above), and then the multiple groups of data to be decompressed are processed from the time dimension according to the unit time to obtain the N channels of video compression data mentioned above.

[0067] It should be noted that in practical applications, Figure 4 As shown, a simulation device, such as an industrial computer, is integrated with a GPU. Furthermore, the industrial computer obtains total compressed data for simulation testing from a data center, and splits the total compressed data to obtain N channels of video compression data, and then sends multiple frame compressed data belonging to the same time point in different channels of video compression data to (for example, serially sent to) the GPU of the industrial computer for decompression.

[0068] In this way, the disclosed solution provides a specific solution for obtaining N-channel video compression data. The solution is simple and efficient, and can quickly split the total compressed data into N-channel video compression data in time sequence. This provides strong support for the subsequent rapid and orderly use of GPU decompression to obtain simulated images.

[0069] Figure 5 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figures 1 to 4 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0070] Further, the method includes at least part of the following contents. Figure 5 As shown, including:

[0071] Step S501: The simulation device obtains N channels of video compression data for simulation testing.

[0072] Here, the N channels of video compression data are used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives during the same period of time; each channel of video compression data in the N channels of video compression data contains multiple frame compression data.

[0073] Here, the specific solution for obtaining N channels of video compression data can be found in the above description and will not be repeated here.

[0074] Step S502: the simulation device determines a plurality of frame compressed data belonging to the same time point from different channels of video compressed data based on the time information of each channel of video compressed data.

[0075] Step S503: the simulation device sends a plurality of frames of compressed data belonging to the same time point to the graphics processor GPU for decompression, so as to obtain a plurality of simulation images for simulating the environment scene of the vehicle at the same time point.

[0076] Here, in one example, after decompressing a simulated image, the GPU may first send the simulated image to the simulated device for storage, for example, Figure 6 As shown, the simulation image successfully decompressed by the GPU is cached in the cache area of ​​the simulation device.

[0077] Step S504: when multiple simulation images belonging to the same time point are obtained after decompression, the simulation device sends the multiple simulation images belonging to the same time point to the board.

[0078] Step S505: The simulation device uses the board card to perform a first time alignment on multiple simulation images belonging to the same time point, so as to synchronously send them to the vehicle for simulation testing after the first time alignment.

[0079] For example, in one example, after the simulation device (such as the CPU of the simulation device) determines that all frame compressed data belonging to the same time point are decompressed successfully, Figure 6 As shown, when the simulation device determines that all simulation images belonging to the same time point are stored in its cache area, all simulation images belonging to the same time point are sent to the board card, and all simulation images belonging to the same time point are aligned in the first time using the board card, and then sent out synchronously after the first time alignment.

[0080] It should be noted that in actual scenarios, the reception time of each simulated image received by the board from the simulation device may be different. Therefore, in order to ensure that all simulated images belonging to the same time point can be sent synchronously and the vehicle can receive the simulated images synchronously (that is, to ensure the time consistency of the simulated images received by the vehicle), so as to successfully complete the test of the simulation link timing and / or the test of the simulated video content, in this example, the simulation device can also use the board to perform a first time alignment on multiple simulated images belonging to the same time point, and after the first time alignment, use the board to synchronously send multiple simulated images belonging to the same time point to the vehicle, thereby ensuring the time consistency of the simulated images received by the vehicle.

[0081] It can be understood that the above-mentioned time alignment operation (i.e., the first time alignment operation) may not change the timestamp of the simulation image, but only align the simulation images received at different times, so that all simulation images received at different times (also referred to as different time points) but belonging to the same time point can be sent out at the same time (i.e., sent synchronously).

[0082] In this way, the disclosed solution utilizes the board card to perform a first time alignment on multiple simulated images at the same time point after decompression, so that multiple simulated images at the same time point can be synchronously sent to the vehicle, effectively improving the time synchronization of the decompressed image data in transmission, thereby providing strong support for reducing the delay of the simulated images received by the vehicle, and at the same time, providing strong support for achieving accurate simulation of high-pixel and high-frequency cameras.

[0083] Further, in a specific example, the simulation device described above uses a board card to perform a first time alignment on a plurality of simulation images belonging to the same time point, so as to synchronously send the images to the vehicle for simulation testing after the first time alignment (for example, the step S505 described above), specifically including:

[0084] Step S505 - 1 : The simulation device determines whether all simulation images belonging to the same time point are stored in the cache unit of the board.

[0085] Here, it should be noted that the board in this example may refer to the general term for the boards integrated in the simulation device. Accordingly, the cache unit of the board in this example may specifically be the total cache unit of all boards integrated in the simulation device.

[0086] For instance, in one example, the simulation device may be integrated into a board card. At this time, after the simulation device sends each simulation image belonging to the same time point to the board card, the simulation images belonging to the same time point may be stored in the cache unit of the board card; accordingly, step S505-1 may be specifically as follows: determining whether all simulation images belonging to the same time point are stored in the cache unit of the board card.

[0087] Alternatively, in another example, the simulation device may integrate two or more boards. In this case, the simulation device may send each simulation image belonging to the same time point to multiple boards. Accordingly, each simulation image belonging to the same time point may be stored in the cache units of different boards. Accordingly, step S505-1 may be specifically as follows: determining whether all simulation images belonging to the same time point are stored in the total cache unit of all boards.

[0088] For example, take the case where there are 6 acquisition devices that need to be simulated and tested (that is, a total of 6 channels of video compression data) and the simulation device is connected to 2 boards. At this time, in one example, each board can be used to receive 3 channels of decompressed simulation images and store the received simulation images in its own cache unit; accordingly, the simulation device can determine whether the total cache unit of the two boards stores 6 simulation images belonging to the same time point.

[0089] It should be noted that, in one example, in order to store simulation images at different time points, multiple cache units may be provided in the board. At this time, simulation images at different time points may be cached in different cache units, so that only images belonging to the same time point are cached in the same cache unit. This provides effective support for efficiently and synchronously sending all simulation images belonging to the same time point.

[0090] Step S505-2: When it is determined that all simulation images belonging to the same time point are stored, the simulation device uses the board card to perform a first time alignment on all simulation images belonging to the same time point, and synchronously sends them to the vehicle for simulation testing after the first time alignment.

[0091] For instance, continuing with the example of a simulation test in which there are a total of 6 acquisition devices and the simulation devices are connected to 2 boards, after determining that the total cache unit of the two boards stores 6 simulation images belonging to the same time point, the two boards can be used to perform a first time alignment on the 6 simulation images at the same time point. For example, each board can be used to perform a first time alignment on the 3 simulation images in its respective cache unit. At this point, it can be considered that all simulation images belonging to the same time point have completed the first time alignment. Alternatively, the two boards can use a time synchronization signal to perform a first time alignment on the 6 simulation images in the total cache unit, and after the first time alignment, the 6 simulation images can be synchronously sent to the vehicle through the two boards for simulation testing.

[0092] In this way, the disclosed solution can use the board to synchronously send all simulated images belonging to the same time point to the vehicle, thereby effectively improving the time synchronization of the decompressed image data in transmission, thereby providing strong support for effectively reducing the delay of the simulated images received by the vehicle, and at the same time, it also provides strong support for achieving accurate simulation of high-pixel and high-frequency cameras.

[0093] Further, in a specific example, the simulation device described above determines whether all simulation images belonging to the same time point are stored in the cache unit of the board (for example, the step S505-1 described above), specifically including:

[0094] When it is determined that the current occupied capacity of the cache unit of the board reaches the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the cache unit of the board.

[0095] Here, the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data in the simulation test.

[0096] It should be noted that, in this example, the board can also be a general term for the boards integrated into the simulation device, and accordingly, the cache unit of the board can refer to the total cache unit of all boards integrated into the simulation device. At this time, "the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data in the simulation test" can be understood as: the maximum capacity of the total cache unit of the board is determined based on the total number of channels of video compression data in the simulation test. For example, in one example, the maximum capacity of the total cache unit of the board = the total number of channels of video compression data, and further, the maximum capacity of the total cache unit of the board is also equal to the number of acquisition devices required for simulation, and further, it is also equal to the number of simulated images belonging to the same time point.

[0097] Furthermore, in the case where the simulation device is integrated with multiple boards, the maximum capacity of the cache unit of each board can be determined based on the total number of channels of video compression data in the simulation test and the total number of boards.

[0098] For example, if there are 6 acquisition devices for simulation test, there are 6 simulation images belonging to the same time point. At this time, the maximum capacity of the total cache unit of the board can be set to accommodate 6 simulation images. In other words, the sum of the maximum capacity of the cache units of each board (that is, the total maximum capacity of the set of cache units of each board) can accommodate the 6 simulation images. Further, if the simulation device is integrated with 2 boards, then in one example, the maximum capacity of the cache unit of each board can accommodate 3 simulation images; at this time, if it is determined that the current occupied capacity of the cache unit of each board has reached the maximum capacity, it can be considered that all simulation images belonging to the same time point have been stored in the cache unit of the board, so that all simulation images belonging to the same time point can be synchronized and sent out after the first time alignment.

[0099] It should be noted that the above is only an exemplary description. In actual applications, the maximum capacities of cache units in different boards may be the same or different, and the disclosed solution does not limit this.

[0100] In this way, the disclosed solution provides a specific solution for quickly determining whether the board has received all the simulated images belonging to the same time point. This provides strong support for ensuring that the vehicle can synchronously receive all the simulated images belonging to the same time point. At the same time, it also provides strong support for further improving the accuracy of time synchronization, thereby laying the foundation for the subsequent smooth completion of the test of the simulation link timing and / or the test of the simulation video content.

[0101] Here, in one example, the board mentioned above can be specifically a video injection board; further, in one example, the video injection board is one of the following: FPGA or a chip capable of implementing video injection requirements. In other words, the disclosed solution is compatible with existing video injection boards, thus improving the scope of application of the disclosed solution, making the disclosed solution both practical and applicable.

[0102] Further, in a specific example, the sending of multiple simulation images belonging to the same time point to the board mentioned above may specifically include:

[0103] The central processing unit (CPU) of the simulation device is used to perform second time alignment on a plurality of simulation images belonging to the same time point, and the images are synchronously sent to the board after the second time alignment.

[0104] That is to say, before the simulation device sends multiple simulation images belonging to the same time point (for example, all simulation images belonging to the same time point) to the board, the central processing unit CPU of the simulation device can first perform a second time alignment on the multiple simulation images belonging to the same time point, and then send them to the board synchronously after the second time alignment. In this way, the second time alignment of image data belonging to the same time point is guaranteed at the CPU level, thereby providing support for ensuring the accuracy of image data synchronization. At the same time, it also lays the foundation for subsequently reducing the time cost required for the board to perform the first time alignment of image data.

[0105] Here, it should be noted that, in one example, the time alignment accuracy that can be achieved by the central processing unit CPU for the multiple simulation images by performing the second time alignment is lower than the time alignment accuracy that can be achieved by the board card for the multiple simulation images by performing the first time alignment. In this way, in the disclosed scheme, multiple simulation images belonging to the same time point will first use the CPU of the simulation device to perform a time alignment with low alignment accuracy (such as milliseconds) (also referred to as "coarse" granularity time alignment), and then the board card will perform a time alignment with high alignment accuracy (such as microseconds) (also referred to as "fine" granularity time alignment) on the multiple simulation images after the "coarse" granularity time alignment. After the "fine" granularity time alignment, all simulation images belonging to the same time point are synchronously sent to the vehicle for simulation testing. In this way, the time synchronization of the image data is further ensured, the delay in the transmission of the image data is greatly reduced, and the foundation is laid for the accurate simulation of high-pixel and high-frequency cameras. At the same time, it also lays the foundation for the automatic driving system to respond to environmental changes more quickly and make accurate decisions.

[0106] Here, it should be noted that, in one example, the bandwidth of the data receiving link of the board is greater than the bandwidth of the data sending link of the board. This further lays the foundation for reducing the transmission delay of image data and thus achieving accurate simulation of high-pixel and high-frequency cameras.

[0107] Furthermore, in a specific example, before using the board to perform a first time alignment on a plurality of simulation images belonging to the same time point, the method further includes:

[0108] The board is used to convert the data formats of multiple simulation images belonging to the same time point into a preset format.

[0109] That is to say, in one example, after receiving multiple simulated images belonging to the same time point sent by the simulation device, the board can first convert the data format of the multiple simulated images into a preset format, and then perform a first time alignment on the multiple simulated images at the same time point in the preset format, and then send them out synchronously after the first time alignment. In this way, the required data format can be effectively simulated, and the data compatibility is stronger, thereby ensuring that the receiving end, such as the vehicle end, can process the received image data accordingly, laying the foundation for realizing efficient simulation of high-pixel and high-frequency cameras.

[0110] Furthermore, in one example, the preset format is the data format output by the acquisition device of the vehicle under simulated test. For example, the preset format is the format output by the GMSL format acquisition device of the vehicle under simulated test, that is, the GMSL format. In this way, accurate and efficient simulation of the GMSL camera is achieved, laying the foundation for subsequent improvement of the performance stability of the entire autonomous driving system.

[0111] For example, Figure 7 A schematic diagram of a data link used for simulation testing of an acquisition device of an autonomous driving HIL system is provided, where a simulation device (such as an industrial computer) integrates a GPU and a CPU, and a board (such as an FPGA injection board) is plugged into a peripheral component interconnect express (PCIE) card slot of the industrial computer.

[0112] Specifically, Figure 7As shown, first, a simulation device such as an industrial computer obtains total compressed data for simulation testing from a data center. For example, the total compressed data may be a compressed video stream obtained by compression using High Efficiency Video Coding (HEVC). The simulation device splits the total compressed data into multiple channels of video compression data according to the number of acquisition devices (such as GMSL cameras) to be simulated, that is, the N channels of video compression data mentioned above are obtained. Secondly, the CPU of the industrial computer determines multiple frame compression data belonging to the same time point from different channels of video compression data according to the time information of each channel of video compression data, and sends the multiple frame compression data belonging to the same time point in series to the GPU for decompression using the GPU. Here, the frame compression data sent to the GPU may also be a compressed video stream compressed by HEVC. After the GPU obtains the simulation image after decompression, it is directly sent to the cache area of ​​the industrial computer for temporary storage. Here, the decompressed simulation image may be in YUV (Y represents the luminance (Luma) component, and U and V represent the chrominance (Chroma) component) format. Furthermore, when the industrial computer (such as the CPU) determines that the cache area obtains all the simulated images at the same time point, it performs a second time alignment on all the simulated images belonging to the same time point and sends them to the FPGA injection board; accordingly, the FPGA injection board converts the data format of the received simulated image (such as the YUV format) into the GMSL format to simulate the real GMSL camera, and caches the simulated image in the GMSL format into its own cache unit. Further, when the FPGA injection board determines that the current occupied capacity of the cache unit is the maximum capacity, it is considered that all the simulated images belonging to the same time point have been obtained, and the data format of all the simulated images belonging to the same time point is the GMSL format, and then all the simulated images at the same time point in the GMSL format in the cache unit are first time aligned, and after the first time alignment, they are synchronously sent to the vehicle to facilitate the simulation test of the vehicle. In this way, the high decompression performance of the GPU is effectively utilized. Compared with the existing solutions, the time cost required for the decompression process is effectively reduced, the decompression efficiency is improved, and efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras) is achieved. For example, accurate data format and timing simulation is achieved. At the same time, precise control of the GMSL camera content is also achieved, laying the foundation for improving the efficiency of vehicle simulation testing.

[0113] Figure 8 This is a schematic flow chart of a simulation data processing method according to an embodiment of the present application. Figure 3 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figures 1 to 7The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0114] Further, the method includes at least part of the following contents. Figure 8 As shown, including:

[0115] Step S801: The simulation device obtains N channels of video compression data for simulation testing.

[0116] Here, the N channels of video compression data are used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives during the same period of time; each channel of video compression data in the N channels of video compression data contains multiple frame compression data.

[0117] Here, the specific solution for obtaining N channels of video compression data can be found in the above description and will not be repeated here.

[0118] Step S802: The simulation device splits the N channels of video compression data into M data blocks based on the time information of each channel of video compression data.

[0119] Here, each data block contains a plurality of frame compressed data belonging to the same time point, and the frame compressed data contained in different data blocks correspond to different time points.

[0120] Further, in one example, the value of M is related to the number of frame compression data contained in the video compression data. In other words, the value of M is related to the number of video frames collected per unit time. For example, the value of M is less than or equal to the number of video frames collected per unit time. Further, in one example, the value of M is equal to the number of video frames collected per unit time.

[0121] For example, each channel of video compression data includes compressed data of multiple frames of images (also called multiple video frames) collected by a collection device in a unit time. Here, the unit time can include multiple time points. For example, for a scene where 10 frames of images can be collected in 1 second, the unit time includes 10 different time points. Specifically, Fig. 9As shown, each of the 6 channels of video compression data includes 10 frames of compressed images collected within 1 second. At this time, the frame compressed images belonging to the same time point can be divided into a data block according to the time points of different frame compressed images in each channel of video compression data. For example, the 11th frame compressed image belonging to time point 1 in the 1st channel of video compression data, the 21st frame compressed image belonging to time point 1 in the 2nd channel of video compression data, ..., the 61st frame compressed image belonging to time point 1 in the 6th channel of video compression data are divided into the 1st data block; the 12th frame compressed image belonging to time point 2 in the 1st channel of video compression data, the 22nd frame compressed image belonging to time point 2 in the 2nd channel of video compression data, ..., the 62nd frame compressed image belonging to time point 2 in the 6th channel of video compression data are divided into the 2nd data block; similarly, the 3rd data block, ..., the 9th data block and the 10th data block can be obtained. In this way, the GPU can decompress compressed data at the same time point, further ensuring time synchronization, providing strong support for the simulation of video link timing, and greatly reducing the delay in image data transmission, thereby laying the foundation for the accurate simulation of high-pixel and high-frequency cameras. At the same time, it also lays the foundation for the autonomous driving system to respond to environmental changes more quickly and make accurate decisions.

[0122] Step S803: The simulation device selects the i-th data block from the M data blocks based on a timing rule.

[0123] Here, the i-th data block is at least one of the currently undecompressed data blocks in the M data blocks. Further, the i-th data block may be the earliest data block in time among all the currently undecompressed data blocks in the M data blocks.

[0124] Step S804: the simulation device sends the multiple frames of compressed data in the i-th data block in series to the GPU for decompression, so as to obtain multiple simulation images for simulating the environment scene of the vehicle at the same time point.

[0125] In this way, the disclosed solution can utilize the time information of each channel of video compression data to split N channels of video compression data according to time points to obtain M data blocks belonging to different time points, and then send multiple frame compressed data in each data block serially to the GPU for decompression. In this way, the GPU memory overflow during decompression is effectively avoided, and at the same time, the foundation is laid for the subsequent rapid and orderly use of the GPU to decompress frame compressed data at different time points.

[0126] Further, in a specific example, multiple frames of compressed data may be sent serially in the following manner; specifically, the above-mentioned simulation device sends multiple frames of compressed data in the i-th data block serially to the GPU for decompression (for example, the above-mentioned step S804), which specifically includes:

[0127] Step S804 - 1 : the simulation device sends the j-th frame compressed data in the i-th data block to the GPU for decompression.

[0128] Here, the j-th frame compressed data is one of the uncompressed frame compressed data in the selected i-th data block.

[0129] Step S804-2: After decompressing to obtain the simulated image of the j-th frame compressed data, for example, after the simulation device receives the response information of the decompression completion from the GPU, or after the simulation device receives the simulated image of the j-th frame compressed data, the j+1-th frame compressed data in the i-th data block is sent.

[0130] Here, the j+1th frame compressed data is one of the uncompressed frame compressed data in the i-th data block.

[0131] That is, the currently undecompressed frame compressed data in the i-th data block is sent to the GPU for decompression, and after the decompression is completed, the next undecompressed frame compressed data is sent to the GPU for decompression, so that the serial decompression can quickly obtain the simulation image. Fig. 9 For example, for the first data block, the 11th frame compressed image can be sent to the GPU for decompression first. After the simulation image of the 11th frame compressed image is obtained after decompression, the 21st frame compressed image in the first data block is sent to the GPU for decompression, and so on, until the last frame (i.e., the 61st frame) compressed image in the first data block is sent to the GPU.

[0132] In this way, the disclosed solution sends compressed data of different frames in the same data block serially to the GPU for decompression, which greatly saves the resource usage of the GPU during decompression and improves the processing efficiency of GPU decompression, thereby ensuring the quality of the simulated image obtained after decompression, laying the foundation for the subsequent efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras).

[0133] In a specific example, after the decompression of the i-th data block is completed, it can be sent to the GPU for decompression in the following manner. For example, after the above-mentioned step S804, it can also specifically include:

[0134] Step S805: After determining that the frame compressed data of the i-th data block is decompressed, multiple frame compressed data in the i+1-th data block are sent serially to the GPU for decompression, and this cycle is repeated, serially sending each data block to the GPU for decompression until the frame compressed data of all data blocks are successfully decompressed.

[0135] Here, the i+1th data block is at least one of the currently undecompressed data blocks in the M data blocks. Further, the i+1th data block may be the earliest data block in time among all the currently undecompressed data blocks in the M data blocks.

[0136] Continue with Fig. 9 For example, when 6 simulation images corresponding to the first data block are obtained after decompression, the second data block will be decompressed, for example, the 12th frame compressed image in the second data block is sent to the GPU for decompression; when 6 simulation images corresponding to the second data block are obtained after decompression, the third data block will be decompressed, for example, the 13th frame compressed image in the third data block is sent to the GPU for decompression, and so on, until the last (i.e., the 10th) data block.

[0137] In this way, the disclosed solution sends different data blocks to the GPU in a serial manner for decompression, thereby reducing the pressure of data transmission and effectively improving the processing efficiency of the GPU, thereby giving full play to the high decompression performance of the GPU, thereby ensuring the quality of the simulated image obtained after decompression, and laying the foundation for the subsequent efficient simulation of high-pixel and high-frequency cameras (such as GMSL cameras).

[0138] The disclosed solution also provides a simulation device, such as Fig.10 As shown, including:

[0139] The data determination unit 1001 is used to obtain N channels of video compression data for simulation testing, wherein the N channels of video compression data are used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives in the same period of time; each channel of video compression data in the N channels of video compression data contains a plurality of frame compression data;

[0140] The data processing unit 1002 is used to determine multiple frame compressed data belonging to the same time point from different video compressed data based on the time information of each channel of video compressed data; and send the multiple frame compressed data belonging to the same time point to the graphics processor GPU for decompression to obtain multiple simulation images for simulating the environmental scenes of the vehicle at the same time point.

[0141] In a specific example of the disclosed solution,

[0142] The data processing unit is also used to send multiple simulation images belonging to the same time point to the board card when multiple simulation images belonging to the same time point are obtained after decompression; use the board card to perform a first time alignment on the multiple simulation images belonging to the same time point, so as to send them to the vehicle for simulation testing after the first time alignment.

[0143] In a specific example of the disclosed solution, the data processing unit is specifically used to:

[0144] Determine whether all simulation images belonging to the same time point are stored in the cache unit of the board;

[0145] When it is determined that all simulation images belonging to the same time point are stored, the board is used to perform a first time alignment on all simulation images belonging to the same time point, and after the first time alignment, they are synchronously sent to the vehicle for simulation testing.

[0146] In a specific example of the disclosed solution, the data processing unit is specifically used to:

[0147] When it is determined that the current occupied capacity of the cache unit of the board has reached the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the cache unit of the board; wherein the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data of the simulation test.

[0148] In a specific example of the disclosed solution, the data processing unit is specifically used to:

[0149] A central processing unit (CPU) is used to perform second time alignment on a plurality of simulation images belonging to the same time point, and the images are synchronously sent to the board after the second time alignment.

[0150] In a specific example of the disclosed solution, the time alignment accuracy that can be achieved by the central processing unit CPU for the multiple simulation images by performing the second time alignment is lower than the time alignment accuracy that can be achieved by the board card for the multiple simulation images by performing the first time alignment.

[0151] In a specific example of the disclosed solution, the data processing unit is further used to:

[0152] Before the board card is used to perform the first time alignment on the multiple simulation images belonging to the same time point, the board card is used to convert the data formats of the multiple simulation images belonging to the same time point into a preset format.

[0153] In a specific example of the disclosed solution, the preset format is the data format output by the acquisition device of the vehicle under simulation test.

[0154] In a specific example of the disclosed solution, the data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

[0155] In a specific example of the disclosed solution, the board is one of the following: a field programmable gate array FPGA or a chip capable of realizing video injection requirements.

[0156] In a specific example of the disclosed solution, the data processing unit is further used to:

[0157] Based on the time information of each channel of video compression data, the N channels of video compression data are split into M data blocks, wherein each data block contains multiple frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of M is related to the number of frame compression data contained in the video compression data.

[0158] In a specific example of the disclosed solution, the data processing unit is specifically used to:

[0159] Based on a timing rule, selecting an i-th data block from the M data blocks; wherein the i-th data block is at least one of the M data blocks that is not currently decompressed;

[0160] The multiple frames of compressed data in the i-th data block are sent serially to the GPU for decompression.

[0161] In a specific example of the disclosed solution, the data processing unit is specifically used to:

[0162] Sending the j-th frame compressed data in the i-th data block to the GPU for decompression; wherein the j-th frame compressed data is one of the undecompressed frame compressed data in the selected i-th data block;

[0163] After obtaining the simulated image of the j-th frame compressed data after decompression, the j+1-th frame compressed data in the ith data block is sent; the j+1-th frame compressed data is one of the undecompressed frame compressed data in the ith data block.

[0164] In a specific example of the disclosed solution, the data processing unit is further used to:

[0165] After determining that the frame compressed data of the i-th data block has been decompressed, multiple frame compressed data in the i+1-th data block are sent serially to the GPU for decompression, and each data block is sent serially to the GPU for decompression; wherein the i+1-th data block is at least one of the currently uncompressed data blocks in the M data blocks.

[0166] In a specific example of the disclosed solution, the data determination unit is specifically used to:

[0167] Obtaining total compressed data for simulation testing; wherein the total compressed data is used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives within a preset period of time;

[0168] The total compressed data is split based on at least N acquisition devices required for simulation test to obtain the N channels of video compression data.

[0169] In a specific example of the disclosed solution, the data determination unit is specifically used to:

[0170] Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N collection devices within the unit time;

[0171] Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data;

[0172] Based on the N acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N channels of video compression data.

[0173] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0174] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0175] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0176] Fig.11 A schematic block diagram of an example electronic device 1100 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0177] like Fig.11As shown, the device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 to a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0178] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0179] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as the simulation data processing method. For example, in some embodiments, the simulation data processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the simulation data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the simulation data processing method in any other appropriate manner (eg, by means of firmware).

[0180] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0181] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0182] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0183] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0184] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0185] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0186] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0187] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A simulation data processing method, applied to a simulation device, comprising: Acquire N channels of video compression data for simulation testing, wherein the N channels of video compression data are used to simulate environmental images captured by N acquisition devices in a vehicle from different acquisition perspectives during the same period of time; each channel of video compression data in the N channels of video compression data contains a plurality of frame compression data; Based on the time information of each channel of video compression data, multiple frame compression data belonging to the same time point are determined from different channels of video compression data; Sending multiple frames of compressed data belonging to the same time point to a graphics processor (GPU) for decompression to obtain multiple simulation images for simulating the environment of the vehicle at the same time point; The method further comprises: In the case where multiple simulation images belonging to the same time point are obtained after decompression, the multiple simulation images belonging to the same time point are sent to the board; The board is used to perform first time alignment on multiple simulation images belonging to the same time point, so that they are synchronously sent to the vehicle for simulation testing after the first time alignment.

2. The method according to claim 1, wherein: The method of using the board to perform a first time alignment on a plurality of simulation images belonging to the same time point, so as to synchronously send the images to the vehicle for simulation testing after the first time alignment, includes: Determine whether all simulation images belonging to the same time point are stored in the cache unit of the board; When it is determined that all simulation images belonging to the same time point are stored, the board is used to perform a first time alignment on all simulation images belonging to the same time point, and after the first time alignment, they are synchronously sent to the vehicle for simulation testing.

3. The method according to claim 2, wherein: The step of determining whether all simulation images belonging to the same time point are stored in the cache unit of the board includes: When it is determined that the current occupied capacity of the cache unit of the board has reached the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the cache unit of the board; wherein the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data of the simulation test.

4. The method according to claim 1, wherein: The sending of multiple simulation images belonging to the same time point to the board includes: The central processing unit (CPU) of the simulation device is used to perform second time alignment on a plurality of simulation images belonging to the same time point, and the plurality of simulation images are synchronously sent to the board after the second time alignment.

5. The method according to claim 4, wherein: The time alignment accuracy that can be achieved by the central processing unit CPU for the multiple simulation images by performing the second time alignment is lower than the time alignment accuracy that can be achieved by the board card for the multiple simulation images by performing the first time alignment.

6. The method according to any one of claims 1 to 5, wherein: Before using the board to perform the first time alignment on a plurality of simulation images belonging to the same time point, the method further includes: The board is used to convert the data formats of multiple simulation images belonging to the same time point into a preset format.

7. The method according to claim 6, wherein: The preset format is the data format output by the acquisition device of the vehicle under simulation test.

8. The method according to claim 7, wherein: The data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

9. The method according to claim 6, wherein: The board is one of the following: a field programmable gate array FPGA or a chip capable of realizing video injection requirements.

10. The method according to any one of claims 1 to 5, further comprising: Based on the time information of each channel of video compression data, the N channels of video compression data are split into M data blocks, wherein each data block contains multiple frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of M is related to the number of frame compression data contained in the video compression data.

11. The method according to claim 10, wherein: The method of determining a plurality of frame compressed data belonging to the same time point from different video compressed data based on the time information of each channel of video compressed data, and sending the plurality of frame compressed data belonging to the same time point to a graphics processor GPU for decompression includes: Based on a timing rule, selecting an i-th data block from the M data blocks; wherein the i-th data block is at least one of the M data blocks that is not currently decompressed; The multiple frames of compressed data in the i-th data block are sent serially to the GPU for decompression.

12. The method according to claim 11, wherein: The step of serially sending the plurality of frame compressed data in the i-th data block to the GPU for decompression includes: Sending the j-th frame compressed data in the i-th data block to the GPU for decompression; wherein the j-th frame compressed data is one of the undecompressed frame compressed data in the selected i-th data block; After decompressing the simulated image of the j-th frame compressed data, the j+1-th frame compressed data in the ith data block is sent; the j+1-th frame compressed data is one of the undecompressed frame compressed data in the ith data block.

13. The method according to claim 11, further comprising: After determining that the frame compressed data of the i-th data block has been decompressed, multiple frame compressed data in the i+1-th data block are sent serially to the GPU for decompression, and each data block is sent serially to the GPU for decompression; wherein the i+1-th data block is at least one of the currently uncompressed data blocks in the M data blocks.

14. The method according to any one of claims 1 to 5, wherein: The step of obtaining N channels of video compression data for simulation testing includes: Obtaining total compressed data for simulation testing; wherein the total compressed data is used to simulate the environmental images captured by N acquisition devices in the vehicle from different acquisition perspectives within a preset period of time; The total compressed data is split based on at least N acquisition devices required for simulation test to obtain the N channels of video compression data.

15. The method according to claim 14, wherein: The method of splitting the total compressed data based on at least the N acquisition devices required for the simulation test to obtain the N channels of video compressed data includes: Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N collection devices within the unit time; Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data; Based on the N acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N channels of video compression data.

16. A simulation device comprising: A data determination unit is used to obtain N channels of video compression data for simulation testing, wherein the N channels of video compression data are used to simulate environmental images captured by N acquisition devices in a vehicle from different acquisition perspectives during the same period of time; each channel of video compression data in the N channels of video compression data contains a plurality of frame compression data; The data processing unit is used to determine a plurality of frame compressed data belonging to the same time point from the different video compressed data based on the time information of each channel of video compressed data; send the plurality of frame compressed data belonging to the same time point to the graphics processor GPU for decompression, so as to obtain a plurality of simulation images for simulating the environment screen of the vehicle at the same time point; Among them, the data processing unit is also used to send multiple simulation images belonging to the same time point to the board card when multiple simulation images belonging to the same time point are obtained after decompression; use the board card to perform a first time alignment on the multiple simulation images belonging to the same time point, so as to send them to the vehicle synchronously for simulation testing after the first time alignment.

17. The device according to claim 16, wherein: The data processing unit is specifically used for: Determine whether all simulation images belonging to the same time point are stored in the cache unit of the board; When it is determined that all simulation images belonging to the same time point are stored, the board is used to perform a first time alignment on all simulation images belonging to the same time point, and after the first time alignment, they are synchronously sent to the vehicle for simulation testing.

18. The apparatus according to claim 17, wherein: The data processing unit is specifically used for: When it is determined that the current occupied capacity of the cache unit of the board has reached the maximum capacity, it is determined that all simulation images belonging to the same time point are stored in the cache unit of the board; wherein the maximum capacity of the cache unit of the board is determined based on the total number of channels of video compression data of the simulation test.

19. The apparatus according to claim 16, wherein: The data processing unit is specifically used for: A central processing unit (CPU) is used to perform second time alignment on a plurality of simulation images belonging to the same time point, and the images are synchronously sent to the board after the second time alignment.

20. The apparatus of claim 19, wherein: The time alignment accuracy that can be achieved by the central processing unit CPU for the multiple simulation images by performing the second time alignment is lower than the time alignment accuracy that can be achieved by the board card for the multiple simulation images by performing the first time alignment.

21. The device according to any one of claims 16 to 20, wherein: The data processing unit is further used for: Before the board card is used to perform the first time alignment on the multiple simulation images belonging to the same time point, the board card is used to convert the data formats of the multiple simulation images belonging to the same time point into a preset format.

22. The apparatus according to claim 21, wherein The preset format is the data format output by the acquisition device of the vehicle under simulation test.

23. The apparatus of claim 22, wherein: The data format output by the acquisition device of the vehicle is global motion blur suppression logic GMSL.

24. The apparatus of claim 21, wherein: The board is one of the following: a field programmable gate array FPGA or a chip capable of realizing video injection requirements.

25. The apparatus according to any one of claims 16 to 20, wherein: The data processing unit is further used for: Based on the time information of each channel of video compression data, the N channels of video compression data are split into M data blocks, wherein each data block contains multiple frame compression data belonging to the same time point, and the frame compression data contained in different data blocks correspond to different time points; the value of M is related to the number of frame compression data contained in the video compression data.

26. The apparatus of claim 25, wherein: The data processing unit is specifically used for: Based on a timing rule, selecting an i-th data block from the M data blocks; wherein the i-th data block is at least one of the M data blocks that is not currently decompressed; The multiple frames of compressed data in the i-th data block are sent serially to the GPU for decompression.

27. The apparatus of claim 26, wherein: The data processing unit is specifically used for: Sending the j-th frame compressed data in the i-th data block to the GPU for decompression; wherein the j-th frame compressed data is one of the undecompressed frame compressed data in the selected i-th data block; After obtaining the simulated image of the j-th frame compressed data after decompression, the j+1-th frame compressed data in the ith data block is sent; the j+1-th frame compressed data is one of the undecompressed frame compressed data in the ith data block.

28. The apparatus of claim 26, wherein: The data processing unit is further used for: After determining that the frame compressed data of the i-th data block has been decompressed, multiple frame compressed data in the i+1-th data block are sent serially to the GPU for decompression, and each data block is sent serially to the GPU for decompression; wherein the i+1-th data block is at least one of the currently uncompressed data blocks in the M data blocks.

29. The apparatus according to any one of claims 16 to 20, wherein: The data determination unit is specifically used to obtain total compressed data for simulation testing; wherein the total compressed data is used to simulate the environmental scenes captured by N acquisition devices in the vehicle from different acquisition perspectives within a preset time period; the total compressed data is split based on at least the N acquisition devices required for the simulation test to obtain the N channels of video compression data.

30. The apparatus of claim 29, wherein: The data determination unit is specifically used for: Based on the multiple unit times included in the preset time period, the total compressed data is divided into multiple groups of data to be decompressed, wherein each group of data to be decompressed includes frame compressed data of video frames that can be collected by the N collection devices within the unit time; Selecting a group of to-be-decompressed data that meets the time requirement from the multiple groups of to-be-decompressed data; Based on the N acquisition devices required for the simulation test, a group of to-be-decompressed data that meets the time requirement is split into the N channels of video compression data.

31. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 15.

32. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-15.

33. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 15.

Images