Multi-type data file storage and playback method, device, equipment and medium

Abstract

The application provides a multi-type data file storage and playback method, device, equipment and medium, applied to the technical field of vehicle-mounted systems, and the multi-type data file storage method comprises the following steps: recording the data collected by M sensors as multi-type data files, and creating a file index, so that after the multi-type data files are stored in a playback processing device, the corresponding file index is obtained according to the multi-type data files; obtaining playback time information; determining a target storage address corresponding to the playback time information according to the playback time information and a timestamp in the file index; and obtaining corresponding data frames based on the target storage address to perform playback. The timestamp of the playback time in the file index is used to obtain the storage address corresponding to the playback time, so that when the data file is read for playback, the data can be found by only traversing the file index, and the playback efficiency is improved because the content stored in the file index is relatively small.

Description

[0001] The application is a divisional application of "Playback method and device of multi-type data file, equipment and medium" (Filing date: November 3, 2022, China application number: 202211370180.3). Figure 1 Technical Field

[0002] This application relates to the field of vehicle system technology, specifically to a method, apparatus, device, and medium for storing and playing back multiple types of data files. Background Technology

[0003] Based on various sensors installed on the vehicle, such as millimeter-wave radar, lidar, monocular / dual-lens cameras, and satellite navigation, the system continuously senses the surrounding environment while the vehicle is in motion, collects data, identifies, detects, and tracks static and dynamic objects, and combines this data with navigation map data to perform system calculations and analysis.

[0004] In existing technologies, after data collection, relevant software can be used to present the data collected on each vehicle in real time or play it back later. When playing back, it is especially necessary to realize the synchronous playback of various data.

[0005] Currently, there are two main methods for data playback: The first method is to save the collected data in a file, sort and store it according to the time inside the file, convert the data file to a supported playback data format before playback, and then play and control the playback speed at fixed time intervals during playback; The second method is to directly locate the playback data using timestamps as indexes, record the collected data in the same file through a bus, strictly sort the data inside the file according to time, add timestamps to the data stored locally, and play it according to the timestamps.

[0006] However, the first method requires converting the data file into a supported format, which takes time and is inefficient. Furthermore, the in-vehicle system collects data from multiple sensors and stores this data in the same file. The internal time of this data file is not strictly ordered, making playback impossible. Playback at fixed time intervals and controlled playback rate require a fixed frame rate, but in the in-vehicle system, bus data is not recorded at a fixed frame rate, leading to playback failures. Playing data for a specific time requires calculating how many data entries to skip, repeatedly reading the data file and skipping the specified number before continuing playback. If the file is large, looping through useless data consumes significant time, degrading the user experience. The second method involves multiple sensors in the in-vehicle system recording data in the same file via the bus. The data from different sensors is interleaved within the file, not a time-series file, which can cause playback failures or require processing large amounts of data, reducing playback efficiency and the user experience. Summary of the Invention

[0007] In view of this, the embodiments of this specification provide a method, apparatus, device, and medium for storing and playing back multiple types of data files, which solves the technical problems of low playback efficiency and poor customer experience in vehicle systems.

[0008] The embodiments in this specification provide the following technical solutions:

[0009] This specification provides an embodiment of a method for storing multiple types of data files, applied to a recording device, including:

[0010] Data collected by M sensors is recorded into multiple types of data files, and a file index is created; wherein, the file index represents the relationship between the timestamp of the data frame in the multiple types of data files and the corresponding storage address, and the timestamp represents the time when the recording device records the corresponding data frame.

[0011] Preferably, the multi-type data file storage method further includes: creating a channel index, wherein the channel index represents the relationship between each sensor and the corresponding recording time information, and the recording time information represents the time period during which the recording device records the data file of the corresponding sensor.

[0012] Preferably, the channel index includes: channel start time and channel end time; wherein, for each channel, the recording time information of the corresponding sensor includes the channel start time and the channel end time;

[0013] Create a channel index, including:

[0014] Based on the file extensions of the data collected by the M sensors in the multi-type data files, create a corresponding channel for each sensor;

[0015] The channel index is created based on the channel start time and channel end time corresponding to each sensor.

[0016] Preferably, the multi-type data file storage method further includes: associating channel indexes and file indexes, wherein a corresponding file index is created for each channel starting at the storage location corresponding to each sensor in the multi-type data file.

[0017] Preferably, the channel index includes the position, file name, and number of file indexes in the first data file; the file index includes: the timestamp and the data offset address;

[0018] Associate the channel index and the file index, including:

[0019] For each channel, the storage address of the data collected by the corresponding sensor in the multi-type data file is determined by the position in the first data file;

[0020] After obtaining the storage address of the data collected by the sensor corresponding to the channel in the multi-type data file, the timestamp, data offset address and data length of the corresponding data frame are obtained starting from the storage address. The file index corresponding to the channel is established, and the number of file indexes and the corresponding file names are established.

[0021] The number of file indices and the file names are written back to the channel index.

[0022] Preferably, in the above scheme, the sensor refers to the hardware device installed in the vehicle system to record data during vehicle operation, including any one or more of the following sensors: millimeter-wave radar, lidar, monocular / dual-lens camera, and satellite navigation;

[0023] And / or, the recording device is directly connected to the vehicle's sensors, or the sensors and recording device are connected to the vehicle's bus.

[0024] This specification also provides a method for playing back multiple types of data files, applied to a playback device, including:

[0025] After the multi-type data files are stored in the playback processing device, a corresponding file index is obtained based on the multi-type data files; wherein, the multi-type data files and the file index are files formed by the multi-type data file storage method described in any one of the claims of this application;

[0026] Obtain playback time information;

[0027] Based on the playback time information and the timestamp in the file index, determine the target storage address corresponding to the playback time information;

[0028] Based on the target storage address, the corresponding data frame is obtained and played back.

[0029] Preferably, when the multi-type data file is indexed by a corresponding channel, the corresponding target sensor is first determined in the channel index based on the playback time information and the recording time information; then, based on the playback time information, the timestamps in the file index corresponding to the target sensor are traversed to determine the target storage address.

[0030] Preferably, when the file index includes the timestamp and the data offset address, the corresponding data offset address is first determined in the file index based on the playback time information and the timestamp; then, the target storage address is determined based on the data offset address.

[0031] Preferably, when the file index also includes a data length, the data frame of the data length is obtained according to the target storage address.

[0032] Preferably, in the above scheme, the file index corresponding to the target sensor is determined according to the file name in the channel index.

[0033] This specification also provides an embodiment of a multi-type data file storage device, including:

[0034] Recording module: Records data collected by M sensors into multiple data files;

[0035] The index module creates file indexes;

[0036] The file index represents the relationship between the timestamps of data frames in the multi-type data files and their corresponding storage addresses, and the timestamp represents the time when the recording device records the corresponding data frame.

[0037] This specification also provides a playback device for multiple types of data files, including:

[0038] Creation module: After the multi-type data files are stored in the playback processing device, a corresponding file index is obtained based on the multi-type data files; wherein, the multi-type data files and the file index are files formed by the multi-type data file storage method described in any one of the claims of this application;

[0039] Acquisition module: Acquires playback time information;

[0040] Addressing module: Determines the storage address corresponding to the playback time information based on the playback time information and the timestamp in the file index;

[0041] Playback module: Based on the target storage address, obtain the corresponding data frame for playback.

[0042] This specification also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described multi-type data file storage method or playback method.

[0043] This specification also provides a computer storage medium that stores computer-executable instructions. When the computer-executable instructions are executed by a processor, the above-described multi-type data file storage method or playback method is performed.

[0044] Compared with the prior art, the beneficial effects that can be achieved by the above-mentioned at least one technical solution adopted in the embodiments of this specification include at least the following: after establishing the file index, the storage address corresponding to the playback time is obtained according to the timestamp of the playback time in the file index, and the data file is read from the corresponding storage address for playback. It is only necessary to traverse the file index to find the data for playback. Since the content stored in the file index is relatively small, the playback efficiency is improved. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 2 This is a flowchart illustrating a method for playing back multiple types of data files provided in an embodiment of this application;

[0047] Figure 3 This is a flowchart illustrating another method for playing back multiple types of data files provided in an embodiment of this application;

[0048] Figure 4 This is a schematic diagram of the structure of a channel index provided in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of a file index structure provided in an embodiment of this application;

[0050] Figure 1 This is a schematic diagram of the structure of a playback device for multiple types of data files provided in an embodiment of this application. Detailed Implementation

[0051] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0052] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0054] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0055] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.

[0056] Various sensors installed on vehicles, such as millimeter-wave radar, lidar, monocular / dual-lens cameras, and satellite navigation systems, continuously sense the surrounding environment while the vehicle is in motion, collect data, identify, detect, and track static and dynamic objects, and perform system calculations and analyses in conjunction with navigation map data. After data collection, relevant software can be used to present the data collected from each vehicle in real time or for later playback. Synchronous playback of various data sets is particularly important during playback.

[0057] To achieve synchronized playback, heterogeneous data needs to be unified to the same timestamp and then aligned for playback. Current playback methods require sorting the collected data by timestamp and processing it into a fixed data format. Data that doesn't meet these requirements cannot be played back. Furthermore, playing data at a specific time requires traversing a large amount of data, which is time-consuming for large files, reducing playback efficiency and user experience. Additionally, if only timestamp indexing is used, retrieving data at a specific time point requires looping through the file. First, the first data entry is read, and its time point is checked against the desired time. If it doesn't match, the entry is skipped, and the next entry continues. If the file is large, typically in gigabytes (G), looping to the desired time point takes a significant amount of time. While binary search or similar looping algorithms can improve efficiency, they still involve multiple file loops. Each time the user specifies a playback time point, this looping process increases exponentially, degrading the user experience and causing playback stuttering.

[0058] Another scenario is indexing solely by timestamps. However, this only supports time-series files, meaning the data in the file strictly increments in time. But in an automotive system, a single bus may record data from multiple sensors. The timing of the data sent by the sensors through the bus is not fixed, so the timing of the data frames recorded in the file is also not fixed. Such files cannot be replayed using only time indexing.

[0059] Based on this, the embodiments of this specification propose a processing scheme: for the acquired multi-type data files, establish corresponding file indexes and channel indexes, obtain the storage address by the channel index and file index, read the data according to the storage address, and replay the data.

[0060] The technical solutions provided by the various embodiments of this application are described below with reference to the accompanying drawings.

[0061] like Figure 2 As shown in the embodiments of this specification, a method for playing back multiple types of data files is provided, including:

[0062] Step 1: Obtain the corresponding file index based on the acquired multi-type data files.

[0063] Specifically, after multiple types of data files are stored in the playback processing device, the corresponding file index is obtained based on the multiple types of data files.

[0064] Among them, the multi-type data files originate from data collected by M sensors and recorded by the recording device; the file index represents the relationship between the timestamp of the data frame in the multi-type data file and the corresponding storage address, and the timestamp represents the time when the recording device records the corresponding data frame.

[0065] In the embodiments of this specification, the recording device is used to acquire data collected by sensors. The recording device can be connected to the vehicle's bus to acquire bus data, etc.

[0066] The recording device described in this specification is not limited; for example, it can be a recorder. A recorder is a data recording device that is connected to a vehicle, such as directly connected to sensors on the vehicle (e.g., radar, cameras, vehicle speed sensors), or: the sensors are connected to the vehicle's bus, and the recorder is connected to the bus to indirectly obtain and record this data.

[0067] In the embodiments of this specification, the playback processing device is used to receive and store data sent by the recording device. For example, the playback processing device can be a host computer, etc.

[0068] In the embodiments of this specification, the source of various types of data files is not limited. For example, it can be data obtained by various sensors in the vehicle system, and then files generated based on the data.

[0069] The sensors may include millimeter-wave radar, lidar, mono / dual-lens cameras, and satellite navigation systems.

[0070] Specifically, a file index is built based on the timestamp and corresponding storage address of each data frame in the acquired multi-type data files. When reading data, the corresponding storage address can be determined based on the timestamp, and then the data frame can be read out based on the storage address.

[0071] The embodiments in this specification do not limit the types of data files. For example, they can be bus data, video data, and radar data, etc.

[0072] The embodiments in this specification do not limit the method of establishing the file index. For example, the file index can be established based on the timestamps of multiple types of data files and the corresponding data offset addresses and data lengths.

[0073] Furthermore, the embodiments in this specification do not limit the format of various types of data files. For example, they can be measurement data format (MDF), binary data format (BLF), American Standard Code for Information Interchange (ASC), multimedia container (MKV), packet capture (PCAP), and audio video interleaved (AVI) files, among other formats.

[0074] Step 2: Obtain playback time information.

[0075] The playback time information refers to the playback time specified by the user.

[0076] In the embodiments of this specification, the playback time information can be a point in time or a period of time, which can be set according to the specific situation.

[0077] Step 3: Determine the target storage address corresponding to the playback time based on the playback time information and the timestamp in the file index.

[0078] Specifically, when a user needs to replay data from a certain time, the corresponding time is the replay time information. Based on the timestamp of the file index and the replay time information, the target storage address corresponding to the replay time is determined.

[0079] In the embodiments of this specification, there are no restrictions on the method of determining the target storage address corresponding to the playback time information based on the timestamp and playback time information. For example, the storage address corresponding to the playback time is determined based on whether the playback time information is equal to the timestamp.

[0080] The target storage address refers to the storage address of the data to be replayed in the multi-type data file. For example, the target storage address can be represented as the nth byte stored in the multi-type data file.

[0081] Step 4: Based on the target storage address, obtain the corresponding data frame for playback.

[0082] Specifically, once the target storage address is obtained, the corresponding data frame is read from the multi-type data file according to the target storage address, and then the data frame is replayed.

[0083] In this embodiment, a file index is created based on multiple types of data files. Then, the corresponding target storage address is determined based on the playback time information and the timestamp in the file index. Finally, the data frame is read from the target storage address and played back. Compared with the prior art, it is not necessary to preprocess and convert multiple types of data files into a fixed format, reducing the processing steps. Furthermore, it is not necessary to sort the multiple types of data files according to time, improving the flexibility of data storage. Moreover, by using the file index, it is not necessary to read multiple types of data files, improving playback efficiency and user experience.

[0084] The playback method for multiple types of data files provided in the embodiments of this specification further includes: Step 5: Obtain the corresponding channel index based on the multiple types of data files.

[0085] The channel index represents the relationship between each sensor and its corresponding recording time information, while the recording time information represents the time period from the recording device to the data file of the corresponding sensor.

[0086] In the embodiments of this specification, there may be multiple sensors. Therefore, in order to distinguish the acquisition channels of each data frame, each sensor can be numbered or marked, and each sensor has a corresponding file index.

[0087] Furthermore, based on the playback time information and the timestamps in the file index, the target storage address corresponding to the playback time information is determined, including: determining the corresponding target sensor in the channel index based on the playback time information and the recording time information; and determining the target storage address by traversing the timestamps in the file index corresponding to the target sensor based on the playback time information.

[0088] In the embodiments of this specification, there is no restriction on the traversal method for determining the target storage address by traversing the timestamps in the file index corresponding to the target sensor based on the playback time information. It can be set according to the specific situation. For example, partial traversal or full traversal can be performed.

[0089] Furthermore, after determining the corresponding target sensor in the channel index based on the playback time information and the recording time information, the method also includes: determining the file index corresponding to the target sensor based on the file name in the channel index.

[0090] By using channel indexes and file indexes, there are no restrictions on the formats of the various data files obtained. Furthermore, the channel indexes and file indexes can be associated, allowing for quick location of the corresponding storage location by retrieving data from the channel indexes and file indexes. This eliminates the need to read multiple data files of different types, improving playback efficiency and user experience.

[0091] In addition, the timestamps of two adjacent timestamps in the file index can be the timestamps of two data frames that are adjacent in time, or the timestamps of two non-adjacent data frames. The time interval between two adjacent timestamps can be uniform or non-uniform. As long as the data offset address is accurate and the data length is appropriate, the data frames required for synchronous playback can be obtained.

[0092] As can be seen, once an index is established, location playback can be achieved. On this basis, in order to facilitate better synchronous playback, when indexing different types of data (such as two or more types of data), the timestamps in the file index can be the same or similar. For example, for data of type A and data of type B, each timestamp in the file index of data of type A can find a timestamp that is the same or similar in the file index of data of type B.

[0093] like Figure 3 The embodiments of this specification provide a method for playing back multiple types of data files, including:

[0094] Step 1: After the multiple types of data files are stored in the playback processing device, obtain the corresponding file index and channel index based on the multiple types of data files.

[0095] Among them, such as Figure 3 As shown, the channel index includes the channel start time and the channel end time.

[0096] Specifically, based on the multi-type data files, the corresponding channel index is obtained, including: creating a corresponding channel for each sensor based on the file extensions of the data collected by the M sensors in the multi-type data files; for each channel, the recording time information of the corresponding sensor includes the channel start time and the channel end time; and creating a channel index based on the channel start time and the channel end time of the corresponding channel for each sensor.

[0097] Each sensor-collected data has a corresponding file extension. In the embodiments of this specification, the corresponding sensor can be determined based on the file extension.

[0098] The file extension based on the sensor can distinguish data collected by different sensors in multiple types of data files. Furthermore, a channel number is assigned to each sensor, a corresponding channel is established, and a corresponding file index is created for each channel starting at the storage location corresponding to each sensor in the type of data file.

[0099] Furthermore, such as Figure 3As shown, the channel index also includes: the position, file name, and number of file indexes in the first data file. For each channel, the storage address of the corresponding sensor-acquired data in the multi-type data file is determined by the position in the first data file; after obtaining the storage address of the sensor-acquired data for the corresponding channel in the multi-type data file, the timestamp, data offset address, and data length of the corresponding data frame are obtained starting from the storage address, a file index for the corresponding channel is established, and the number of file indexes and the corresponding file name are obtained; the number of file indexes and the file name are then written back to the channel index.

[0100] Specifically, for each channel, the position in the first data file indicates the storage location of the first data entry for that channel within the multi-type data file. By determining the position of the first data entry for each channel within the multi-type data file, the storage location of each channel's data file can be identified, allowing for the rapid creation of a corresponding file index for each channel.

[0101] Furthermore, after determining the storage location of the corresponding channel's data file through the position in the first data file, the data file is read from the corresponding storage location to obtain the corresponding timestamp, data offset address, and data length. The corresponding file index is then established, and after the file index is created, the number of file indexes and the corresponding file names are written back to the channel index.

[0102] The file name is determined by renaming the file index during its creation and then naming corresponding files in multiple data types with the same file name, so as to determine the file index through the channel index.

[0103] In the embodiments described in this specification, such as Figure 4 As shown, the channel index also includes whether the file index creation is complete, which is used to indicate whether the file index creation in the corresponding channel is complete. When it is displayed as "yes", it means that the file index has been created and can be accessed normally. When it is displayed as "no", it means that the file index has not been created and needs to be accessed after creation.

[0104] Step 2: Obtain playback time information.

[0105] Step 31: Determine the corresponding data offset address in the file index based on the playback time information and timestamp.

[0106] Specifically, such as Figure 4 As shown, the file index includes: timestamp and data offset address.

[0107] Step 32: Determine the target storage address based on the data offset address.

[0108] Step 41: Obtain the data frame of the data length according to the target storage address.

[0109] Among them, such as Figure 3 As shown, the file index also includes the data length.

[0110] Step 42: Play back the data frame.

[0111] In this context, sensors refer to hardware devices installed in the vehicle system to record data during vehicle operation, including radar and cameras. Data files refer to the data collected by sensors and recorded by the recorder during vehicle operation; this data includes various types such as bus, radar, and video data. The recorder is the system that receives data from sensors and writes it to a file; it can be a hardware device or a software system. A sensor channel refers to the channel number assigned to each sensor device after it is connected to the recorder, facilitating data recording and differentiation; this channel number is the sensor channel.

[0112] It is known that a timestamp refers to the total number of milliseconds since the recorder recorded that frame of data at 08:00:00 Beijing time on January 1, 1970 (or any other time) until now. Due to the high precision of bus data, the timestamp precision used in this application is in the millisecond range. Each piece of data has a timestamp. If the timestamp is the time of the first piece of data, it is the start time of the file; if the timestamp is the time of the last piece of data, it is the end time of the file.

[0113] like Figure 4 As shown, the channel index includes: channel start time, channel end time, file name, whether file index creation is complete, number of file indexes, and position of the first data file.

[0114] The recorded time information includes the channel start time and channel end time. The position in the first data file indicates the point in time when the first valid data is stored in the data file. The number of file indexes indicates the number of data frames in the file index. The file name indicates the name of the file generated by the recorder for this channel. Whether the file index ends indicates whether the file index generation for this channel has ended; yes means the file index generation is complete, no means the file index is being generated.

[0115] Specifically, by traversing the channel index, when the playback time is found to be between the start and end times of the sensor channel corresponding to a certain recording device, it can be determined that the playback time data is recorded in the file index corresponding to the recording device.

[0116] like Figure 5As shown, the file index includes: timestamp, data offset address, and data length. The target storage address is determined by the data offset address.

[0117] In the embodiments of this specification, the file index is associated with the channel index, that is, each channel in the corresponding channel index has a corresponding file index, and the corresponding file index can be found through the file name in the channel index.

[0118] Specifically, the data in the file index can be read, and then it can be determined whether the corresponding timestamp is equal to or falls within the playback time information. If so, the data offset address and data length can be obtained to get the target storage address of the data.

[0119] In the embodiments of this specification, there are no restrictions on the method of reading data in the file index. For example, reading can start from the first data in the file index.

[0120] The playback method for multiple types of data files provided in the embodiments of this specification further includes: marking the time point of each frame of data in the acquired multiple types of data files on the same time axis. For example, the time point of each frame of data in the acquired multiple types of data files can be marked on the same time axis by obtaining a timestamp (representing the time when the recording device recorded the corresponding data frame), and then the timestamp is used as the marking time point.

[0121] Specifically, on the same timeline, a time point is assigned to each frame of data for each type of data.

[0122] This specification does not limit the method of calibrating time points on the same time axis in the embodiments. For example, after acquiring bus data, video data, and radar data, a time point can be calibrated for each frame of each type of data on the same time axis. For example, video data may be 30 frames per second, and the time point corresponding to each frame can be calibrated on the same time axis; radar may be 10 frames per second, and the time point corresponding to each frame can be calibrated on the same time axis; the same applies to bus data, although the amount of data per second may be larger.

[0123] In practical applications, sensors in vehicle systems record real-time vehicle operation data into files via a recorder. These files include various formats such as MDF, BLF, ASC, MKV, PCAP, and AVI. According to the file format protocol, the recorder writes the time of data occurrence into the file header or data header information when recording data. The playback system receives this file and reads the start and end times from the file header or data header information. Multiple files can yield multiple sets of start and end time information. By sorting these time information, the start and end times of the entire playback item can be obtained and mapped onto the same timeline.

[0124] Then, based on the various types of data files after the calibrated time points, file indexes and channel indexes are created.

[0125] Specifically, the various types of data files after the calibration time point are mapped, and the data files are associated with sensor channels to obtain channel indexes and file indexes.

[0126] The embodiments in this specification do not restrict the creation method of channel index and file index. For example, the software can automatically create sensor channels by data file name, or the user can manually create sensor channels. After the sensor channels are created, the data file mapping association of the corresponding sensor channels is established by file name, or the user can manually configure it through the interface.

[0127] The channel index includes the following fields: channel file start time, channel file end time, whether file index creation is complete, number of file indexes, whether the device has created an index, and data file name; the file index includes the following fields: current frame timestamp, current frame offset address in the data file, and current frame data length in the data file.

[0128] The playback method for multiple data files provided in this specification marks time points on the same timeline for multiple data files, facilitating user verification of whether the actual vehicle operation in the experiment matches the sensor-recorded data, thus enabling quantitative analysis of vehicle operation. For example, vehicle driving records are stored as MKV files, and wiper data is stored in MDF files. With multiple data files calibrated on the same timeline, when playing the data, if the wipers activate in the video and the signal values ​​related to the wipers in the MDF file change and reach the correct values, it indicates that the wiper test matches the actual situation, and the user can subsequently use this recorded data for quantitative analysis.

[0129] In the embodiments described in this specification, the user selects the target time T0 to be played back in the human-computer interaction interface, i.e., the playback time information. The program can first check this target time T0 in the channel index to determine whether each sensor channel has data that can be played back. For example, if the channel index records that a certain sensor channel A received data during the period from the channel start time T1 to the channel end time T2, it is necessary to determine whether T0 falls between T1 and T2. If it falls within this interval, it means that there is data that can be played back. If there is data that can be played back, the corresponding data file of the sensor channel can be located, and then subsequent logic processing can be performed to realize the parsing and playback of the data.

[0130] In a specific implementation, the channel index is traversed. When the target time is found to be between the start and end times of a certain sensor channel, it indicates that the data file mapped to that sensor channel contains data for the target time. The next step is to search the file index corresponding to that sensor channel, read the first data entry of the file index, and compare the target time with the index data timestamp. If they are greater, continue reading the next file index entry and comparing the target time with the index data timestamp again. If they are greater, repeat the above steps until the read file index timestamp equals the target time. The file index is then searched according to this pattern to find the index point. The index data stores file information, file offset address, and data length. Using this information, the data of the corresponding length is retrieved from the specified offset address in the file for data parsing and playback.

[0131] In the embodiments described in this specification, data can be found simply by traversing the file index. Because the file index stores relatively little content, playback efficiency is very high. The file index structure is simple, retrieval is fast, and the file is small. The file index only stores the timestamp, file offset address, and data length. This structure can meet the needs of scenarios requiring rapid information retrieval. For example, a 1.65G video file generates a channel index of only 150 bytes and a file index of only 156K.

[0132] In the embodiments described in this specification, when reading data, given a playback time point, the system first searches the channel index to see if this time point is within the start and end time range of the data recorded in that channel. If not, it checks the next channel, and so on. When the time point is found in a channel index, the next step is to search in the file index corresponding to that channel. The file index is traversed until the index point is found. The index data stores file information, file offset address, and data length. Using this information, the system retrieves the corresponding length of data from the specified offset address in the file for playback.

[0133] In one implementation, for the first data frame of the first data file of the first type and the second data frame of the second data file of the second type in the multi-type data files, the number of data files stored during recording can be the same, for example, both are N, and the timestamps of the first data frames in the same data file match. This result can be achieved when recording data files. For example, when generating a first data file based on the acquired first data frame each time, a second data file is also generated for the acquired second data frame. The first data frame of the first type of data is, for example, image data acquired by an image acquisition component, and the second data frame of the second type is, for example, point cloud data acquired by radar. For example, if 5 minutes of image data can be divided into 5 data files, then the first data file contains the image data of the 1st minute, the second data file contains the image data of the 2nd minute, and so on; the point cloud data is similar. In this way, for any kth position in the N first data files and N second data files, the timestamp of the first frame image data in the kth first data file and the timestamp of the first frame point cloud data in the kth second data file are the same or similar, where similar can be understood as the difference being less than a threshold.

[0134] Each first data file may have a first-order identifier written in its name and / or attributes to identify its position among N first data files. Similarly, each second data file may have a sequence identifier written in its name and / or attributes to identify its position among N second data files.

[0135] Before replaying the corresponding data frame, the process further includes: if the indexed target storage address includes the target storage address of the data frame of the first data file and the target storage address of the second data file, then before replaying, the process further includes: extracting the order identifier of the first data file and the second data file, verifying whether the two order identifiers are the same, and only extracting the data frame for replaying if they are the same, and not extracting the data frame for replaying if they are different, and an error can be reported.

[0136] By using the above methods, the synchronization of playback can be effectively guaranteed, and the verification can be completed before the data frame is extracted, avoiding the synchronization of asynchronous data due to errors in index creation.

[0137] like ​ As shown in the embodiments of this specification, a playback device for multiple types of data files is also provided, including:

[0138] Creation Module: After the multi-type data files are stored in the playback processing device, the corresponding file index is obtained based on the multi-type data files. The multi-type data files originate from data collected by M sensors and recorded by the recording device. The file index represents the relationship between the timestamp of the data frame in the multi-type data file and the corresponding storage address. The timestamp represents the time when the recording device recorded the corresponding data frame.

[0139] Preferably, the creation module further includes: obtaining the corresponding channel index based on multiple types of data files, wherein the channel index represents the relationship between each sensor and the corresponding recording time information, and the recording time information represents the time period during which the recording device records the data file of the corresponding sensor.

[0140] Preferably, the channel index includes: channel start time and channel end time; the creation module includes: a first creation submodule: creating a corresponding channel for each sensor based on the file extensions of the data collected by M sensors in the multi-type data files; a second creation submodule: for each channel, the recording time information of the corresponding sensor includes the channel start time and channel end time; a third creation submodule: creating a channel index based on the channel start time and channel end time of the corresponding channel for each sensor.

[0141] Preferably, the channel index further includes: the position, file name, and number of file indexes in the first data file. The playback device for multiple types of data files further includes: a processing module: for each channel, determining the storage address of the corresponding sensor-collected data in the multiple types of data files based on the position in the first data file; after obtaining the storage address of the sensor-collected data for the corresponding channel in the multiple types of data files, obtaining the timestamp, data offset address, and data length of the corresponding data frame starting from the storage address, establishing a file index for the channel, obtaining the number of established file indexes and the corresponding file names; and writing the number of file indexes and file names back to the channel index.

[0142] Preferably, it further includes: an indexing module: determining the file index corresponding to the target sensor based on the file name in the channel index.

[0143] Acquisition module: Acquires playback time information.

[0144] Addressing module: Determines the storage address corresponding to the playback time information based on the playback time information and the timestamp in the file index.

[0145] The addressing module includes: a first addressing submodule: determining the corresponding target sensor in the channel index based on the playback time information and the recording time information; and a second addressing submodule: determining the target storage address by traversing the timestamps in the file index corresponding to the target sensor based on the playback time information.

[0146] Preferably, the file index includes: a timestamp and a data offset address; the addressing module includes:

[0147] The third addressing module determines the corresponding data offset address in the file index based on the playback time information and timestamp; the fourth addressing module determines the target storage address based on the data offset address.

[0148] Playback module: Based on the target storage address, it retrieves the corresponding data frames for playback.

[0149] Preferably, the file index further includes data length; the playback module includes: a first playback submodule: obtaining a data frame with the data length according to the target storage address; and a second playback submodule: playing back the data frame.

[0150] This specification also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described method for playing back multiple types of data files.

[0151] This specification also provides a computer storage medium that stores computer-executable instructions, which, when executed by a processor, perform the aforementioned method for playing back multiple types of data files.

[0152] In this specification, the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the product embodiments described later, since they correspond to the methods, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions in the system embodiments.

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for storing multiple types of data files, characterized in that, Applied to recording devices, including: Data collected by M sensors is recorded into multiple data files of various types, and file indexes and channel indexes are created. The file index represents the relationship between the timestamps of data frames in the multiple data files and their corresponding storage addresses; the timestamps represent the time when the recording device records the corresponding data frame. The channel index represents the relationship between each sensor and its corresponding recording time information; the recording time information represents the time period during which the recording device records data from the corresponding sensor's data file. The file index includes the timestamps and data offset addresses. The channel index includes: channel start time, channel end time, position in the first data file, file name, and number of file indexes; wherein, for each channel, the recording time information of the corresponding sensor includes the channel start time and the channel end time; The process of creating a channel index includes: creating a corresponding channel for each sensor based on the file extension of the data collected by the M sensors in the multi-type data file; and creating the channel index based on the channel start time and channel end time corresponding to each sensor. Associating the channel index and the file index includes: In the multi-type data files, a corresponding file index is created for each channel, starting at the storage location corresponding to each sensor. For each channel, the storage address of the data collected by the corresponding sensor in the multi-type data file is determined by the position in the first data file; After obtaining the storage address of the data collected by the sensor corresponding to the channel in the multi-type data file, the timestamp, data offset address and data length of the corresponding data frame are obtained starting from the storage address. The file index corresponding to the channel is established, and the number of file indexes and the corresponding file names are established. The number of file indices and the file names are written back to the channel index.

2. The method for storing multiple types of data files according to any one of claims 1, characterized in that, Sensors refer to hardware devices installed in vehicle systems to record data during vehicle operation, including any one or more of the following sensors: millimeter-wave radar, lidar, mono / dual-lens cameras, and satellite navigation. And / or, the recording device is directly connected to the vehicle's sensors, or the sensors and recording device are connected to the vehicle's bus.

3. A method for playing back multiple types of data files, characterized in that, Used in playback devices, including: After the multi-type data files are stored in the playback processing device, a corresponding file index is obtained based on the multi-type data files; wherein the multi-type data files and the file index are files formed by the multi-type data file storage method according to any one of claims 1-2; Obtain playback time information; Based on the playback time information and the timestamp in the file index, determine the target storage address corresponding to the playback time information; Based on the target storage address, the corresponding data frame is obtained and played back.

4. The method for playing back multiple types of data files according to claim 3, characterized in that, When the multi-type data file is indexed by a corresponding channel, the corresponding target sensor is first determined in the channel index based on the playback time information and the recording time information; then, based on the playback time information, the timestamps in the file index corresponding to the target sensor are traversed to determine the target storage address.

5. The method for playing back multiple types of data files according to claim 3, characterized in that, When the file index includes the timestamp and the data offset address, the corresponding data offset address is first determined in the file index based on the playback time information and the timestamp. Then, the target storage address is determined based on the data offset address.

6. The method for playing back multiple types of data files according to claim 5, characterized in that, When the file index also includes data length, the data frame of the data length is obtained according to the target storage address.

7. The method for playing back multiple types of data files according to any one of claims 4-6, characterized in that, The file index corresponding to the target sensor is determined based on the file name in the channel index.

8. A multi-type data file storage device, characterized in that, include: Recording module: Records data collected by M sensors into multiple data files; The index module creates file indexes and channel indexes; Wherein, the file index represents the relationship between the timestamp of the data frame in the multi-type data file and the corresponding storage address, the timestamp representing the time when the recording device records the corresponding data frame; the channel index represents the relationship between each sensor and the corresponding recording time information, the recording time information representing the time period during which the recording device records the data file of the corresponding sensor; the file index includes: the timestamp and the data offset address; The channel index includes: channel start time, channel end time, position in the first data file, file name, and number of file indexes; wherein, for each channel, the recording time information of the corresponding sensor includes the channel start time and the channel end time; The process of creating a channel index includes: creating a corresponding channel for each sensor based on the file extension of the data collected by the M sensors in the multi-type data file; and creating the channel index based on the channel start time and channel end time corresponding to each sensor. The association module, used to associate the channel index and the file index, includes: In the multi-type data files, a corresponding file index is created for each channel, starting at the storage location corresponding to each sensor. For each channel, the storage address of the data collected by the corresponding sensor in the multi-type data file is determined by the position in the first data file; After obtaining the storage address of the data collected by the sensor corresponding to the channel in the multi-type data file, the timestamp, data offset address and data length of the corresponding data frame are obtained starting from the storage address. The file index corresponding to the channel is established, and the number of file indexes and the corresponding file names are established. The number of file indices and the file names are written back to the channel index.

9. A playback device for multiple types of data files, characterized in that, include: Creation module: After the multi-type data files are stored in the playback processing device, a corresponding file index is obtained based on the multi-type data files; wherein, the multi-type data files and the file index are files formed by the multi-type data file storage method according to any one of claims 1-2; Acquisition module: Acquires playback time information; Addressing module: Determines the target storage address corresponding to the playback time information based on the playback time information and the timestamp in the file index; Playback module: Based on the target storage address, obtain the corresponding data frame for playback.

10. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform: the multi-type data file storage method of any one of claims 1-2, or the multi-type data file playback method of any one of claims 3-7.

11. A computer storage medium, characterized in that, The computer storage medium stores computer-executable instructions, which, when executed by a processor, perform: the method for storing multiple types of data files according to any one of claims 1-2, or the method for playing back multiple types of data files according to any one of claims 3-7.

Images