A data storage method, apparatus, device, and computer-readable storage medium

Abstract

This invention discloses a data storage method, apparatus, device, and computer-readable storage medium, applied in the field of computer technology. The method includes: receiving a video stream and a bus message stream; the video stream includes encoded video frames carrying timestamps, and the bus message stream includes bus messages; using the timestamps of the encoded video frames as a reference, uniformly time-stamping the bus message stream to obtain a bus message stream with a unified time reference; performing sample processing on the bus message stream with a unified time reference based on a dynamic boundary strategy to generate bus samples; encapsulating the encoded video frames and bus samples into video tracks and bus tracks respectively, and writing the encapsulated track data into a multimedia container file according to timing rules. This invention achieves integrated and orderly storage of video data and bus data by adding an independent bus data track, enabling collaborative encapsulation and storage with the video track while ensuring universal format compatibility.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data storage method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] Currently, multi-source heterogeneous time-series data is crucial in scenarios such as status monitoring, accident backtracking algorithm iteration, and liability determination, and requires strict alignment on the timeline. Therefore, electronic devices need to record and manage this data synchronously. However, due to limitations such as system architecture, data format differences, and compatibility with general containers, separate storage in independent files is currently commonly used. During later data analysis, different files need to be read separately, and the data needs to be correlated and aligned again based on timestamps. This increases system I / O (input / output) overhead and processing complexity, and is also prone to alignment deviations due to system clock drift or data frame loss, leading to data timing errors and inaccurate backtracking.

[0003] Therefore, how to achieve high-precision synchronous storage of multi-source heterogeneous time-series data is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a data storage method, apparatus, device and computer-readable storage medium, which solves the problem of asynchronous multi-source heterogeneous time-series data storage in the prior art.

[0005] To address the aforementioned technical problems, the present invention provides a data storage method, comprising: Receive video stream and bus message stream; the video stream includes encoded video frames carrying timestamps, and the bus message stream includes bus messages; Using the timestamp of the encoded video frame as a reference, the bus message stream is uniformly time-stamped to obtain a bus message stream with a unified time reference. Based on a dynamic boundary strategy, the bus message stream with the unified time base is sampled to generate bus samples. The encoded video frame and the bus sample are respectively encapsulated into video track and bus track, and the encapsulated track data is written into a multimedia container file according to timing rules.

[0006] Optionally, the bus message stream with the unified time base is sampled based on a dynamic boundary strategy to generate bus samples, including: Calculate the cumulative duration and cumulative data volume of the current bus messages to be aggregated, and monitor the bus message identifiers simultaneously; When the cumulative duration reaches a preset time threshold, or the cumulative data volume reaches a preset capacity threshold, or when a bus message carrying an event trigger identifier is identified in the current bus message to be aggregated, the current bus message to be aggregated is sampled to generate the bus sample.

[0007] Optionally, after sampled the bus message stream with the unified time base based on the dynamic boundary strategy to generate bus samples, the method further includes: Obtain the decoding time of the bus sample on the multimedia container timeline, and use the decoding time as the global reference time of the bus sample; Calculate the time offset of each bus message within the bus sample relative to the global reference time, and perform timing compression on the bus sample based on the time offset; And / or, a preset compression algorithm is used to compress the payload data of the bus sample to obtain a compressed bus sample; Accordingly, the encoded video frames and the bus samples are respectively encapsulated into video tracks and bus tracks, and the encapsulated track data is written into a multimedia container file according to timing rules, including: The encoded video frame and the compressed bus sample are respectively encapsulated into video track and bus track, and the encapsulated track data is written into a multimedia container file according to the timing rules.

[0008] Optionally, the encoded video frame and the bus sample are respectively encapsulated into video tracks and bus tracks, and the encapsulated track data is written into a multimedia container file according to timing rules, including: The encoded video frames and the bus samples are respectively encapsulated into video tracks and bus tracks, and cached to the corresponding tracks; When the accumulated buffer data of the video track or bus track reaches the preset window duration, the timing progress of the video track and the bus track being written to the multimedia container file is compared. The track data with lagging timing progress is written to the multimedia container file in the form of data blocks until the file writing timing progress of the bus track and the video track is consistent.

[0009] Optionally, after performing video track encapsulation and bus track encapsulation on the encoded video frame and the bus sample respectively, and writing the encapsulated track data into the multimedia container file according to timing rules, the method further includes: Receive the target query time point; By using a binary search method, the coded video frame file offset corresponding to the target query time point and the bus sample file offset covering the target query time point are retrieved from the video track time mapping table and the bus track time mapping table, respectively. Based on the encoded video frame file offset and the bus file offset, jump to the corresponding storage location of the multimedia container file and read the corresponding video data and bus data.

[0010] Optionally, before performing video track encapsulation and bus track encapsulation on the encoded video frame and the bus sample respectively, and writing the encapsulated track data into the multimedia container file according to timing rules, the method further includes: If the multimedia container file to be written is MP4, then the created bus track is configured as a timed metadata track, the track type identifier of the bus track is set to metadata type, and a custom sample entry is added to the sample description table and configured with a unique four-character identifier. If the multimedia container file being written is in MKV format, then a private encoding identifier is configured for the bus track, and a sample format is defined through a private data field, which is used in conjunction with an index table to complete the track index configuration; and a time scale independent of the video track is configured for the bus track, with the time scale of the bus track used as the unit of measurement for the bus data time parameter.

[0011] Optionally, after sampled the bus message stream with the unified time base based on the dynamic boundary strategy to generate bus samples, the method further includes: The bus sample generated at the moment is written to a pre-writing decision by a preset cost function. If the bus sample is determined to be in a write-ready state, then the steps of encapsulating the bus sample into a bus track and writing it into a multimedia container file according to timing rules are executed.

[0012] The present invention also provides a data storage device, comprising: A receiving module is used to receive video streams and bus message streams; the video streams include encoded video frames carrying timestamps, and the bus message streams include bus messages; The synchronization module is used to use the timestamp of the encoded video frame as a reference to perform unified time stamping on the bus message stream, so as to obtain a bus message stream with a unified time reference. The sample processing module is used to perform sample processing on the bus message stream with the unified time base based on the dynamic boundary strategy to generate bus samples. The storage module is used to encapsulate the encoded video frame and the bus sample into video track and bus track respectively, and write the encapsulated track data into a multimedia container file according to timing rules.

[0013] The present invention also provides a data storage device, comprising: Memory, used to store computer programs; A processor for implementing the data storage method described above when executing the computer program.

[0014] The present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the steps of the data storage method described above.

[0015] The present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the data storage method described above.

[0016] As can be seen from the above technical solution, this invention receives video streams and bus message streams; the video stream includes encoded video frames carrying timestamps, and the bus message stream includes bus messages; using the timestamps of the encoded video frames as a reference, the bus message stream is uniformly time-stamped to obtain a bus message stream with a unified time reference; based on a dynamic boundary strategy, the bus message stream with a unified time reference is sampled to generate bus samples; the encoded video frames and bus samples are respectively encapsulated into video tracks and bus tracks, and the encapsulated track data is written into a multimedia container file according to timing rules. The beneficial effects of this invention are: this invention utilizes the native multi-track extension capability of the multimedia container without changing the original specifications and architecture of the container, and completes the extended encapsulation by adding independent bus tracks, possessing good standard compatibility; constraining two types of heterogeneous data with a unified timing reference, and aggregating and storing video and bus data in the same container file through dual-track independent encapsulation and timing fusion writing, realizing integrated and orderly storage of multi-source heterogeneous data.

[0017] In addition, the present invention also provides a data storage device, apparatus, and computer-readable storage medium, which also have the above-mentioned beneficial effects. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a data storage method provided in an embodiment of the present invention; Figure 2 An example diagram of a sampling processing method based on a dynamic boundary strategy provided in an embodiment of the present invention; Figure 3 A timestamp comparison diagram provided for an embodiment of the present invention; Figure 4 An example diagram of a sample structure provided in an embodiment of the present invention; Figure 5 A flowchart illustrating a data storage method provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a data storage device provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of a data storage device provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] First, several terms used in this invention will be explained: Sample: Logical data unit within the track.

[0022] Data block: A contiguous unit of storage when a file is physically written. When writing a file, one or more consecutive samples are grouped together into a chunk (data block) and written to disk.

[0023] Video track: One sample typically corresponds to one frame or one access unit.

[0024] Bus track: A sample typically corresponds to a group of aggregated bus messages.

[0025] With the widespread adoption of intelligent driving assistance functions (such as AEBS (Advanced Emergency Braking System) and ACC (Adaptive Cruise Control), in-vehicle electronic devices need to simultaneously record and manage information from multiple data sources. Video streams are used to reconstruct the visual environment surrounding the vehicle, while CAN (Controller Area Network) / CAN-FD (CAN with Flexible Data Rate) bus message streams are used to record critical data such as vehicle status, sensor targets, and control decisions. These two types of data have irreplaceable value in scenarios such as accident investigation, algorithm iteration, and liability determination, and must be strictly aligned on the timeline. Due to limitations such as system architecture, data format differences, and compatibility with general containers, video data and CAN bus message data are typically stored independently. Video is stored as MP4 or H.264 files, while CAN messages are stored as binary logs or database files. This separate storage method requires reading two files separately during verification and analysis, and then performing secondary correlation and alignment through timestamps. This not only increases I / O overhead, but also often causes alignment errors due to system clock drift or frame loss, resulting in data timing disorder problems such as "audio and video desynchronization".

[0026] To address the aforementioned issues, this invention provides a data storage method that, without violating standard multimedia container specifications, encapsulates bus message streams as independent data tracks and video tracks, utilizes the container's native timeline and indexing mechanisms to achieve high-precision synchronous storage and rapid positioning of heterogeneous vehicle data (video streams and bus data streams), and reduces storage overhead through an optimized sample structure.

[0027] Please refer to the details. Figure 1 , Figure 1 A flowchart illustrating a data storage method provided in an embodiment of the present invention. The method may include: S101: Receive video stream and bus message stream; the video stream includes encoded video frames carrying timestamps, and the bus message stream includes bus messages.

[0028] Each step in this embodiment can be performed by a designated electronic device. This method is applicable to all in-vehicle electronic devices that require synchronized storage of video data and vehicle bus data. Examples include in-vehicle dashcams, ADAS (Advanced Driver Assistance System) active safety terminals, commercial vehicle video surveillance mainframes (MDVR), and DMS (Driver Monitoring System) / OMS (Occupant Monitoring System) devices. In this embodiment, the bus message can be a CAN bus message; the bus message stream in this embodiment can be a CAN bus message stream (or CAN message stream).

[0029] For example, the system receives the H.264 / H.265 bitstream and timestamp output from the video encoder, and simultaneously receives the CAN message stream output from the CAN controller. The video bitstream includes multiple encoded video frames (such as H.264 / H.265 encoded video frames) and their display timestamps (PTS). The bus message stream includes multiple bus messages.

[0030] S102: Using the timestamp of the encoded video frame as a reference, the bus message stream is uniformly time-stamped to obtain a bus message stream with a unified time reference.

[0031] Since the video is already a standard bitstream with its own timestamp, while the bus message stream does not have a standard timestamp, this embodiment uses the timestamp carried by each frame during video encoding as a unified reference standard, and adds a time tag aligned with it to each of the originally scattered bus messages, thereby placing the video stream and the bus data stream in the same time coordinate system.

[0032] S103: Based on the dynamic boundary strategy, sample the bus message stream with unified time base to generate bus samples.

[0033] It should be noted that video streams use encoded frames as naturally independent structured data units, with built-in standard timestamps and complete encapsulation attributes, which can be directly adapted to the writing specifications of multimedia container tracks. Therefore, no secondary aggregation processing is required, and video frames can be directly used as video samples. However, CAN bus messages are short, have a high transmission frequency, and are fragmented and disordered, lacking unified encapsulation boundaries and regular storage units. If they are written directly line by line, it will cause serious data fragmentation, high storage redundancy, and difficulty in accurately matching the video timing. Therefore, it is necessary to sample and integrate the bus message stream to form standardized data units in order to achieve multi-track collaborative encapsulation and synchronous storage.

[0034] To adapt to the sudden changes in bus message transmission and reception rhythm and data fluctuations, and to ensure regular packet assembly timing while balancing storage efficiency and video frame timing alignment, this embodiment employs a sampled processing method using a dynamic boundary strategy, rather than a fixed "frame rate" concept. This embodiment does not impose specific limitations on the dynamic boundary strategy. For example, it could be based on dynamic adjustment of the time window, such as automatically scaling the time segmentation interval according to the density of bus message transmission and reception—shrinking the time window when messages are dense and expanding it when messages are sparse; or it could be based on adaptive segmentation based on timing deviations, such as dynamically correcting the bus packet assembly segmentation point by combining the video frame time reference to ensure strict alignment of the dual-track timing.

[0035] Furthermore, the above-mentioned sample processing of the bus message stream with unified time base based on the dynamic boundary strategy to generate bus samples may specifically include: counting the cumulative duration and cumulative data volume of the current bus messages to be aggregated, while monitoring the bus message identifier; when the cumulative duration reaches a preset time threshold, or the cumulative data volume reaches a preset capacity threshold, or a bus message carrying an event trigger identifier is identified in the current bus messages to be aggregated, the current bus messages to be aggregated are sampled to generate bus samples.

[0036] Specifically, you can refer to Figure 2 , Figure 2 This diagram illustrates a sample processing method based on a dynamic boundary strategy, provided by an embodiment of the present invention. The sample boundary is determined using a logical "OR" of three conditions: .

[0037] in, Sample splitting trigger condition (Boolean value, range of values) ), The cumulative duration of the current bus messages to be aggregated (in bus track ticks, with a value range of...). ); The preset time threshold (in terms of bus track ticks; a typical value is the number of ticks corresponding to 500ms). The cumulative data volume (in bytes) of the current bus messages to be aggregated, with a value range of... ); The preset capacity threshold (bytes, typically 4096). Event trigger flag (Boolean value, range of values) ).

[0038] Thus, the preset time threshold To ensure that bus samples are generated periodically even when the bus is idle, and to avoid excessively large index intervals affecting positioning; a preset capacity threshold is used. To prevent bus storms from causing individual bus samples to become too large, affecting read buffers and parsing latency; event trigger flags. When a critical message (such as AEBS startup or a collision signal) is detected, it is set to true to immediately truncate the bus message and encapsulate it to generate a new independent bus sample. Furthermore, for easier indexing later, bus samples triggered by critical / special events are marked as critical bus samples (Sync Sample / KeyFrame) in the container index, supporting fast navigation.

[0039] Furthermore, after sampled processing of the bus message stream with unified time base based on the dynamic boundary strategy to generate bus samples, the process can further include: obtaining the decoding time corresponding to the bus sample on the multimedia container timeline, and using the decoding time as the global reference time of the bus sample; calculating the time offset of each bus message within the bus sample relative to the global reference time, and performing timing compression on the bus sample based on the time offset; and / or, using a preset compression algorithm to compress the payload data of the bus sample to obtain a compressed bus sample; correspondingly, video track encapsulation and bus track encapsulation are performed on the encoded video frames and bus samples respectively, and the encapsulated track data is written to the multimedia container file according to timing rules. Specifically, this can include: performing video track encapsulation and bus track encapsulation on the encoded video frames and compressed bus samples respectively, and writing the encapsulated track data to the multimedia container file according to timing rules. This embodiment achieves timing compression by combining the global reference time with the bus message time offset, and can further perform payload data compression to reduce the bus data storage capacity and reduce file redundancy.

[0040] Accordingly, the sample structure of the bus sample after timing compression and data compression includes a sample header and a sample body; the sample header includes the global reference time, the number of bus messages, and the preset compression algorithm identifier; the sample body includes N bus messages within a preset time period, and each bus message includes the content after timing compression and data compression.

[0041] For details, please refer to [link / reference]. Figure 3 and Figure 4 , Figure 3 This invention provides a timestamp comparison diagram. In terms of timestamp design, the traditional scheme uses a 64-bit absolute timestamp (occupying 8 bytes) for each bus message, while this scheme uses a 16-bit relative time offset (occupying only 2 bytes), which significantly reduces the size of the timestamp field. Figure 4This is an example diagram of a sample structure provided in an embodiment of the present invention. A bus sample contains all bus messages within a certain period of time (e.g., within 500ms, or an equivalent tick span). Its structure is as follows: The sample header contains BaseTime (optional, global reference time, used for recording or correction), MsgCount (number of bus messages), and CodecID (preset compression algorithm identifier). The sample body contains... A series of tightly packed bus messages. The time offset of each bus message. The calculation formula is: .

[0042] All times below are expressed in tick, the unit of the bus track time scale: The timestamp (range of values) is the time of acquisition of this bus message under a unified reference clock mapped to the bus track. ); The bus sample reference time (range of values) By default, the decoding time (DTS) of the bus sample on the bus track time axis is taken (in terms of the track tick). If necessary, it can be overwritten or corrected by the BaseTime in the Header. The offset of the bus message relative to the sample reference time (range of values) , This is a preset time threshold. Since the sample span is typically short (e.g., no more than 1 second),... It can be represented with fewer bits (such as a 16-bit unsigned integer in ticks), thereby reducing the size of the timestamp field and the sample size.

[0043] Furthermore, after the bus message stream with unified time base is sampled based on the dynamic boundary strategy to generate bus samples, the process further includes: making a pre-write decision judgment on the currently generated bus samples through a preset cost function; if the bus sample is determined to be in a state to be written, then performing bus track encapsulation on the bus sample and writing it into the multimedia container file according to the timing rules.

[0044] It should be noted that this section is primarily used in the write triggering decision stage after bus data sampling, especially in scenarios where system resources are limited and small bus samples are frequently generated, to reduce indexing overhead and the number of writes. This optimization is a pre-decision before interleaved writes: it first uses a preset cost function. It determines whether the currently generated bus sample has entered the write-ready state, and then the track interleaving write mechanism and video track data jointly perform physical write scheduling. A preset cost function is used. The formula is: .

[0045] in, The cost of writing (dimensionless, range of values) ); This is the data volume weighting coefficient (dimensionless, typically 0.6). This is the time weighting coefficient (dimensionless, typically 0.4). The cumulative data volume (in bytes) of the current bus messages to be aggregated, with a value range of... ); The preset capacity threshold (bytes, typically 4096). The cumulative duration of the current bus messages to be aggregated (in bus track ticks, with a value range of...). ); This is a preset time threshold (in bus track ticks; a typical value is the number of ticks corresponding to 500ms). This implementation can be used only... Writes may only be executed when a high-priority event is detected. This can be achieved by adjusting the weighting coefficients. It can find the best balance between storage compactness and real-time performance.

[0046] S104: Perform video track encapsulation and bus track encapsulation on the encoded video frames and bus samples respectively, and write the encapsulated track data into the multimedia container file according to the timing rules.

[0047] This step encapsulates the encoded video frames and the sampled bus data into separate video tracks and custom bus tracks, respectively. Then, the data from both tracks is written sequentially into the same multimedia container file, achieving synchronized and integrated storage of heterogeneous video and bus data. Specifically, the video track encapsulation of encoded video frames can be performed according to a common video track format, including track structure configuration, sample regularization, and timing field encapsulation, which will not be detailed here. However, when encapsulating bus samples into bus tracks, an independent bus track structure needs to be created, and the encapsulation configuration needs to be adapted to the current multimedia container format. Therefore, this independent bus track needs to be defined in advance. It should also be noted that this embodiment does not limit the specific format of the multimedia container file, as long as the container has the capability for multi-track encapsulation, time synchronization, index positioning, and the ability to carry custom data tracks—that is, a standard multimedia container file. For example, common media containers such as MP4 (the product / file format of ISO BMFF) and MKV (MKV media files) can be used. Among them, MP4 is the most mainstream implementation of the ISO basic media file format (ISO BMFF), while MKV is the file carrier corresponding to the Matroska container format.

[0048] For example, two standard multimedia container formats are illustrated here as methods for defining tracks to achieve the same storage: (1) If the multimedia container file to be written is MP4, then the created bus track is configured as a timed metadata track, the track type identifier of the bus track is set to metadata type, and a custom sample entry is added to the sample description table and configured with a unique four-character identifier.

[0049] In MP4 (ISO BMFF) containers, bus tracks can be encapsulated as Timed Metadata Tracks. The track type identifier (hdlr) is set to the meta metadata type within the track (trak) structure, improving compatibility with general-purpose players. Simultaneously, a custom Sample Entry is defined in the STSD (Sample Description Box), and a unique FourCC four-character identifier (cand) is set for dedicated tools to quickly identify and parse bus data. Ordinary players automatically ignore the unknown custom metadata payload, ensuring no interference with the decoding and playback of normal video tracks.

[0050] (2) If the multimedia container file is written in MKV format, then a private encoding identifier is configured for the bus track, and a sample format is defined through the private data field, and the track index configuration is completed in conjunction with the index table; and a time scale independent of the video track is configured for the bus track, and the time scale of the bus track is used as the unit of measurement for the bus data time parameter.

[0051] Within the MKV (Matroska) container, a private CodecID can be used in the TrackEntry (track description information), and a custom-formatted binary description information can be carried through the CodecPrivate field, enabling random access to bus data in conjunction with the Cues (index table). Considering the transmission characteristics of bus data, it is recommended to set the bus track's timescale to 1000 (milliseconds) or 10000 (0.1 milliseconds), separating it from the 90000 commonly used for video tracks. To avoid measurement ambiguities caused by mixing different track timescales, this embodiment separates the bus track-related time parameters (such as t...). msg t base T max (etc.), uniformly represented by the smallest time scale unit (tick) of the track: when Timescale=1000, 1 tick=1ms; when Timescale=10000, 1 tick=0.1ms.

[0052] Furthermore, to optimize the write performance of the hard disk drive and embedded storage, the encoded video frames and bus samples are respectively encapsulated as video tracks and bus tracks, and the encapsulated track data is written to the multimedia container file according to timing rules. This includes: encapsulating the encoded video frames and bus samples as video tracks and bus tracks respectively, and caching them to the corresponding tracks; when the accumulated buffer data of the video track or bus track reaches the preset window duration, comparing the timing progress of the video track and bus track written to the multimedia container file, and writing the track data with lagging timing progress to the multimedia container file in the form of data blocks, until the file writing timing progress of the bus track and video track is consistent.

[0053] It should be noted that this embodiment first logically assigns the two types of data to their respective tracks, and then physically writes them to the same container file in an interleaved manner. That is, caching to the corresponding track is used to define the logical channel to which the data belongs; the physical writing is used to standardize the physical storage arrangement order of various types of data within the file. The physical writing process is explained in detail below: The preset time window can be understood as an "interleaving window" (e.g., 1 second). When the accumulated buffer data of any track reaches the preset window duration, a write decision is triggered: the current write progress of the video track and the bus track is compared; data from the track with a lagging timestamp is written to the file as a chunk until that track catches up with the progress of the other track. This ensures that temporally adjacent video frames and bus samples are also as close as possible to each other in terms of file physical address. The distance metric can be formalized as follows: .

[0054] in, For time points Nearby video data blocks; For time points Nearby bus data blocks; The starting byte offset of the data block in the file (range of values) ); It is a distance metric (byte difference) in the physical layout of a file, with the goal of getting as close to 0 as possible.

[0055] Furthermore, after performing video track encapsulation and bus track encapsulation on the encoded video frames and bus samples respectively, and writing the encapsulated track data into the multimedia container file according to timing rules, the process may further include: receiving the target query time point; retrieving the encoded video frame file offset and the bus sample file offset covering the target query time point from the video track time mapping table and the bus track time mapping table respectively using a binary search method; and jumping to the corresponding storage location in the multimedia container file based on the encoded video frame file offset and the bus file offset to read the corresponding video data and bus data. For example, when a user requests to locate a time point... A binary search is performed in the video track time map table to find... The corresponding video frames and their file offsets; simultaneously, a binary search is performed in the CAN track time map table to find the overlay. The bus sample and its file offset are obtained; finally, the data is read directly from the corresponding position using a Seek, and O(logN) time positioning is achieved by utilizing the standard container's native index structure and time mapping mechanism, which is much faster than the linear scanning of traditional log files (complexity...). This improves efficiency by several orders of magnitude.

[0056] The data storage method provided in this embodiment of the invention proceeds as follows: S101: Receive video stream and bus message stream; the video stream includes encoded video frames carrying timestamps, and the bus message stream includes bus messages; S102: Use the timestamps of the encoded video frames as a reference to uniformly time-stamp the bus message stream, resulting in a bus message stream with a unified time reference; S103: Perform sample processing on the bus message stream with a unified time reference based on a dynamic boundary strategy to generate bus samples; S104: Perform video track encapsulation and bus track encapsulation on the encoded video frames and bus samples respectively, and write the encapsulated track data into a multimedia container file according to timing rules. This method utilizes the native multi-track extension capability of the multimedia container without altering the original specifications and architecture of the container. It completes the extended encapsulation by adding independent bus tracks, exhibiting good standard compatibility. By constraining two types of heterogeneous data with a unified timing reference, and through dual-track independent encapsulation and timing fusion writing, it aggregates and stores video and bus data in the same container file, achieving integrated and orderly storage of multi-source heterogeneous data. Furthermore, a variable boundary sampling strategy is adopted, dynamically determining the bus sample splitting boundary based on three conditions: time span threshold, cumulative data volume threshold, and key event trigger signal. This offers the advantage of balancing index size and real-time performance, adapting to bus traffic fluctuations. Additionally, an interleaving write mechanism is employed, alternately writing video data blocks and bus data blocks according to the timestamp progress of each track. This ensures that data with similar timestamps are physically adjacent in the file. During reading, the file pointer can perform more sequential scanning, reducing frequent seeks between distant positions within the file, thereby improving playback smoothness. Simultaneously, it ensures sequential data write to disk, effectively reducing... This approach mitigates the risk of frame loss during multi-channel high-bandwidth data writing, extending the lifespan of the storage medium. Furthermore, it utilizes container extension points to define bus track identifiers and encoding / decoding descriptions (using hdlr / stsd for MP4 and TrackEntry / CodecID / CodecPrivate for MKV). Additionally, it employs a relative timestamp compression structure within the bus samples, storing only the offset relative to the sample's base time, thus reducing timestamp redundancy and allowing for optional load compression, significantly lowering storage overhead. Moreover, it leverages the standard container's native index structure and time mapping mechanism to achieve O(logN) level time positioning, a significant improvement over the linear scanning of traditional log files (complexity...). This improves efficiency by several orders of magnitude; furthermore, by using container indexes in conjunction with event-triggered slicing, key event samples are marked as Sync Samples, enabling the verification process to quickly jump to the moment the event occurred and accurately trace back.

[0057] For a clearer understanding of this invention, please refer to the following details. Figure 5 , Figure 5This is a flowchart illustrating a data storage method provided in an embodiment of the present invention. The data receiving and time synchronization module receives video frames and their display timestamps (PTS) output by the video encoder, and simultaneously receives the CAN message stream output by the CAN controller, uniformly stamping them with the same system timestamp as the video to ensure time consistency. The sample processing module buffers and aggregates the CAN messages, performing variable boundary segmentation. By calculating the relative time offset between the sample base time and the message, a bus compression sample is constructed, and optionally, lz4 (compression algorithm) / zstd (compression algorithm) compression can be applied to the bus sample load. The container encapsulation module maintains the container structure (using moov / mdat for MP4 as an example, and Segment / Cluster structures for MKV), and is responsible for multi-track interleaving and multiplexing, maintaining index and offset information (Chunk offset on the MP4 side, Cluster / Block positioning information on the MKV side). This solution utilizes the concept of "containerized multi-track fusion," which differs from traditional physical splicing or separate storage methods. Instead, it extends and defines independent bus tracks within ISO BMFF or Matroska standard containers using metadata / private data tracks. This multiplexes and encapsulates the CAN message stream and video stream, allowing mainstream players to typically play the video track normally while ignoring the private bus track. This ensures that the basic file flow and viewing are not affected, maintaining compatibility with the existing multimedia ecosystem. Furthermore, each track is mapped to its respective timescale, achieving alignment on the same timeline. The two tracks share a unified encapsulation index, reducing clock drift, frame loss interpolation errors, and alignment uncertainties caused by secondary alignment after file storage, thus improving alignment accuracy and consistency.

[0058] The data storage device provided in the embodiments of the present invention will be described below. The data storage device described below can be referred to in correspondence with the data storage method described above.

[0059] Please refer to the details. Figure 6 , Figure 6 A schematic diagram of a data storage device provided in an embodiment of the present invention may include: The receiving module 100 is used to receive video streams and bus message streams; the video streams include encoded video frames carrying timestamps, and the bus message streams include bus messages; Synchronization module 200 is used to use the timestamp of the encoded video frame as a reference to perform unified time stamping on the bus message stream to obtain a bus message stream with unified time reference. The sample processing module 300 is used to perform sample processing on the bus message stream with the unified time base based on the dynamic boundary strategy to generate bus samples. The storage module 400 is used to encapsulate the encoded video frame and the bus sample into video track and bus track respectively, and write the encapsulated track data into a multimedia container file according to timing rules.

[0060] Furthermore, based on the above embodiments, the sample processing module 300 may include: The statistical monitoring unit is used to count the cumulative duration and cumulative data volume of the current bus messages to be aggregated, and at the same time monitor the bus message identifier; The sample processing unit is used to sample the current bus message to be aggregated and generate the bus sample when the cumulative duration reaches a preset time threshold, or the cumulative data volume reaches a preset capacity threshold, or when a bus message carrying an event triggering identifier is identified in the current bus message to be aggregated.

[0061] Furthermore, based on the above embodiments, the data storage device may further include: The global reference time determination module is used to sample the bus message stream with the unified time reference based on the dynamic boundary strategy, generate bus samples, obtain the decoding time corresponding to the bus sample on the multimedia container timeline, and use the decoding time as the global reference time of the bus sample. The timing compression module is used to calculate the time offset of each bus message within the bus sample relative to the global reference time, and to perform timing compression on the bus sample based on the time offset; And / or, a data compression module is used to compress the payload data of the bus sample using a preset compression algorithm to obtain a compressed bus sample; Correspondingly, the storage module 400 may include: The storage unit is used to encapsulate the encoded video frame and the compressed bus sample into video track and bus track respectively, and write the encapsulated track data into a multimedia container file according to timing rules.

[0062] Furthermore, based on the above embodiments, the storage module 400 may include: The track encapsulation unit is used to encapsulate the encoded video frame and the bus sample into video track and bus track respectively, and cache them into the corresponding track; The cross-writing unit is used to compare the timing progress of the video track and the bus track in writing to the multimedia container file when the accumulated buffer data of the video track or the bus track reaches a preset window duration, and write the track data with the lagging timing progress to the multimedia container file in the form of data blocks until the file writing timing progress of the bus track and the video track are consistent.

[0063] Furthermore, based on the above embodiments, the data storage device may further include: The receiving module is used to receive the target query time point after performing video track encapsulation and bus track encapsulation on the encoded video frame and the bus sample respectively, and writing the encapsulated track data into a multimedia container file according to the timing rules. The retrieval module is used to retrieve the encoded video frame file offset corresponding to the target query time point and the bus sample file offset covering the target query time point from the video track time mapping table and the bus track time mapping table respectively by binary search. The reading module is used to jump to the corresponding storage location of the multimedia container file according to the offset of the encoded video frame file and the offset of the bus file, and read the corresponding video data and bus data.

[0064] Furthermore, based on any of the above embodiments, the data storage device may further include: The first track definition module is used to configure the created bus track as a timed metadata track if the multimedia container file being written is MP4, set the track type identifier of the bus track as metadata type, add a custom sample entry to the sample description table and configure a unique four-character identifier. The second track definition module is used to configure a private encoding identifier for the bus track if the multimedia container file being written is in MKV format, and to define a sample format through a private data field, and to complete the track index configuration in conjunction with an index table; and to configure a time scale for the bus track that is independent of the video track, and to use the time scale of the bus track as the unit of measurement for the bus data time parameter.

[0065] Furthermore, based on any of the above embodiments, the data storage device may further include: The decision module is used to sample the bus message stream with the unified time base based on the dynamic boundary strategy, generate bus samples, and then make a pre-writing decision judgment on the currently generated bus samples through a preset cost function. The writing module is used to perform the steps of encapsulating the bus sample on a bus track and writing it into a multimedia container file according to timing rules if the bus sample is determined to be in a write-ready state.

[0066] It should be noted that the order of the modules and units in the aforementioned data storage device can be changed without affecting the logic.

[0067] The data storage device provided in this embodiment of the invention includes a receiving module 100 for receiving video streams and bus message streams; the video stream includes encoded video frames carrying timestamps, and the bus message stream includes bus messages; a synchronization module 200 is used to uniformly time-stamp the bus message stream using the timestamps of the encoded video frames as a reference, resulting in a bus message stream with a unified time reference; a sample processing module 300 is used to perform sample processing on the bus message stream with a unified time reference based on a dynamic boundary strategy, generating bus samples; and a storage module 400 is used to encapsulate the encoded video frames and the bus samples into video tracks and bus tracks respectively, and write the encapsulated track data into a multimedia container file according to timing rules. This device utilizes the native multi-track extension capability of the multimedia container without altering the original specifications and architecture of the container, completing the extended encapsulation by adding independent bus tracks, and possesses good standard compatibility; constraining two types of heterogeneous data with a unified timing reference, and aggregating and storing video and bus data in the same container file through dual-track independent encapsulation and timing fusion writing, it achieves integrated and orderly storage of multi-source heterogeneous data. Furthermore, a variable boundary sampling strategy is adopted, dynamically determining the bus sample splitting boundary based on three conditions: time span threshold, cumulative data volume threshold, and key event trigger signal. This offers the advantage of balancing index size and real-time performance, adapting to bus traffic fluctuations. Additionally, an interleaving write mechanism is employed, alternately writing video data blocks and bus data blocks according to the timestamp progress of each track. This ensures that data with similar timestamps are physically adjacent in the file. During reading, the file pointer can perform more sequential scanning, reducing frequent seeks between distant positions within the file, thereby improving playback smoothness. Simultaneously, it ensures sequential data write to disk, effectively reducing... This approach mitigates the risk of frame loss during multi-channel high-bandwidth data writing, extending the lifespan of the storage medium. Furthermore, it utilizes container extension points to define bus track identifiers and encoding / decoding descriptions (using hdlr / stsd for MP4 and TrackEntry / CodecID / CodecPrivate for MKV). Additionally, it employs a relative timestamp compression structure within the bus samples, storing only the offset relative to the sample's base time, thus reducing timestamp redundancy and allowing for optional load compression, significantly lowering storage overhead. Moreover, it leverages the standard container's native index structure and time mapping mechanism to achieve O(logN) level time positioning, a significant improvement over the linear scanning of traditional log files (complexity...). This improves efficiency by several orders of magnitude; furthermore, by using container indexes in conjunction with event-triggered slicing, key event samples are marked as Sync Samples, enabling the verification process to quickly jump to the moment the event occurred and accurately trace back.

[0068] The data storage device provided in the embodiments of the present invention will be described below. The data storage device described below can be referred to in correspondence with the data storage method described above.

[0069] Please refer to Figure 7 , Figure 7 A schematic diagram of a data storage device provided in an embodiment of the present invention may include: Memory 10 is used to store computer programs; Processor 20 is used to execute computer programs to implement the data storage method described above.

[0070] The memory 10, processor 20, and communication interface 31 all communicate with each other through the communication bus 32.

[0071] In this embodiment of the invention, the memory 10 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 10 may store programs for implementing the following functions: Receive video streams and bus message streams; the video streams include encoded video frames carrying timestamps, and the bus message streams include bus messages; Using the timestamps of encoded video frames as a reference, the bus message stream is uniformly time-stamped to obtain a bus message stream with a unified time reference. Based on a dynamic boundary strategy, bus message streams with unified time bases are sampled to generate bus samples. The encoded video frames and bus samples are encapsulated into video tracks and bus tracks respectively, and the encapsulated track data is written into the multimedia container file according to the timing rules.

[0072] In one possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0073] Furthermore, memory 10 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores operating systems and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and handling hardware-based tasks.

[0074] Processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic device. Processor 20 can be a microprocessor or any conventional processor. Processor 20 can call programs stored in memory 10.

[0075] Communication interface 31 can be an interface for the communication module, used to connect with other devices or systems.

[0076] Of course, it should be noted that, Figure 7 The structure shown does not constitute a limitation on the data storage device in the embodiments of the present invention. In practical applications, the data storage device may include more than Figure 7 More or fewer components as shown, or combinations of certain components.

[0077] It is understood that if the data storage method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk, or optical disk, and other media capable of storing program code.

[0078] Based on this, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data storage method described above.

[0079] The following describes a computer program product provided by an embodiment of this application. The computer program product described below can be referred to in conjunction with other embodiments described herein.

[0080] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned disclosed data storage method.

[0081] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0082] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0083] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0084] The present invention has provided a detailed description of a data storage method, apparatus, device, and computer-readable storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data storage method, characterized in that, include: Receive video stream and bus message stream; the video stream includes encoded video frames carrying timestamps, and the bus message stream includes bus messages; Using the timestamp of the encoded video frame as a reference, the bus message stream is uniformly time-stamped to obtain a bus message stream with a unified time reference. Based on a dynamic boundary strategy, the bus message stream with the unified time base is sampled to generate bus samples. The encoded video frame and the bus sample are respectively encapsulated into video track and bus track, and the encapsulated track data is written into a multimedia container file according to timing rules.

2. The data storage method according to claim 1, characterized in that, Based on a dynamic boundary strategy, the bus message stream with the unified time base is sampled to generate bus samples, including: Calculate the cumulative duration and cumulative data volume of the current bus messages to be aggregated, and monitor the bus message identifiers simultaneously; When the cumulative duration reaches a preset time threshold, or the cumulative data volume reaches a preset capacity threshold, or when a bus message carrying an event trigger identifier is identified in the current bus message to be aggregated, the current bus message to be aggregated is sampled to generate the bus sample.

3. The data storage method according to claim 1, characterized in that, After sampled the bus message stream with the unified time base based on the dynamic boundary strategy to generate bus samples, the process further includes: Obtain the decoding time of the bus sample on the multimedia container timeline, and use the decoding time as the global reference time of the bus sample; Calculate the time offset of each bus message within the bus sample relative to the global reference time, and perform timing compression on the bus sample based on the time offset; And / or, a preset compression algorithm is used to compress the payload data of the bus sample to obtain a compressed bus sample; Accordingly, the encoded video frames and the bus samples are respectively encapsulated into video tracks and bus tracks, and the encapsulated track data is written into a multimedia container file according to timing rules, including: The encoded video frame and the compressed bus sample are respectively encapsulated into video track and bus track, and the encapsulated track data is written into a multimedia container file according to the timing rules.

4. The data storage method according to claim 1, characterized in that, The encoded video frame and the bus sample are respectively encapsulated into video track and bus track, and the encapsulated track data is written into a multimedia container file according to timing rules, including: The encoded video frames and the bus samples are respectively encapsulated into video tracks and bus tracks, and cached to the corresponding tracks; When the accumulated buffer data of the video track or bus track reaches the preset window duration, the timing progress of the video track and the bus track being written to the multimedia container file is compared. The track data with lagging timing progress is written to the multimedia container file in the form of data blocks until the file writing timing progress of the bus track and the video track is consistent.

5. The data storage method according to claim 1, characterized in that, After performing video track encapsulation and bus track encapsulation on the encoded video frame and the bus sample respectively, and writing the encapsulated track data into the multimedia container file according to timing rules, the method further includes: Receive the target query time point; By using a binary search method, the coded video frame file offset corresponding to the target query time point and the bus sample file offset covering the target query time point are retrieved from the video track time mapping table and the bus track time mapping table, respectively. Based on the encoded video frame file offset and the bus file offset, jump to the corresponding storage location of the multimedia container file and read the corresponding video data and bus data.

6. The data storage method according to any one of claims 1 to 5, characterized in that, Before performing video track encapsulation and bus track encapsulation on the encoded video frame and the bus sample respectively, and writing the encapsulated track data into the multimedia container file according to timing rules, the process also includes: If the multimedia container file being written is in MP4 format, then the created bus track is configured as a timed metadata track, the track type identifier of the bus track is set to metadata type, and a custom sample entry is added to the sample description table and configured with a unique four-character identifier. If the multimedia container file being written is in MKV format, then a private encoding identifier is configured for the bus track, and a sample format is defined through a private data field, which is used in conjunction with an index table to complete the track index configuration; and a time scale independent of the video track is configured for the bus track, with the time scale of the bus track used as the unit of measurement for the bus data time parameter.

7. The data storage method according to claim 1, characterized in that, After sampled the bus message stream with the unified time base based on the dynamic boundary strategy to generate bus samples, the process further includes: The bus sample generated at the moment is written to a pre-writing decision by a preset cost function. If the bus sample is determined to be in a write-ready state, then the steps of encapsulating the bus sample into a bus track and writing it into a multimedia container file according to timing rules are executed.

8. A data storage device, characterized in that, include: A receiving module is used to receive video streams and bus message streams; the video streams include encoded video frames carrying timestamps, and the bus message streams include bus messages; The synchronization module is used to use the timestamp of the encoded video frame as a reference to perform unified time stamping on the bus message stream, so as to obtain a bus message stream with a unified time reference. The sample processing module is used to perform sample processing on the bus message stream with the unified time base based on the dynamic boundary strategy to generate bus samples. The storage module is used to encapsulate the encoded video frame and the bus sample into video track and bus track respectively, and write the encapsulated track data into a multimedia container file according to timing rules.

9. A data storage device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data storage method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the steps of the data storage method as described in any one of claims 1 to 7.

Images