A video data self-description method and reconstruction method based on a global timeline

Abstract

The application provides a video data self-description method and reconstruction method based on a global time axis, and belongs to the technical field of video processing.The method comprises the following steps: constructing a global time axis through a system clock; acquiring at least two video streams; for each video stream, after detecting a starting signal of a video frame in the video stream, determining a current time axis value based on the global time axis, and taking the current time axis value as a timestamp of the video frame; and associating the timestamp with the video frame to obtain a self-description video.The method of the application can establish a unified time reference from the source by constructing a global time axis and assigning a unified timestamp to a video frame to form a self-description video, thereby avoiding the problems of insufficient software timestamp precision, delay jitter and non-unified time reference, and simultaneously realizing the alignment of multiple video streams by acquiring and analyzing the self-description video without relying on external hardware trigger signals and complex software scheduling.

Description

Technical Field

[0001] This invention belongs to the field of video processing technology, and specifically relates to a self-describing and reconstruction method for video data based on a global time axis. Background Technology

[0002] In multi-source video acquisition and fusion applications, such as infrared and visible light dual-spectrum imaging systems, frame-level synchronization of video data from different sensors is typically required to meet the needs of subsequent image fusion, target detection, and multimodal display. Because different video sources differ in acquisition clocks, transmission paths, and processing flows, time offsets and jitter often exist between multiple video streams. Therefore, achieving high-precision, low-latency video synchronization has become a key technical challenge in this field.

[0003] In existing technologies, a common approach is a synchronization mechanism based on software timestamps. This involves the CPU or driver obtaining the system time after video data enters system memory and appending a timestamp to each frame. While simple to implement, this method suffers from significant uncertainty due to the reliance on operating system scheduling for timestamp generation. Furthermore, multiple video streams are typically processed by different threads or units, making it difficult to maintain strict consistency in their time bases, thus hindering high-precision frame alignment.

[0004] Another approach is synchronization based on external hardware trigger signals. This method uses GPIO or dedicated synchronization signal lines to control multiple sensors to begin exposure or data output simultaneously. While this method provides a degree of consistency at the initial acquisition stage, its synchronization effect is limited to the moment of acquisition and cannot eliminate delays in subsequent data transmission, buffering, and processing. Furthermore, this approach increases system wiring complexity and places additional demands on the hardware interface, making it difficult to implement in complex systems.

[0005] In addition, some solutions construct a cache queue at the reconstruction end to sort and match different video streams, achieving time-based "soft alignment." However, this type of method heavily relies on the accuracy of timestamps, and existing timestamp mechanisms themselves suffer from insufficient precision. Furthermore, the cache alignment process often introduces additional latency, which can easily lead to system instability in high-bandwidth or high-frame-rate scenarios.

[0006] As can be seen, existing technologies generally suffer from problems such as inconsistent time bases, insufficient timestamp accuracy, reliance on software processing for synchronization, and high system complexity, making it difficult to meet the application requirements of high-precision video synchronization. Summary of the Invention

[0007] To address the problems in the background technology, this invention proposes a video data self-description method and reconstruction method based on a global time axis.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention proposes a video data self-description method based on a global time axis, comprising: Construct a global timeline using the system clock; At least two video streams are acquired. For each video stream, after detecting the start signal of a video frame in the video stream, the current time axis value is determined based on the global time axis, and the current time axis value is used as the timestamp of the video frame. By associating the timestamp with the video frame, a self-describing video is obtained.

[0009] Preferably, the global timeline is implemented using a counter, which continuously increments based on the system clock; and / or, The counter has a bit width of 64 bits.

[0010] Preferably, before detecting the start signal of a video frame in the video stream, the method further includes: The video frames of the video stream are preprocessed.

[0011] Preferably, detecting the start signal of a video frame in the video stream specifically involves detecting the start signal of a video frame in the video stream using an edge detection circuit.

[0012] Preferably, associating the timestamp with the video frame to obtain a self-describing video specifically involves: encapsulating the timestamp into a data header and inserting the data header before the video frame to obtain a self-describing video.

[0013] Preferably, the data header includes a synchronization identifier field, a video stream identifier field, a frame sequence number field, a timestamp field, and a data length field.

[0014] Preferably, when multiple video streams share the same transmission channel, they are sent sequentially according to the order of timestamps in the self-describing video.

[0015] Secondly, this invention proposes a video data reconstruction method based on a global time axis, comprising: Obtain the self-describing video for each video stream; The self-describing video is parsed to obtain a timestamp; Each video stream is aligned based on the timestamp.

[0016] Preferably, the step of parsing the self-describing video to obtain the timestamp specifically involves: The self-describing video is parsed to obtain the data header, and the timestamp field of the data header is extracted to obtain the timestamp.

[0017] Preferably, the matching of video streams based on the timestamp specifically includes: Map the self-describing video of each video stream to a unified timeline; Frame-level alignment is performed based on the difference in timestamps of each video stream on the unified timeline.

[0018] The beneficial effects of this invention are: The method of this invention constructs a global timeline and assigns a unified timestamp to video frames to form a self-describing video. This establishes a unified time reference from the source, avoiding problems such as insufficient software timestamp accuracy, latency jitter, and inconsistent time references. At the same time, by acquiring and parsing the self-describing video to achieve alignment of multiple video streams, it eliminates time offset and latency differences generated during the acquisition, transmission, and processing of multiple video streams without relying on external hardware trigger signals and complex software scheduling. This improves the synchronization accuracy of multiple video streams and the stability of the system.

[0019] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description and the drawings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram of the overall structure of the video data self-describing and reconstruction system of the present invention is shown; Figure 2 A schematic diagram of the FPGA internal video data timing calibration and packaging structure of the present invention is shown; Figure 3 A schematic diagram of the video data self-describing packet structure of the present invention is shown; Figure 4 This diagram illustrates the timing sequence of frame start detection and timestamp latching according to the present invention. Figure 5 A schematic diagram of video frame alignment based on a unified time axis according to the present invention is shown; Figure 6A flowchart of the video data reconstruction process based on timestamps of the present invention is shown; Figure 7 A flowchart of a video data self-description method based on a global time axis according to the present invention is shown; Figure 8 A flowchart of a video data reconstruction method based on a global time axis according to the present invention is shown. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Reference Figure 7 As shown, a self-description method for video data based on a global time axis specifically includes the following steps: To achieve frame-level time calibration of multiple video streams under a unified time reference, it is first necessary to establish a globally unified time reference system. Therefore, the methods mentioned above include: S11. Construct a global timeline using the system clock.

[0024] After the global timeline is constructed, it can provide a unified time reference for all video streams, thereby ensuring the consistency and comparability of subsequent timestamps.

[0025] To accurately time-stamp each frame of each video stream, after constructing a global timeline, the above method also includes: S12. Acquire at least two video streams. For each video stream, after detecting the start signal of a video frame in the video stream, determine the current time axis value based on the global time axis, and use the current time axis value as the timestamp of the video frame.

[0026] Through the above steps, each frame of video data in each video stream is assigned a unique and unified time identifier, providing a foundation for subsequent multi-stream video alignment.

[0027] In order to enable video frames to carry complete time information even without an external synchronization signal, thus achieving the self-describing characteristic of the data, the above method also includes: S13. Associate the timestamp with the video frame to obtain a self-describing video.

[0028] Self-describing videos can be directly used for transmission, parsing, and reconstruction, achieving frame-level alignment of multiple video streams without the need for additional synchronization mechanisms.

[0029] Reference Figure 8 As shown, a video data reconstruction method based on a global time axis specifically includes the following steps: To perform time-series reconstruction and frame-level synchronization on multiple time-calibrated self-describing videos, it is first necessary to acquire video data carrying a unified timestamp. Therefore, the methods mentioned above include: S21. Obtain the self-describing video of each video stream.

[0030] After obtaining the self-describing video from each video stream, the data already contains time information for synchronization alignment, providing a data foundation for subsequent parsing and matching.

[0031] To extract uniform time reference information from the encapsulated self-describing video, the above method further includes the following after acquiring the self-describing video: S22. Parse the self-describing video to obtain the timestamp.

[0032] Through the above steps, the timestamp corresponding to each frame can be separated from the self-describing video, so that the various video streams have a basis for comparison and matching under a unified time dimension.

[0033] To eliminate time offsets generated during the acquisition, transmission, and processing of multiple video streams and achieve high-precision frame-level synchronization, the above method also includes: S23. Align each video stream based on the timestamp.

[0034] The aligned multi-channel video streams have consistent timing and can be directly used for image fusion, display, or subsequent algorithm processing to complete the temporal reconstruction of the entire video data.

[0035] This invention constructs a global timeline and assigns a unified timestamp to video frames to form self-describing videos. This establishes a unified time reference from the source, avoiding problems such as insufficient software timestamp accuracy, latency jitter, and inconsistent time references. At the same time, by acquiring and parsing self-describing videos to achieve multi-channel video stream alignment, it eliminates time offset and latency differences generated during the acquisition, transmission, and processing of multiple video streams without relying on external hardware trigger signals and complex software scheduling. This improves the synchronization accuracy of multiple video streams and the stability of the system.

[0036] like Figure 1As shown, in a specific embodiment, both the aforementioned global time-axis-based video data self-describing method and reconstruction method are deployed in a field-programmable gate array (FPGA). This FPGA is located between the video sensor interface and the high-speed data transmission interface, and is used for timestamping and self-describing encapsulation of the input video data. The system supports at least two video inputs, including infrared video streams and visible light video streams. The overall system consists of a video input source, an FPGA processing system, and a data output and reconstruction terminal. After processing by the FPGA, the video data is transmitted to the reconstruction terminal via a high-speed interface for further processing.

[0037] First, a global timeline is constructed using the system clock. For example... Figure 2 As shown, specifically, this is achieved through a global timestamp generator module. This module generates a free-running counter based on the system clock. The counter's bit width is preferably 64 bits to avoid overflow issues during long-term operation. This counter continuously increments after system startup and provides a unified time reference signal to all video processing channels, thereby ensuring that different video streams have a comparable time base under the same global timeline.

[0038] Subsequently, the system acquires at least two video streams and performs frame start signal detection and timestamp latching at each video input interface. For example... Figure 2 and Figure 4 As shown, this module performs synchronization and filtering (preprocessing) on ​​the valid signals of the input video frames, and identifies stable frame start events through an edge detection circuit. Upon detecting the rising edge of the frame start, a timestamp latch operation is immediately triggered, reading the current global timeline value and using it as the timestamp for the video frame. This timestamp remains unchanged throughout the entire frame transmission, thus completing the time calibration of the video frame.

[0039] After obtaining the timestamp, the system associates the timestamp with the video frame to obtain a self-describing video. For example... Figure 2 and Figure 3 As shown, this is specifically achieved through a data encapsulation and timestamp embedding module: Before each frame of video data is output, a data header with a fixed format is first generated and inserted before the video frame to form a self-describing video with a timestamp. The data header includes a synchronization identifier field, a video stream identifier field, a frame sequence number field, a timestamp field, and a data length field. Among them, the timestamp field is used to indicate the global timeline position corresponding to the frame, which is the key basis for realizing the synchronization and alignment of subsequent multi-channel video.

[0040] The encapsulated self-describing video is output through a standard streaming interface and then enters the subsequent transmission module. For example... Figure 2As shown, in a multi-channel video system, a data scheduling module is further configured to manage the self-describing videos from different channels in a unified manner. This module optimizes the data transmission order based on the chronological order of the timestamps in the self-describing videos, ensuring that earlier data enters the transmission channel first, thereby reducing cross-channel transmission latency differences.

[0041] At the reconstruction end, the system acquires the self-describing video of each video stream, parses the self-describing video, and extracts the timestamp from the data header. For example... Figure 6 As shown, the reconstruction end first parses the self-describing video, establishes a cache queue for each video stream, and then matches the self-describing videos of different video streams based on timestamp information.

[0042] Furthermore, such as Figure 5 As shown, the reconstruction end maps self-describing videos from different video streams to a unified timeline and performs frame-level alignment based on the differences between timestamps, thereby obtaining time-synchronized video frame pairs. The aligned self-describing videos can be directly used for image fusion, display, or subsequent algorithm processing.

[0043] Based on the same inventive concept as the above method, this invention also proposes a self-describing system for global timeline video data, comprising: The global timeline construction module is used to construct a global timeline using the system clock. The current timeline value is determined and used to acquire at least two video streams. For each video stream, after detecting the start signal of the video frame in the video stream, the current timeline value is determined based on the global timeline, and the current timeline value is used as the timestamp of the video frame. The association module is used to associate the timestamp with the video frame to obtain a self-describing video.

[0044] Based on the same inventive concept as the above method, this invention also proposes a global time-axis video data reconstruction system, comprising: The self-describing video acquisition module is used to acquire the self-describing video of each video stream; The parsing module is used to parse the self-describing video and obtain a timestamp; An alignment module is used to align each video stream based on the timestamp.

[0045] Based on the same inventive concept as the above method, the present invention also proposes a device including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and the processor executes the above-described video data self-description method based on a global time axis or a video data reconstruction method based on a global time axis when executing the computer instructions.

[0046] Based on the same inventive concept as the above method, the present invention also proposes a computer-readable storage medium storing computer instructions thereon, which, when executed, can implement the above-described video data self-description method based on a global time axis or a video data reconstruction method based on a global time axis.

[0047] Any references to memory, storage, database, or other media used in the embodiments provided in this invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0048] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0049] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A self-describing method for video data based on a global time axis, characterized in that, include: Construct a global timeline using the system clock; At least two video streams are acquired. For each video stream, after detecting the start signal of a video frame in the video stream, the current time axis value is determined based on the global time axis, and the current time axis value is used as the timestamp of the video frame. By associating the timestamp with the video frame, a self-describing video is obtained.

2. The self-describing method for video data based on a global time axis according to claim 1, characterized in that, The global timeline is implemented using a counter, which continuously increments based on the system clock; and / or, The counter has a bit width of 64 bits.

3. The self-describing method for video data based on a global time axis according to claim 1, characterized in that, Before detecting the start signal of a video frame in the video stream, the method further includes: The video frames of the video stream are preprocessed.

4. The self-describing method for video data based on a global time axis according to claim 1, characterized in that, The detection of the start signal of a video frame in the video stream specifically involves detecting the start signal of a video frame in the video stream using an edge detection circuit.

5. The self-describing method for video data based on a global time axis according to claim 1, characterized in that, The step of associating the timestamp with the video frame to obtain a self-describing video specifically involves: encapsulating the timestamp into a data header and inserting the data header before the video frame to obtain a self-describing video.

6. The self-describing method for video data based on a global time axis according to claim 5, characterized in that, The data header includes a synchronization identifier field, a video stream identifier field, a frame sequence number field, a timestamp field, and a data length field.

7. The self-describing method for video data based on a global time axis according to claim 1, characterized in that, When multiple video streams share the same transmission channel, they are sent sequentially according to the order of timestamps in the self-describing video.

8. A video data reconstruction method based on a global time axis, characterized in that, include: Obtain the self-describing video for each video stream; The self-describing video is parsed to obtain a timestamp; Each video stream is aligned based on the timestamp.

9. The video data reconstruction method based on a global time axis according to claim 8, characterized in that, The process of parsing the self-describing video to obtain the timestamp is as follows: The self-describing video is parsed to obtain the data header, and the timestamp field of the data header is extracted to obtain the timestamp.

10. A video data reconstruction method based on a global time axis according to claim 8, characterized in that, The matching of video streams based on the timestamp specifically includes: Map the self-describing video of each video stream to a unified timeline; Frame-level alignment is performed based on the difference in timestamps of each video stream on the unified timeline.

Images