Multi-channel event video synchronous playback method and system

Abstract

The embodiment of the invention discloses a multi-channel event video synchronous playback method and system, and relates to the technical field of video monitoring. The method comprises the following steps: creating a global synchronization control instance and a plurality of decoding playing instances, wherein each instance corresponds to a video channel and is registered to the synchronization control instance; and establishing an independent video frame buffer queue for each channel. And starting the decoding playing instance to enter a decoding operation state, and sending a rendering request containing a timestamp of a frame to the synchronous control instance before the video frame obtained by decoding is rendered. The synchronization control instance executes synchronization control logic and compares the timestamp with a globally maintained current playing timestamp. And through centralized arbitration, the corresponding decoding playing instance is approved to render the frame only when the difference value of the two is within the preset fault-tolerant range. According to the method and the device, frame-by-frame synchronous control on multi-channel video playback is realized, each channel is forced to advance according to a unified timeline, and strict synchronization and experience continuity of cross-channel playback pictures are ensured.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology, and in particular to a method and system for synchronous playback of multi-channel event recording. Background Technology

[0002] Traditional multi-channel video synchronous playback solutions typically employ independent downloading and simultaneous playback for each channel. This works correctly when all channels have complete and continuous video recordings (e.g., 24 / 7 uninterrupted recording). As long as the playback rhythm of each player remains consistent, the picture will remain synchronized.

[0003] However, in practical intelligent monitoring applications, to save cloud storage space and bandwidth, devices often adopt event-triggered recording mode. This means that only channels that detect an event (such as motion detection) will trigger recording and upload it to the cloud, while channels without events will not upload still images. This results in the recordings stored in the cloud for each channel consisting of multiple, discontinuous event segments.

[0004] In this situation, the shortcomings of traditional synchronization solutions become apparent. Because events occur at different times on each channel, the uploaded video clips cannot be aligned on the timeline, resulting in situations where only some channels have video data at certain points in time, while others have none. If simple synchronous playback is still used, the interleaved "data available" and "data not available" intervals in the video streams of each channel lead to severely inconsistent playback progress, thus ruining the observation effect of multi-screen synchronous playback. Summary of the Invention

[0005] This application provides a method and system for synchronous playback of multi-channel event recordings, which can solve the problem in the prior art where the playback progress cannot be synchronized due to the non-overlapping recording times of each channel in multi-channel, event-triggered non-continuous recording scenarios.

[0006] The technical solution is as follows: Firstly, this application provides a method for synchronous playback of multi-channel event recordings, comprising the following steps: Create a synchronization control instance for global control, and create multiple decoding playback instances to correspond to multiple video channels respectively and register them all with the synchronization control instance; Acquire recording segments for the multiple video channels, and establish an independent frame queue for each video channel to buffer the video frame data of that channel; Each of the aforementioned decoding and playback instances is started to enter the decoding running state, and the video data belonging to its corresponding video channel is decoded to obtain video frames. Before rendering the obtained video frames, a rendering request for the video frame is sent to the synchronization control instance. The rendering request contains at least the timestamp of the video frame. The synchronization control instance receives and responds to the rendering request sent by any decoding playback instance, executes synchronization control logic, compares the video frame timestamp in the rendering request with the globally maintained current playback timestamp, and determines whether to allow the decoding playback instance to render the corresponding video frame based on whether the difference between the two is within a preset allowable error range.

[0007] Furthermore, the synchronization control logic also includes: Before the comparison and judgment, check the playback control status; If the playback control status is paused, the comparison and judgment are paused, and after waiting for a preset frame interval, the process returns to the start step.

[0008] Furthermore, the synchronization control logic also includes: Periodically traverse all video channels, query the data status of the frame queue corresponding to each channel, and count the number of channels whose frame queue is currently empty; If the number of empty channels in the frame queue equals the total number of channels, it is determined that playback has ended or data buffering is required, and the buffering wait mechanism is triggered.

[0009] Furthermore, the synchronization control logic also includes: If the number of channels with empty frame queues is less than the total number of channels, then iterate through all video channels and query the timestamp corresponding to the earliest frame in the frame queue of each channel. From all the earliest timestamps retrieved, determine the minimum value as the global earliest timestamp; The current playback timestamp of the global system is updated based on the earliest global frame timestamp.

[0010] Furthermore, updating the global current playback timestamp based on the earliest global frame timestamp includes: Compare the global earliest frame timestamp with the global current playback timestamp; If the global earliest frame timestamp is greater than the global current playback timestamp, then the global current playback timestamp is set to be equal to the global earliest frame timestamp.

[0011] Furthermore, the acquisition of video clips from multiple video channels includes: Create at least one download control instance; The download control instance queries the storage location corresponding to each video channel for available recording segments and forms a list of recording segments for each channel. Based on the list of video clips, control each channel to download data for at least one video clip.

[0012] Furthermore, before establishing an independent frame queue for each video channel, the method further includes: The downloaded video clip data is parsed, and each video frame and its corresponding timestamp information are extracted from the frame header of the data frame. The extracted video frames are stored in the frame queue established for each video channel according to their respective video channels.

[0013] Furthermore, the preset allowable error range is a value set based on a user-perceptible playback asynchrony time threshold.

[0014] Furthermore, the video recordings from the multiple video channels are discontinuous recordings uploaded to the cloud storage server based on an event-triggered mechanism.

[0015] Secondly, this application provides a system for synchronous playback of multi-channel event video recordings, comprising: The synchronization control module is configured to maintain the global current playback timestamp and execute synchronization control logic; Multiple decoding and playback modules, each corresponding to multiple video channels, are all communicatively connected to the synchronization control module; The decoding and playback module is configured to decode the recording data of its corresponding video channel, and after decoding the video frame but before rendering the video frame, send a rendering request for the video frame to the synchronization control module. The request includes the timestamp of the video frame. The synchronization control module is configured to, in response to a rendering request from any decoding playback module, execute the synchronization control logic to compare the video frame timestamp in the rendering request with the globally maintained current playback timestamp, and determine whether to allow the decoding playback module to render the corresponding video frame based on whether the difference between the two is within a preset allowable error range.

[0016] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following: By using a centralized synchronization control instance, arbitration is performed before the decoding and playback instances prepare to render each frame. Each instance must report the timestamp of its frame, and the synchronization control instance compares it with the global playback time benchmark, approving rendering only when the time deviation is within a preset tolerance range. Through this frame-by-frame control, this method forces all channels to advance according to a unified timeline, effectively solving the problem of fragmented and asynchronous playback progress caused by different start and end times of event-triggered recording, thus ensuring a consistent playback experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating the multi-channel event recording synchronous playback method provided in this application embodiment; Figure 2 A flowchart illustrating the process of handling video frames and constructing rendering requests for the decoding and playback example provided in this application. Detailed Implementation

[0019] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0020] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, systems, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0021] It should be noted that the data acquisition or information collection in this embodiment is performed after authorization by the user or the object of collection, and its process and purpose strictly follow the relevant regulations.

[0022] The block diagrams shown in the attached figures are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more hardware modules composed of smart chips, smart integrated circuits, or application-specific integrated circuits (ASICs), or in different network and / or processor devices and / or microcontroller devices.

[0023] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0024] like Figure 1 As shown, this application provides a method for synchronous playback of multi-channel event recordings, including the following steps: Step S1: Create a synchronization control instance for global control, and create multiple decoding playback instances to correspond to multiple video channels respectively and register them all with the synchronization control instance; Step S2: Acquire recording segments for the multiple video channels, and establish an independent frame queue for each video channel to buffer the video frame data of that channel; Step S3: Start each of the decoding and playback instances to enter the decoding running state, decode the video data belonging to its corresponding video channel to obtain video frames, and before rendering the obtained video frames, send a rendering request for the video frame to the synchronization control instance. The rendering request shall at least contain the timestamp of the video frame. Step S4: The synchronization control instance receives and responds to the rendering request sent by any decoding playback instance, executes synchronization control logic, compares the video frame timestamp in the rendering request with the globally maintained current playback timestamp, and determines whether to allow the decoding playback instance to render the corresponding video frame based on whether the difference between the two is within the preset allowable error range.

[0025] This embodiment creates a centralized synchronization control instance and submits the timestamp of each video frame to this instance for verification before each decoding playback instance renders the video frame. The synchronization control instance compares the reported timestamp with the globally maintained current playback timestamp, and only approves the rendering of the frame if the time difference between the two is within a preset allowable error range. This method centrally and in real-time arbitrates and controls the playback timing of each frame from the final stage of rendering output, thereby ensuring that the video frames of multiple channels can maintain strict alignment and coherence in the time dimension in cloud storage and non-continuous recording scenarios, improving the synchronization consistency of multi-screen playback and the user experience.

[0026] In one embodiment, step S1: creating a synchronization control instance for global control, and creating multiple decoding playback instances to correspond to multiple video channels respectively and registering them all with the synchronization control instance, may specifically include the following: Step S11: The program starts the initialization process, creating multiple download instances, multiple decoding and playback instances, and one synchronization control instance; The multi-channel download instance is used to concurrently manage the download tasks of recording data from multiple video channels; each decoding and playback instance is pre-assigned to handle the video decoding and playback tasks of a specific video channel; and the synchronization control instance is designed as the central coordinator for the entire synchronous playback process.

[0027] Step S12: By calling the management interface provided by the synchronization control instance, each created decoding playback instance is added to the internal management list of the synchronization control instance one by one, so as to establish a centralized control relationship between the synchronization control instance and all decoding playback instances, and ensure that subsequent synchronization instructions can be sent to each decoding playback unit.

[0028] In one embodiment, step S2: acquiring recording segments for the plurality of video channels, and establishing an independent frame queue for buffering the video frame data of each video channel, specifically includes the following steps: Step S21: Using the created multi-channel download instance, a query request is sent to the cloud storage server for each video channel that needs to be played back, in order to obtain a list of discontinuous video clips recorded by the channel based on the event triggering mechanism within a specified time period, and an independent list of video clips is maintained for each channel. Step S22: Iterate through the list of video clips acquired for each channel, control the download logic to ensure that each channel successfully completes the download of at least one video clip, and acquire the downloaded video data for each channel. This step ensures that all channels participating in synchronous playback have a usable video data source at the initial moment, avoiding the inability to start the synchronization process due to a channel having no data at all; Step S23: Parse the downloaded video data of each channel to obtain frame header information and corresponding video frame data; Specifically, the frame header information of the recorded data is first parsed. The frame header contains key metadata such as the sequence number of the video frame, the relative timestamp relative to the start point of the segment, the absolute timestamp, and the frame data length. After parsing the frame header information, the corresponding video frame data is then extracted. Step S24: Create and maintain an independent frame queue data structure in memory for each video channel; Specifically, the video frame data parsed in step S23 and its corresponding timestamp information are stored in the frame queue established for that channel, according to the channel identifier parsed from the frame header. The frame queue serves as a buffer to temporarily store video frame data that has been parsed but has not yet been processed by the decoding playback instance.

[0029] In one embodiment, such as Figure 2 As shown, step S3: Start each of the decoding and playback instances to enter the decoding running state, decode the video data belonging to its corresponding video channel to obtain video frames, and before rendering the obtained video frames, send a rendering request for the video frame to the synchronization control instance. The rendering request includes at least the timestamp of the video frame. Specifically, it includes the following steps: Step S31: Start all registered decoding and playback instances, so that each decoding and playback instance enters an independent continuous running thread, and each decoding and playback instance continuously monitors the frame queue status of its corresponding video channel; Step S32: When the decoding playback instance detects that there is video frame data to be processed in the frame queue of its corresponding channel, it retrieves the next frame data from the frame queue, performs video decoding operation, and converts the compressed encoded video data into original image frames that can be directly used for screen rendering. Step S33: After successfully decoding the video frame, the decoding and playback instance constructs a rendering request and sends it to the synchronization control instance; Specifically, to ensure global synchronization of multi-channel images, the decoding and playback instance must generate a structured request message before initiating any rendering operation. The core data payload of this rendering request message is the timestamp information carried by the currently ready and pending video frame. This timestamp is an inherent time attribute of the video frame, and its value comes from the key metadata field extracted when parsing the recording data frame header in step S23.

[0030] In one embodiment, step S4: The synchronization control instance receives and responds to the rendering request sent by any decoding playback instance, executes synchronization control logic, compares the video frame timestamp in the rendering request with the globally maintained current playback timestamp, and determines whether to allow the decoding playback instance to render the corresponding video frame based on whether the difference between the two is within a preset allowable error range. This synchronization control logic specifically includes the following sub-steps: Step S41: The synchronous control instance continuously listens for and receives rendering requests from each decoding and playback instance. The synchronous control instance executes real-time arbitration logic and global maintenance logic. Through the parallel or alternating execution of these two control flows, it ultimately determines and returns a rendering approval or rejection instruction for the corresponding video frame to the decoding and playback instance. S41A: Real-time arbitration process for single-frame rendering requests: The synchronization control instance continuously listens for and receives rendering requests from each decoding and playback instance. Each received rendering request is processed according to the following procedure: The system checks the initialization status of the synchronization control instance itself and the user-defined global playback control status. If the instance has not completed initialization, or the current global playback status is "paused," subsequent synchronization checks for this rendering request are paused, and the system enters a sleep state for a preset duration (e.g., the default frame interval of 40 milliseconds). After the sleep state ends, the processing flow for this request terminates. This mechanism implements the playback pause function.

[0031] If the status check passes, the timestamp of the video frame to be rendered is extracted from the rendering request. This timestamp is compared with the globally maintained current playback timestamp, and the absolute time difference between the two is calculated. Then, it is determined whether this time difference is within a preset allowable error range (e.g., 1 second).

[0032] Based on the judgment result, a command is returned to the decoding playback instance that sent the rendering request: If the time difference is within the allowable error range, the "Allow Rendering" command is returned.

[0033] If the time difference exceeds the allowable error range, a "Rendering not allowed" instruction will be returned.

[0034] S41B: Periodic global state maintenance process; specifically, the synchronization control instance periodically performs the following tasks to calibrate the time base and manage system buffers: S41B1: Traverse all video channels, query the data status of the corresponding frame queue for each channel, and count the number of channels whose current frame queue is empty.

[0035] If the number of empty channels in the frame queue equals the total number of channels, then playback is considered to have ended or data buffering is required, triggering the buffering wait mechanism.

[0036] If the number of channels with empty frame queues is less than the total number of channels, then try to traverse all channels and query the timestamp corresponding to the earliest frame (i.e., the first frame) in the frame queue of each channel.

[0037] If, during the query process, a valid timestamp cannot be obtained because the frame queue of any channel is empty, the current process is interrupted, triggering a buffer wait for that channel.

[0038] If all channels can provide valid timestamps, the minimum value among these timestamps is determined as the global earliest frame timestamp.

[0039] S41B2: Compare the global earliest frame timestamp with the global current playback timestamp.

[0040] If the earliest global frame timestamp is greater than the current global playback timestamp, then the current global playback timestamp will be updated to the earliest global frame timestamp.

[0041] If the earliest global frame timestamp is less than or equal to the current global playback timestamp, then the current global playback timestamp remains unchanged.

[0042] S41B3: After completing the above checks and synchronization, if the global current playback timestamp is not the initial value (e.g., not 0), it is incremented by a preset unit (e.g., by 1 second); then, the synchronization control instance enters a sleep period of a fixed duration to control its loop execution frequency.

[0043] Step S42: Based on the rendering approval or rejection instruction for the corresponding video frame received from the synchronization control instance, the decoding playback instance executes the following corresponding instruction operations: If a "permit rendering" instruction is received, the decoded video frames are submitted to the graphics rendering engine for output.

[0044] If a "rendering not allowed" instruction is received, it will be handled according to the preset strategy, and a re-initiation of the rendering request can be made thereafter; the preset strategy is to discard the frame, buffer and wait, or continue decoding the next frame; Secondly, this application provides a system for synchronous playback of multi-channel event video recordings, comprising: The synchronization control module is configured to maintain the global current playback timestamp and execute synchronization control logic; Multiple decoding and playback modules, each corresponding to multiple video channels, are all communicatively connected to the synchronization control module; The decoding and playback module is configured to decode the recording data of its corresponding video channel, and after decoding the video frame but before rendering the video frame, send a rendering request for the video frame to the synchronization control module. The request includes the timestamp of the video frame. The synchronization control module is configured to, in response to a rendering request from any decoding playback module, execute the synchronization control logic to compare the video frame timestamp in the rendering request with the globally maintained current playback timestamp, and determine whether to allow the decoding playback module to render the corresponding video frame based on whether the difference between the two is within a preset allowable error range.

[0045] The functions of each module in the above-mentioned multi-channel event recording synchronous playback system correspond to the steps in the above-mentioned multi-channel event recording synchronous playback method embodiment, and their functions and implementation processes will not be described in detail here.

[0046] Thirdly, embodiments of this application provide a multi-channel event recording synchronous playback device, which can be a personal computer (PC), laptop computer, server, or other device with data processing capabilities.

[0047] In this embodiment of the application, the multi-channel event recording synchronous playback device may include a processor, a memory, a communication interface, and a communication bus.

[0048] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0049] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the multi-channel event recording and playback device, as well as interfaces used for interconnecting the multi-channel event recording and playback device with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0050] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0051] The processor can be a general-purpose processor, which can call the multi-channel event recording synchronous playback program stored in the memory and execute the multi-channel event recording synchronous playback method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the multi-channel event recording synchronous playback program is called can be referred to the various embodiments of the multi-channel event recording synchronous playback method of this application, and will not be repeated here.

[0052] Fourthly, embodiments of this application also provide a readable storage medium.

[0053] This application stores a multi-channel event recording synchronous playback program on a readable storage medium, wherein when the multi-channel event recording synchronous playback program is executed by a processor, it implements the steps of the multi-channel event recording synchronous playback method described above.

[0054] The method implemented when the multi-channel event recording synchronous playback program is executed can be referred to in the various embodiments of the multi-channel event recording synchronous playback method of this application, and will not be repeated here.

[0055] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for synchronous playback of multi-channel event recordings, characterized in that, Includes the following steps: Create a synchronization control instance for global control, and create multiple decoding playback instances to correspond to multiple video channels respectively and register them all with the synchronization control instance; Acquire recording segments for the multiple video channels, and establish an independent frame queue for each video channel to buffer the video frame data of that channel; Each of the aforementioned decoding and playback instances is started to enter the decoding running state, and the video data belonging to its corresponding video channel is decoded to obtain video frames. Before rendering the obtained video frames, a rendering request for the video frame is sent to the synchronization control instance. The rendering request contains at least the timestamp of the video frame. The synchronization control instance receives and responds to the rendering request sent by any decoding playback instance, executes synchronization control logic, compares the video frame timestamp in the rendering request with the globally maintained current playback timestamp, and determines whether to allow the decoding playback instance to render the corresponding video frame based on whether the difference between the two is within a preset allowable error range.

2. The multi-channel event recording synchronous playback method according to claim 1, characterized in that, The synchronization control logic also includes: Before the comparison and judgment, check the playback control status; If the playback control status is paused, the comparison and judgment are paused, and after waiting for a preset frame interval, the process returns to the start step.

3. The multi-channel event recording synchronous playback method according to claim 1, characterized in that, The synchronization control logic also includes: Periodically traverse all video channels, query the data status of the frame queue corresponding to each channel, and count the number of channels whose frame queue is currently empty; If the number of empty channels in the frame queue equals the total number of channels, it is determined that playback has ended or data buffering is required, and the buffering wait mechanism is triggered.

4. The multi-channel event recording synchronous playback method according to claim 3, characterized in that, The synchronization control logic also includes: If the number of channels with empty frame queues is less than the total number of channels, then iterate through all video channels and query the timestamp corresponding to the earliest frame in the frame queue of each channel. From all the earliest timestamps retrieved, determine the minimum value as the global earliest timestamp; The current playback timestamp of the global system is updated based on the earliest global frame timestamp.

5. The multi-channel event recording synchronous playback method according to claim 4, characterized in that, The step of updating the global current playback timestamp based on the earliest global frame timestamp includes: Compare the global earliest frame timestamp with the global current playback timestamp; If the global earliest frame timestamp is greater than the global current playback timestamp, then the global current playback timestamp is set to be equal to the global earliest frame timestamp.

6. The multi-channel event recording synchronous playback method according to claim 1, characterized in that, The process of acquiring video clips from multiple video channels includes: Create at least one download control instance; The download control instance queries the storage location corresponding to each video channel for available recording segments and forms a list of recording segments for each channel. Based on the list of video clips, control each channel to download data for at least one video clip.

7. The multi-channel event recording synchronous playback method according to claim 6, characterized in that, Before establishing an independent frame queue for each video channel, the method further includes: The downloaded video clip data is parsed, and each video frame and its corresponding timestamp information are extracted from the frame header of the data frame. The extracted video frames are stored in the frame queue established for each video channel according to their respective video channels.

8. The multi-channel event recording synchronous playback method according to claim 7, characterized in that, The preset allowable error range is a value set based on the threshold of playback asynchrony time that is perceptible to the user.

9. The multi-channel event recording synchronous playback method according to claim 7, characterized in that, The video clips from the multiple video channels are discontinuous video clips uploaded to the cloud storage server based on an event-triggered mechanism.

10. A system for synchronous playback of multi-channel event recordings, characterized in that, The system is used to implement the multi-channel event recording synchronous playback method as described in any one of claims 1-9; the system includes: The synchronization control module is configured to maintain the global current playback timestamp and execute synchronization control logic; Multiple decoding and playback modules, each corresponding to multiple video channels, are all communicatively connected to the synchronization control module; The decoding and playback module is configured to decode the recording data of its corresponding video channel, and after decoding the video frame but before rendering the video frame, send a rendering request for the video frame to the synchronization control module. The request includes the timestamp of the video frame. The synchronization control module is configured to, in response to a rendering request from any decoding playback module, execute the synchronization control logic to compare the video frame timestamp in the rendering request with the globally maintained current playback timestamp, and determine whether to allow the decoding playback module to render the corresponding video frame based on whether the difference between the two is within a preset allowable error range.

Images