Video stream synchronization method and apparatus, platform, and storage medium

By introducing a main control module and a frame synchronization control module into the video stream synchronization platform, and adopting a video frame cutting and message packet transmission mechanism, the synchronization problem between local video streams and network video streams is solved, and efficient video image synchronization display is achieved.

WO2024230176A1PCT designated stage expired Publication Date: 2024-11-14ZHEJIANG UNIVIEW TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2023/140139
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2023-12-20
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing technologies struggle to achieve synchronized processing of local and network video streams, resulting in fragmented video images. Furthermore, existing solutions are either costly or highly complex.

Method used

By introducing a main control module, a frame synchronization control module, input nodes, and output nodes into the video stream synchronization platform, and adopting a video frame cutting and message packet transmission mechanism, a target frame synchronization signal is generated to ensure the synchronous transmission and display of video blocks.

Benefits of technology

It enables the synchronous display of local video streams and network video streams, avoiding a sense of disconnect, reducing costs, and improving the real-time performance of processing devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023140139_14112024_PF_FP_ABST
    Figure CN2023140139_14112024_PF_FP_ABST
Patent Text Reader

Abstract

A video stream synchronization method and apparatus, a platform, and a storage medium. The method comprises: an input node respectively transmits at least two video blocks generated by cutting a video frame and a message packet to output nodes corresponding to the at least two video blocks, and sends the message packet to a frame synchronization control module when the transmission of the at least two video blocks and the message packet is completed; the frame synchronization control module processes an initial frame synchronization signal according to a first video identifier and a first frame number identifier in the message packet to generate a target frame synchronization signal, and respectively sends the target frame synchronization signal to the output nodes corresponding to the at least two video blocks; each output node parses the target frame synchronization signal to determine a second video identifier and a second frame number identifier, and when the first video identifier is the same as the second video identifier, and the first frame number identifier is the same as the second frame number identifier, the output node sends the video block of the first frame number identifier to a display device.
Need to check novelty before this filing date? Find Prior Art

Description

Video stream synchronization method, device, platform and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 11, 2023, with application number 202310529739.0, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of video processing technology, and for example, to a video stream synchronization method, device, platform and storage medium. Background Art

[0003] Figure 1 is a functional block diagram of a simple video integration platform provided by related technologies. As shown in Figure 1, the video integration platform can receive local and network video streams and perform processing such as cutting, splicing, scaling, and overlaying on the multiple input video streams. It can output the processed video images through a local video output interface or send the entire video image as a network stream. Complex video integration platforms can simultaneously input and output dozens or even hundreds of video streams.

[0004] Figure 2 is a schematic diagram of the framework of the video integration platform provided by the related art. As shown in Figure 2, the video integration platform includes a local stream input node, a local stream output node, a network stream input node and a network stream output node. Among them, the local stream input node is responsible for operations such as cutting and superimposing the local video stream, and sends the processed video stream to the local stream output node for display or encodes and sends it to the network stream output node. The network stream input node is responsible for decoding the network video stream into the local video stream and sending it to the local stream output node for display. The local stream output node sends the local video stream sent by the local stream input node and the network stream input node from the video output interface, and the network stream output node encodes the local video stream sent by the local stream input node and sends it out. All nodes are connected through a bus matrix (such as a high-speed peripheral component interconnect Express (PCIE) bus).

[0005] Local video streams refer to videos transmitted through common video interfaces and related cables. Common interfaces include Video Graphics Array (VGA), Display Port (DP), and High Definition Multimedia Interface (HDMI). Local video streams are characterized by stable transmission frame rates, uncompressed or lightly compressed video data, large data bandwidth, low latency, and short transmission distances. Network video streams refer to video streams that have undergone deep image compression and are transmitted over a network path. Common video compression methods include H.264 and H.265, which are characterized by high video compression ratios, small data bandwidths, unstable frame rates, high latency, and suitability for long-distance transmission.

[0006] The network video stream is displayed from the video interface and needs to be decoded and converted into a local video stream first. The local video stream needs to be encoded to convert it into a network video stream. Mainstream chip platforms, such as the Central Processing Unit (CPU), Digital Signal Processor (DSP) and Graphics Processing Unit (GPU), include encoding and decoding functions and can process network video streams and local video stream data. However, Field Programmable Gate Array (FPGA) devices do not have encoding and decoding functions and can only process local video streams but not network video streams. The video integration platform needs to process local video streams and network video streams to achieve the effect of synchronized image playback, but the local video stream and network video stream are processed differently, and special considerations are required to ensure that the output from different video output ports remains synchronized after image cutting.

[0007] Summary of the Invention

[0008] The present application provides a video stream synchronization method, device, platform and storage medium to avoid the sense of fragmentation in the video image generated by the display device by splicing video blocks received from multiple output nodes, thereby achieving synchronization of video streams.

[0009] According to one aspect of the present application, a video stream synchronization method is provided, which is applied to a video stream synchronization platform. The video stream synchronization platform includes a main control module, a frame synchronization control module, an input node, and an output node. The method includes:

[0010] For each video frame in the video stream, the input node cuts the current video frame to generate at least two video blocks;

[0011] For each video block, the input node transmits the current video block to the output node corresponding to the current video block, and when the transmission of the current video block is completed, sends a message packet to the output node corresponding to the current video block, wherein the video blocks correspond to the output nodes one-to-one, and the message packet includes the first video identifier of the current video frame and the first frame number identifier of the current video block, the first video identifier of multiple video frames in the same video stream is the same, and the first frame number identifier of at least two video blocks generated by cutting the same video frame is the same;

[0012] When at least two video blocks generated by cutting the same video frame and the message packet are transmitted, the input node sends the message packet to the frame synchronization control module through the main control module;

[0013] In response to the message packet, the frame synchronization control module processes the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet to generate a target frame synchronization signal, and sends the target frame synchronization signal to the output nodes corresponding to the at least two video blocks respectively;

[0014] For each output node, the current output node parses the target frame synchronization signal, determines the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier, and the first frame number identifier is the same as the second frame number identifier, sends the video block corresponding to the first frame number identifier in the current output node to the display device.

[0015] According to another aspect of the present application, a video stream synchronization device is provided, which is applied to a video stream synchronization platform. The video stream synchronization platform includes a main control module, a frame synchronization control module, an input node, and an output node. The device includes:

[0016] a video frame cutting unit configured to cut each video frame in the video stream through the input node to generate at least two video blocks;

[0017] a video block transmission unit configured to transmit, for each video block, the current video block to an output node corresponding to the current video block via the input node, and send a message packet to the output node corresponding to the current video block when the transmission of the current video block is completed, wherein the video blocks correspond to the output nodes in a one-to-one relationship, and the message packet includes a first video identifier of the current video frame and a first frame number identifier of the current video block, wherein the first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are the same;

[0018] a message packet sending unit, configured to send the message packet to the frame synchronization control module via the input node and the main control module when the transmission of at least two video blocks generated by cutting the same video frame and the message packet are completed;

[0019] a frame synchronization signal generating unit, configured to, in response to the message packet, process an initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet through the frame synchronization control module to generate a target frame synchronization signal, and send the target frame synchronization signal to output nodes corresponding to the at least two video blocks respectively;

[0020] The first video block sending unit is configured to parse the target frame synchronization signal through the current output node for each output node, determine the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier and the first frame number identifier is the same as the second frame number identifier, send the video block corresponding to the first frame number identifier in the current output node to the display device.

[0021] According to another aspect of the present application, a video stream synchronization platform is provided, which includes a main control module, a frame synchronization control module, an input node and an output node; wherein,

[0022] The input node is configured to, for each video frame in the video stream, cut the current video frame to generate at least two video blocks;

[0023] The input node is further configured to, for each video block, transmit the current video block to an output node corresponding to the current video block, and send a message packet to the output node corresponding to the current video block when the transmission of the current video block is completed, wherein the video blocks correspond to the output nodes in a one-to-one manner, and the message packet includes a first video identifier of the current video frame and a first frame number identifier of the current video block, wherein the first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are the same;

[0024] The input node is further configured to send the message packet to the frame synchronization control module through the main control module when the at least two video blocks generated by cutting the same video frame and the message packet are transmitted;

[0025] The frame synchronization control module is configured to, in response to the message packet, process the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet to generate a target frame synchronization signal, and send the target frame synchronization signal to the output nodes corresponding to the at least two video blocks respectively;

[0026] For each output node, the current output node parses the target frame synchronization signal, determines the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier, and the first frame number identifier is the same as the second frame number identifier, sends the video block corresponding to the first frame number identifier in the current output node to the display device.

[0027] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video stream synchronization method described in any embodiment of the present application when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG1 is a functional schematic block diagram of a simple video integration platform provided by the related art;

[0029] FIG2 is a schematic diagram of a framework of a video integration platform provided by related art;

[0030] FIG3A is a schematic diagram showing the effect of video stream synchronization based on a frame synchronization signal;

[0031] FIG3B is another schematic diagram showing the effect of video stream synchronization based on a frame synchronization signal;

[0032] FIG4 is a flowchart of a video stream synchronization method provided according to Embodiment 1 of the present application;

[0033] FIG5 is a schematic structural diagram of a video stream synchronization platform provided in an embodiment of the present application;

[0034] FIG6 is a schematic diagram of a target frame synchronization signal provided in an embodiment of the present application;

[0035] FIG7 is a schematic diagram of a video stream synchronization process provided by an embodiment of the present application;

[0036] FIG8 is a schematic structural diagram of a video stream synchronization platform provided in an embodiment of the present application;

[0037] FIG9 is a structural diagram of a video stream synchronization device provided in Example 2 of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. The described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0039] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. The numbers used in this way can be interchanged where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units listed, but may include other steps or units that are not listed or that are inherent to these processes, methods, products or devices.

[0040] In the related art, two approaches are used to achieve synchronized video stream output: Option 1: Video stream synchronization based on frame synchronization signals; Option 2: Video stream synchronization based on timestamps. Option 1 places high real-time demands on the processing components for local and network video streams, requiring all video stream operations to be completed between the pulse intervals of the frame synchronization signal. However, typical processing components lack real-time performance and are prone to interruptions and blockages due to unexpected tasks. Figure 3A illustrates the effect of video stream synchronization based on frame synchronization signals. As shown in Figure 3A, upon receiving the second frame of the video stream, the first frame of the video stream is displayed, and upon receiving the third frame of the video stream, the second frame of the video stream is displayed. This increases the decoding workload involved in processing the network video stream. Furthermore, due to the non-real-time operation of the device, once the processing of a network video stream is interrupted by the frame synchronization signal, desynchronization occurs. Figure 3B illustrates another approach to video stream synchronization based on frame synchronization signals. As shown in Figure 3B, the processing of the second frame of the video stream is interrupted by the frame synchronization signal, resulting in a fragmented display of the second frame of the video stream. To address the technical issues of solution one, the network video stream can be converted into a local video stream, which can then be processed using an FPGA device to ensure real-time video display. However, this approach increases costs and introduces higher latency. Solution two, using timestamps to synchronize video streams, doesn't fundamentally address the issue; it simply doesn't rely on an external frame synchronization signal, and using timestamps across different devices is inherently complex.

[0041] Example 1

[0042] FIG4 is a flowchart of a video stream synchronization method according to a first embodiment of the present application. This embodiment is applicable to situations where video streams are synchronized. The method can be performed by a video stream synchronization device, which can be implemented in hardware and / or software. The video stream synchronization device can be configured in a video stream synchronization platform, which includes a main control module, a frame synchronization control module, an input node, and an output node. As shown in FIG4 , the method includes the following steps.

[0043] S410 : For each video frame in the video stream, the input node cuts the current video frame to generate at least two video blocks.

[0044] FIG5 is a schematic structural diagram of a video stream synchronization platform provided in an embodiment of the present application. As shown in FIG5 , the video stream synchronization platform 500 includes a main control module 510, a frame synchronization control module 520, an input node 530, and an output node 540. Optionally, the input node includes a local stream input node and a network stream input node, and the output node includes a local stream output node. The local stream input node and the network stream input node can be one or more, and the output node can be multiple. The local stream input node can be implemented by a device with high real-time performance (such as an FPGA), and the network stream input node can be implemented by a device with low real-time performance (such as a CPU, DSP, or GPU).

[0045] In an embodiment of the present application, an input node receives a video stream, wherein when the input node is a local stream input node, the local stream input node receives the video stream (which may be referred to as a local video stream) through a common video interface and related cables; when the input node is a network stream input node, the network stream input node receives a video stream (which may be referred to as a network video stream) that has undergone deep image compression and is transmitted through a network path, and decodes the network video stream. Each video stream includes multiple video frames. For each video frame in the video stream, the input node cuts the current video frame and generates at least two video blocks corresponding to the current video frame. The video frame can be cut based on a preset cutting algorithm, or it can be cut randomly. The embodiment of the present application does not limit the cutting method of the video frame.

[0046] S420. For each video block, the input node transmits the current video block to the output node corresponding to the current video block, and when the transmission of the current video block is completed, sends a message packet to the output node corresponding to the current video block, wherein the video blocks correspond to the output nodes one-to-one, and the message packet includes a first video identifier of the current video frame and a first frame number identifier of the current video block. The first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are the same.

[0047] In an embodiment of the present application, for each video frame in a video stream, an input node transmits at least two video blocks corresponding to the current video frame to output nodes corresponding to at least two video blocks, wherein the video blocks and the output nodes are in a one-to-one correspondence. Exemplarily, at least two output nodes having a mapping relationship with the input node are determined, and the output node having a mapping relationship with the input node is an output node for receiving the video stream output by the input node. The at least two video blocks generated by cutting the video frame are respectively sent to the corresponding output nodes having a mapping relationship with the input node, wherein the number of output nodes having a mapping relationship with the input node is the same as the number of video blocks generated by cutting the video frame, and the two are in one-to-one correspondence.

[0048] When an input node transmits a video block to an output node corresponding to the video block, that is, when it determines that a video block has been successfully transmitted to the output node corresponding to the video block, the input node sends a message packet to the output node corresponding to the video block. The message packet includes the first video identifier of the video frame that generates the video block and the first frame number identifier of the video block. The first video identifier is the video stream identifier before the video frame is cut, and the first frame number identifier is the identifier of the video block after the video frame is cut. The first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are the same. Therefore, the message packets received by the output nodes corresponding to the at least two video blocks generated by cutting the video frame are the same, and both contain the same first video identifier and the same first frame number identifier.

[0049] In an embodiment of the present application, the first video identifiers of multiple video frames of the same video stream are the same, wherein the first video identifier can be a first video identity identifier (IDentifier, ID). The first frame number identifiers of the multiple video frames of the video stream can be a first frame number identity identifier (ID), and the first frame number ID is an ID that is cyclically counted by a cyclic counter, and the first frame number ID is cyclically counted in sequence according to the sequence of video frames in the video stream. For example, the first frame number ID can be an ID that is counted by a 2-bit cyclic counting counter, and the first frame number ID of the video frame continuously changes cyclically as the number of video frames in the video stream increases.

[0050] S430: When the at least two video blocks generated by cutting the same video frame and the message packet are transmitted completely, the input node sends the message packet to the frame synchronization control module through the main control module.

[0051] Because the sizes of the at least two video blocks generated by cutting a video frame are not identical, the transmission speeds at which the input node transmits the at least two video blocks to the corresponding output nodes are not identical. Therefore, the time points at which the input node transmits the at least two video blocks and the message packet to the output nodes corresponding to the at least two video blocks are not necessarily the same. When it is determined that the at least two video blocks and the message packet generated by cutting the same video frame have been transmitted to the corresponding output nodes, the input node sends the message packet corresponding to the video frame to the main control module. When the main control module receives the message packet, it can be determined that the input node has transmitted the at least two video blocks generated by cutting the video frame corresponding to the message packet to the corresponding output nodes. At this time, the main control module can send the message packet to the frame synchronization control module.

[0052] S440. In response to the message packet, the frame synchronization control module processes the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet, generates a target frame synchronization signal, and sends the target frame synchronization signal to the output nodes corresponding to the at least two video blocks respectively.

[0053] When the frame synchronization control module receives a message packet sent by the main control module, it encodes the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet, and uses the encoded initial frame synchronization signal as the target frame synchronization signal. The initial frame synchronization signal is a fixed-period pulse signal, such as a fixed-period high-level pulse (i.e., a high-level pulse signal with a fixed frame rate). A common video stream is a 60-frame video, so the frame pulse period of the initial frame synchronization signal can be 16.67ms. The target frame synchronization signal is a pulse signal with an irregular period.

[0054] Optionally, the first video identifier includes a first video ID; the frame synchronization control module processes the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet to generate a target frame synchronization signal, including: in accordance with the preset order of the first video ID, after the preset duration of the falling edge of the initial frame synchronization signal, the frame synchronization control module sends a target pulse signal corresponding to the first frame number identifier to the initial frame synchronization signal according to the first frame number identifier corresponding to the first video ID within a preset period to generate a target frame synchronization signal.

[0055] Among them, the preset order of the first video ID can be the order of the first video ID from small to large, or it can be the order of the first video ID from large to small. The embodiment of the present application does not limit the preset order. In the embodiment of the present application, according to the order of the first video ID from small to large, that is, according to the sequence of the video stream, at the preset duration of the falling edge of the initial frame synchronization signal (that is, a fixed delay time), with a fixed time unit as a period, according to the first frame number identifier corresponding to the first video ID, a pulse signal corresponding to the first frame number identifier is sent to the initial frame synchronization signal, thereby generating a target frame synchronization signal. Exemplarily, Figure 6 is a schematic diagram of a target frame synchronization signal provided by an embodiment of the present application. As shown in Figure 6, with 1us as the preset duration, after the time period of 1us of the falling edge of the initial frame synchronization signal, a low-level pulse of 500ns (considered to be 0) or a high-level pulse of 500ns (considered to be 1) is continuously sent, and each 1us pulse signal can be used as the first frame number identifier (such as the first frame number ID) corresponding to a first video ID. For example, if a video stream is 8-bit, a matching pulse signal can be continuously sent to the initial frame synchronization signal for 256 μs, thereby generating a target frame synchronization signal corresponding to the video stream. As shown in Figure 6, 1 μs after the falling edge of the initial frame synchronization signal, the high and low levels of the continuously sent pulse signal (500 ns as a period) are: low, low (the first frame ID of the first video ID 0 is 00), low, high (the first frame ID of the first video ID 1 is 1), high, low (the first frame ID of the first video ID 2 is 2), high, high (the first frame ID of the first video ID 3 is 3), low, low (the first frame ID of the first video ID 4 is 00), and so on.

[0056] The frame synchronization control module sends target frame synchronization signals to the output nodes corresponding to at least two video blocks generated by video frame segmentation. As time passes, the target frame synchronization signals generated by the frame synchronization control module are continuously updated as video frames in the video stream are continuously transmitted. Therefore, the target frame synchronization signals sent by the frame synchronization control module at different time points may not be the same.

[0057] S450. For each output node, the current output node parses the target frame synchronization signal to determine the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier and the first frame number identifier is the same as the second frame number identifier, the video block corresponding to the first frame number identifier in the current output node is sent to the display device.

[0058] In an embodiment of the present application, the output nodes corresponding to at least two video blocks respectively receive the target frame synchronization signal sent by the frame synchronization control module, and parse the target frame synchronization signal to parse out the second video identifier and the second frame number identifier from the target frame synchronization signal. Exemplarily, a process opposite to the process of encoding the initial frame synchronization signal according to the first video identifier and the first frame number identifier can be adopted to parse out the second video identifier and the second frame number identifier from the target frame synchronization signal. Exemplarily, the target frame synchronization signal can be parsed based on the characteristics of the initial frame synchronization signal and the target frame synchronization signal (the initial frame synchronization signal is a pulse signal with a fixed period, and the target frame synchronization signal is a pulse signal with an irregular period), and the second video identifier and the second frame number identifier can be determined. Optionally, each output node can also receive the initial frame synchronization signal and the target frame synchronization signal sent by the frame synchronization control module, compare and analyze the initial frame synchronization signal and the target frame synchronization signal, and thus parse out the second video identifier and the second frame number identifier from the target frame synchronization signal.

[0059] For the output node corresponding to each video block, each output node determines whether the second video identifier parsed from the target frame synchronization signal is the same as the first video identifier received from the input node, and whether the second frame number identifier parsed from the target frame synchronization signal is the same as the first frame number identifier received from the input node. When the second video identifier is the same as the first video identifier, and the second frame number identifier is also the same as the first frame number identifier, the output node sends the video block with the first frame number identifier received from the input node to the display device. This setting can effectively ensure that the video blocks received by the display device from multiple output nodes are video blocks generated by cutting from the same video frame, so as to avoid the sense of fragmentation in the video image generated by the display device by splicing the video blocks received from multiple output nodes, thereby achieving synchronization of the video streams.

[0060] Optionally, the method further includes: for each output node, when the current output node determines that the first video identifier is the same as the second video identifier, and the first frame number identifier is different from the second frame number identifier, the video block corresponding to the second frame number identifier in the current output node is sent to the display device. Since the input node first sends a message packet to the output node corresponding to each video block, and then sends a message packet to the frame synchronization control module through the main control module, there are two relationships between the first frame number identifier received by the output node from the input node and the second frame number identifier parsed from the target frame synchronization signal sent by the frame synchronization control module, one is that the first frame number identifier is consistent with the second frame number identifier, and the other is that the first frame number identifier is greater than the second frame number identifier. When the second video identifier is the same as the first video identifier, but the second frame number identifier is different from the first frame number identifier, the output node sends the video block corresponding to the second frame number identifier to the display device.

[0061] When the second video identifier is the same as the first video identifier, and the second frame number identifier is also the same as the first frame number identifier, the output node sends the video frame corresponding to the first frame number identifier for display, that is, the video frame corresponding to the first frame number identifier is displayed on the display device. When the second video identifier is the same as the first video identifier, but the second frame number identifier is different from the first frame number identifier, the output node sends the video block corresponding to the second frame number identifier for display, that is, the video block corresponding to the second frame number identifier is displayed on the display device. Figure 7 is a schematic diagram of a video stream synchronization process provided by an embodiment of the present application. As shown in Figure 7, the first frame number ID of the video block received by the output node from the input node is 2, and the corresponding second frame number ID parsed from the target frame synchronization signal is 1. The two are inconsistent. Therefore, the output node sends the video block with the second frame number ID of 1 to the display device, that is, the display device displays the video frame corresponding to the video block with the second frame number ID of 1.

[0062] The video stream synchronization method of the embodiment of the present application is applied to a video stream synchronization platform, which includes a main control module, a frame synchronization control module, an input node and an output node. The method includes: for each video frame in the video stream, the input node cuts the current video frame to generate at least two video blocks; for each video block, the input node transmits the current video block to the output node corresponding to the current video block, and when the transmission of the current video block is completed, sends a message packet to the output node corresponding to the current video block, wherein the message packet includes the first video identifier of the current video frame and the first frame number identifier of the current video block, the first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are The same; when at least two video blocks and message packets generated by cutting the same video frame are transmitted, the input node sends the message packet to the frame synchronization control module through the main control module; in response to the message packet, the frame synchronization control module processes the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet, generates a target frame synchronization signal, and sends the target frame synchronization signal to the output nodes corresponding to at least two video blocks respectively; for each output node, the current output node parses the target frame synchronization signal, determines the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier, and the first frame number identifier is the same as the second frame number identifier, the video block corresponding to the first frame number identifier in the current output node is sent to the display device. The technical solution provided by the embodiment of the present application can effectively ensure that the video blocks received by the display device from multiple output nodes are video blocks generated by cutting from the same video frame, so as to avoid the sense of fragmentation of the video image generated by the display device splicing the video blocks received from multiple output nodes, thereby achieving synchronization of the video streams.

[0063] In some embodiments, for each video block, the input node transmits the current video block to the output node corresponding to the current video block, including: obtaining pre-stored service configuration information, wherein the service configuration information includes a mapping relationship between the input node and at least two output nodes; determining an output node having a mapping relationship with the input node based on the service configuration information; and for each video block, the input node transmits the current video block to the output node corresponding to the current video block, wherein the output node corresponding to the current video block is the output node having a mapping relationship with the input node. Exemplarily, the input node obtains the pre-stored service configuration information, wherein the input node can obtain the service configuration information sent by the main control module. Exemplarily, a user can configure the service at the main control module. Since the video stream synchronization platform includes multiple inputs and multiple outputs, during service configuration, the user can configure which output interfaces the video streams output by the multiple input nodes are transmitted to, and which display devices the video streams are displayed on via the output interfaces, thereby generating the service configuration information. The multiple output nodes for receiving the video streams transmitted by the input node are the output nodes having a mapping relationship with the input node. Therefore, the service configuration information includes the mapping relationship between the input node and the at least two output nodes. The input node sends at least two video blocks generated by cutting the video frame to output nodes that have a mapping relationship with the input node, wherein the video blocks correspond to the output nodes one by one.

[0064] Optionally, the service configuration information also includes the synchronized video ID of the input node; and before the input node splits the current video frame for each video frame in the video stream to generate at least two video chunks, the service configuration information further includes: in response to a video stream synchronization event being triggered, the main control module uses the synchronized video ID as the first video identifier for multiple video frames in the video stream. Exemplarily, when performing service configuration, the main control module may also configure a synchronized video ID for the input node, and thus the service configuration information may also include the synchronized video ID of the input node. Upon receiving a video stream synchronization instruction, the main control module sends the synchronized video ID to the input node and uses the synchronized video ID as the first video identifier for multiple video frames in the video stream output by the input node. Optionally, the synchronized video ID pre-configured by the main control module may be an 8-bit value, which is assigned to the input node and used as the first video identifier for each video frame in the video stream. Optionally, if splitting of video frames in the video stream is not required and video stream synchronization is not a concern, the input node may be assigned no synchronized video ID or a default video ID, such as 0, may be assigned.

[0065] For example, FIG8 is a structural diagram of a video stream synchronization platform provided in an embodiment of the present application. The video stream synchronization process can be understood in combination with FIG8 and the description of the above embodiment, and will not be repeated here.

[0066] Example 2

[0067] Figure 9 is a schematic diagram of the structure of a video stream synchronization device provided in Example 2 of the present application. The video stream synchronization device is applied to a video stream synchronization platform, which includes a main control module, a frame synchronization control module, an input node, and an output node. As shown in Figure 9, the device includes the following units.

[0068] The video frame cutting unit 910 is configured to cut each video frame in the video stream through the input node to generate at least two video blocks;

[0069] The video block transmission unit 920 is configured to transmit the current video block to the output node corresponding to the current video block through the input node for each video block, and send a message packet to the output node corresponding to the current video block when the transmission of the current video block is completed, wherein the video blocks correspond to the output nodes in a one-to-one relationship, and the message packet includes a first video identifier of the current video frame and a first frame number identifier of the current video block, wherein the first video identifier of multiple video frames in the same video stream is the same, and the first frame number identifier of at least two video blocks generated by cutting the same video frame is the same;

[0070] The message packet sending unit 930 is configured to send the message packet to the frame synchronization control module via the input node and the main control module when the at least two video blocks generated by cutting the same video frame and the message packet are transmitted;

[0071] a frame synchronization signal generating unit 940, configured to, in response to the message packet, process an initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet through the frame synchronization control module to generate a target frame synchronization signal, and send the target frame synchronization signal to the output nodes corresponding to the at least two video blocks respectively;

[0072] The first video block sending unit 950 is configured to parse the target frame synchronization signal through the current output node for each output node, determine the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier and the first frame number identifier is the same as the second frame number identifier, send the video block corresponding to the first frame number identifier in the current output node to the display device.

[0073] Optionally, the first video identifier includes a first video ID;

[0074] The frame synchronization signal generating unit 940 is configured to:

[0075] According to the preset order of the first video ID, after the preset duration of the falling edge of the initial frame synchronization signal, the frame synchronization control module sends a target pulse signal corresponding to the first frame number identifier to the initial frame synchronization signal according to the first frame number identifier corresponding to the first video ID within a preset period to generate a target frame synchronization signal.

[0076] Optionally, the device further includes:

[0077] The second video block sending unit is configured to send, for each output node, a video block corresponding to the second frame number identifier in the current output node to the display device when the current output node determines that the first video identifier is the same as the second video identifier and the first frame number identifier is different from the second frame number identifier.

[0078] Optionally, the first frame number identifier is a first frame number ID, and the first frame number ID is an ID cyclically counted by a cyclic counter, and the first frame number ID is cyclically counted in sequence according to the sequence of video frames in the same video stream.

[0079] Optionally, the input node includes a local flow input node and a network flow input node, and the output node includes a local flow output node.

[0080] Optionally, the video block transmission unit 920 is configured to:

[0081] Acquire pre-stored service configuration information, wherein the service configuration information includes a mapping relationship between an input node and at least two output nodes;

[0082] Determine, according to the service configuration information, an output node having a mapping relationship with the input node;

[0083] For each video block, the input node transmits the current video block to an output node corresponding to the current video block, wherein the output node corresponding to the current video block is an output node having a mapping relationship with the input node.

[0084] Optionally, the service configuration information further includes a synchronous video ID of the input node;

[0085] The device further comprises:

[0086] The first video identifier determination unit is configured to, before the input node cuts the current video frame to generate at least two video blocks for each video frame in the video stream, in response to a video stream synchronization event being triggered, cause the main control module to use the synchronization video ID as the first video identifier of multiple video frames in the video stream.

[0087] The video stream synchronization device provided in the embodiments of the present application can execute the video stream synchronization method provided in any embodiment of the present application, and has the corresponding functional modules and effects of the execution method.

[0088] Example 3

[0089] An embodiment of the present application provides a video stream synchronization platform. As shown in FIG5 , the video stream synchronization platform 500 includes a main control module 510 , a frame synchronization control module 520 , an input node 530 and an output node 540 .

[0090] The input node 530 is configured to, for each video frame in the video stream, split the current video frame to generate at least two video blocks;

[0091] The input node 530 is further configured to, for each video block, transmit the current video block to the output node 540 corresponding to the current video block, and send a message packet to the output node 540 corresponding to the current video block when the transmission of the current video block is completed, wherein the video blocks correspond to the output nodes in a one-to-one manner, and the message packet includes a first video identifier of the current video frame and a first frame number identifier of the current video block, wherein the first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are the same;

[0092] The input node 530 is further configured to send the message packet to the frame synchronization control module 520 via the main control module 510 when at least two video blocks generated by cutting the same video frame and the message packet are transmitted.

[0093] The frame synchronization control module 520 is configured to, in response to the message packet, process the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet, generate a target frame synchronization signal, and send the target frame synchronization signal to the output nodes 540 corresponding to the at least two video blocks respectively;

[0094] For each output node 540, the current output node parses the target frame synchronization signal, determines the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier, and the first frame number identifier is the same as the second frame number identifier, sends the video block corresponding to the first frame number identifier in the current output node to the display device.

[0095] The video stream synchronization platform 500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The video stream synchronization platform 500 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0096] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A machine-readable storage medium includes an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The storage medium can be a non-transitory storage medium.

[0098] To provide interaction with a user, the systems and techniques described herein can be implemented on a video streaming synchronization platform having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the video streaming synchronization platform. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0099] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0100] A computing system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship arises through computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of physical hosts and virtual private server (VPS) services.

[0101] The various forms of processes shown above can be used to reorder, add, or delete steps. For example, the multiple steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This document is not limited here.

Claims

1. A video stream synchronization method, applied to a video stream synchronization platform, the video stream synchronization platform comprising a main control module, a frame synchronization control module, an input node and an output node, the method comprising: For each video frame in the video stream, the input node cuts the current video frame to generate at least two video blocks; For each video block, the input node transmits the current video block to the output node corresponding to the current video block, and sends a message packet to the output node corresponding to the current video block when the transmission of the current video block is completed, wherein the video block corresponds to the output node one-to-one, and the message packet includes a first video identifier of the current video frame and a first frame number identifier of the current video block, the first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are the same; When at least two video blocks generated by cutting the same video frame and the message packet are transmitted, the input node sends the message packet to the frame synchronization control module through the main control module; In response to the message packet, the frame synchronization control module processes the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet, generates a target frame synchronization signal, and sends the target frame synchronization signal to the output nodes corresponding to the at least two video blocks respectively; For each output node, the current output node parses the target frame synchronization signal, determines the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier and the first frame number identifier is the same as the second frame number identifier, sends the video block corresponding to the first frame number identifier in the current output node to the display device.

2. The method according to claim 1, wherein: The first video identifier includes a first video ID; The frame synchronization control module processes the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet to generate a target frame synchronization signal, including: According to the preset order of the first video ID, after the preset duration of the falling edge of the initial frame synchronization signal, the frame synchronization control module sends a target pulse signal corresponding to the first frame number identifier to the initial frame synchronization signal according to the first frame number identifier corresponding to the first video ID within a preset period, thereby generating the target frame synchronization signal.

3. The method according to claim 1, further comprising: For each output node, when the current output node determines that the first video identifier is the same as the second video identifier, and the first frame number identifier is different from the second frame number identifier, the video block corresponding to the second frame number identifier in the current output node is sent to the display device.

4. The method according to claim 1, wherein: The first frame number identifier is a first frame number ID, and the first frame number ID is an ID cyclically counted by a cyclic counter. The first frame number ID is cyclically counted in sequence according to the sequence of video frames in the same video stream.

5. The method according to claim 1, wherein: The input nodes include local flow input nodes and network flow input nodes, and the output nodes include local flow output nodes.

6. The method according to claim 1, wherein: For each video block, the input node transmits the current video block to the output node corresponding to the current video block, including: Acquire pre-stored service configuration information, wherein the service configuration information includes a mapping relationship between the input node and at least two output nodes; Determine, according to the service configuration information, an output node having a mapping relationship with the input node; For each of the video blocks, the input node transmits the current video block to an output node corresponding to the current video block, wherein the output node corresponding to the current video block is an output node having a mapping relationship with the input node.

7. The method according to claim 6, wherein: The service configuration information also includes the synchronous video ID of the input node; Before the input node cuts the current video frame for each video frame in the video stream to generate at least two video blocks, the method further includes: In response to a video stream synchronization event being triggered, the main control module uses the synchronization video ID as a first video identifier of multiple video frames in the video stream.

8. A video stream synchronization device, applied to a video stream synchronization platform, the video stream synchronization platform comprising a main control module, a frame synchronization control module, an input node and an output node, the device comprising: A video frame cutting unit, configured to cut the current video frame through the input node for each video frame in the video stream to generate at least two video blocks; a video block transmission unit, configured to transmit the current video block to the output node corresponding to the current video block through the input node for each video block, and send a message packet to the output node corresponding to the current video block when the transmission of the current video block is completed, wherein the video block corresponds to the output node one-to-one, the message packet includes a first video identifier of the current video frame and a first frame number identifier of the current video block, the first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are the same; a message packet sending unit, configured to send the message packet to the frame synchronization control module through the input node and the main control module when at least two video blocks generated by cutting the same video frame and the message packet are transmitted; The frame synchronization signal generating unit is configured to respond to the message packet through the frame synchronization control module The block processes the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet to generate a target frame synchronization signal, and sends the target frame synchronization signal to the output nodes corresponding to the at least two video blocks respectively; The first video block sending unit is configured to parse the target frame synchronization signal through the current output node for each output node, determine the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier and the first frame number identifier is the same as the second frame number identifier, send the video block corresponding to the first frame number identifier in the current output node to the display device.

9. A video stream synchronization platform, wherein: The video stream synchronization platform includes a main control module, a frame synchronization control module, an input node and an output node; wherein, The input node is configured to, for each video frame in the video stream, cut the current video frame to generate at least two video blocks; The input node is further configured to, for each video block, transmit the current video block to the output node corresponding to the current video block, and send a message packet to the output node corresponding to the current video block when the transmission of the current video block is completed, wherein the video block corresponds to the output node one-to-one, the message packet includes the first video identifier of the current video frame and the first frame number identifier of the current video block, the first video identifiers of multiple video frames in the same video stream are the same, and the first frame number identifiers of at least two video blocks generated by cutting the same video frame are the same; The input node is further configured to send the message packet to the frame synchronization control module through the main control module when at least two video blocks generated by cutting the same video frame and the message packet are transmitted completely; The frame synchronization control module is configured to, in response to the message packet, process the initial frame synchronization signal according to the first video identifier and the first frame number identifier in the message packet, generate a target frame synchronization signal, and send the target frame synchronization signal to the output nodes corresponding to the at least two video blocks respectively; For each output node, the current output node parses the target frame synchronization signal, determines the second video identifier and the second frame number identifier, and when the first video identifier is the same as the second video identifier and the first frame number identifier is the same as the second frame number identifier, sends the video block corresponding to the first frame number identifier in the current output node to the display device.

10. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the video stream synchronization method according to any one of claims 1 to 7 when executed.

Citation Information

Patent Citations

  • Output signal synchronism detection method and device, electronic equipment and storage medium

    CN110324609A

  • Video source synchronization method and device

    CN112911346A

  • High-resolution VR cloud game solving method based on scattered coding and computer equipment

    CN115225902A

  • Video frame extraction analysis method and device, equipment and storage medium

    CN115866331A

  • Video stream synchronization method, device and platform and storage medium

    CN116847140A