Multi-type video signal synchronous alignment method, system and medium

Abstract

The invention discloses a multi-type video signal synchronous alignment method and system, and a medium. The method comprises the following steps: an NDI camera and an ST2110 camera receive PTP time sent by a PTP clock source through a network so as to synchronize equipment time; when the same object is shot, the PTP timestamps during shooting are embedded into video signals respectively, and the video signals are transmitted to a video switching table through a network; and the video switching station reads the PTP timestamps in the NDI signal and the ST2110 signal, calculates the delay frame number according to the difference between the two, and caches the video signal according to the delay frame number so as to realize synchronous alignment. Wherein a timestamp is embedded in an SEI field by the NDI camera, and a timestamp is embedded in an RTP packet header by the ST2110 camera. According to the invention, through a unified time reference and a differentiated cache mechanism, the synchronization problem caused by different compression transmission time consumption of heterogeneous video signals is effectively solved, and accurate synchronization of broadcast signals is ensured.

Description

Technical Field

[0001] This invention relates to the field of online video signal production technology, specifically to a method, system, and medium for synchronizing and aligning multiple types of video signals. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] With the continuous development of online video signal production technology, it is often necessary to process various video signals of different transmission types in studio or on-site production environments. In multi-camera shooting scenarios, ensuring the synchronous alignment of all video signals is a key factor in guaranteeing the quality of program broadcast.

[0004] Currently, for multiple video signals of different transmission types, the existing synchronization alignment method mainly relies on manual calibration. Specifically, operators visually observe the monitor screen, judge the time difference of each signal based on experience, and manually set the signal delay time to complete the synchronization alignment.

[0005] However, this traditional processing method has the following significant technical drawbacks: 1. Reliance on subjective experience and insufficient precision: This method mainly relies on the operator's personal experience and subjective visual judgment, which makes it difficult to achieve precise alignment at the microsecond or frame level and is prone to human error.

[0006] 2. Inefficient and cumbersome process: The process of repeated manual debugging is time-consuming, which significantly increases the time cost of system setup and debugging, and the operation process is complicated.

[0007] 3. High personnel requirements: This process usually requires the presence of highly experienced professionals, which limits the flexibility of system deployment.

[0008] 4. High broadcast risk: If the calibration is not good when shooting from multiple positions, it will lead to serious problems such as misalignment of the picture (such as lip-syncing and disjointed action), which will directly affect the final broadcast effect.

[0009] Therefore, there is an urgent need for a method that can automatically, accurately and efficiently achieve synchronous alignment of multiple types of video transmission signals in order to solve the above-mentioned technical problems. Summary of the Invention

[0010] The purpose of this invention is to address the technical problems of current methods for synchronizing multiple signals of different transmission types by visual observation and setting signal delay times, which result in inaccurate synchronization alignment, long processing times, complex operations, and easy image misalignment. This invention provides a method, system, and medium for synchronizing and aligning multiple types of video signals. By using a unified high-precision PTP timestamp to synchronize and align the times of each signal, and calculating the number of delayed frames based on the timestamp difference at the receiving end for buffering and alignment, this invention solves the technical problem of high-precision automatic synchronization being impossible due to delay differences in encoding / decoding and transmission mechanisms of different types of video transmission signals such as NDI and ST2110.

[0011] The technical solution of the present invention is as follows: A method for synchronizing and aligning multiple types of video signals, comprising: PTP Time Synchronization: The NDI camera and ST2110 camera receive the PTP time sent by the PTP clock source through the network and synchronize the device time. Signal acquisition and embedding timestamp: The NDI camera and the ST2110 camera capture the same object in real time and embed the PTP timestamp of the capture into the video signal; Signal transmission: The NDI camera and the ST2110 camera transmit the video signal to the video switcher via a network; Signal synchronization alignment: The video switcher reads the PTP timestamps from the NDI signal transmitted by the NDI camera and the ST2110 signal transmitted by the ST2110 camera, calculates the number of delayed frames based on the difference between the two PTP timestamps, and buffers the video signal based on the number of delayed frames to achieve synchronization alignment.

[0012] Furthermore, embedding the PTP timestamp at the time of shooting into the video signal specifically includes: The NDI camera embeds the PTP timestamp in the SEI field of the video stream, and the PTP timestamp is bound to the video frame; The ST2110 camera embeds the PTP timestamp in the RTP packet header.

[0013] Furthermore, the signal transmission specifically includes: The NDI camera performs shallow signal compression using the NDI|FULL protocol and then transmits the signal to the video switcher via a signal switch. The ST2110 camera uses an uncompressed ST2110-20 signal, which is transmitted to the video switcher via the signal exchange.

[0014] Furthermore, the step of calculating the number of delayed frames based on the difference between the two PTP timestamps, and buffering the video signal according to the number of delayed frames to achieve synchronization alignment, specifically includes: Calculate the difference between the PTP timestamp in the currently received ST2110 signal and the PTP timestamp in the NDI signal; The difference is converted into the number of delayed frames based on the signal frame rate; The ST2110 signal is buffered with video frames according to the number of delayed frames, and the buffered frames are stored in the ST2110 signal buffer queue.

[0015] Furthermore, after buffering the ST2110 signal for video frames according to the number of delayed frames, the method further includes: The video switcher reads frame data from the ST2110 signal buffer queue in a first-in-first-out manner; The video switcher directly reads the network data of the NDI signal; The video switcher switches the aligned NDI signal and ST2110 signal to output a broadcast signal.

[0016] Furthermore, the PTP time synchronization specifically includes: Enable the embedded PTP timecode function of the NDI camera to receive the PTP time via the network; Turn on the external synchronization switch of the ST2110 camera to synchronize the local time with the PTP time of the PTP clock source.

[0017] This invention also proposes a multi-type video signal synchronization alignment system, including: an NDI camera, an ST2110 camera, and a video switcher; Both the NDI camera and the ST2110 camera are configured to receive PTP time from a PTP clock source via the network to synchronize device time, capture real-time images of the same object, embed the PTP timestamp at the time of capture into the video signal, and transmit the video signal to the video switcher via the network. The video switcher is configured to read the PTP timestamps from the NDI signal transmitted by the NDI camera and the ST2110 signal transmitted by the ST2110 camera, calculate the number of delayed frames based on the difference between the two PTP timestamps, and buffer the video signal based on the number of delayed frames to achieve synchronization alignment.

[0018] Furthermore, the NDI camera is configured to embed the PTP timestamp in the SEI field of the video stream, and the PTP timestamp is bound to the video frame; The ST2110 camera is configured to embed the PTP timestamp in the RTP packet header.

[0019] Furthermore, the video switcher is specifically configured as follows: Calculate the difference between the PTP timestamp in the currently received ST2110 signal and the PTP timestamp in the NDI signal; The difference is converted into the number of delayed frames based on the signal frame rate; The ST2110 signal is buffered with video frames according to the number of delayed frames, and the buffered frames are stored in the ST2110 signal buffer queue.

[0020] The present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the multi-type video signal synchronization alignment method described above.

[0021] Compared with existing technologies, the advantages of this invention are: 1. Solves the problem of synchronization in mixed production of heterogeneous video signal sources: This invention addresses the signal asynchrony problem caused by the different transmission mechanisms and processing times of the NDI protocol (which typically involves intra-frame compression and software decompression, resulting in significant latency) and the ST2110 protocol (uncompressed, hardware-IP-based, with extremely low latency). A complete alignment scheme is proposed to solve this problem. This scheme enables precise synchronization of two different types of cameras in the same production environment, breaking the traditional limitation of studios that can only use a single type of signal source.

[0022] 2. High-precision time alignment achieved through PTP and specific field embedding: This invention utilizes PTP (Precise Time Protocol) to unify the time base of all devices, ensuring the uniqueness and high precision of the time source. Simultaneously, tailored to the characteristics of different protocols, PTP timestamps are embedded in the SEI field of the NDI signal and the RTP packet header of the ST2110 signal, strongly binding time information to video frame data. This "carrying identification (timestamp)" method allows the switcher to reconstruct the absolute moment of capture based on the timestamp, regardless of network jitter or path length, ensuring accurate synchronization.

[0023] 3. Automated Delay Calculation and Buffer Alignment: This invention automatically calculates the timestamp difference between the ST2110 signal and the NDI signal through the video switcher, and dynamically converts this time difference into delayed frame counts. This allows for targeted buffering of the "fast-moving" ST2110 signal (constructing virtual delay). This mechanism eliminates the need for manual adjustment of delay parameters and can adapt to minor fluctuations caused by changes in network environment or equipment performance in real time, ensuring that the final output broadcast signal exhibits seamless picture transitions and strict audio-visual synchronization during switching. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0025] Figure 1 This is a flowchart illustrating the method for synchronizing and aligning multiple types of video transmission signals in an embodiment of the present invention. Figure 2 This is a schematic diagram of the system setup for the method of synchronizing and aligning multiple types of video transmission signals in an embodiment of the present invention; Figure 3 This is a schematic diagram of the processing flow of the method for synchronizing and aligning multiple types of video transmission signals in an embodiment of the present invention. Detailed Implementation

[0026] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0027] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0028] Example 1 This embodiment provides a method for synchronizing and aligning various types of video signals, such as... Figure 1 and Figure 3 As shown, the method first includes: PTP Time Synchronization: The NDI camera and the ST2110 camera receive PTP time transmitted by a PTP clock source via the network and synchronize their device times. In this embodiment, it should be noted that the PTP time synchronization specifically includes: enabling the embedded PTP timecode function of the NDI camera to receive the PTP time via the network; and enabling the external synchronization switch of the ST2110 camera to synchronize its local time using the PTP time from the PTP clock source. That is, the core function of this step is to establish a unified time reference. Since NDI devices are typically based on software network protocols, while ST2110 devices are typically based on hardware IP-based broadcast standards, their clock mechanisms differ. By introducing a PTP (Precision Time Protocol) clock source, it is possible to ensure that heterogeneous cameras in the same network environment have "absolute time" with microsecond-level or even nanosecond-level synchronization accuracy, providing accurate reference coordinates for subsequent alignment calculations.

[0029] Subsequently, signal acquisition and timestamp embedding are performed: the NDI camera and the ST2110 camera capture images of the same object in real time, and embed the PTP timestamp at the time of capture into the video signal. Further, embedding the PTP timestamp at the time of capture into the video signal specifically includes: the NDI camera embeds the PTP timestamp in the SEI field of the video stream, and the PTP timestamp is bound to the video frame; the ST2110 camera embeds the PTP timestamp in the RTP packet header. In this embodiment, it should be noted that because the two signals have different encapsulation formats, the embedding position must conform to their respective protocol standards. For the NDI signal, using the SEI (Supplemental Enhancement Information) field of the video stream to carry time information ensures that the timestamp is tightly bound to the specific image frame and is not lost during encoding and decoding; while for the ST2110 signal, which is based on RTP (Real-time Transport Protocol) transmission, directly embedding the PTP timestamp in the RTP packet header conforms to the requirements of the SMPTE ST 2110-10 standard, facilitating rapid parsing by the receiving end.

[0030] Next, signal transmission occurs: the NDI camera and the ST2110 camera transmit the video signal to the video switcher via the network. Further, the signal transmission specifically includes: the NDI camera performs shallow compression using the NDI|FULL protocol and then transmits the signal to the video switcher via a signal switch; the ST2110 camera uses an uncompressed ST2110-20 signal and transmits it to the video switcher via the signal switch. In this embodiment, this step reveals the root cause of the signal asynchrony. Although NDI|FULL is a high-bandwidth version, it still belongs to intra-frame compression (shallow compression), and its encoding and packaging process inevitably incurs a certain processing time; while the ST2110-20 transmits an uncompressed raw video stream, with extremely low processing latency. Therefore, even if both cameras capture the same object at the same physical moment (i.e., the same PTP time point), due to the difference in transmission protocols and compression mechanisms, the arrival times at the video switcher will inevitably differ, with the ST2110 signal usually arriving before the NDI signal.

[0031] Finally, signal synchronization alignment is performed: the video switcher reads the PTP timestamps from the NDI signal transmitted by the NDI camera and the ST2110 signal transmitted by the ST2110 camera, calculates the number of delayed frames based on the difference between the two PTP timestamps, and buffers the video signal according to the number of delayed frames to achieve synchronization alignment. Further, the step of calculating the number of delayed frames based on the difference between the two PTP timestamps and buffering the video signal according to the number of delayed frames to achieve synchronization alignment specifically includes: calculating the difference between the PTP timestamp in the currently received ST2110 signal and the PTP timestamp in the NDI signal; converting the difference into the number of delayed frames based on the signal frame rate; buffering the ST2110 signal video frames according to the number of delayed frames, and storing the buffered frames in the ST2110 signal buffer queue. In this embodiment, it should be noted that because the NDI signal lags, in order to achieve image synchronization, the "fast-moving" ST2110 signal must be made to "stop and wait". By extracting and comparing the original capture timestamps (PTP timestamps) from the two received signals, the transmission and processing delay difference between the two can be accurately calculated. That is, assuming the PTP time of the current frame of the NDI signal is T1, while the PTP time of the arriving ST2110 signal is already T2 (T2>T1), it indicates that ST2110 is ahead. The system calculates this difference and divides it by the frame interval (e.g., 20ms for 50fps) to obtain the number of frames that ST2110 needs to delay (buffer), thus constructing a virtual delay line for ST2110 in the buffer queue.

[0032] Furthermore, after buffering the ST2110 signal according to the number of delayed frames, the process further includes: the video switcher reading frame data from the ST2110 signal buffer queue in a first-in-first-out manner; the video switcher directly reading the network data of the NDI signal; and the video switcher switching the aligned NDI signal and the ST2110 signal to output the broadcast signal. In this embodiment, after the above buffering process, the physical shooting time corresponding to the frame data read from the ST2110 buffer queue will be consistent with the physical shooting time of the currently directly read NDI signal. At this time, the video switcher can then perform special effects switching or mixed output on the two signals to ensure lip-sync and motion synchronization of the picture, avoiding the problems of "inaccurate audio-visual alignment" or "uneven picture speed" caused by different signal transmission mechanisms.

[0033] Example 2 Based on the same inventive concept, this embodiment also proposes a synchronization alignment system for multiple types of video signals. For example... Figure 2 As shown, the system includes: an NDI camera, an ST2110 camera, and a video switcher; Both the NDI camera and the ST2110 camera are configured to receive PTP time from a PTP clock source via the network to synchronize device time, capture real-time images of the same object, embed the PTP timestamp at the time of capture into the video signal, and transmit the video signal to the video switcher via the network. The video switcher is configured to read the PTP timestamps from the NDI signal transmitted by the NDI camera and the ST2110 signal transmitted by the ST2110 camera, calculate the delay frame number based on the difference between the two PTP timestamps, and buffer the video signal based on the delay frame number to achieve synchronization alignment.

[0034] In this embodiment, the specific configurations of each module are as follows: The NDI camera is configured to embed the PTP timestamp in the SEI field of the video stream, and the PTP timestamp is bound to the video frame; The ST2110 camera is configured to embed the PTP timestamp in the RTP packet header.

[0035] Regarding the processing logic of the video switcher, the video switcher is specifically configured as follows: calculate the difference between the PTP timestamp in the currently received ST2110 signal and the PTP timestamp in the NDI signal; convert the difference into the number of delayed frames based on the signal frame rate; buffer the ST2110 signal for video frames according to the number of delayed frames, and store the buffered frames in the ST2110 signal buffer queue.

[0036] Example 3 This embodiment also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for synchronizing and aligning multiple types of video signals. However, the computer-readable storage medium of the present invention is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0037] A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0038] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0039] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0040] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0041] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A method for synchronizing and aligning multiple types of video signals, characterized in that, include: PTP Time Synchronization: The NDI camera and ST2110 camera receive the PTP time sent by the PTP clock source through the network and synchronize the device time. Signal acquisition and embedding timestamp: The NDI camera and the ST2110 camera capture the same object in real time and embed the PTP timestamp of the capture into the video signal; Signal transmission: The NDI camera and the ST2110 camera transmit the video signal to the video switcher via a network; Signal synchronization alignment: The video switcher reads the PTP timestamps from the NDI signal transmitted by the NDI camera and the ST2110 signal transmitted by the ST2110 camera, calculates the number of delayed frames based on the difference between the two PTP timestamps, and buffers the video signal based on the number of delayed frames to achieve synchronization alignment.

2. The method for synchronizing and aligning multiple types of video signals according to claim 1, characterized in that, The specific steps of embedding the PTP timestamp at the time of shooting into the video signal include: The NDI camera embeds the PTP timestamp in the SEI field of the video stream, and the PTP timestamp is bound to the video frame; The ST2110 camera embeds the PTP timestamp in the RTP packet header.

3. The method for synchronizing and aligning multiple types of video signals according to claim 1, characterized in that, The signal transmission specifically includes: The NDI camera performs shallow signal compression using the NDI|FULL protocol and then transmits the signal to the video switcher via a signal switch. The ST2110 camera uses an uncompressed ST2110-20 signal, which is transmitted to the video switcher via the signal exchange.

4. The method for synchronizing and aligning multiple types of video signals according to claim 1, characterized in that, The step of calculating the number of delayed frames based on the difference between the two PTP timestamps, and buffering the video signal based on the number of delayed frames to achieve synchronization alignment, specifically includes: Calculate the difference between the PTP timestamp in the currently received ST2110 signal and the PTP timestamp in the NDI signal; The difference is converted into the number of delayed frames based on the signal frame rate; The ST2110 signal is buffered with video frames according to the number of delayed frames, and the buffered frames are stored in the ST2110 signal buffer queue.

5. The method for synchronizing and aligning multiple types of video signals according to claim 4, characterized in that, After buffering the ST2110 signal for video frames according to the number of delayed frames, the method further includes: The video switcher reads frame data from the ST2110 signal buffer queue in a first-in-first-out manner; The video switcher directly reads the network data of the NDI signal; The video switcher switches the aligned NDI signal and ST2110 signal to output a broadcast signal.

6. The method for synchronizing and aligning multiple types of video signals according to claim 1, characterized in that, The PTP time synchronization specifically includes: Enable the embedded PTP timecode function of the NDI camera to receive the PTP time via the network; Turn on the external synchronization switch of the ST2110 camera to synchronize the local time with the PTP time of the PTP clock source.

7. A multi-type video signal synchronization and alignment system, characterized in that, include: NDI cameras, ST2110 cameras, and video switchers; Both the NDI camera and the ST2110 camera are configured to receive PTP time from a PTP clock source via the network to synchronize device time, capture real-time images of the same object, embed the PTP timestamp at the time of capture into the video signal, and transmit the video signal to the video switcher via the network. The video switcher is configured to read the PTP timestamps from the NDI signal transmitted by the NDI camera and the ST2110 signal transmitted by the ST2110 camera, calculate the number of delayed frames based on the difference between the two PTP timestamps, and buffer the video signal based on the number of delayed frames to achieve synchronization alignment.

8. A multi-type video signal synchronization and alignment system according to claim 7, characterized in that, The NDI camera is configured to embed the PTP timestamp in the SEI field of the video stream, and the PTP timestamp is bound to the video frame; The ST2110 camera is configured to embed the PTP timestamp in the RTP packet header.

9. A multi-type video signal synchronization and alignment system according to claim 7, characterized in that, The video switcher is specifically configured as follows: Calculate the difference between the PTP timestamp in the currently received ST2110 signal and the PTP timestamp in the NDI signal; The difference is converted into the number of delayed frames based on the signal frame rate; The ST2110 signal is buffered with video frames according to the number of delayed frames, and the buffered frames are stored in the ST2110 signal buffer queue.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of a method for synchronizing and aligning multiple types of video signals as described in any one of claims 1-6.

Images