Acquisition card synchronization method and device and computer program product

Abstract

The invention discloses an acquisition card synchronization method and device and a computer program product. The method comprises the following steps: enabling rendering periods of media players to be consistent; respectively starting an acquisition thread and a rendering thread on each media player, and setting a buffer queue; after the buffer queue is filled, only the current newest frame is reserved in the buffer queue, and the current newest frame is used as the first frame of each corresponding acquisition card; setting the same timestamp for the first frame of each acquisition card; if the sizes of the buffer queues are greater than the number of the acquisition cards, all the media players obtain the first frames of the corresponding acquisition cards for rendering; if the media player meeting the preset redrawing condition exists, rendering the current latest frame once again by the rendering threads of all the media servers; and if the media player meeting the preset overflow condition exists, the buffer queues of all the media servers discard the oldest frame. According to the acquisition card synchronization scheme provided by the invention, an expensive hardware synchronization solution is replaced by a software method, so that the cost is greatly reduced.

Description

Technical Field

[0001] The present application relates to the technical field of video signal processing, and in particular to a capture card synchronization method, device and computer program product. Background Art

[0002] In the broadcast control industry, the video playback system is a complex system composed of multiple components, mainly including video acquisition, processing, transmission and display. The system usually manages the transmission and playback of video signals through a media server, such as Figure 1 As shown in the figure, the capture card is a key hardware device that converts the video signals generated by external devices into a format that can be processed by the computer. The capture card captures these signals in real time, and the media server further processes these signals, decodes and synchronizes them, and renders and plays them on the display device.

[0003] During playback, if the content captured by different capture cards is related, the system needs to ensure frame synchronization of each video stream, that is, the capture cards need to be synchronized to provide a smooth visual experience. In other words, in a video playback system, especially when processing video streams from multiple capture cards, capture card synchronization is particularly important. The core purpose of synchronization is to ensure the temporal consistency of video streams from different capture cards to avoid inconsistent or delayed images. If the signal frames of each capture card are not synchronized, it will cause problems with image dislocation or asynchrony, affecting the continuity of the video and the audience's viewing experience.

[0004] At present, the synchronization of acquisition cards in the industry mainly relies on hardware solutions, and the signal synchronization of multiple acquisition cards is achieved through special hardware. However, the inventors have realized that although these hardware solutions can solve the frame synchronization problem to a certain extent, there are few manufacturers that can provide such acquisition synchronization functions, and the cost of hardware synchronization equipment is high. The equipment is relatively expensive, resulting in an increase in the overall system investment, which makes the industry need an economical software solution to achieve synchronous acquisition of multiple acquisition cards. Summary of the invention

[0005] The present application provides an acquisition card synchronization method, device and computer program product, aiming to solve the technical problem that the existing solution of realizing signal synchronization of multiple acquisition cards through special hardware is relatively costly.

[0006] In a first aspect, a capture card synchronization method is applied to a video playback system, wherein the video playback system includes at least two media players, each of which is provided with a capture card; the method includes:

[0007] S1, making the rendering cycles of the at least two media players consistent;

[0008] S2, on each media player, respectively enabling an acquisition thread and a rendering thread, and setting a buffer queue, wherein data is transferred between the acquisition thread and the rendering thread through the buffer queue;

[0009] S3, when the acquisition card is turned on, the acquisition thread fills the acquired data into the corresponding buffer queue in sequence; after the buffer queue is filled, only the current latest frame is retained in the buffer queue, and the current latest frame is used as the first frame of each corresponding acquisition card;

[0010] S4, setting the same timestamp for the first frame of each acquisition card, and setting a timestamp for each frame subsequently acquired by each acquisition card in the buffer queue;

[0011] S5, each media player compares the size of the corresponding buffer queue with the number of acquisition cards. If the size of the buffer queue is greater than the number of acquisition cards, the rendering threads of all media players obtain the first frame of the corresponding acquisition card from the corresponding buffer queue for rendering, and render each subsequent frame acquired in the buffer queue according to the timestamp;

[0012] S6, each media player checks the size of the corresponding buffer queue in each rendering cycle; if there is a media player that meets the preset redrawing condition, the current media player sends a first synchronization instruction to all media servers, so that the rendering threads of all media servers re-render the current latest frame; if there is a media player that meets the preset overflow condition, the current media player sends a second synchronization instruction to all media servers, so that the buffer queues of all media servers discard the oldest frame.

[0013] In the above solution, optionally, in step S1, the rendering cycles of the at least two media players are made consistent specifically through a synchronization card.

[0014] In the above solution, further optionally, the synchronization card uses the PTP protocol.

[0015] In the above solution, optionally, in step S4, a frame counting method is adopted to set a timestamp for each frame subsequently acquired by each acquisition card.

[0016] In the above solution, optionally, in step S6, if the size of the buffer queue of the media player is less than or equal to the number of capture cards, it is determined that the media player meets the preset redrawing condition.

[0017] In the above solution, optionally, in step S6, if the remaining size of the buffer queue of the media player is less than or equal to the number of acquisition cards, it is determined that the media player meets the preset overflow condition.

[0018] In the above solution, optionally, both the first synchronization instruction and the second synchronization instruction are marked with a PTP clock.

[0019] In a second aspect, a capture card synchronization device is provided in a video playback system, wherein the video playback system includes at least two media players, each of which is provided with a capture card; the device includes:

[0020] A rendering cycle setting module, used to make the rendering cycles of the at least two media players consistent;

[0021] A thread enabling module, used to enable a collection thread and a rendering thread respectively on each media player, and to set a buffer queue, through which data is transferred between the collection thread and the rendering thread;

[0022] The first frame synchronization module is used for the acquisition thread to fill the acquired data into the corresponding buffer queue in sequence after the acquisition card is turned on; after the buffer queue is filled, only the current latest frame is retained in the buffer queue, and the current latest frame is used as the first frame of each corresponding acquisition card;

[0023] A timestamp setting module is used to set the same timestamp for the first frame of each acquisition card, and to set a timestamp for each frame subsequently acquired by each acquisition card in the buffer queue;

[0024] The playback module is used for each media player to compare the size of the corresponding buffer queue with the number of acquisition cards. If the size of the buffer queue is greater than the number of acquisition cards, the rendering threads of all media players obtain the first frame of the corresponding acquisition card from the corresponding buffer queue for rendering, and render each subsequent frame acquired in the buffer queue according to the timestamp;

[0025] The synchronization adjustment module is used for each media player to check the size of the corresponding buffer queue in each rendering cycle; if there is a media player that meets the preset redrawing condition, the current media player sends a first synchronization instruction to all media servers, so that the rendering threads of all media servers re-render the current latest frame; if there is a media player that meets the preset overflow condition, the current media player sends a second synchronization instruction to all media servers, so that the buffer queues of all media servers discard the oldest frame.

[0026] In a third aspect, a computer program product comprises a computer program / instruction, wherein the computer program / instruction implements the steps of the method described in the first aspect when executed by a processor.

[0027] Compared with the prior art, this application has at least the following beneficial effects:

[0028] The technical solution provided in this application,

[0029] In the acquisition card synchronization solution provided in the present application, by ensuring that the rendering cycle of the media server is consistent, the acquisition thread and the rendering thread are enabled on the media server, and a buffer queue is set to manage the acquisition data; the timestamp of the first frame is synchronized to ensure the synchronization of the acquisition card frame, and the dynamic adjustment mechanism of the buffer queue (redraw or drop frame instructions) is used to cope with the slight difference between the acquisition frequency and the rendering frequency, thereby maintaining a stable synchronization state; the frame data of the acquisition card is aligned by timestamp to ensure that all players render the same frame at the same time, and the playback strategy is automatically adjusted according to the state of the buffer queue, such as dropping frames or redrawing. This mechanism enables the buffer to remain stable between different acquisition and rendering frequencies, reducing the problem of picture asynchrony caused by frequency mismatch, thereby achieving efficient multimedia data acquisition and rendering synchronization; the technical solution provided in the present application uses synchronization instructions to dynamically adjust the buffer, so that the buffer queue is always in a stable state, ensuring the data synchronization and playback continuity of multiple acquisition cards; the acquisition card synchronization solution provided in the present application replaces expensive hardware synchronization solutions with software methods, which greatly reduces costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of an application environment of a capture card synchronization method provided by an embodiment of the present application;

[0031] Figure 2 A flowchart of a method for synchronizing an acquisition card provided by an embodiment of the present application;

[0032] Figure 3 A block diagram of the module architecture of a capture card synchronization device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] In the description of this application: unless otherwise specified, the meaning of "plurality" is two or more. The terms "first", "second", "third", etc. in this application are intended to distinguish the objects referred to, and do not have special meanings in terms of technical connotations (for example, they should not be understood as emphasizing the importance or order, etc.). Expressions such as "including", "comprising", "having", etc. also mean "not limited to" (certain units, components, materials, steps, etc.).

[0035] In one embodiment, the present application provides a capture card synchronization method, which can be applied to Figure 1In the video playing system shown, the video playing system includes at least two media players, and each media player is provided with a capture card.

[0036] like Figure 2 As shown, the acquisition card synchronization method provided by the present application includes the following steps:

[0037] S1, make the rendering cycles of at least two media players consistent.

[0038] Furthermore, in step S1, the rendering cycles of at least two media players are made consistent through a synchronization card, wherein the synchronization card uses the PTP protocol.

[0039] Capture cards cannot synchronize signals, but multiple media servers can use synchronization cards to ensure consistent rendering cycles. In this way, different capture cards can be in the same rendering cycle, and the PTP protocol can be used between media servers to ensure clock synchronization.

[0040] Use the PTP protocol to ensure that the internal clock of each media server is consistent with the clocks of other servers. This is the basis for ensuring that all servers operate in the same clock cycle. The rendering threads of all media servers will be scheduled based on the synchronized clock signal to ensure that they render in the same time interval. Through the PTP clock, the server can ensure that the rendering operation is performed in the same clock cycle, avoiding the problem of rendering asynchrony caused by clock deviation.

[0041] S2, on each media player, enable the acquisition thread and rendering thread respectively, and set a buffer queue; in other words, on the media server, enable two threads, one for rendering and one for acquisition; the acquisition thread and the rendering thread transfer data through a buffer queue.

[0042] That is to say, on each media player, there are two threads: one is responsible for acquiring data from the capture card (capture thread), and the other is responsible for rendering this data to the display device (rendering thread). The acquisition thread puts the data frame into a buffer queue, and the rendering thread obtains data from this buffer queue for rendering.

[0043] The capture thread continuously obtains new frames from the capture card and puts them into the buffer queue. The buffer queue temporarily stores all captured frames; the purpose of the buffer queue is to smooth the data transmission process and allow the rendering thread to read data from it.

[0044] S3, when the acquisition card is turned on, the acquisition thread fills the acquired data into the corresponding buffer queue in sequence; after the buffer queue is filled, only the current latest frame is retained in the buffer queue, and the current latest frame is used as the first frame of each corresponding acquisition card.

[0045] In other words, when the capture card is turned on, it does not start rendering immediately. It waits until the buffer queue is full, then only the latest frame is retained and other frames are discarded. In this way, the current capture signal is the latest frame while offsetting the fluctuation of the capture cycle, which indirectly synchronizes the first frame of the capture signal of multiple capture cards. The latest frame here is the latest frame in the queue when the buffer queue is full.

[0046] S4, setting the same timestamp for the first frame of each acquisition card, and setting a timestamp for each frame subsequently acquired by each acquisition card in the buffer queue.

[0047] In order to ensure that the playback process is always in synchronization, it is necessary to set a timestamp for the captured frames. The timestamp uses the frame count method. Because the capture card clocks may be inconsistent. When the timestamp of the first frame is the same, set the first frame to 0, and the timestamp of each subsequent frame is accumulated by 1. This ensures that the timestamps of multiple capture cards are consistent. Because the acquisition cycle of the capture card may not be synchronized, but the acquisition frequency can be guaranteed to be the same, so as long as the first frame is synchronized, each subsequent frame with the same timestamp is synchronized.

[0048] S5, each media player compares the size of the corresponding buffer queue with the number of capture cards. If the size of the buffer queue is greater than the number of capture cards, the rendering threads of all media players obtain the first frame of the corresponding capture card from the corresponding buffer queue for rendering, and render each subsequent frame captured in the buffer queue according to the timestamp.

[0049] In other words, comparing the size of the buffer queue with the number of capture cards is the start-up logic, that is, the judgment logic for whether to render the first frame, and each subsequent frame is rendered by a timestamp.

[0050] S6, each media player checks the size of the corresponding buffer queue in each rendering cycle; if there is a media player that meets the preset redrawing condition, the current media player sends a first synchronization instruction to all media servers, so that the rendering threads of all media servers re-render the current latest frame; if there is a media player that meets the preset overflow condition, the current media player sends a second synchronization instruction to all media servers, so that the buffer queues of all media servers discard the oldest frame.

[0051] Further, if the size of the buffer queue of a media player is less than or equal to the number of capture cards, it is determined that a media player meets the preset redraw condition. If the remaining size of the buffer queue of a media player is less than or equal to the number of capture cards, it is determined that a media player meets the preset overflow condition.

[0052] In other words, due to the slight difference between the rendering frequency and the acquisition frequency, when the acquisition frequency is greater than the rendering frequency, the buffer queue will gradually increase, and vice versa. When the buffer queue is full, the acquisition thread will be blocked or frame loss will occur, so the buffer queue cannot offset the difference in acquisition cycle fluctuations; and when the buffer queue is empty, it cannot offset acquisition fluctuations either.

[0053] To solve this problem, after the first frame is synchronized, it is not played immediately. It is necessary to wait until the buffer queue is larger than the number of synchronized capture cards before starting to play. Then the size of the buffer queue is checked in each rendering cycle. When the buffer queue is less than or equal to the number of synchronized capture cards, the current media server notifies all media servers to redraw the current frame; when the buffer queue is close to full, the current media server notifies all media servers to discard the oldest frame in the buffer queue. This mechanism ensures synchronization while keeping the buffer queue stable.

[0054] Among them, when the remaining size of the buffer queue is less than or equal to the number of synchronous acquisition cards, it is considered to be about to overflow and the frame drop operation needs to be triggered.

[0055] For example, two capture cards are currently synchronized, and the buffer queue is designed to be 6 frames. After the first frame is synchronized, it is necessary to wait until the buffer queue is greater than 2 before playing. During the playback process, when the buffer queue is less than or equal to 2, all servers are notified to redraw a frame. When the buffer queue is greater than 4 frames, all servers are notified to drop frames.

[0056] Furthermore, both the first synchronization instruction and the second synchronization instruction are marked with a PTP clock.

[0057] In other words, media servers need to communicate with each other to control a series of commands such as playback, frame dropping, and redrawing. In order to ensure that multiple servers can execute commands in the same clock cycle, the sender needs to add the PTP clock to the command and specify the execution time. The receiver uses the PTP clock and command time in the rendering thread to determine whether the command is executed. In this way, multiple servers can drop frames and redraw in the same rendering cycle to ensure the synchronization of the screen.

[0058] The synchronization of multiple acquisition cards requires solving the following two problems:

[0059] 1. How to determine whether multiple acquired signals are the same frame signal;

[0060] 2. During playback, due to the asynchrony of the acquisition cycles among multiple acquisition cards, the different rendering frequencies of the acquisition cards and the media server will cause intermittent asynchrony, that is, the images will be synchronized for a while and then out of sync.

[0061] The difficulty of acquisition synchronization technology lies in the inconsistency of the acquisition cycles of multiple acquisition cards. In this case, a buffer is used to offset the fluctuation. At the same time, the inconsistency between the acquisition frequency and the rendering frequency will cause the buffer to overflow or become empty, thus losing the buffer function. The technical solution proposed in this application uses synchronization instructions to dynamically adjust the buffer, so that the buffer is always in a stable state to ensure the synchronization of acquisition.

[0062] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0063] In one embodiment, Figure 3 As shown, a capture card synchronization device is provided, which is arranged in a video playback system. The video playback system includes at least two media players, and each media player is provided with a capture card; the device includes:

[0064] A rendering cycle setting module 301, used to make the rendering cycles of at least two media players consistent;

[0065] The thread enabling module 302 is used to enable the acquisition thread and the rendering thread on each media player, and set a buffer queue, so that data is transferred between the acquisition thread and the rendering thread through the buffer queue;

[0066] The first frame synchronization module 303 is used for, when the acquisition card is turned on, the acquisition thread fills the acquired data into the corresponding buffer queue in sequence; after the buffer queue is filled, only the current latest frame is retained in the buffer queue, and the current latest frame is used as the first frame of each corresponding acquisition card;

[0067] The timestamp setting module 304 is used to set the same timestamp for the first frame of each acquisition card, and to set the timestamp for each frame subsequently acquired by each acquisition card in the buffer queue;

[0068] The playback module 305 is used for each media player to compare the size of the corresponding buffer queue with the number of acquisition cards. If the size of the buffer queue is greater than the number of acquisition cards, the rendering threads of all media players obtain the first frame of the corresponding acquisition card from the corresponding buffer queue for rendering, and render each subsequent frame acquired in the buffer queue according to the timestamp;

[0069] The synchronization adjustment module 306 is used for each media player to check the size of the corresponding buffer queue in each rendering cycle; if there is a media player that meets the preset redrawing condition, the current media player sends a first synchronization instruction to all media servers, so that the rendering threads of all media servers re-render the current latest frame; if there is a media player that meets the preset overflow condition, the current media player sends a second synchronization instruction to all media servers, so that the buffer queues of all media servers discard the oldest frame.

[0070] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which involves all or part of the processes in the above-mentioned embodiment method.

[0071] In one embodiment, a computer program product is also provided, including a computer program / instruction, which implements all or part of the process in the above-mentioned embodiment method when executed by a processor.

[0072] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for synchronizing an acquisition card, characterized in that: Applied to a video playback system, the video playback system includes at least two media players, each of which is provided with a capture card; the method includes: S1, making the rendering cycles of the at least two media players consistent; S2, on each media player, respectively enabling an acquisition thread and a rendering thread, and setting a buffer queue, wherein data is transferred between the acquisition thread and the rendering thread through the buffer queue; S3, when the acquisition card is turned on, the acquisition thread fills the acquired data into the corresponding buffer queue in sequence; after the buffer queue is filled, only the current latest frame is retained in the buffer queue, and the current latest frame is used as the first frame of each corresponding acquisition card; S4, setting the same timestamp for the first frame of each acquisition card, and setting a timestamp for each frame subsequently acquired by each acquisition card in the buffer queue; S5, each media player compares the size of the corresponding buffer queue with the number of acquisition cards. If the size of the buffer queue is greater than the number of acquisition cards, the rendering threads of all media players obtain the first frame of the corresponding acquisition card from the corresponding buffer queue for rendering, and render each subsequent frame acquired in the buffer queue according to the timestamp; S6, each media player checks the size of the corresponding buffer queue in each rendering cycle; if there is a media player that meets the preset redrawing condition, the current media player sends a first synchronization instruction to all media servers, so that the rendering threads of all media servers re-render the current latest frame; if there is a media player that meets the preset overflow condition, the current media player sends a second synchronization instruction to all media servers, so that the buffer queues of all media servers discard the oldest frame.

2. The acquisition card synchronization method according to claim 1, characterized in that: In step S1, the rendering cycles of the at least two media players are made consistent through a synchronization card.

3. The acquisition card synchronization method according to claim 2, characterized in that: The synchronization card uses the PTP protocol.

4. The acquisition card synchronization method according to claim 1, characterized in that: In step S4, a frame counting method is adopted to set a timestamp for each frame subsequently acquired by each acquisition card.

5. The acquisition card synchronization method according to claim 1, characterized in that: In step S6, if the size of the buffer queue of the media player is less than or equal to the number of capture cards, it is determined that there is a media player that meets the preset redrawing condition.

6. The acquisition card synchronization method according to claim 1, characterized in that: In step S6, if the remaining size of the buffer queue of the media player is less than or equal to the number of capture cards, it is determined that the media player meets the preset overflow condition.

7. The acquisition card synchronization method according to claim 1, characterized in that: The first synchronization instruction and the second synchronization instruction are both marked with a PTP clock.

8. A data acquisition card synchronization device, characterized in that: The device is arranged in a video playing system, wherein the video playing system comprises at least two media players, each of which is provided with a capture card; the device comprises: A rendering cycle setting module, used to make the rendering cycles of the at least two media players consistent; A thread enabling module, used to enable a collection thread and a rendering thread respectively on each media player, and to set a buffer queue, through which data is transferred between the collection thread and the rendering thread; The first frame synchronization module is used for the acquisition thread to fill the acquired data into the corresponding buffer queue in sequence after the acquisition card is turned on; after the buffer queue is filled, only the current latest frame is retained in the buffer queue, and the current latest frame is used as the first frame of each corresponding acquisition card; A timestamp setting module is used to set the same timestamp for the first frame of each acquisition card, and to set a timestamp for each frame subsequently acquired by each acquisition card in the buffer queue; The playback module is used for each media player to compare the size of the corresponding buffer queue with the number of acquisition cards. If the size of the buffer queue is greater than the number of acquisition cards, the rendering threads of all media players obtain the first frame of the corresponding acquisition card from the corresponding buffer queue for rendering, and render each subsequent frame acquired in the buffer queue according to the timestamp; The synchronization adjustment module is used for each media player to check the size of the corresponding buffer queue in each rendering cycle; if there is a media player that meets the preset redrawing condition, the current media player sends a first synchronization instruction to all media servers, so that the rendering threads of all media servers re-render the current latest frame; if there is a media player that meets the preset overflow condition, the current media player sends a second synchronization instruction to all media servers, so that the buffer queues of all media servers discard the oldest frame.

9. A computer program product comprising a computer program / instructions, characterized in that The computer program / instructions implement the steps of the method as claimed in claim 1 when executed by a processor.

Images