A frame synchronization method, video splicing wall output method and system

By obtaining the frame rate of the video stream and uniformly processing the frame rate, the problem of high FPGA cost and inconsistent frame rate synchronization is solved, the unified and synchronization of video frame rate is achieved, the R&D cost is reduced, and a variety of encoders and decoders are adapted to multiple encoders and decoders. .

CN115883749BActive Publication Date: 2024-12-06VTRON GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211491329.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-25
Publication Date
2024-12-06
Estimated Expiration
2042-11-25

AI Technical Summary

Technical Problem

The use of FPGAs in existing video processing technologies leads to high R&D costs, and FPGAs cannot change the video frame rate and cannot achieve synchronous requirements for inconsistent frame rates.

Method used

By obtaining the original frame rate and target frame rate of the input video stream, determine the number of unnecessary frames that need to be extracted, use the preset frame extraction rules to extract excess frames from the original video frame, reduce the frame rate, and use the time calibration tool to perform the following Synchronize, avoid using FPGA for frame rate compression and overlay.

Benefits of technology

It realizes the unification of video frame rates, reduces hardware R&D costs, simplifies the synchronization process, adapts to different types of encoders and decoders, and avoids the picture tearing of the splicing wall display.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115883749B_ABST
    Figure CN115883749B_ABST
Patent Text Reader

Abstract

The present invention provides a frame synchronization method, a video splicing wall output method and a system, wherein the frame synchronization method comprises: obtaining the original video frame of the input video stream and its input frame rate F in ; Determine the target frame rate F of the output video stream out , determine the number of redundant frames that need to be removed per unit time F extr ; According to the number of redundant frames that need to be removed per unit time F extr , extract redundant frames from the original video frames, specifically including: in And the number of redundant frames that need to be removed per unit time F extr Extract the greatest common divisor G and divide the original video frame into G frame extraction intervals. Each frame extraction interval includes F in / G video frames, using an incremental periodic extraction method to extract redundant frames from each frame extraction interval. Compared with the existing technology, this frame reduction method makes the frame rate of each video stream uniform, and does not need to use FPGA to compress and superimpose the frame rate of the video, which reduces the R&D cost and can easily achieve time synchronization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of video processing technology, and more specifically, to a frame synchronization method, a video splicing wall output method and a system. Background Art

[0002] With the continuous development of video processing technology, various users of splicing walls, such as combat centers, railway management centers, etc., have increasing demands for high resolution and high operability of large walls. Video splicing walls are composed of multiple display units spliced ​​together to form a large screen for display. Existing video processing technology generally receives the encoded audio and video streaming media stream by the decoder, then decodes and outputs it to the back-end FPGA. After the FPGA scales the video, after superimposing the algorithm, it is divided and output to each display unit for output display. Generally, only the synchronization of each display unit needs to be considered. However, the extensive use of FPGA will lead to increased R&D costs, which will not be conducive to the large-scale business promotion of video processing technology.

[0003] In addition, when the business requires the "seat + video wall" mode, using seats to synchronously display and control the video wall, and the frame rates of the seats and the video wall are inconsistent, for this kind of encoding of different frame rate code streams for the same video source, it is necessary to change the frame rate of the video stream to achieve synchronization. However, FPGA cannot change the frame rate of each code stream, so it is impossible to achieve the synchronization requirement in this case. Summary of the invention

[0004] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art and provide a frame synchronization method, a video splicing wall output method and a system for solving the problems that the existing video processing technology uses FPGA, resulting in high R&D costs, and the FPGA cannot change the video frame rate to achieve synchronization.

[0005] The technical solution adopted by the present invention is:

[0006] A frame synchronization method, comprising:

[0007] S1: Get the original video frame of the input video stream and its input frame rate F in ;

[0008] S2: Determine the target frame rate F of the output video stream out , and according to the input frame rate F in and target frame rate F out , determine the number of redundant frames that need to be removed per unit time F extr ;

[0009] S3: according to the number of redundant frames F to be removed within the unit time extr, using a preset frame extraction rule to extract redundant frames from the original video frames to form an output video stream;

[0010] According to the number of redundant frames to be removed per unit time, F extr , using a preset frame extraction rule to extract redundant frames from the original video frame, specifically including:

[0011] S31: Determine the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Is there a common divisor? If yes, execute S311; if no, execute S312;

[0012] S311: Input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Extract the greatest common divisor G, divide the original video frames in each unit time into G frame extraction intervals, each frame extraction interval includes F in / G video frames, using incremental periodic extraction method, extract F from each frame extraction interval extr / G frame extra frame;

[0013] S312: directly use the original video frames in each unit time as frame extraction intervals, each frame extraction interval includes F in Frame video frame, using incremental periodic extraction method, extract F from each frame extraction interval extr Frames too many frames.

[0014] In step S1, according to the time when the original video stream arrives at the encoder and the input frame rate F in To calculate the frame number of the generated video stream, and input the number of frames per second N in For a set of determined original video frames, the calculation formula is [T (graphics card output) / T (interval between frames)]%F (frame rate)=N (frame number), where the number of milliseconds that have passed since 1970 is used as the timestamp.

[0015] The present invention removes redundant frames from the original video frame, achieving the effect of reducing the original video frame. It does not require the participation of FPGA, and only needs to use a time calibration tool to time-calibrate each video frame after frame reduction, thereby ensuring the synchronization of multi-head videos and reducing the cost of research and development and production. The frame extraction interval is divided evenly, that is, when there is a common divisor, the original video frame is evenly divided into G frame extraction intervals. Because the original video frame is evenly divided, and frame extraction is performed on each frame extraction area, the smoothness of the encoding is guaranteed as much as possible; at the same time, the reduction in frame rate enables the encoding stream to adapt to encoders of various performances.

[0016] Furthermore, the incremental extraction method is specifically as follows:

[0017] A1: Set the number of video frames in the frame extraction interval to F1, the number of redundant frames to be extracted in the frame extraction interval to F2, and N is a natural number starting from 1;

[0018] A2: Determine whether F1 / N is less than or equal to F2. If so, execute A21; otherwise, execute A22.

[0019] A21: Taking N video frames as a extraction period, dividing the frame extraction interval into extraction periods, extracting one video frame from each extraction period, and recording the number of all video frames extracted in each extraction period at the current moment as C, and then executing step A3;

[0020] A22: Set N=N+1, and jump back to A2 to re-execute the determination of whether F1 / N is less than or equal to F2;

[0021] A3: Determine whether there is a remainder in F1 / N at the current moment, and whether F2-C is greater than or equal to 1. If so, execute A31; otherwise, execute A32.

[0022] A31: extract one more video frame from the remaining video frames after the frame extraction interval is divided into periods, and set C=C+1 at the current moment, and then jump to step A32;

[0023] A32: Determine whether F2-C at the current moment is equal to 0;

[0024] Determine whether F2-C at the current moment is equal to 0. If so, execute A321; otherwise, execute A322;

[0025] A321: End redundant frame extraction;

[0026] A322: Set the current F2 = F2-C, and jump back to A2 to re-execute the determination of whether F1 / N is less than or equal to F2.

[0027] Further, extracting one video frame from each extraction period, and extracting one video frame from the remaining video frames after the frame extraction interval is divided into periods, specifically including:

[0028] Video frames are extracted starting from the first frame of each extraction cycle or from the first frame of the remaining video frames after the frame extraction interval is divided into periods. If the first video frame is extracted, the next video frame is extracted in frame order. If the next video frame is also extracted, the video frames are continuously extracted until a video frame can be extracted.

[0029] The method extracts one video frame from each extraction cycle, starting from the first frame of the extraction cycle. This extraction allows a certain interval between the extracted frames, minimizes the continuity between the extracted frames, and further ensures the smoothness of the extracted video frames.

[0030] The present invention also provides a frame synchronization system, comprising:

[0031] The input video stream acquisition module is used to obtain the original video frames of the input video stream and its input frame rate F in ;

[0032] The redundant frame determination module is used to determine the target frame rate F of the output video stream. out , and according to the input frame rate F in and target frame rate F out , determine the number of redundant frames that need to be removed per unit time F extr ;

[0033] The redundant frame extraction module is used to extract the redundant frames according to the number of frames F to be extracted in the unit time. extr , using a preset frame extraction rule to extract redundant frames from the original video frames to form an output video stream;

[0034] The redundant frame removal module specifically includes:

[0035] The common divisor judgment module is used to judge the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Is there a common divisor?

[0036] The first sub-module is used for determining the input frame rate F when the common divisor determination module determines the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr When there is a common divisor, the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Extract the greatest common divisor G, divide the original video frames in each unit time into G frame extraction intervals, each frame extraction interval includes F in / G video frames, using incremental periodic extraction method, extract F from each frame extraction interval extr / G frame extra frame;

[0037] The second submodule is used for determining the input frame rate F when the common divisor determination module determines the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr When there is no common divisor, the original video frames in each unit time are directly used as frame extraction intervals. Each frame extraction interval includes Fin Frame video frame, using incremental periodic extraction method, extract F from each frame extraction interval extr Frames too many frames.

[0038] Through the frame synchronization system, the original video frames in the video stream are reduced to compress the frame rate, and the frame rate of each video stream can be unified after frame reduction. There is no need to use FPGA for frame compression and video superposition, which reduces the cost of hardware research and development. At the same time, the video frames processed by the frame synchronization system only need to synchronize the time of each encoder through the time synchronization tool to synchronize the encoding output, reducing unnecessary interactions and simple implementation. In addition, the redundant frame extraction module can be set, that is, the bit rate to be extracted can be set. For the same video source, different frame rate bit streams can be encoded, which can adapt to various types of decoders.

[0039] The present invention also provides a video splicing wall output method, comprising:

[0040] B1: Obtain several video streams with different frame rates from several output heads of the computer graphics card;

[0041] B2: using the frame synchronization method to unify the frame rates of the plurality of video streams with different frame rates, thereby obtaining a plurality of video streams with unified frame rates;

[0042] B3: inputting the plurality of video streams with unified frame rates into corresponding encoders for encoding to obtain a plurality of encoded video streams;

[0043] B4: inputting the plurality of encoded video streams into corresponding decoders for decoding to obtain a plurality of decoded video streams;

[0044] B5: The plurality of decoded video streams are spliced ​​together through a splicing processor to obtain a video splicing wall for output display.

[0045] The frame synchronization method is used to reduce the frame rate of the bitstreams in each video stream and unify the frame rate of each bitstream, thereby ensuring the synchronization of the video frame rate and avoiding tearing of the image displayed on the splicing wall. At the same time, there is no need to set up an additional FPGA for frame rate compression and FPGA programming, reducing hardware and development costs.

[0046] Further, the plurality of video streams with unified frame rates are input into corresponding encoders for encoding to obtain a plurality of encoded video streams, specifically including:

[0047] Using ntp (Network Time Protocol) to perform time synchronization calibration on the plurality of video streams with unified frame rates, to obtain a plurality of video streams after synchronization calibration;

[0048] The plurality of synchronously calibrated video streams are input into corresponding encoders for encoding to obtain a plurality of encoded video streams.

[0049] Video stream synchronization can be achieved by simply using NTP, which reduces additional equipment, improves efficiency and reduces costs.

[0050] The present invention also provides a video splicing wall output system, comprising:

[0051] A video stream acquisition module is used to acquire a plurality of video streams with different frame rates from a plurality of output heads of a computer graphics card;

[0052] A frame rate unification module, used to unify the frame rates of the plurality of video streams with different frame rates by using the frame synchronization method described above, so as to obtain the plurality of video streams with unified frame rates;

[0053] An encoding module, used for inputting the plurality of video streams with unified frame rates into a corresponding encoder for encoding to obtain a plurality of encoded video streams;

[0054] A decoding module, used for inputting the plurality of encoded video streams into corresponding decoders for decoding to obtain a plurality of decoded video streams;

[0055] The splicing display module is used to splice the plurality of decoded video streams through a splicing processor to obtain a video splicing wall for output display.

[0056] Furthermore, the encoding module specifically includes:

[0057] A synchronization calibration module is used to perform time synchronization calibration on the plurality of video streams with unified frame rates by using NTP to obtain the plurality of video streams after synchronization calibration;

[0058] The encoding submodule is used to input the plurality of synchronously calibrated video streams into corresponding encoders for encoding to obtain a plurality of encoded video streams.

[0059] After the frame unification module performs frame reduction processing on each video stream and unifies them to the same frame rate, a video stream with a lower frame rate is obtained, which can adapt to various types of encoders and decoders, making it easier to set up the subsequent encoding module and decoding module.

[0060] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the frame synchronization method or the video splicing wall output method as described above when executing the computer program.

[0061] The present invention also provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the frame synchronization method or the video splicing wall output method as described above is implemented.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The present invention can extract redundant frames from video streams of various frame rates through a frame synchronization method to achieve an arbitrary uniform frame rate while ensuring the smoothness of the video stream;

[0064] 2. The present invention also uses a frame synchronization method to reduce the frame rate of the video stream before encoding and unify the frame rate, which can meet the requirements of various decoders and only needs to use ntp for time synchronization. There is no need to use FPGA to scale and superimpose the video after decoding, which reduces the cost of research and development. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flow chart of the steps of the frame synchronization method in Embodiment 1 of the present invention.

[0066] Figure 2 It is a flowchart of the steps of the incremental extraction method in Example 1 of the present invention.

[0067] Figure 3 This is an example of embodiment 1 of the present invention. Figure 1 .

[0068] Figure 4 This is an example of embodiment 1 of the present invention. Figure 2 .

[0069] Figure 5 This is an example of embodiment 1 of the present invention. Figure 3 .

[0070] Figure 6 This is a system structure diagram in Example 2 of the present invention.

[0071] Figure 7 This is a flow chart of the steps of the video splicing wall output method in Example 3 of the present invention.

[0072] Figure 8 This is a system structure diagram in Example 4 of the present invention.

[0073] Fig. 9 This is an example of embodiment 1 of the present invention. Figure 4 . DETAILED DESCRIPTION

[0074] The drawings of the present invention are only for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the drawings may be omitted.

[0075] Example 1

[0076] like Figure 1 FIG. 1 is a flowchart of a frame synchronization method according to Embodiment 1 of the present invention. Figure 1 Embodiment 1 of the present invention provides a frame synchronization method, including:

[0077] S1: Get the original video frame of the input video stream and its input frame rate F in ;

[0078] In this step, according to the time when the original video stream arrives at the encoder and the input frame rate F in To calculate the frame number of the generated video stream, and input the number of frames per second N in As a group, from 0 to (N in -1) is a set of frame numbers to determine the original video frame. Specifically, the number of milliseconds that have passed since 1970 is used as the timestamp, and the frame number is calculated by the formula [T (graphics card output) / T (interval between frames)] % F (frame rate) = N (frame number), such as Fig. 9 As shown, assuming that the time it takes for a video frame to arrive at the encoder is 7128ms and the output frame rate is 50 frames, substituting it into the formula yields: [7128 / (1000 / 50)]%50=6; that is, the frame number with a timestamp of 7128ms is 6, and then each frame number 0-49 is grouped together as the original video frame for processing.

[0079] S2: Determine the target frame rate F of the output video stream out , and according to the input frame rate F in and target frame rate F out , determine the number of redundant frames that need to be removed per unit time F extr ;

[0080] S3: according to the number of redundant frames F to be removed within the unit time extr , using a preset frame extraction rule to extract redundant frames from the original video frame to form an output video stream, specifically including:

[0081] S31: Determine the input frame rate F in And the number of redundant frames that need to be removed per unit time F extrIs there a common divisor? If yes, execute S311; if no, execute S312;

[0082] S311: Input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Extract the greatest common divisor G, divide the original video frames in each unit time into G frame extraction intervals, each frame extraction interval includes F in / G video frames, using incremental periodic extraction method, extract F from each frame extraction interval extr / G frame redundant frames, the frame extraction intervals after the redundant frames are removed form the output video stream;

[0083] S312: directly use the original video frames in each unit time as frame extraction intervals, each frame extraction interval includes F in Frame video frame, using incremental periodic extraction method, extract F from each frame extraction interval extr Frame redundant frames, the frame extraction intervals after removing the redundant frames will form the output video stream.

[0084] Specifically, the incremental extraction method in S311 and S312 is:

[0085] A1: Set the number of video frames in the frame extraction interval to F1, the number of redundant frames to be extracted in the frame extraction interval to F2, and N is a natural number starting from 1; wherein, when executing step S1032, the value of F2 is F extr / G; when executing step S1033, the value of F2 is F extr ;

[0086] A2: Determine whether F1 / N is less than or equal to F2. If so, execute A21; otherwise, execute A22.

[0087] A21: Taking N video frames as a extraction period, dividing the frame extraction interval into extraction periods, extracting one video frame from each extraction period, and recording the number of all video frames extracted in each extraction period at the current moment as C, and then executing step A3;

[0088] A22: Set N=N+1, and jump back to A2 to re-execute the determination of whether F1 / N is less than or equal to F2;

[0089] A3: Determine whether there is a remainder in F1 / N at the current moment, and whether F2-C is greater than or equal to 1. If so, execute A31; otherwise, execute A32.

[0090] A31: Extract one more video frame from the remaining video frames after the frame extraction interval is divided into periods. Specifically, extract the video frames from the first frame of each extraction period or from the first frame of the remaining video frames after the frame extraction interval is divided into periods. If the first video frame is extracted, extract the next video frame in the frame sequence. If the next video frame is also extracted, continue to extract the video frames until the video frame can be extracted. Then set C = C + 1 at the current moment, and jump to step A32;

[0091] A32: Determine whether F2-C at the current moment is equal to 0;

[0092] Determine whether F2-C at the current moment is equal to 0. If so, execute A321; otherwise, execute A322;

[0093] A321: End redundant frame extraction;

[0094] A322: Set the current F2 = F2-C, and jump back to A2 to re-execute the determination of whether F1 / N is less than or equal to F2.

[0095] In a specific embodiment, Figure 2 As shown in Figure 2, the light color represents the frames that need to be extracted. Assuming that the video input is 50 frames, F in =50; need to reduce to 30 frames output, that is, F out =30; then we need to extract 20 frames from 50 frames, F extr =20; According to the above frame synchronization method, input frame rate F in =50 and extra frame F extr =20, there is a common divisor G = 10, that is, the original video is divided into 10 frame extraction intervals, each frame extraction interval includes F in / G=50 / 10=F1=5 frames, F needs to be extracted from 5 frames extr / G=20 / 10=F2=2 frames;

[0096] In order to achieve uniform extraction, an incremental extraction method is used. Specifically, first take N=1, then F1 / N=5 / 1=5. Because 5>F2=2, skip N=1, and then N+1 to get N=2. At this time, F1 / N=5 / 2=2 with a remainder of 1, and 2<=F2=2, so 5 frames are divided into a cycle of 2 frames, and 1 frame is extracted from the first frame in each cycle; at the same time, because there is a remainder of 1 in the F1 / N formula, but F2-C=0<1, no additional extraction is required. Then perform the same extraction operation on each frame extraction interval until it is reduced to 30 frames for output.

[0097] The above assumptions take into account the situation where there is a common divisor between the video input frame rate and the redundant frames. It is also necessary to consider the situation where there is no common divisor between the input frame rate and the redundant frames, such as Figure 3 As shown, assuming the input frame rate is 49, that is, F in =49, the output frame rate is required to be 30 frames, that is, F out =30; then 19 frames need to be extracted from 49 frames, F extr =19, because the input frame rate F in =49 and extra frame F extr =19, there is no common divisor between them. In order to achieve uniform extraction, the incremental extraction method is directly adopted. Specifically, F1 = 49, F2 = F extr =19, first take N=1, F1 / N=49 / 1=49>19, skip N=1; take N=2, F1 / N=49 / 2=24 remainder 1, because 24>19, so skip N=2, then N+1=3, F1 / N=49 / 3=16 remainder 1, because 16<19, so extract from 49 frames with every 3 frames as a cycle.

[0098] And because F1 / N=49 / 3=16 has a remainder 1, through F2-C=19-16=3>=1, that is, F2-C>=1, then 1 more frame can be extracted from 49 frames, that is, 17 frames can be extracted from 49 frames, and 2 frames need to be extracted, then C=C+1=17; and because F2-C is not equal to 0, F2=F2-C=2 is set, then N=3, 49 / 3=16 has a remainder 1, because 16>2, so the case of N=3 is skipped, and then N=N+1; 49 / 4=12 has a remainder 1, because 12>2, so the case of N=4 is skipped, and then N=N+1; and so on, the calculation is carried out until 49 / 17=2 has a remainder 15, 2<=2 satisfies the condition. At this time, if all the divisors that satisfy the quotient equal to 2 are found, the calculation is as follows: 49 / 18 = 2 with a remainder of 13, 2 <= 2 meets the condition; 49 / 19 = 2 with a remainder of 11, 2 <= 2 meets the condition; 49 / 20 = 2 with a remainder of 9, 2 <= 2 meets the condition; 49 / 21 = 2 with a remainder of 7, 2 <= 2 meets the condition; 49 / 22 = 2 with a remainder of 5, 2 <= 2 meets the condition; 49 / 23 = 2 with a remainder of 3, 2 <= 2 meets the condition; 49 / 24 = 2 with a remainder of 1, 2 <= 2 meets the condition; until 49 / 25 = 1 with a remainder of 24 does not meet the condition that the divisor is 2. Therefore, the 24 frames with the smallest remainder are taken as a cycle for extraction, such as Figure 4 As shown in the figure, starting from the first frame of the cycle, if the first frame of the cycle is extracted, the next frame is extracted in sequence. If the next frame is also extracted, one frame is pushed back until it can be extracted. At this point, the remaining 2 frames are also extracted. At this time, the number of frames is reduced to 30, which meets the output conditions, and also tries to ensure the extraction interval between frames, ensuring the smoothness of the video.

[0099] Example 2

[0100] like Figure 4 As shown, this embodiment provides a frame synchronization system, including:

[0101] Input video stream acquisition module 1, used to obtain the original video frame of the input video stream and its input frame rate F in ;

[0102] The redundant frame determination module 2 is used to determine the target frame rate F of the output video stream. out , and according to the input frame rate F in and target frame rate F out , determine the number of redundant frames that need to be removed per unit time F extr ;

[0103] The redundant frame extraction module 3 is used to extract the redundant frames according to the number of frames F to be extracted in the unit time. extr , using a preset frame extraction rule to extract redundant frames from the original video frames to form an output video stream;

[0104] The redundant frame removal module 3 specifically comprises:

[0105] The common factor determination module 31 is used to determine the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Is there a common divisor?

[0106] The first sub-module 32 is used for determining the input frame rate F when the common divisor determination module 31 determines the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr When there is a common divisor, the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Extract the greatest common divisor G, divide the original video frames in each unit time into G frame extraction intervals, each frame extraction interval includes F in / G video frames, using incremental periodic extraction method, extract F from each frame extraction interval extr / G frame extra frame;

[0107] The second sub-module 33 is used for determining the input frame rate F when the common divisor determination module 31 determines the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr When there is no common divisor, the original video frames in each unit time are directly used as frame extraction intervals. Each frame extraction interval includes F in Frame video frame, using incremental periodic extraction method, extract F from each frame extraction intervalextr Frames too many frames.

[0108] Through the frame synchronization system, the original video frames in the video stream are reduced to a frame rate, the frame rate is compressed, and the frame rates of each video stream can be unified after frame reduction. There is no need to use FPGA to compress frames and superimpose videos, which reduces the cost of hardware research and development. At the same time, the video frames processed by the frame synchronization system only need to synchronize the time of each encoder through the time synchronization tool to synchronize the encoding output, reducing unnecessary interactions and making the implementation simple. In addition, the redundant frame extraction module 3 can be set, that is, the bit rate to be extracted can be set. For the same video source, bit streams with different frame rates can be encoded, which can adapt to various types of decoders.

[0109] Example 3

[0110] like Figure 7 As shown, this embodiment provides a video splicing wall output method, including:

[0111] B1: Obtain several video streams with different frame rates from several output heads of the computer graphics card;

[0112] B2: using the frame synchronization method of Example 1 to unify the frame rates of the plurality of video streams with different frame rates, thereby obtaining a plurality of video streams with unified frame rates;

[0113] B3: Input the plurality of video streams with unified frame rate into corresponding encoders for encoding to obtain a plurality of encoded video streams; specifically, using NTP to perform time synchronization calibration on the plurality of video streams with unified frame rate to obtain a plurality of synchronized and calibrated video streams, and then input the plurality of synchronized and calibrated video streams into corresponding encoders for encoding to obtain a plurality of encoded video streams.

[0114] B4: inputting the plurality of encoded video streams into corresponding decoders for decoding to obtain a plurality of decoded video streams;

[0115] B5: The plurality of decoded video streams are spliced ​​together through a splicing processor to obtain a video splicing wall for output display.

[0116] The frame synchronization method of Example 1 is used to reduce the frame rate of the code stream in each video stream and unify the frame rate of each code stream, thereby ensuring the synchronization of the video frame rate and avoiding tearing of the picture displayed on the splicing wall. At the same time, there is no need to additionally set up FPGA to compress the frame rate and program the FPGA. After removing the FPGA, only ntp needs to be used to synchronize the time of the video stream, thereby reducing the hardware and development costs.

[0117] Example 4

[0118] like Figure 8 As shown, this embodiment provides a video splicing wall output system, including:

[0119] The video stream acquisition module 4 is used to acquire a plurality of video streams with different frame rates from a plurality of output heads of a computer graphics card;

[0120] A frame rate unification module 5 is used to unify the frame rates of the plurality of video streams with different frame rates by using the frame synchronization method in Embodiment 1, so as to obtain the plurality of video streams with unified frame rates;

[0121] The encoding module 6 is used to input the multiple video streams with unified frame rate into the corresponding encoder for encoding, so as to obtain multiple encoded video streams; specifically, the encoding module 6 includes a synchronization calibration module and an encoding submodule, wherein the synchronization calibration module is used to use NTP to perform time synchronization calibration on the multiple video streams with unified frame rate, so as to obtain multiple synchronized and calibrated video streams, and the encoding submodule is used to input the multiple synchronized and calibrated video streams into the corresponding encoder for encoding, so as to obtain multiple encoded video streams.

[0122] A decoding module 7, used for inputting the plurality of encoded video streams into corresponding decoders for decoding to obtain a plurality of decoded video streams;

[0123] The splicing display module 8 is used to splice the plurality of decoded video streams through a splicing processor to obtain a video splicing wall for output display.

[0124] After the frame unification module performs frame reduction processing on each video stream and unifies them to the same frame rate, a video stream with a lower frame rate is obtained, which can adapt to various types of encoders and decoders, making it easier to set up the encoding module 6 and decoding module 7 set up later.

[0125] Example 5

[0126] This embodiment provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the frame synchronization method of embodiment 1 or the video splicing wall output method of embodiment 3 is implemented.

[0127] Example 6

[0128] This embodiment provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the frame synchronization method of embodiment 1 or the video splicing wall output method of embodiment 3 is implemented.

[0129] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A frame synchronization method, characterized in that: include: S1: Get the original video frame of the input video stream and its input frame rate F in ; S2: Determine the target frame rate F of the output video stream out , and according to the input frame rate F in and target frame rate F out , determine the number of redundant frames that need to be removed per unit time F extr ; S3: according to the number of redundant frames F to be removed within the unit time extr , using a preset frame extraction rule to extract redundant frames from the original video frames to form an output video stream; According to the number of redundant frames to be removed per unit time, F extr , using a preset frame extraction rule to extract redundant frames from the original video frame, specifically including: S31: Determine the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Is there a common divisor? If yes, execute S311; if no, execute S312; S311: Input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Extract the greatest common divisor G, divide the original video frames in each unit time into G frame extraction intervals, each frame extraction interval includes F in / G video frames, using incremental periodic extraction method, extract F from each frame extraction interval extr / G frame extra frame; S312: directly use the original video frames in each unit time as frame extraction intervals, each frame extraction interval includes F in Frame video frame, using incremental periodic extraction method, extract F from each frame extraction interval extr Frames are superfluous; The incremental periodic extraction method is specifically as follows: A1: Set the number of video frames in the frame extraction interval to F1, the number of redundant frames to be extracted in the frame extraction interval to F2, and N is a natural number starting from 1; A2: Determine whether F1 / N is less than or equal to F2. If so, execute A21; otherwise, execute A22. A21: Taking N video frames as a extraction period, dividing the frame extraction interval into extraction periods, extracting one video frame from each extraction period, and recording the number of all video frames extracted in each extraction period at the current moment as C, and then executing step A3; A22: Set N=N+1, and jump back to A2 to re-execute the check whether F1 / N is less than or equal to F2; A3: Determine whether there is a remainder in F1 / N at the current moment, and whether F2-C is greater than or equal to 1. If so, execute A31; otherwise, execute A32. A31: extract one more video frame from the remaining video frames after the frame extraction interval is divided into periods, and set C=C+1 at the current moment, and then jump to step A32; A32: Determine whether F2-C at the current moment is equal to 0; Determine whether F2-C at the current moment is equal to 0. If so, execute A321; otherwise, execute A322; A321: End redundant frame extraction; A322: Set the current F2 to F2-C, and jump back to A2 to re-execute the determination of whether F1 / N is less than or equal to F2.

2. A frame synchronization method according to claim 1, characterized in that: Extracting one video frame from each extraction period, and extracting one video frame from the remaining video frames after the frame extraction interval is divided into periods, specifically including: Video frames are extracted starting from the first frame of each extraction cycle or from the first frame of the remaining video frames after the frame extraction interval is divided into periods. If the first video frame is extracted, the next video frame is extracted in frame order. If the next video frame is also extracted, the video frames are continuously extracted until a video frame can be extracted.

3. A frame synchronization system, characterized in that: include: The input video stream acquisition module is used to obtain the original video frames of the input video stream and its input frame rate F in ; The redundant frame determination module is used to determine the target frame rate F of the output video stream. out , and according to the input frame rate F in and target frame rate F out , determine the number of redundant frames that need to be removed per unit time F extr ; The redundant frame extraction module is used to extract the redundant frames according to the number of frames F to be extracted in the unit time. extr , using a preset frame extraction rule to extract redundant frames from the original video frames to form an output video stream; The redundant frame removal module specifically includes: The common divisor judgment module is used to judge the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Is there a common divisor? The first sub-module is used for determining the input frame rate F when the common divisor determination module determines the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr When there is a common divisor, the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr Extract the greatest common divisor G, divide the original video frames in each unit time into G frame extraction intervals, each frame extraction interval includes F in / G video frames, using incremental periodic extraction method, extract F from each frame extraction interval extr / G frame extra frame; The second submodule is used for determining the input frame rate F when the common divisor determination module determines the input frame rate F in And the number of redundant frames that need to be removed per unit time F extr When there is no common divisor, the original video frames in each unit time are directly used as frame extraction intervals. Each frame extraction interval includes F in Frame video frame, using incremental periodic extraction method, extract F from each frame extraction interval extr Frames are superfluous; The incremental periodic extraction method is specifically as follows: A1: Set the number of video frames in the frame extraction interval to F1, the number of redundant frames to be extracted in the frame extraction interval to F2, and N is a natural number starting from 1; A2: Determine whether F1 / N is less than or equal to F2. If so, execute A21; otherwise, execute A22. A21: Taking N video frames as a extraction period, dividing the frame extraction interval into extraction periods, extracting one video frame from each extraction period, and recording the number of all video frames extracted in each extraction period at the current moment as C, and then executing step A3; A22: Set N=N+1, and jump back to A2 to re-execute the check whether F1 / N is less than or equal to F2; A3: Determine whether there is a remainder in F1 / N at the current moment, and whether F2-C is greater than or equal to 1. If so, execute A31; otherwise, execute A32. A31: extract one more video frame from the remaining video frames after the frame extraction interval is divided into periods, and set C=C+1 at the current moment, and then jump to step A32; A32: Determine whether F2-C at the current moment is equal to 0; Determine whether F2-C at the current moment is equal to 0. If so, execute A321; otherwise, execute A322; A321: End redundant frame extraction; A322: Set the current F2 to F2-C, and jump back to A2 to re-execute the determination of whether F1 / N is less than or equal to F2.

4. A video splicing wall output method, characterized in that: include: B1: Obtain several video streams with different frame rates from several output heads; B2: using the frame synchronization method as described in any one of claims 1 to 2 to unify the frame rates of the plurality of video streams with different frame rates, thereby obtaining a plurality of video streams with unified frame rates; B3: inputting the plurality of video streams with unified frame rates into corresponding encoders for encoding to obtain a plurality of encoded video streams; B4: inputting the plurality of encoded video streams into corresponding decoders for decoding to obtain a plurality of decoded video streams; B5: The plurality of decoded video streams are spliced ​​together through a splicing processor to obtain a video splicing wall for output display.

5. A video splicing wall output method according to claim 4, characterized in that: Inputting the plurality of video streams with unified frame rates into corresponding encoders for encoding to obtain the plurality of encoded video streams specifically includes: Using a network time protocol to perform time synchronization calibration on the plurality of video streams with unified frame rates, to obtain a plurality of video streams after synchronization calibration; The plurality of synchronously calibrated video streams are input into corresponding encoders for encoding to obtain a plurality of encoded video streams.

6. A video splicing wall output system, characterized in that: include: A video stream acquisition module is used to acquire a plurality of video streams with different frame rates from a plurality of output heads of a computer graphics card; A frame rate unification module, configured to unify the frame rates of the plurality of video streams with different frame rates by using the frame synchronization method as claimed in claim 2, so as to obtain the plurality of video streams with unified frame rates; An encoding module, used for inputting the plurality of video streams with unified frame rates into a corresponding encoder for encoding to obtain a plurality of encoded video streams; A decoding module, used for inputting the plurality of encoded video streams into corresponding decoders for decoding to obtain a plurality of decoded video streams; The splicing display module is used to splice the plurality of decoded video streams through a splicing processor to obtain a video splicing wall for output display.

7. A video splicing wall output system according to claim 6, characterized in that: The encoding module specifically includes: A synchronization calibration module, used for performing time synchronization calibration on the plurality of video streams with unified frame rates by using a network time protocol, so as to obtain a plurality of video streams after synchronization calibration; The encoding submodule is used to input the plurality of synchronously calibrated video streams into corresponding encoders for encoding to obtain a plurality of encoded video streams.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the frame synchronization method according to any one of claims 1 to 2 or the video splicing wall output method according to any one of claims 4 to 5 is implemented.

9. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the frame synchronization method according to any one of claims 1 to 2 or the video splicing wall output method according to any one of claims 4 to 5 is implemented.