Video stream processing method, video stream display control method and related components

Abstract

The invention is suitable for the technical field of display screens, and provides a video stream processing method, a video stream display control method and related components. The video stream processing method comprises the following steps: acquiring N to-be-processed video streams, N being a positive integer greater than 1; a fusion video stream is obtained, so that the fusion video stream is displayed on a display screen, the fusion video stream is used for being displayed on the display screen, and continuous N frames of video pictures of the fusion video stream are from different to-be-processed video streams respectively, when the same display area in the display screen displays the continuous N frames of video pictures, N-1 frames of display pictures are masked pictures, and the positions of the masked pictures of all frames of video pictures in the same to-be-processed video stream are the same. The embodiment of the invention can alleviate the flicker problem when the display screen displays the fused video stream.

Description

Technical Field

[0001] This application belongs to the technical field of display screens, and particularly relates to a method for processing a video stream, a method for displaying and controlling a video stream, and related components. Background Art

[0002] In recent years, with the rapid development of technology, many manufacturers have combined shooting technology with video stream transmission and display technology to achieve functions such as virtual shooting and sports event live broadcast. LED (Light-Emitting Diode) display screens are often used to support the realization of functions such as virtual shooting and sports event live broadcast due to their advantages of high brightness, wide viewing angle, long lifespan, etc.

[0003] Functions such as virtual shooting and sports event live broadcast support multiple cameras to simultaneously shoot different scenes. When multiple cameras shoot different scenes, the LED display screen can play a fused video stream of multiple scenes obtained by fusion processing by a video processing device. Taking virtual shooting as an example, each frame of the fused video stream can be obtained by fusing the same foreground (such as a person) with different backgrounds. At this time, the foreground only needs to be shot once, which helps to reduce shooting costs and post-production costs. Since the fused video stream is composed of different backgrounds, obvious flicker often occurs when the display screen displays the fused video stream. Summary of the Invention

[0004] Embodiments of this application provide a method for processing a video stream, a method for displaying and controlling a video stream, and related components, which can solve the problem that obvious flicker often occurs when a display screen displays a fused video stream in related technologies.

[0005] In a first aspect of the embodiments of this application, a method for processing a video stream is provided, including: obtaining N video streams to be processed, where N is a positive integer greater than 1; obtaining a fused video stream so that the fused video stream is displayed on a display screen, and consecutive N video frames of the fused video stream respectively come from different ones of the video streams to be processed, and N - 1 of the displayed frames of the same display area on the display screen are masked frames when displaying the consecutive N video frames, and the positions of the masked frames of each frame of video in the same video stream to be processed are the same.

[0006] In some embodiments of the first aspect, the obtaining of the fused video stream includes: respectively performing masking processing on the video frames of the N video streams to be processed to obtain N masked video streams, so that there are N - 1 masked video streams with masked frames for the video frames of the same image area in the N masked video streams, where each image area corresponds to one of the display areas; performing interval frame interpolation on the N masked video streams to obtain the fused video stream, and thus the masking processing of the frames can be implemented through image processing.

[0007] In some embodiments of the first aspect, each frame of the video picture in the video stream to be processed has been scaled, and the resolution of the scaled picture is the same as the resolution of the display screen.

[0008] In some embodiments of the first aspect, each frame of the video picture in the video stream to be processed has been scaled and spliced. The resolution of the scaled picture is 1 / M of the resolution of the display screen, where M is a positive integer greater than 1. The replication process is used to replicate and splice M - 1 image regions in the neighborhood for each image region. In this way, while adapting to the resolution of the display screen, the loss of image content can be reduced and the display effect can be improved.

[0009] A method for displaying and controlling a video stream provided in the second aspect of the embodiments of the present application includes: receiving a fused video stream, where consecutive N frames of video pictures of the fused video stream are respectively from different video streams to be processed; controlling a display screen to display the fused video stream, where N - 1 frames of the display pictures are masked pictures when the same display area of the display screen displays the consecutive N frames of video pictures, and the positions of the masked pictures of each frame of the video picture in the same video stream to be processed are the same.

[0010] In some embodiments of the second aspect, controlling the display screen to display the fused video stream includes: generating control electrical parameters for each display area of the display screen, and controlling each of the display areas to display the fused video stream according to the control electrical parameters. The control electrical parameters are used to adjust the brightness of the light points in the display area when displaying the corresponding display picture, so that picture masking can be achieved during the display control process.

[0011] A video stream processing device provided in the third aspect of the embodiments of the present application includes: a video stream acquisition unit for acquiring N video streams to be processed, where N is a positive integer greater than 1; a video stream processing unit for acquiring a fused video stream so that the fused video stream is displayed on a display screen. Consecutive N frames of video pictures of the fused video stream are respectively from different video streams to be processed, and N - 1 frames of the display pictures are masked pictures when the same display area of the display screen displays the consecutive N frames of video pictures, and the positions of the masked pictures of each frame of the video picture in the same video stream to be processed are the same.

[0012] A display control device for a video stream provided in the fourth aspect of the embodiments of the present application includes: a video stream receiving unit configured to receive a fused video stream, where consecutive N video frames of the fused video stream respectively come from different video streams to be processed; a display control unit configured to control a display screen to display the fused video stream, where N - 1 display frames of the consecutive N video frames displayed in the same display area of the display screen are masked frames, and the positions of the masked frames of each video frame in the same video stream to be processed are the same.

[0013] A video processing device provided in the fifth aspect of the embodiments of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The video processing device is connected to a display screen, or the video processing device is connected to the display screen through a video display control device. When the processor executes the computer program, the steps of the video stream processing method described in the first aspect are implemented.

[0014] A video display control device provided in the sixth aspect of the embodiments of the present application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The video display control device is connected to the display screen. When the processor executes the computer program, the steps of the video stream display control method described in the second aspect are implemented.

[0015] A display system provided in the seventh aspect of the embodiments of the present application includes the video processing device, and the video processing device is connected to a display screen through a video display control device; where the video processing device is configured to execute the steps of the video stream processing method described in the first aspect, and / or, the video display control device is configured to execute the steps of the video stream display control method described in the second aspect; the display screen is configured to display a fused video stream. Or, the display system includes the video processing device, and the video processing device is connected to the display screen, where the video processing device is configured to execute the steps of the video stream processing method described in the first aspect, and / or, the video processing device is further configured to execute the steps of the video stream display control method described in the second aspect; the display screen is configured to display a fused video stream.

[0016] In the eighth aspect of the embodiments of the present application, a virtual shooting system is provided, which includes a collection device, a video processing device, and a display screen. The video processing device is connected to the display screen. The video processing device processes the to-be-processed video according to the video stream processing method described in the first aspect to obtain a fused video stream, and controls the display screen to display according to the video stream display control method described in the second aspect. Alternatively, the video processing device receives the to-be-processed video stream, processes the to-be-processed video according to the video stream processing method described in the first aspect to obtain a fused video stream, and the fused video stream is used for display on the display screen. The collection device is used to shoot the foreground and the fused video stream displayed on the display screen to obtain a shot video stream.

[0017] In the ninth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps of the above video stream processing method, or when the computer program is executed by a processor, it implements the steps of the above video stream display control method.

[0018] In the tenth aspect of the embodiments of the present application, a computer program product is provided. When the computer program product runs on a video processing device / video display control device, it causes the video processing device to execute the steps of the above video stream processing method, or causes the video processing device / video display control device to execute the steps of the above video stream display control method.

[0019] In the embodiments of the present application, by obtaining N to-be-processed video streams and obtaining a fused video stream for display on the display screen, consecutive N video frames of the fused video stream are respectively from different to-be-processed video streams. Among the consecutive N video frames displayed in the same display area of the display screen, N - 1 displayed frames are masked frames, and the positions of the masked frames of each video frame in the same to-be-processed video stream are the same. Compared with continuously displaying N different displayed frames, since N - 1 displayed frames are masked frames, when the display area displays consecutive N video frames, it can reduce the flicker when the display screen displays the fused video stream. Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 is a schematic diagram of a fused video stream in the related art;

[0022] Figure 2 It is a schematic flowchart of the implementation process of a video stream processing method provided by an embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of interval frame interpolation provided by an embodiment of the present application;

[0024] Figure 4 It is a schematic flowchart of the specific implementation process of step S202 provided by an embodiment of the present application;

[0025] Figure 5 It is provided by an embodiment of the present application for processing two video streams to be processed according to Figure 4 The corresponding effect schematic diagram of the processing in the shown manner;

[0026] Figure 6 It is provided by an embodiment of the present application for processing three video streams to be processed according to Figure 4 The corresponding effect schematic diagram of the processing in the shown manner;

[0027] Figure 7 It is provided by an embodiment of the present application for processing two video streams to be processed by means of scaling and copying and Figure 4 The corresponding first effect schematic diagram of the processing in the shown manner;

[0028] Figure 8 It is provided by an embodiment of the present application for processing two video streams to be processed by means of scaling and copying and Figure 4 The corresponding second effect schematic diagram of the processing in the shown manner;

[0029] Figure 9 It is a schematic flowchart of the implementation process of the video stream display control method provided by an embodiment of the present application;

[0030] Figure 10 It is a schematic structural diagram of a video stream processing device provided by an embodiment of the present application;

[0031] Figure 11 It is a schematic structural diagram of a video stream display control device provided by an embodiment of the present application;

[0032] Figure 12 It is a schematic structural diagram of a video processing device provided by an embodiment of the present application;

[0033] Figure 13 It is a schematic structural diagram of a video display control device provided by an embodiment of the present application;

[0034] Figure 14 And Figure 15 It is a schematic structural diagram of a display system provided by an embodiment of the present application;

[0035] Figure 16It is a schematic structural diagram of a virtual shooting system provided by an embodiment of the present application. Detailed implementation manners

[0036] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts belong to the protection scope of the present application.

[0037] Functions such as virtual shooting and event live broadcast support multiple cameras to simultaneously shoot different scenes. When multiple cameras shoot different scenes, the LED display screen can play a fused video stream of multiple scenes obtained by fusion processing by a video processing device. Taking virtual shooting as an example, when multiple cameras shoot different scenes, the LED display screen can play a fused video stream of multiple scenes. Please refer to Figure 1 , each scene of the fused video stream can be obtained by fusing the same foreground (such as a person, an animal, etc.) with different backgrounds ( Figure 1 where A, B, and C in the figure respectively represent a background). At this time, the foreground only needs to be shot once, which helps to reduce the shooting cost and post-production cost. Since the fused video stream is formed by fusing different backgrounds, when the display screen plays the fused video stream, it is necessary to sequentially play the scene A' corresponding to the background A, the scene B' corresponding to the background B, and the scene C' corresponding to the background C. By jumping between the three scenes of scene A', scene B', and scene C', the user can view the scene X formed by fusing the three scenes of scene A', scene B', and scene C'. Since the display screen needs to quickly jump between the backgrounds of different scenes, obvious flicker is likely to occur.

[0038] In view of this, the present application proposes a method for processing a video stream and a method for controlling the display of a video stream, which can control the display screen to process the scenes in some display areas as masked scenes when playing the fused video stream, so as to reduce the situation of jumping between the backgrounds of different scenes and improve the flicker problem of the display screen.

[0039] In order to illustrate the technical solution of the present application, the following will be described through specific embodiments.

[0040] Figure 2 shows a schematic implementation flowchart of a method for processing a video stream provided by an embodiment of the present application. This method can be applied to a video processing device and is applicable to the situation where the flicker problem of the display screen needs to be improved.

[0041] A video processing device can be specifically used for video processing and send the processed video stream to a display screen for video display. For example, it can be a video processor, a media server, or a computer. In an embodiment of the present application, the video processing device can be connected to the display screen through a video display control device or directly connected to the display screen, and the present application does not limit this. Among them, the video display control device is a device for controlling the display screen, which can refer to a sending card, a scanning card, a receiving card, or other devices. In some embodiments, the video display control device can specifically refer to a chip, such as a T-con chip, or other chips capable of transmitting images to the display screen, etc.

[0042] Specifically, the above method for processing the video stream can include the following steps S201 to S202.

[0043] Step S201: Obtain N video streams to be processed.

[0044] Among them, the video stream to be processed refers to the video stream that needs to be fused into the same video stream. In an embodiment of the present application, N is a positive integer greater than 1, that is, multiple video streams to be processed can be obtained for processing.

[0045] In some embodiments of the present application, the N video streams to be processed can be N video streams obtained by respectively fusing the same foreground with N backgrounds. Specifically, the video processing device can control the camera to perform video shooting on N backgrounds and one foreground, and respectively fuse the video images of the foreground with the video images of the N backgrounds to obtain N video streams to be processed with the same foreground and different backgrounds. In other embodiments, the N video streams to be processed can also be arbitrary video streams. For example, N authorized video streams can be downloaded as the N video streams to be processed, or the N video streams respectively shot by N cameras can be used as the N video streams to be processed, etc. The present application does not limit the acquisition method of the N video streams to be processed.

[0046] Step S202: Obtain a fused video stream so that the fused video stream is displayed on the display screen.

[0047] In an embodiment of the present application, the consecutive N video frames of the fused video stream respectively come from different video streams to be processed. Specifically, the fused video stream can be obtained by means of interval frame interpolation. Please refer to Figure 3 , Figure 3The case of N = 3 is shown. The 1st, 4th, 7th, ……, N-2th frames of the fused video stream all come from the video stream A to be processed, the 2nd, 5th, 8th, ……, N-1th frames all come from the video stream B to be processed, and the 3rd, 6th, 9th, ……, Nth frames all come from the video stream C to be processed. At this time, the display screen can display the fused video stream according to the frame sequence. When the display screen displays continuous N frames of video images in the same display area, N-1 frames of the display images are masked images.

[0048] Specifically, the display screen can be divided into several display areas, each display area is respectively used to display a part of a single-frame video image, and all the display areas included in the display screen can be spliced to display a complete single-frame video image. Among them, the display image refers to the image displayed on the display screen. For a single display area, when it displays continuous N frames of video images of the fused video stream (the continuous N frames of video images come from N video streams to be processed respectively), the display image includes N-1 masked images and one image that is the same as the original video image (which can be called an unmasked image).

[0049] Among them, the masked image refers to the image after masking processing, which can be a solid-color image or can be the image at the corresponding position in the video stream to be processed after brightness reduction processing. The solid-color image can include but is not limited to black images, gray images, etc.

[0050] Moreover, in the embodiments of the present application, the positions of the masked images of each frame of video image in the same video stream to be processed are the same. That is to say, when the display screen displays each frame of video image from the same video stream to be processed (for example Figure 3 the 1st, 4th, 7th, ……, N-2th frames in the fused video stream), the positions of the masked images are the same.

[0051] It can be understood that if the continuous N frames of display images are all the same as the original video image, among the continuous N frames of display images continuously displayed by the display screen, each light point usually needs to jump between the N frames of video images. Since the N frames of video images come from different video streams to be processed, their colors usually have relatively large changes. At this time, the display screen will have relatively obvious flickering. However, in the embodiments of the present application, N-1 frames of the display images are processed as masked images. When the display screen displays continuous N frames of display images, the light points only need to jump between the original video image and the masked image, and the degree of color change is relatively small. When N is greater than 2, there may be some display areas where the continuous two frames of display images are both masked images. Then, when these display areas display these two frames of display images, the colors of the light points hardly need to change. Therefore, the flickering problem of the display screen can be alleviated.

[0052] In an embodiment of the present application, by obtaining N video streams to be processed and obtaining a fused video stream for display on a display screen, consecutive N video frames of the fused video stream are respectively from different video streams to be processed. For the same display area on the display screen, N - 1 of the consecutive N video frames displayed are masked frames, and the positions of the masked frames of each video frame in the same video stream to be processed are the same. Compared with continuously displaying N different display frames, since N - 1 of the display frames are masked frames, when the display area displays consecutive N video frames, it can reduce the flicker when the display screen displays the fused video stream.

[0053] Specifically, please refer to Figure 4 , in some embodiments of the present application, controlling the display screen to display the fused video stream may include the following steps S401 to step S402.

[0054] Step S401: Mask the video frames of the N video streams to be processed respectively to obtain N masked video streams, so that there are N - 1 masked video streams with masked video frames for the video frames in the same image area among the N masked video streams.

[0055] For the convenience of processing, in some embodiments of the present application, for each video frame of the same video stream to be processed, masked frames may be generated in the same image area. Wherein, each image area may include at least one pixel, and each image area may respectively correspond to a display area.

[0056] Step S402: Perform interval frame interpolation on the N masked video streams to obtain a fused video stream.

[0057] Specifically, for a single video stream to be processed, processing may be performed on the original video frame so that at least part of the image area within each video frame is processed as a masked frame. The N video streams to be processed may divide the image area in the same way. Then, for the same image area, among the N video streams to be processed, N - 1 video streams to be processed are processed as masked frames. Performing interval frame interpolation on the N masked video streams can obtain a fused video stream, so that consecutive N video frames in the fused video stream are respectively from different video streams to be processed, and for a single image area within the consecutive N video frames, there are N - 1 video frames that are masked frames and one original video frame.

[0058] When the display screen displays the fused video stream, each display area of the display screen can display the corresponding image area. Then, when a single display area displays consecutive N video frames, N - 1 of the display frames will be masked frames.

[0059] It should be noted that the masking process described in this application can be to overlay a solid color block on the screen of a partial image area, or to perform a brightness reduction process on the screen, or other processing methods. And it can be understood that the fewer the number of pixels in a single image area, the better the display effect.

[0060] For a single masked video stream, the image area for the masking process can be adjusted according to actual needs. In a single masked video stream, the total area size of the masked video frames and the total area size of the unmasked video frames in each video frame can be the same or different. Among them, the total area size of the masked video frames can refer to the total area size of the image areas that are processed as masked video frames. The total area size of the unmasked video frames can refer to the total area size of the image areas that are not processed as masked video frames.

[0061] Exemplarily, in a single masked video stream, the masked video frames and the unmasked video frames in each video frame are alternately distributed. In this way, a video frame similar to a checkerboard can be formed.

[0062] Or, the image area types of adjacent columns in each video frame of a single masked video stream are different; or, the image area types of adjacent rows in each video frame of a single masked video stream are different. Among them, the image area type is a masked video frame or an unmasked video frame. In this way, a video frame similar to a barcode can be formed.

[0063] Or, the image area types of adjacent rows in each video frame of a single masked video stream are different, and the image area types of adjacent columns are different, and the image area type is the masked video frame or the unmasked video frame.

[0064] Or, in a single masked video stream, there are i (0 < i < J*J) masked video frames and J*J - i unmasked video frames in every J*J (J is a positive integer greater than 1) image areas.

[0065] For ease of explanation, please refer to Figure 5 , Figure 5 shows the two video streams to be processed according to Figure 4Schematic diagram of the corresponding effects of processing in the shown manner. The video stream A to be processed can be masked in the form of a checkerboard, that is, the masked and unmasked images are alternately distributed. A black color block is superimposed on the first image area of the first row, and a transparent color block (or no processing) is superimposed on the first image area of the second row and the second image area of the first row, and so on, to form a masked image effect in which one of the two adjacent image areas is a black color block and the other is the original video image. After processing, the masked video stream A' is obtained. The video stream B to be processed is the inverse of the video stream A to be processed. The masked and unmasked images are also alternately distributed. A transparent color block (or no processing) is superimposed on the first image area of the first row, and a black color block is superimposed on the first image area of the second row and the second image area of the first row, and so on, to obtain the masked video stream B'. The video images of the masked video stream A' and the masked video stream B' are alternately interpolated to obtain a fused video stream with the frame sequence of "A'-B'-...-A'-B'". At this time, the first display area of the first row of the display screen is a masked image when displaying the first frame of the image, and the original video image from the video stream B to be processed when displaying the second frame of the image. For any display area, there is a masked image when displaying two consecutive frames of images. Compared with jumping between two frames of original video images, jumping between one frame of original video image and one frame of masked image has a smaller degree of jump, so the flicker problem can be effectively reduced.

[0066] Correspondingly, please refer to Figure 6 , Figure 6 which shows the processing of three video streams to be processed according to Figure 4Schematic diagram of the corresponding effects processed in the shown manner. The video stream A to be processed can also be masked in the form of a checkerboard, that is, the masked and unmasked images are also alternately distributed. A black color block is superimposed on the first image area in the first row, and a transparent color block (or no processing) is superimposed on the first image area in the second row and the second image area in the first row, and so on, to form an effect where one of the adjacent two image areas is a black color block and the other is the original video image. After processing, the masked video stream A' is obtained. In the mask of the video stream B to be processed, all odd rows are superimposed with black color blocks, and the even rows are the opposite of the video stream A to be processed. After processing, the masked video stream B' is obtained, so that there are three masked images and one unmasked image in every 2*2 image areas in a single masked video stream. In the mask of the video stream C to be processed, all even rows are superimposed with black color blocks, and the odd rows are the opposite of the video stream A to be processed. After processing, the masked video stream C' is obtained, so that there are three masked images and one unmasked image in every 2*2 image areas in a single masked video stream. The video images of the masked video stream A', the masked video stream B' and the masked video stream C' are inserted with frames at intervals to obtain a fused video stream with the frame sequence of "A'-B'-C'-…-A'-B'-C'". At this time, the first display area in the first row of the display screen is a masked image when displaying the first and second video images, and is the original video image from the video stream C to be processed when displaying the third video image. For the first display area in the first row, the light points do not need to jump when displaying the first and second video images, so the flicker problem can be further reduced.

[0067] In the embodiments of the present application, in order to adapt to the display of the display screen, before the masking process, each frame of the video image in each video stream to be processed can also be preprocessed.

[0068] Specifically, in some embodiments of the present application, the preprocessing may refer to scaling processing. That is, each frame of the video image in the video stream to be processed is scaled, and the resolution of the scaled image is the same as the resolution of the display screen. In other words, the resolution of each frame of the video image in N video streams to be processed can be scaled to the resolution of the display screen.

[0069] In other embodiments of the present application, the preprocessing may include scaling processing and splicing processing. That is, the resolution of the scaled image is 1 / M of the resolution of the display screen, where M is a positive integer greater than 1, and the replication process is used to splice and replicate M-1 copies of the image area in the neighborhood for each image area. In other words, the resolution of each frame of the video image in N video streams to be processed can be scaled to 1 / M of the resolution of the display screen to obtain a scaled video frame, and then the scaled video frame is replicated to obtain a video image with the same resolution as the display screen.

[0070] Wherein, M is a positive integer greater than 1.

[0071] Specifically, for each image region of the scaled video frame, M - 1 copies of the image region obtained by copying the image region can be spliced on the neighborhood to obtain a video picture with the same resolution as the display screen. Taking M = 4 as an example, after scaling the resolution of each frame of the video picture in a single video stream to be processed to 1 / M of the resolution of the display screen, the first image region in the first row of the scaled video frame can be copied, and the first image region in the first row and the copied image region are spliced to form 2×2 image regions with the same content. After the splicing of each image region is completed, a video stream can be obtained in which the resolution of all video pictures is the same as the resolution of the display screen. It can be understood that if the picture resolution before the scaling process is less than 1 / M of the resolution of the display screen, the scaling process is an enlargement process; if the picture resolution before the scaling process is greater than 1 / M of the resolution of the display screen, the scaling process is a reduction process.

[0072] Please refer to Figure 7 , Figure 7 which shows the first schematic diagram of the effect corresponding to the processing of two video streams to be processed by means of scaling and copying and Figure 4 the method shown. Among them, the masking processing methods of the two video streams to be processed can refer to the descriptions of the video to be processed B and the video to be processed C in Figure 6 respectively. Taking the first frame of the video picture as an example, in Figure 5 , since the first image region in the first row is processed as a black color block, the original image content of this image region has been discarded during display. Compared with Figure 5 , Figure 7 is the masking process after the scaling process and the splicing process. At this time, the first image region in the first row is processed into 2×2 image regions with the same content. During the masking process, only part of the 2×2 image regions are processed as black color blocks. Therefore, this part of the image content can be retained during display, so as to achieve a better visual effect.

[0073] Please refer to Figure 8 , Figure 8 which shows the second schematic diagram of the effect corresponding to the processing of two video streams to be processed by means of scaling and copying and Figure 4 the method shown. Among them, the method of the masking process can refer to the description in Figure 5 . Taking the first frame of the video picture as an example, in Figure 5 , since the first image region in the first row is processed as a black color block, the original image content of this image region has been discarded during display. Compared with Figure 5 , Figure 8It is to perform masking processing on the scaled and spliced processing. At this time, the first image area in the first row is processed into 2×2 image areas with the same content. During the masking processing, only part of the 2×2 image areas are processed into black color blocks. Therefore, this part of the image content can be retained during display, thus achieving a better visual effect. And, compared with Figure 7 , since multiple copies of the image areas with the same image content are retained, a higher brightness of the image content will be felt when viewed with the naked eye. That is, the brightness can be increased compared to only displaying one image area.

[0074] In some other embodiments of the present application, please refer to Figure 9 , which is a schematic flowchart of the implementation of a video stream display control method provided by an embodiment of the present application. This method can be applied to a video processing device or a video display control device and is applicable to the situation where it is necessary to improve the problem of display screen flicker.

[0075] Specifically, the video stream display control method may include the following steps S901 to step S902.

[0076] Step S901, receive a fused video stream.

[0077] In the embodiments of the present application, the consecutive N video frames of the fused video stream are respectively from different video streams to be processed. It can be understood that the obtaining method of the fused video stream in step S901 can refer to the description in Figure 3 , that is, by performing interval frame interpolation processing on N video streams to be processed, a fused video stream can be obtained. At this time, the video frames in the fused video stream may not be subjected to masking processing. For a video display control device, it can obtain the fused video stream sent by the video processing device.

[0078] Step S902, control the display screen to display the fused video stream.

[0079] Among them, for the same display area on the display screen, N - 1 display frames are masked frames when displaying consecutive N video frames, and the positions of the masked frames of each video frame in the same video stream to be processed are the same. In step S902, the video display control device can control the display screen so that the display screen shows masked frames when displaying the fused video stream.

[0080] Specifically, step S902 may include: for each display area, generate control electrical parameters, and according to the control electrical parameters, control each display area to display the fused video stream.

[0081] In an embodiment of the present application, the control electrical parameter can be used to adjust the brightness of the light points in the corresponding display area when displaying N - 1 frames of display pictures, specifically, it can be a control voltage and / or a control current. For the N - 1 frames of display pictures that need to be processed as masked pictures, the brightness of the corresponding light beads can be changed by controlling the current / voltage of the corresponding light points, so that the brightness of these display pictures is lower than that of the original video pictures of the fused video stream, achieving the same effect as the masking process described above. Of course, for the 1 frame of display picture that does not need to be processed as a masked picture, the corresponding light beads can be normally lit by controlling the current / voltage of the corresponding light points.

[0082] It can be understood that before the interval frame interpolation, the video stream to be processed can also be pre - processed. And for the distribution of masked and un - masked pictures in a single frame, reference can be made to the relevant descriptions above. The present application does not make any restrictions or elaborations on this.

[0083] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, some steps can be carried out in other sequences.

[0084] As Figure 10 shown is a schematic structural diagram of a video stream processing device 1000 provided by an embodiment of the present application. The video stream processing device 1000 is configured on a video processing device.

[0085] Among them, the video stream processing device 1000 may include:

[0086] A video stream acquisition unit 1001, configured to acquire N video streams to be processed, where N is a positive integer greater than 1;

[0087] A video stream processing unit 1002, configured to acquire a fused video stream so that the fused video stream is displayed on a display screen. The consecutive N - frame video pictures of the fused video stream respectively come from different video streams to be processed. For the N - 1 frames of display pictures when the same display area on the display screen displays the consecutive N - frame video pictures, they are masked pictures, and the positions of the masked pictures in each frame of video picture in the same video stream to be processed are the same.

[0088] In some embodiments of the present application, the above-mentioned video stream processing unit 1002 may specifically be configured to: perform masking processing on the video frames of the N to-be-processed video streams respectively to obtain N masked video streams, such that for the video frames of the same image region, N - 1 of the N masked video streams have the masked frames, where each image region corresponds to one of the display regions; perform interval frame interpolation on the N masked video streams to obtain the fused video stream.

[0089] In some embodiments of the present application, in a single masked video stream, the total area size of the masked frames and the total area size of the unmasked frames of each video frame may be the same or different.

[0090] In some embodiments of the present application, in a single masked video stream, the masked frames and the unmasked frames of each video frame are alternately distributed; or, for each video frame in a single masked video stream, the picture types of the image regions in adjacent columns are different, and the picture type is either a masked frame or an unmasked frame; or, for each video frame in a single masked video stream, the picture types of the image regions in adjacent rows are different, and the picture type is either a masked frame or an unmasked frame; or, for each video frame in a single masked video stream, the picture types of the image regions in adjacent rows are different and the picture types of the image regions in adjacent columns are different, and the picture type is either a masked frame or an unmasked frame; or, in every J*J image regions in a single masked video stream, there are i masked frames and J*J - i unmasked frames, where J is a positive integer greater than 1 and 0 < i < J*J.

[0091] In some embodiments of the present application, each video frame in the to-be-processed video stream has been scaled, and the resolution of the scaled frame is the same as the resolution of the display screen.

[0092] In some embodiments of the present application, each video frame in the to-be-processed video stream has been scaled and spliced. The resolution of the scaled frame is 1 / M of the resolution of the display screen, where M is a positive integer greater than 1, and the replication process is used to replicate and splice M - 1 copies of the image region in the neighborhood for each image region.

[0093] In some embodiments of the present application, the masked frame is a solid-color frame or a frame at the corresponding position in the to-be-processed video stream with reduced brightness.

[0094] In some embodiments of the present application, the N to-be-processed video streams are N video streams obtained by respectively fusing the same foreground with N backgrounds.

[0095] It should be noted that, for the convenience and conciseness of description, the specific working process of the above video stream processing device 1000 can be referred to Figures 2 to 8 the corresponding process of the method described above, which will not be elaborated here.

[0096] As Figure 11 shown, it is a schematic structural diagram of a video stream display control device 1100 provided by an embodiment of the present application. The video stream display control device 1100 is configured on a video processing device or a video display control device.

[0097] Among them, the video stream display control device 1100 may include:

[0098] A video stream receiving unit 1101, configured to receive a fused video stream, where consecutive N video frames of the fused video stream respectively come from different video streams to be processed;

[0099] A display control unit 1102, configured to control a display screen to display the fused video stream. Among them, in the same display area of the display screen, N - 1 display frames are masked frames when displaying the consecutive N video frames, and the positions of the masked frames of each video frame in the same video stream to be processed are the same.

[0100] In some embodiments of the present application, the display control unit 1102 may specifically be configured to: generate control electrical parameters for each display area of the display screen, and control each display area to display the fused video stream according to the control electrical parameters, where the control electrical parameters are used to adjust the brightness of the light points in the display area when displaying the corresponding display frame.

[0101] It should be noted that, for the convenience and conciseness of description, the specific working process of the above video stream display control device 1100 can be referred to Figure 9 the corresponding process of the method described above, which will not be elaborated here.

[0102] As Figure 12 shown, it is a schematic diagram of a video processing device provided by an embodiment of the present application. Specifically, the video processing device 12 may include: a processor 120, a memory 121, and a computer program 122 stored in the memory 121 and executable on the processor 120, such as a video stream processing program. When the processor 120 executes the computer program 122, it implements the steps in the above-mentioned embodiments of the video stream processing method, such as Figure 2 the steps S201 to S202 shown. Or, when the processor 120 executes the computer program 122, it implements the functions of each module / unit in the above-mentioned device embodiments, such as Figure 10 the functions of the video stream acquisition unit 1001 and the video stream processing unit 1002 shown.

[0103] The computer program can be divided into one or more modules / units, which are stored in the memory 121 and executed by the processor 120 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the video processing device.

[0104] For example, the computer program can be divided into: a video stream acquisition unit and a video stream processing unit. The specific functions of each unit are as follows: The video stream acquisition unit is used to acquire N video streams to be processed, where N is a positive integer greater than 1; the video stream processing unit is used to acquire a fused video stream so that the fused video stream is displayed on the display screen. The consecutive N video frames of the fused video stream are respectively from different video streams to be processed. In the same display area of the display screen, N - 1 display frames are masked frames when displaying the consecutive N video frames, and the positions of the masked frames of each video frame in the same video stream to be processed are the same.

[0105] The video processing device may include, but is not limited to, a processor 120 and a memory 121. Those skilled in the art can understand that Figure 12 merely examples of the video processing device, which do not constitute a limitation on the video processing device. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, the video processing device may further include input / output devices, network access devices, buses, etc.

[0106] The so-called processor 120 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), off-the-shelf programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0107] The memory 121 may be an internal storage unit of the video processing device, such as a hard disk or memory of the video processing device. The memory 121 may also be an external storage device of the video processing device, such as a plug-in hard disk equipped on the video processing device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 121 may also include both an internal storage unit and an external storage device of the video processing device. The memory 121 is used to store the computer program and other programs and data required by the video processing device. The memory 121 may also be used to temporarily store data that has been output or will be output.

[0108] It should be noted that for the convenience and simplicity of description, the structure of the above video processing device may also refer to the specific description of the structure in the method embodiments, which will not be elaborated here.

[0109] As Figure 13 shown, it is a schematic diagram of a video display control device provided by an embodiment of the present application. Specifically, the video display control device 13 may include: a processor 130, a memory 131, and a computer program 132 stored in the memory 131 and executable on the processor 130, such as a display control program for video streams. When the processor 130 executes the computer program 132, it implements the steps in the above-mentioned method embodiments for displaying and controlling each video stream, such as Figure 9 the steps S901 to S902 shown. Alternatively, when the processor 130 executes the computer program 132, it implements the functions of each module / unit in the above-mentioned device embodiments, such as Figure 11 the functions of the video stream receiving unit 1101 and the display control unit 1102 shown.

[0110] It should be noted that for the convenience and simplicity of description, the structure of the above video display control device may also refer to the specific description of the structure in the method embodiments. The processor 130, the memory 131, and the computer program 132 may also refer to Figure 12 the relevant descriptions, which will not be elaborated here.

[0111] Please refer to Figures 14 to 15 which respectively show two situations of the display system provided by the present application.

[0112] Refer to Figure 14 , the display system may include a video processing device, and the video processing device is connected to a display screen through a video display control device. Among them, the video processing device can be used to execute Figures 2 to 8Steps of the method for processing the video stream; the display screen can be used to display the fused video stream. And / or, the video display control device is used to execute as Figure 9 the steps of the method for controlling the display of the video stream.

[0113] At this time, the display screen can display the fused video stream under the control of the video display control device.

[0114] Reference Figure 15 , the display system may include a video processing device, the video processing device is connected to the display screen, and at this time, the video processing device has the functions of the video display control device. Among them, the video processing device can be used to execute as Figures 2 to 8 the steps of the method for processing the video stream, and / or, the video processing device can be used to execute as Figure 9 the steps of the method for controlling the display of the video stream.

[0115] At this time, the display screen can be used to display the fused video stream under the control of the video processing device.

[0116] It should be understood that Figure 14 and Figure 15 are only examples of the display system, and do not constitute a limitation on the display system. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the display system may also include a camera, a host computer, etc.

[0117] Please refer to Figure 16 , Figure 16 shows a virtual shooting system provided by the present application. The virtual shooting system may include an acquisition device, a video processing device, and a display screen. Among them, the video processing device is connected to the display screen.

[0118] Among them, the acquisition device can be used to shoot the background to obtain a background video stream.

[0119] The video processing device can be used to process the to-be-processed video according to Figures 2 to 8 the method for processing the video stream shown, to obtain a fused video stream, and control the display screen to display according to Figure 9 the method for controlling the display of the video stream shown.

[0120] Alternatively, the video processing device can be used to receive the to-be-processed video stream, and process the to-be-processed video according to Figures 2 to 8 the method for processing the video stream shown, to obtain a fused video stream, and the fused video stream is used to be displayed on the display screen.

[0121] Among them, the acquisition device can be one or more, which is used to capture the foreground and the fused video stream displayed on the display screen to obtain a captured video stream. In this way, the fused video stream displayed on the LED display screen can be used as the background and fused with the foreground to achieve virtual shooting.

[0122] It should be noted that for the convenience and simplicity of description, the structures of the above display system and virtual shooting system can also refer to the specific descriptions of the structures in the method embodiments, which will not be elaborated here.

[0123] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working processes of the units and modules in the above system can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated here.

[0124] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0125] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0126] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the apparatus or unit can be in electrical, mechanical or other forms.

[0127] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0128] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0129] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above method embodiments of the present application, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0130] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included within the protection scope of the present application.

Claims

1. A method for processing video streams, characterized in that, it includes: obtaining N video streams to be processed, where N is a positive integer greater than 1; obtaining a fused video stream so that the fused video stream is displayed on a display screen, and the consecutive N video frames of the fused video stream are respectively from different ones of the video streams to be processed. Among the consecutive N video frames displayed in the same display area on the display screen, N - 1 frames of the display pictures are masked pictures, and the positions of the masked pictures of each frame of video picture in the same video stream to be processed are the same.

2. The method for processing video streams according to claim 1, characterized in that, the obtaining of the fused video stream includes: respectively performing masking processing on the video pictures of the N video streams to be processed to obtain N masked video streams, so that there are N - 1 masked video streams among the N masked video streams for the video pictures in the same image area being the masked pictures, where each image area respectively corresponds to one of the display areas; performing interval frame interpolation on the N masked video streams to obtain the fused video stream.

3. The method for processing video streams according to claim 2, characterized in that, in a single masked video stream, the total area size of the masked pictures and the total area size of the unmasked pictures of each frame of video picture are the same or different.

4. The method for processing video streams according to claim 2, characterized in that, in a single masked video stream, the masked pictures and the unmasked pictures of each frame of video picture are alternately distributed; alternatively, in a single masked video stream, the picture types of the image area pictures in adjacent columns of each frame of video picture are different, and the picture type is the masked picture or the unmasked picture; alternatively, in a single masked video stream, the picture types of the image area pictures in adjacent rows of each frame of video picture are different, and the picture type is the masked picture or the unmasked picture; alternatively, in a single masked video stream, the picture types of the image area pictures in adjacent rows are different, and the picture types of the image area pictures in adjacent columns are also different, and the picture type is the masked picture or the unmasked picture; alternatively, in a single masked video stream, there are i masked pictures and J * J - i unmasked pictures in every J * J image areas, where J is a positive integer greater than 1, and 0 < i < J * J.

5. The method for processing video streams according to claim 1, characterized in that, each frame of video picture in the video stream to be processed has been subjected to scaling processing, and the resolution of the picture after the scaling processing is the same as the resolution of the display screen.

6. The method for processing video streams according to claim 1, characterized in that, each frame of video picture in the video stream to be processed has been subjected to scaling processing and splicing processing. The resolution of the picture after the scaling processing is 1 / M of the resolution of the display screen, where M is a positive integer greater than 1, and the replication processing is used to splice and replicate M - 1 copies of the image area for each image area in the neighborhood.

7. The method for processing video streams according to any one of claims 1 to 6, characterized in that, The masked screen is a solid-color screen or a screen with reduced brightness at the corresponding position in the video stream to be processed.

8. The method for processing a video stream according to any one of claims 1 to 6, wherein, The N video streams to be processed are N video streams obtained by respectively fusing the same foreground with N backgrounds.

9. A method for controlling the display of a video stream, wherein, It includes: Receiving a fused video stream, where consecutive N video frames of the fused video stream are respectively from different video streams to be processed; Controlling a display screen to display the fused video stream, wherein, when the consecutive N video frames are displayed in the same display area of the display screen, N - 1 display frames are masked screens, and the positions of the masked screens of each video frame in the same video stream to be processed are the same.

10. The method for controlling the display of a video stream according to claim 9, wherein, The controlling the display screen to display the fused video stream includes: Generating control electrical parameters for each display area of the display screen, and controlling each display area to display the fused video stream according to the control electrical parameters, where the control electrical parameters are used to adjust the brightness of the light points when the display area displays the corresponding display screen.

11. A video stream processing device, wherein, It includes: A video stream acquisition unit, configured to acquire N video streams to be processed, where N is a positive integer greater than 1; A video stream processing unit, configured to acquire a fused video stream so that the fused video stream is displayed on a display screen, where consecutive N video frames of the fused video stream are respectively from different video streams to be processed, and when the consecutive N video frames are displayed in the same display area of the display screen, N - 1 display frames are masked screens, and the positions of the masked screens of each video frame in the same video stream to be processed are the same.

12. A video stream display control device, wherein, It includes: A video stream receiving unit, configured to receive a fused video stream, where consecutive N video frames of the fused video stream are respectively from different video streams to be processed; A display control unit, configured to control a display screen to display the fused video stream, wherein, when the consecutive N video frames are displayed in the same display area of the display screen, N - 1 display frames are masked screens, and the positions of the masked screens of each video frame in the same video stream to be processed are the same.

13. A video processing device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, The video processing device is connected to a display screen, or the video processing device is connected to the display screen through a video display control device, and when the processor executes the computer program, the steps of the method for processing a video stream according to any one of claims 1 to 8 are implemented.

14. A video display control device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, The video display control device is connected to the display screen, and when the processor executes the computer program, it implements the steps of the video stream display control method according to any one of claims 9 to 10.

15. A display system, characterized in that the display system includes a video processing device, a video display control device and a display screen, and the video processing device is connected to the display screen through the video display control device; wherein, the video processing device is used to execute the steps of the video stream processing method according to any one of claims 1 to 8, and / or, the video display control device is used to execute the steps of the video stream display control method according to any one of claims 9 to 10; the display screen is used to display the fused video stream; alternatively, the display system includes the video processing device, and the video processing device is connected to the display screen, wherein, the video processing device is used to execute the steps of the video stream processing method according to any one of claims 1 to 8, and / or, the video processing device is used to execute the steps of the video stream display control method according to any one of claims 9 to 10; the display screen is used to display the fused video stream.

16. A virtual shooting system, characterized in that it includes a collection device, a video processing device, and a display screen, and the video processing device is connected to the display screen; the video processing device processes the video to be processed according to the video stream processing method according to any one of claims 1 to 8 to obtain a fused video stream, and controls the display screen to display according to the video stream display control method according to any one of claims 9 to 10; alternatively, the video processing device receives the video stream to be processed, and processes the video to be processed according to the video stream processing method according to any one of claims 1 to 8 to obtain a fused video stream, and the fused video stream is used to be displayed on the display screen; the collection device is used to shoot the foreground and the fused video stream displayed on the display screen to obtain a shooting video stream.

17. A computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, it implements the steps of the video stream processing method according to any one of claims 1 to 8, or when the computer program is executed by a processor, it implements the steps of the video stream display control method according to any one of claims 9 to 10.

Images