Dynamic mask blending method, device and equipment for ultra-high definition video and medium

Abstract

The application discloses a dynamic mask mixing method, device and equipment for ultra-high-definition video and a medium, and relates to the technical field of video processing. The method comprises the following steps: acquiring at least two video materials, and filling the at least two video materials in respective corresponding display windows on an operation interface, wherein there is an overlapping area between the display windows; in response to a selection result of a user on a layer mask mode of an upper display window in the overlapping area, mixing a layer of the upper display window with a pure white background to generate a mask layer, processing video materials of a lower display window in the overlapping area through the mask layer to obtain a rendering result, and the layer mask mode is used to determine the number of masks of the lower display window in the overlapping area; and converting the rendering result from a linear color space to a standard color space, and generating a target video according to the resolution of an output device. The method can set a dynamic mask for the ultra-high-definition video, and the use range is expanded.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to a dynamic mask mixing method, device and equipment for ultra-high-definition video and a medium. BACKGROUND

[0002] Mask is a kind of mask technology in image processing, which realizes non-destructive editing content by controlling the visibility of different areas, that is, the original data is lossless. However, the mask in the related art is a static mask, and the range of use is limited. SUMMARY

[0003] In view of the above defects or deficiencies in the related art, it is desirable to provide a dynamic mask mixing method, device and equipment for ultra-high-definition video and a medium, which can set a dynamic mask for ultra-high-definition video and expand the range of use.

[0004] In a first aspect, the present application provides a dynamic mask mixing method for ultra-high-definition video, the dynamic mask mixing method comprising:

[0005] obtaining at least two video materials, and filling the at least two video materials in respective corresponding display windows on an operation interface, wherein there is an overlapping area between the display windows;

[0006] in response to a user's selection result of a layer mask mode of a display window above the overlapping area, mixing a layer of the display window above the overlapping area with a pure white background to generate a mask layer, and processing a video material of a display window below the overlapping area through the mask layer to obtain a rendering result, wherein the layer mask mode is used to determine the number of masks of the display window below the overlapping area;

[0007] converting the rendering result from a linear color space to a standard color space, and generating a target video according to the resolution of an output device.

[0008] Optionally, in some embodiments of the present application, the processing of the video material of the display window below the overlapping area through the mask layer to obtain the rendering result comprises:

[0009] calculating a first mask value using a mask channel value and a mask color value of the mask layer;

[0010] calculating a second mask value based on the transparency corresponding to the first mask value and the mask color value, and calculating a mask result based on the second mask value and a video material color value of the display window below the overlapping area;

[0011] mixing and rendering the mask result according to an image mixing mode of the display window below the overlapping area to obtain the rendering result.

[0012] Optionally, the mixing and rendering of the mask result to obtain the rendering result in some embodiments of the present application comprises:

[0013] Optionally, the mixing and rendering of the mask result to obtain the rendering result in some embodiments of the present application comprises:

[0014] Optionally, the image mixing mode in some embodiments of the present application comprises an overlay mode and a superimposition mode.

[0015] Optionally, the filling of the at least two video materials in respective display windows on the operation interface in some embodiments of the present application further comprises:

[0016] Optionally, the filling of the at least two video materials in respective display windows on the operation interface in some embodiments of the present application further comprises:

[0017] Optionally, the method in some embodiments of the present application further comprises:

[0018] Optionally, the method in some embodiments of the present application further comprises:

[0019] Optionally, the method in some embodiments of the present application further comprises:

[0020] In a second aspect, the present application provides a dynamic mask mixing device for ultra-high-definition video, the dynamic mask mixing device comprising:

[0021] a filling module configured to obtain at least two video materials and fill the at least two video materials in respective display windows on an operation interface, wherein the display windows have an overlapping area;

[0022] a mask processing module configured to, in response to a user's selection result of a layer mask mode of a display window above the overlapping area, mix a layer of the display window above the overlapping area with a pure white background to generate a mask layer, and process a video material of a display window below the overlapping area through the mask layer to obtain a rendering result, wherein the layer mask mode is used to determine a number of masks of the display window below the overlapping area;

[0023] a generating module configured to convert the rendering result from a linear color space to a standard color space, and generate a target video according to a resolution of an output device.

[0024] Optionally, the mask processing module in some embodiments of the present application is specifically configured to calculate a first mask value by using a mask channel value and a mask color value of the mask layer.

[0025] a second mask value is calculated based on the first mask value and the transparency corresponding to the mask color value, and a mask result is calculated based on the second mask value and a video material color value of the display window below the overlapping area;

[0026] The mask result is mixed and rendered according to the image mixing mode of the display window below the overlapping area to obtain the rendering result.

[0027] In a third aspect, the present application provides a terminal device, which comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the steps of the dynamic mask mixing method according to any one of the first aspect.

[0028] In a fourth aspect, the present application provides a computer readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the dynamic mask mixing method according to any one of the first aspect.

[0029] From the above technical solutions, it can be seen that the embodiments of the present application have the following advantages:

[0030] The embodiments of the present application provide a dynamic mask mixing method, device, equipment and medium for ultra-high definition video. First, a plurality of video materials are respectively filled in the display windows corresponding thereto on an operation interface, and there is an overlapping area between the display windows. Then, according to the selection result of the user on the display window above the overlapping area in the layer mask mode, the layer of the display window above the overlapping area is mixed with a pure white background to generate a mask layer. The layer mask mode is used to determine the number of masks of the display window below the overlapping area, that is, the video material played is used as a dynamic mask. Further, the video material of the display window below the overlapping area is processed through the mask layer to obtain a rendering result and generate a target video, thereby expanding the use range. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 A flowchart of a dynamic mask mixing method for ultra-high definition video provided by the embodiments of the present application;

[0033] Figure 2 A display window distribution schematic diagram on an operation interface provided by an embodiment of the present application;

[0034] Figure 3 Another display window distribution schematic diagram on an operation interface provided by an embodiment of the present application;

[0035] Figure 4 Still another display window distribution schematic diagram on an operation interface provided by an embodiment of the present application;

[0036] Figure 5 A structural block diagram of a dynamic mask mixing device for ultra-high-definition video provided by an embodiment of the present application;

[0037] Figure 6 A structural block diagram of a terminal device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0039] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0040] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the following embodiments will be described in detail. Figures 1 to 6 The dynamic mask mixing method, device, equipment and medium for ultra-high-definition video provided by the embodiments of the present application are described in detail.

[0041] Please refer to Figure 1 A flowchart of a dynamic mask mixing method for ultra-high-definition video provided by an embodiment of the present application, which specifically includes the following steps:

[0042] S101, at least two video materials are acquired, and the at least two video materials are respectively filled in respective corresponding display windows on an operation interface, and there is an overlapping area between the display windows.

[0043] In some embodiments of the present application, before filling the at least two video materials in the respective display windows on the operation interface, the at least two video materials can be respectively subjected to a transparency pre-multiplication operation and a position clipping operation. The position clipping operation refers to adjusting the position of the video material to the position set by the user on the operation interface, i.e., the position manually operated by the user. Of course, the system window position can also be used. The display window can be rectangular or circular, etc. The video material can be a television program or a short video shot by the user, etc. The transparency pre-multiplication operation is calculated by formula (1), i.e.:

[0044] Color = (R×α, G×α, B×α, α) (1)

[0045] In formula (1), R, G and B represent the color value components of the video material, and a represents the transparency of the material. The advantage of this setting is that it can avoid the color becoming lighter and lighter caused by subsequent multiple mixing, and ensure the accuracy of the result.

[0046] In some embodiments of the present application, the video source resolution corresponding to the at least two video materials can also be detected respectively. If the video source resolution is greater than or equal to a preset resolution threshold, the video material corresponding to the video source resolution is subjected to clustering processing. For example, the preset resolution threshold is 8K. Clustering refers to equally dividing the frame picture of the video material and performing GPU rendering and other processing procedures in batches, such as dividing the frame picture into four blocks in the shape of a cross-shaped grid with the cross center as the dividing line. In this way, the bandwidth bottleneck caused by high-resolution materials can be avoided, and the frame rate can be effectively improved.

[0047] In S102, in response to the selection result of the user on the layer mask mode of the display window above the overlapping area, the layer of the display window above the overlapping area is mixed with a pure white background to generate a mask layer, and the video material of the display window below the overlapping area is processed through the mask layer to obtain a rendering result. The layer mask mode is used to determine the number of masks of the display window below the overlapping area.

[0048] In some embodiments of the present application, the layer mask mode includes but is not limited to a disable mode, a below one mode and a below all mode, etc. The disable mode means that no mask processing is performed on the layer. The below one mode means that the video material of one display window below the overlapping area is processed. The below all mode means that the video materials of all display windows below the overlapping area are processed.

[0049] For example Figure 2As shown, window W1 corresponds to the window displayed above the overlapping area, and window W2 corresponds to the window displayed below the overlapping area. In the "below one" mode, the layer displaying the window above the overlapping area is first alpha-blended with the pure white background image (1,1,1,1) to generate a mask layer with an alpha channel. Next, the first mask value, maskValue1, is calculated using the mask channel value `channelMask` and the mask color value `maskColor`, for example, `float maskValue1=dot(maskColor, channelMask)`, where `channelMask=`...

[0050] (0.3, 0.59, 0.11, 0.0). Then, based on the first mask value maskValue1 and the transparency maskColor.a corresponding to the mask color value, the second mask value maskValue2 is calculated, for example, maskValue2=mix(1.0,maskValue1, maskColor.a), the specific calculation process is 1.0*(1.0-maskColor.a)+maskValue1*maskColor.a, and based on the second mask value maskValue2 and the video material color value inputColor of the display window below the overlapping area, the mask result fragColor is calculated, for example, fragColor=inputColor*maskValue2. Then, according to the image blending mode of the display window below the overlapping area, the masking result is blended and rendered to obtain the rendering result. For example, the image blending mode includes, but is not limited to, overlay mode and blending mode. Overlay mode means that the color of the upper layer replaces the color of the lower layer, while blending mode means that the color of the upper layer is mixed with the color of the lower layer to produce an intermediate color. Each display window corresponds to one layer. For example, when blending and rendering, a pure black background image (0,0,0,0) is created as the initial rendering target. Based on the pure black background image, the display windows and masking results on the pure black background image are blended and rendered layer by layer to obtain the rendering result. The masking result can be located in the middle processing layer or the top processing layer. The blending and rendering can be performed in sequence. That is to say, assuming that the layer stacking order is L3->L2->L1 from top to bottom, and the image blending mode is blending -> overlay -> blending, the blending and rendering order is pure black background image -> (overlay) L1 -> (overlay) L2 -> (overlay) L3.

[0051] For example Figure 3As shown, the window W3 corresponds to the display window above the overlapping area, the window W4 and the window W5 correspond to the display window below the overlapping area, and in the below all mode, the layer of the display window above the overlapping area is also Alpha-blended with the pure white background (1, 1, 1, 1) to generate a mask layer with a transparent channel, and then the mask layer is mixed with the main rendering result of the remaining layers as the top layer, that is, in this case, the window W4 and the window W5 correspond to the remaining layers, and the main rendering result of the remaining layers is obtained according to the image blending mode of each layer, that is, the main rendering result of the remaining layers can be obtained before the mask layer is generated, of course, the above-mentioned sequential blending rendering can also be performed after the mask layer is obtained, and as Figure 4 As shown, when there are multiple overlapping areas, the mask processing of two display windows is first performed respectively, such as the window W6 and the window W7, the window W7 and the window W8, and the window W8 and the window W9, and then the corresponding mask results are sequentially blended and rendered. For example, the disable mode means that the mask processing flow is skipped, and the main rendering result of each layer is directly used, thereby meeting the diversified use requirements.

[0052] S103, converting the rendering result from the linear color space to the standard color space, and generating a target video according to the resolution of the output device.

[0053] In some embodiments of the present application, the standard color space is an sRGB standard space, which facilitates different output devices to display the same color, so that dynamic scaling is performed based on the resolution of the output device to generate a target video.

[0054] The dynamic mask blending method for ultra-high-definition video provided by the embodiments of the present application first fills a plurality of video materials in respective display windows corresponding to the display windows on an operation interface, and there are overlapping areas between the display windows, and then according to the selection result of the user on the display window above the overlapping area, the layer of the display window above the overlapping area is mixed with a pure white background to generate a mask layer, wherein the layer mask mode is used to determine the number of masks of the display window below the overlapping area, that is, the video material played as a dynamic mask is used to process the video material of the display window below the overlapping area through the mask layer to obtain a rendering result and generate a target video, thereby expanding the use range.

[0055] Based on the foregoing embodiments, the embodiments of the present application provide a dynamic mask blending device for ultra-high-definition video. The dynamic mask blending device 100 can be applied to Figures 1 to 4 The dynamic mask blending method of the corresponding embodiments. Please refer to Figure 5 The dynamic mask blending device 100 includes:

[0056] The filling module 101 is configured to acquire at least two video materials, and fill the at least two video materials in respective corresponding display windows on an operation interface, wherein an overlapping area exists between the display windows.

[0057] The mask processing module 102 is configured to, in response to a selection result of a user on a layer mask mode of a display window above the overlapping area, mix a layer of the display window above the overlapping area with a pure white background to generate a mask layer, and process a video material of a display window below the overlapping area through the mask layer to obtain a rendering result, wherein the layer mask mode is used to determine a number of masks of the display window below the overlapping area.

[0058] The generating module 103 is configured to convert the rendering result from a linear color space to a standard color space, and generate a target video according to a resolution of an output device.

[0059] Optionally, the mask processing module 102 is specifically configured to calculate a first mask value by using a mask channel value and a mask color value of the mask layer.

[0060] A second mask value is calculated based on a transparency corresponding to the first mask value and the mask color value, and a mask result is calculated based on the second mask value and a video material color value of the display window below the overlapping area.

[0061] The mask result is mixed and rendered according to an image mixing mode of the display window below the overlapping area to obtain the rendering result.

[0062] Optionally, the mask processing module 102 is further specifically configured to create a pure black background as an initial rendering target, and mix and render each display window and the mask result on the pure black background layer by layer to obtain the rendering result.

[0063] Optionally, the image mixing mode includes a cover mode and an overlay mode.

[0064] Optionally, the filling module 101 is specifically configured to perform a transparency pre-multiplication operation and a position clipping operation on the at least two video materials respectively.

[0065] Optionally, the filling module 101 is further specifically configured to detect video source resolutions corresponding to the at least two video materials respectively.

[0066] If the video source resolution is greater than or equal to a preset resolution threshold, the video material corresponding to the video source resolution is clustered.

[0067] It should be noted that the descriptions of the same steps and contents in other embodiments are referred to the descriptions in other embodiments, and will not be repeated here.

[0068] The device for dynamic mask mixing of ultra-high definition video provided by the embodiment of the present application firstly fills a plurality of video materials in respective corresponding display windows on an operation interface, and there is an overlapping area between the display windows, then according to a selection result of a user on a layer mask mode of an upper display window in the overlapping area, a layer of the upper display window in the overlapping area is mixed with a pure white background to generate a mask layer, wherein the layer mask mode is used to determine a number of masks of a lower display window in the overlapping area, that is, a played video material is used as a dynamic mask, and then a video material of the lower display window in the overlapping area is processed through the mask layer to obtain a rendering result and generate a target video, thereby expanding the use range.

[0069] Based on the foregoing embodiment, an embodiment of the present application provides a terminal device. Please refer to Figure 6 The terminal device 200 can include a processor 201 and a memory 202. The memory 202 stores at least one instruction, at least one program, a code set or an instruction set, which is loaded and executed by the processor 201 to realize Figures 1 to 4 The steps of the dynamic mask mixing method of the corresponding embodiment.

[0070] As another aspect, an embodiment of the present application provides a computer readable storage medium for storing program code, the program code being used to execute any one of the foregoing Figures 1 to 4 The dynamic mask mixing method of the corresponding embodiment.

[0071] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and module can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

[0072] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. For example, the above-described device embodiment is only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed ones can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms. The modules shown as separate components can be or can not be physically separated, and the components shown as modules can be or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the embodiment scheme.

[0073] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present separately, or two or more units can be integrated in one module. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.

[0074] Based on such understanding, the technical solutions of the present application, essentially or in the form of a contribution to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the dynamic mask mixing method of various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0075] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0076] The principles and implementation modes of the present application are described by using specific examples in this paper, and the above embodiment descriptions are only used to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A dynamic mask blending method for ultra-high definition video, characterized in that, The dynamic mask mixing method comprises: acquiring at least two video materials, and filling the at least two video materials in respective corresponding display windows on an operation interface, wherein there is an overlapping area between the display windows; in response to a selection result of a user on a layer mask mode of a display window above the overlapping area, mixing a layer of the display window above the overlapping area with a pure white background to generate a mask layer, and processing a video material of a display window below the overlapping area through the mask layer to obtain a rendering result, wherein the layer mask mode is used to determine a number of masks of the display window below the overlapping area; converting the rendering result from a linear color space to a standard color space, and generating a target video according to a resolution of an output device; wherein the processing of the video material of the display window below the overlapping area through the mask layer to obtain the rendering result comprises: calculating a first mask value by using a mask channel value and a mask color value of the mask layer; calculating a second mask value based on a transparency corresponding to the first mask value and the mask color value, and calculating a mask result based on the second mask value and a color value of the video material of the display window below the overlapping area; and performing mixed rendering on the mask result according to an image mixing mode of the display window below the overlapping area to obtain the rendering result.

2. The dynamic mask blending method of claim 1, wherein, The mixed rendering of the mask result to obtain the rendering result comprises: creating a pure black background as an initial rendering target, and performing mixed rendering on each of the display windows and the mask result on the pure black background layer by layer to obtain the rendering result.

3. The dynamic mask blending method of claim 1, wherein, The image mixing mode comprises an overlay mode and a superimposition mode.

4. The dynamic mask blending method of any one of claims 1-3, wherein, Before the filling of the at least two video materials in respective corresponding display windows on the operation interface, the method further comprises: performing a transparency pre-multiplication operation and a position cropping operation on the at least two video materials respectively.

5. The dynamic mask blending method of claim 4, wherein, The method further comprises: detecting video source resolutions corresponding to the at least two video materials respectively; if the video source resolutions are greater than or equal to a preset resolution threshold, performing cluster processing on the video materials corresponding to the video source resolutions.

6. An apparatus for dynamic mask blending of ultra-high definition video, the apparatus comprising: The dynamic mask mixing device comprises: a filling module configured to acquire at least two video materials, and fill the at least two video materials in respective corresponding display windows on an operation interface, wherein there is an overlapping area between the display windows; a mask processing module configured to, in response to a selection result of a user on a layer mask mode of a display window above the overlapping area, mix a layer of the display window above the overlapping area with a pure white background to generate a mask layer, and process a video material of a display window below the overlapping area through the mask layer to obtain a rendering result, wherein the layer mask mode is used to determine a number of masks of the display window below the overlapping area; a generating module configured to convert the rendering result from a linear color space to a standard color space, and generate a target video according to a resolution of an output device; The mask processing module is specifically configured to calculate a first mask value by using a mask channel value and a mask color value of the mask layer; calculate a second mask value based on a transparency corresponding to the first mask value and the mask color value, and calculate a mask result based on the second mask value and a video material color value of a display window below the overlapping area; and perform mixed rendering on the mask result according to an image mixing mode of the display window below the overlapping area to obtain the rendering result.

7. A terminal device, characterized by comprising: The terminal device comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, which are loaded and executed by the processor to implement the steps of the dynamic mask mixing method in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the dynamic mask mixing method in any one of claims 1 to 5.

Images