Special effect video processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a special effect video processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a color channel image and a transparency channel image contained in each video image frame in an original special effect video; reducing the transparency channel image, and filling the reduced transparency channel image in the first image area; filling the color channel image in the second image area, and combining the color channel image contained in the second image area and the reduced transparency channel image contained in the first image area to generate a target image frame corresponding to each video image frame; and generating a target special effect video by using each target image frame. The volume of the special effect video file can be reduced, so that the file transmission efficiency is improved, and the storage cost of the video file is reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of video processing, and particularly relates to a special effect video processing method, apparatus, electronic device, and storage medium. Background Art

[0002] With the development of video processing technology, a technology for adding special effect videos to short videos and live broadcast rooms has emerged. The special effect videos carry a transparent effect and can be rendered on a display screen, thereby improving the display effect of short videos and live broadcast rooms.

[0003] In related technologies, a special effect video with a transparent effect can usually be achieved by adding an Alpha channel to an mp4 video. The Alpha channel is a channel used to represent transparency information in a video or image, and this channel can be used in combination with the RGB channels used to represent color information, so that some parts of the video can display the underlying content through, achieving a richer visual effect.

[0004] However, the above-mentioned special effect video file with an alpha channel usually has a large file size, which will reduce the efficiency of file transmission and increase the storage cost of video files. Summary of the Invention

[0005] The present disclosure provides a special effect video processing method, apparatus, electronic device, and storage medium to at least solve the problem that the special effect video file mentioned in related technologies is too large. The technical solutions of the present disclosure are as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a special effect video processing method is provided, including:

[0007] Obtaining color channel images and transparency channel images included in each video image frame of an original special effect video;

[0008] Performing a shrinking process on the transparency channel image, and filling and setting the shrunk transparency channel image in a first image area;

[0009] Filling and setting the color channel image in a second image area, and merging the color channel image included in the second image area and the shrunk transparency channel image included in the first image area to generate a target image frame corresponding to each video image frame;

[0010] Generating a target special effect video by using each of the target image frames.

[0011] In an exemplary embodiment, the regional height of the second image region is the same as that of the first image region; the processing of reducing the transparency channel image and filling the reduced transparency channel image in the first image region includes: reducing the transparency channel image according to a preset reduction mode to obtain the reduced transparency channel image; setting the color channel image in the second image region, and filling and setting the reduced transparency channel image in the first image region according to a filling method matching the reduction mode; the merging of the color channel image included in the second image region and the reduced transparency channel image included in the first image region to generate a target image frame corresponding to each video image frame includes: horizontally splicing the color channel image included in the second image region and the reduced transparency channel image included in the first image region to generate the target image frame.

[0012] In an exemplary embodiment, the reducing the transparency channel image according to a preset reduction mode to obtain the reduced transparency channel image includes: when the reduction mode is the first reduction mode, horizontally compressing each pixel included in the transparency channel image according to a first compression ratio to obtain the reduced transparency channel image; the filling and setting the reduced transparency channel image in the first image region according to a filling method matching the reduction mode includes: filling and setting the reduced transparency channel image in the first image region; wherein the regional width of the first image region matches the image width of the reduced transparency channel image.

[0013] In an exemplary embodiment, the reducing the transparency channel image according to a preset reduction mode to obtain the reduced transparency channel image includes: when the reduction mode is the second reduction mode, horizontally compressing each pixel included in the transparency channel image according to a second compression ratio, and vertically compressing each pixel included in the horizontally compressed transparency channel image according to a preset compression ratio to obtain the reduced transparency channel image; the filling and setting the reduced transparency channel image in the first image region according to a filling method matching the reduction mode includes: vertically splitting the reduced transparency channel image according to the preset compression ratio to obtain at least two sub-transparency channel images; arranging each of the sub-transparency channel images vertically in the first image region to fill and set each of the sub-transparency channel images in the first image region.

[0014] In an exemplary embodiment, the preset compression ratio is 50%; the step of longitudinally splitting the reduced transparency channel image according to the preset compression ratio to obtain at least two sub-transparency channel images includes: equally dividing the reduced transparency channel image longitudinally to obtain a first sub-transparency channel image and a second sub-transparency channel image; wherein the first sub-transparency channel image is the image corresponding to the left half region of the reduced transparency channel image, and the second sub-transparency channel image is the image corresponding to the right half region of the reduced transparency channel image; the step of longitudinally arranging each of the sub-transparency channel images in the first image region includes: flipping the second sub-transparency channel image horizontally, and longitudinally arranging the first sub-transparency channel image and the horizontally flipped second sub-transparency channel image in the first image region.

[0015] In an exemplary embodiment, after generating the target special effect video by using each of the target image frames, the method further includes: obtaining the region position information of the color channel image and the reduced transparency channel image in the target image frame; based on the region position information, obtaining sampling coordinate information, and using the sampling coordinate information to perform rendering processing on the target special effect video.

[0016] In an exemplary embodiment, the step of obtaining the region position information of the color channel image and the reduced transparency channel image in the target image frame includes: obtaining the preset vertex coordinates of the color channel image and the reduced transparency channel image in the target image frame, and obtaining the image width and image height of the color channel image and the reduced transparency channel image in the target image frame; using the preset vertex coordinates, the image width and the image height as the region position information.

[0017] According to a second aspect of the embodiments of the present disclosure, there is provided a special effect video processing apparatus, including:

[0018] A channel image extraction unit configured to obtain the color channel image and the transparency channel image included in each video image frame of the original special effect video;

[0019] A channel image reduction unit configured to perform reduction processing on the transparency channel image and fill and set the reduced transparency channel image in a first image region;

[0020] A target image generation unit configured to fill and set the color channel image in a second image region, and merge the color channel image included in the second image region and the reduced transparency channel image included in the first image region to generate a target image frame corresponding to each video image frame;

[0021] The special effect video generation unit is configured to execute generating a target special effect video by using each of the target image frames.

[0022] In an exemplary embodiment, the regional height of the first image region is the same as the regional height of the second image region; the channel image reduction unit is further configured to execute reducing the transparency channel image according to a preset reduction mode to obtain the reduced transparency channel image; filling and setting the reduced transparency channel image in the first image region according to a filling method matching the reduction mode; the target image generation unit is further configured to execute horizontally splicing the color channel image included in the second image region and the reduced transparency channel image included in the first image region to generate the target image frame.

[0023] In an exemplary embodiment, the channel image reduction unit is further configured to execute, when the reduction mode is the first reduction mode, horizontally compressing each pixel included in the transparency channel image according to a first compression ratio to obtain the reduced transparency channel image; filling and setting the reduced transparency channel image in the first image region; wherein the regional width of the first image region matches the image width of the reduced transparency channel image.

[0024] In an exemplary embodiment, the channel image reduction unit is further configured to execute, when the reduction mode is the second reduction mode, horizontally compressing each pixel included in the transparency channel image according to a second compression ratio, and vertically compressing each pixel included in the horizontally compressed transparency channel image according to a preset compression ratio to obtain the reduced transparency channel image; vertically splitting the reduced transparency channel image according to the preset compression ratio to obtain at least two sub-transparency channel images; vertically arranging each of the sub-transparency channel images in the first image region so as to fill and set each of the sub-transparency channel images in the first image region.

[0025] In an exemplary embodiment, the preset compression ratio is 50%; the channel image reduction unit is further configured to perform vertical equal division on the reduced transparency channel image to obtain a first sub-transparency channel image and a second sub-transparency channel image; wherein the first sub-transparency channel image is an image corresponding to the left half region of the reduced transparency channel image, and the second sub-transparency channel image is an image corresponding to the right half region of the reduced transparency channel image; flip the second sub-transparency channel image horizontally, and arrange the first sub-transparency channel image and the horizontally flipped second sub-transparency channel image vertically in the first image region.

[0026] In an exemplary embodiment, the special effect video processing device further includes: a special effect video rendering unit configured to obtain the regional position information of the color channel image and the reduced transparency channel image in the target image frame; based on the regional position information, obtain sampling coordinate information, and use the sampling coordinate information to perform rendering processing on the target special effect video.

[0027] In an exemplary embodiment, the special effect video rendering unit is further configured to obtain the preset vertex coordinates of the color channel image and the reduced transparency channel image in the target image frame, and obtain the image width and image height of the color channel image and the reduced transparency channel image in the target image frame; use the preset vertex coordinates, the image width and the image height as the regional position information.

[0028] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the instructions to implement the special effect video processing method according to any one of the embodiments in the first aspect.

[0029] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the special effect video processing method according to any one of the embodiments in the first aspect.

[0030] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, the computer program product includes instructions, when the instructions are executed by a processor of an electronic device, enabling the electronic device to execute the special effect video processing method according to any one of the embodiments in the first aspect.

[0031] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:

[0032] By obtaining the color channel images and alpha channel images included in each video image frame of the original special effect video; performing a shrinking process on the alpha channel images, and filling and setting the shrunk alpha channel images in a first image area; filling and setting the color channel images in a second image area, and merging the color channel images included in the second image area and the shrunk alpha channel images included in the first image area to generate target image frames corresponding to each video image frame; generating a target special effect video by using the target image frames. The present disclosure obtains the color channel images and alpha channel images of each video image frame in the original special effect video, and then can shrink the alpha channel images, and fill and set the shrunk alpha channel images in the first image area, and fill and set the color channel images in the second image area, so as to merge the color channel images included in the second image area and the shrunk alpha channel images included in the first image area to generate target image frames corresponding to each video image frame, and further generate a target special effect video. It can be seen that the present disclosure can shrink the alpha channel images in the video image frames, and can fill and set the shrunk alpha channel images in the first image area, so that the storage space of the alpha channel images can be effectively saved, the volume of the special effect video file can be reduced, the file transmission efficiency can be improved, and the storage cost of the video file can be reduced.

[0033] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure and do not constitute an improper limitation to the present disclosure.

[0035] Figure 1 is a flowchart of a method for processing a special effect video shown according to an exemplary embodiment.

[0036] Figure 2 is a flowchart of generating a target image frame corresponding to a video image frame shown according to an exemplary embodiment.

[0037] Figure 3 is a schematic diagram of a preview effect of a target image frame shown according to an exemplary embodiment.

[0038] Figure 4 is a schematic diagram of a preview effect of a target image frame shown according to another exemplary embodiment.

[0039] Figure 5 is a schematic diagram of a preview effect of a target image frame shown according to still another exemplary embodiment.

[0040] Figure 6 It is a flowchart showing the rendering process of a target special effect video according to an exemplary embodiment.

[0041] Figure 7 It is an overall flowchart of a rendering optimization method with a transparent effect according to an exemplary embodiment.

[0042] Figure 8 It is a schematic diagram showing the corresponding relationship of pixel transparency values according to an exemplary embodiment.

[0043] Figure 9 It is a block diagram of a special effect video processing device according to an exemplary embodiment.

[0044] Figure 10 It is a block diagram of an electronic device according to an exemplary embodiment. Detailed implementation manners

[0045] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described here can be implemented in an order different from those illustrated or described here. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are only examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0047] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties.

[0048] Figure 1 It is a flowchart of a special effect video processing method according to an exemplary embodiment. As Figure 1 shown, the special effect video processing method is used in a terminal and includes the following steps.

[0049] In step S101, color channel images and transparency channel images included in each video image frame of the original special effect video are obtained.

[0050] Among them, the original special effect video refers to the unprocessed special effect video for rendering on the display screen. This special effect video can be an mp4 video file, which can be composed of multiple video image frames. Each video image frame is an individual video image frame in the original special effect video. The color channel image refers to the image corresponding to the color channel, that is, the RGB channel, in the video image frame, and the transparency channel image is the image corresponding to the transparency channel, that is, the alpha channel, in the video image frame. The extraction of the color channel image and the transparency channel image can be achieved through the filter tool in the audio-video processing tool.

[0051] Specifically, after obtaining the original special effect video, the original special effect video can be input into the audio-video processing tool, which can extract the RGB channel map and the alpha channel map contained in each video image frame of the original special effect video, as the color channel image and the transparency channel image respectively.

[0052] In step S102, the transparency channel image is scaled down and filled and set in the first image area.

[0053] The first image area refers to the image area for setting the scaled-down transparency channel image, and the setting method of the scaled-down transparency channel image in the first image area is filling and setting. This filling and setting method means that after the scaled-down transparency channel image is set in the first image area, the first image area does not contain blank areas, that is, the first image area is completely filled by the scaled-down transparency channel image.

[0054] Specifically, after the terminal obtains the transparency channel images of each video image frame, the transparency channel images can be scaled down respectively. This scaling down can refer to compressing the transparency channel image to reduce its size. Then, the color channel image can be set in the second image area, and at the same time, the scaled-down transparency channel image is filled and set in the first image area.

[0055] In step S103, the color channel image is filled and set in the second image area, and the color channel image contained in the second image area and the scaled-down transparency channel image contained in the first image area are merged to generate the target image frame corresponding to each video image frame;

[0056] In step S104, a target special effect video is generated using each target image frame.

[0057] The second image area refers to the image area used to set the color channel image, the target image frame is the new image frame generated after processing each video image frame, and the target special effect video is the special effect video composed of each target image frame. Specifically, the terminal can fill and set the color channel image obtained in step S101 in the second image area, and merge the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate the target image frame corresponding to each video image frame, so as to generate the target special effect video by using each target image frame.

[0058] For example, the color channel image included in the second image area can be set in the left area of the target image frame, and the reduced transparency channel image included in the first image area can be set in the right area of the target image frame, and the left area and the right area are spliced to obtain the target image frame.

[0059] In the above special effect video processing method, by obtaining the color channel image and the transparency channel image included in each video image frame of the original special effect video; performing a reduction process on the transparency channel image and filling and setting the reduced transparency channel image in the first image area; filling and setting the color channel image in the second image area, and merging the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate the target image frame corresponding to each video image frame; using each target image frame to generate the target special effect video. The present disclosure obtains the color channel image and the transparency channel image of each video image frame in the original special effect video, and then the transparency channel image can be reduced, and the reduced transparency channel image can be filled and set in the first image area, and the color channel image can be filled and set in the second image area, so as to merge the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate the target image frame corresponding to each video image frame, and further generate the target special effect video. It can be seen that the present disclosure can reduce the transparency channel image in the video image frame, and the reduced transparency channel image can be filled and set in the first image area, so the storage space of the transparency channel image can be effectively saved, thereby reducing the volume of the special effect video file, further improving the file transmission efficiency, and reducing the storage cost of the video file.

[0060] In an exemplary embodiment, the regional height of the first image area is the same as that of the second image area; as Figure 2 shown, step S102 may further include:

[0061] In step S201, the transparency channel image is scaled down according to a pre-set scaling mode to obtain a scaled-down transparency channel image.

[0062] Among them, the scaling mode can be used to represent the scaling method for scaling down the transparency channel image. In this embodiment, the scaling methods of the transparency channel image can include multiple types, and multiple scaling methods can be represented by a pre-set scaling mode. The terminal can scale down the transparency channel image according to the pre-set scaling mode, so as to obtain a scaled-down transparency channel image.

[0063] In step S202, the scaled-down transparency channel image is filled and set in the first image area according to a filling method matching the scaling mode.

[0064] The filling method refers to the method of filling and setting the scaled-down transparency channel image in the first image area, and this method also needs to match the scaling mode. Specifically, the terminal can set the color channel image in the second image area, and at the same time, according to the filling method matching the scaling mode, fill and set the scaled-down transparency channel image in the first image area.

[0065] Step S103 can further include:

[0066] Step S203, horizontally splice the color channel image included in the second image area and the scaled-down transparency channel image included in the first image area to generate a target image frame.

[0067] In this embodiment, since the region height of the second image area is the same as the region height of the first image area, that is, the vertical side of the second image area is the same as the length of the vertical side of the first image area, the merging method of the color channel image and the scaled-down transparency channel image can be horizontal splicing, that is, splicing the vertical side of the second image area with the vertical side of the first image area. For example, the second image area can be set on the left side of the target image frame, and the first image area can be set on the right side of the target image frame, so as to horizontally splice the second image area and the first image area to generate a target image frame.

[0068] In this embodiment, the scaling mode can also be set, so that the transparency channel image can be scaled down according to the scaling mode, and the filling and setting method can also match the scaling mode. By this method, the diversity of the scaling methods of the transparency channel image can be increased, and the splicing method can be to horizontally splice the color channel image in the second image area and the scaled-down transparency channel image included in the first image area. Therefore, the generation efficiency of the target image frame can be improved.

[0069] Further, step S201 may further include: when the reduction mode is the first reduction mode, horizontally compressing each pixel included in the transparency channel image according to a first compression ratio to obtain a reduced transparency channel image; step S202 may further include: filling and setting the reduced transparency channel image in a first image area; wherein the area width of the first image area matches the image width of the reduced transparency channel image.

[0070] Among them, the first reduction mode refers to an image reduction mode that only compresses the image width of the transparency channel image. In the first reduction mode, each image pixel in the transparency channel image will be horizontally compressed, and the first compression ratio is the horizontal compression ratio in the first reduction mode. Specifically, if the reduction mode is the first reduction mode, the terminal can first horizontally compress each pixel included in the transparency channel image according to the first compression ratio. At this time, the obtained reduced transparency channel image is the transparency channel image horizontally compressed according to the first compression ratio. Then, the transparency channel image can be filled and set in the first image area, and at this time, the area width of the first image area matches the image width of the reduced transparency channel image, so as to realize filling and setting the reduced transparency channel image in the first image area.

[0071] For example, the size of the transparency channel image before compression is 60*30, that is, the width is 60 and the height is 30. Assuming the first compression ratio is 50%, that is, the width of the transparency channel image is compressed to half. At this time, the size of the reduced transparency channel image is 30*30, then the size of the first image area used is also 30*30. Compared with the original transparency channel image that needs to occupy an area of 60*30, the range of the first image area occupied is significantly reduced, so the file size of the target special effect video can be reduced.

[0072] As Figure 3 shown, before reducing the transparency channel image, the size of the image frame can be as Figure 3 shown. At this time, a second image area with a color channel image is set on the left, and the size is the same as that of the first image area with the transparency channel image set on the right. Then the terminal can horizontally compress the transparency channel image and reduce the width of the first image area to half. At this time, the size of the target image frame can be as Figure 4 shown.

[0073] In this embodiment, if the reduction mode is the first reduction mode, each pixel included in the transparency channel image can be horizontally compressed according to the first compression ratio, and the reduced transparency channel image can be filled and set in the first image area with the same area width. By this method, the occupied range of the first image area can be reduced, thereby reducing the file size of the target special effect video.

[0074] In addition, step S201 may further include: when the reduction mode is the second reduction mode, horizontally compressing each pixel included in the transparency channel image according to a second compression ratio, and vertically compressing each pixel included in the horizontally compressed transparency channel image according to a preset compression ratio to obtain a reduced transparency channel image; step S202 may further include: vertically splitting the reduced transparency channel image according to the preset compression ratio to obtain at least two sub-transparency channel images; and arranging the sub-transparency channel images vertically in the first image area to fill and set the sub-transparency channel images in the first image area.

[0075] The second reduction mode refers to an image reduction mode in which both the image width and the image height of the transparency channel image are compressed. In the second reduction mode, each image pixel in the transparency channel image will be compressed both horizontally and vertically. The second compression ratio is the horizontal compression ratio in the second reduction mode, and the preset compression ratio is a pre-fixed vertical compression ratio. Specifically, if the reduction mode is the second reduction mode, after the terminal horizontally compresses each pixel included in the transparency channel image according to the second compression ratio, it can further vertically compress each pixel included in the horizontally compressed transparency channel image according to the preset compression ratio to obtain a reduced transparency channel image.

[0076] Continuing with the size of the transparency channel image before compression being 60*30, that is, 60 in width and 30 in height, if both the second compression ratio and the preset compression ratio are 50%, that is, both the width and the height of the transparency channel image are compressed to half, the size of the reduced transparency channel image is 30*15 at this time.

[0077] The sub-transparency channel image refers to the transparency channel image obtained by vertically splitting the reduced transparency channel image, and the vertical splitting method can match the vertical compression ratio, that is, the preset compression ratio. Since the image height of the reduced transparency channel image will be correspondingly reduced after vertical compression, and the area height of the first image area is the same as the area height of the second image area, if the reduced transparency channel image is directly set in the first image area, blank areas will inevitably appear, thus wasting the image area space. Therefore, in order to further improve the utilization rate of the image space to reduce the file size, in this embodiment, the reduced transparency channel image can also be vertically split according to the preset compression ratio. After obtaining multiple sub-transparency channel images, the sub-transparency channel images are arranged vertically in the first image area, so as to fill and set the sub-transparency channel images in the first image area.

[0078] Taking the size of the reduced transparency channel image as 30*15 as an example, since the regional height of the first image region is the same as that of the second image region, both being 30, the reduced transparency channel image cannot complete the filling of the region at this time. That is, at least a first image region of 30*30 size is required, and there will be blank regions at this time. Therefore, the reduced transparency channel image can be vertically split. For example, it can be split into two 15*15 sub-transparency channel images. Then, the two sub-transparency channel images can be vertically arranged, so that each sub-transparency channel image is filled and set in the first image region. At this time, the width of the first image region can be the width of the sub-transparency channel image, that is, the size of the first image region at this time is 15*30. Compared with the size of the first image region obtained in the first reduction mode, which is 30*30, this embodiment can further reduce the file size of the target special effect video.

[0079] In this embodiment, if the reduction mode is the second reduction mode, each pixel included in the transparency channel image can be horizontally compressed according to the second compression ratio, and vertically compressed according to the preset compression ratio. And the reduced transparency channel image is vertically split according to the preset compression ratio, and each sub-transparency channel image after splitting is vertically arranged in the first image region. By this way, the occupied range of the first image region can be further reduced, thereby further reducing the file size of the target special effect video.

[0080] Further, the preset compression ratio is 50%; vertically splitting the reduced transparency channel image according to the preset compression ratio to obtain at least two sub-transparency channel images, including: vertically equally dividing the reduced transparency channel image to obtain a first sub-transparency channel image and a second sub-transparency channel image; where the first sub-transparency channel image is the image corresponding to the left half region of the reduced transparency channel image, and the second sub-transparency channel image is the image corresponding to the right half region of the reduced transparency channel image; vertically arranging each sub-transparency channel image in the first image region can further include: horizontally flipping the second sub-transparency channel image, and vertically arranging the first sub-transparency channel image and the horizontally flipped second sub-transparency channel image in the first image region.

[0081] In this embodiment, the preset compression ratio is 50%, so the number of sub-transparency channel images obtained is 2, namely the first sub-transparency channel image and the second sub-transparency channel image. Among them, the first sub-transparency channel image refers to the left half part of the reduced transparency channel image, while the second sub-transparency channel image is the right half part of the reduced transparency channel image.

[0082] Specifically, after obtaining the reduced transparency channel image, the reduced transparency channel image can be evenly divided into left and right parts to obtain a first sub-transparency channel image, which can be defined as alpha channel-left, and a second sub-transparency channel image, which can be defined as alpha channel-right.

[0083] After that, the second sub-transparency channel image can be flipped left and right, and the first sub-transparency channel image and the second sub-transparency channel image after being flipped left and right can be arranged vertically in the first image area. For example, the alpha channel-right can be flipped left and right to obtain alpha channel-right-reversed-X, and then the upper half of the first image area can be set as alpha channel-left, and the lower half can be set as alpha channel-right-reversed-X. By flipping left and right, it is possible to avoid rendering errors caused by interference from the color channel image included in the second image area when rendering the target special effect video, so the rendering effect of the target special effect video can be further improved.

[0084] For example, in the case where the reduction mode is the second reduction mode, and both the second compression ratio and the preset compression ratio are 50%, the size of the generated target image frame can be as Figure 5 shown, where the left side of the target image frame is the color channel image, and the right side is two sub-transparency channel images. The first sub-transparency channel image is set in the upper right area, and the second sub-transparency channel image is set in the lower right area after being flipped left and right.

[0085] In this embodiment, the preset compression ratio can be set to 50%. After evenly dividing the reduced transparency channel image vertically to obtain the first sub-transparency channel image corresponding to the left half area and the second sub-transparency channel image corresponding to the right half area, the second sub-transparency channel image can also be flipped left and right, so as to arrange the first sub-transparency channel image and the second sub-transparency channel image after being flipped left and right vertically in the first image area. By this method, it is possible to avoid rendering errors caused by interference from the color channel image included in the second image area when rendering the target special effect video, so the rendering effect of the target special effect video can be further improved.

[0086] In an exemplary embodiment, after step S104, as Figure 6 shown, it may further include:

[0087] In step S601, obtain the regional position information of the color channel image and the reduced transparency channel image in the target image frame.

[0088] The regional position information refers to the regional position of the color channel image and the downscaled transparency channel image in the target image frame. This regional position can be characterized in the form of a rendering region structure, which can be composed of the preset vertex coordinates, image width, and image height of the color channel image and the downscaled transparency channel image in the target image frame. The preset vertex coordinates can be the lower left corner vertex coordinates of the color channel image and the downscaled transparency channel image.

[0089] For example, the definition of this structure is as follows:

[0090] Rect{

[0091] X coordinate of the lower left corner vertex,

[0092] Y coordinate of the lower left corner vertex,

[0093] Width of the rendering region,

[0094] Height of the rendering region

[0095] }

[0096] Taking the first downscaling mode as an example, the regional position information of the color channel image and the downscaled transparency channel image in the target image frame can be shown as follows:

[0097] "rgbRect":

[0098] 0,

[0099] 0,

[0100] 2.0 / 3.0, 1.0

[0102] ,

[0103] "alphaRect":

[0104] 2.0 / 3.0,

[0105] 0,

[0106] 1.0 / 3.0, 1.0

[0109] Among them, rgbRect represents the regional position information of the color channel image, and alphaRect represents the regional position information of the downscaled transparency channel image.

[0110] If the downscaling mode is the second downscaling mode, the regional position information of the color channel image and the downscaled transparency channel image in the target image frame can be shown as follows:

[0111] ​"rgbRect":

[0112] 0,

[0113] 0,

[0114] 0.8, 1.0

[0116] ,

[0117] "alphaRectLeft":

[0118] 0.8,

[0119] 0,

[0120] 0.4, 0.5

[0122] ,

[0123] "alphaRectRight":

[0124] 1.2,

[0125] 0.5,

[0126] -0.4, 0.5

[0129] Among them, rgbRect represents the regional position information of the color channel image, alphaRectLeft represents the regional position information of the first sub-transparency channel image, and alphaRectRight represents the second sub-transparency channel image after left-right flipping.

[0130] In step S602, based on the regional position information, sampling coordinate information is obtained, and the obtained sampling coordinate information is used to perform rendering processing on the target special effect video.

[0131] After obtaining the regional position information, the regional position information can be used to calculate the sampling coordinate information. The sampling coordinate information refers to the sampling coordinates of the OpenGL fragment shader, which can make the RGB pixels and Alpha correspond one by one, and finally render the mp4 with a transparent channel onto the screen.

[0132] Taking the first reduction mode as an example of the reduction mode, the calculation formula for the sampling coordinates corresponding to the RGB color sampling is as follows:

[0133] vec4 rgbFrame = vec4(0.0,0.0,2.0 / 3.0,1.0)

[0134] ​vec2 rgbCoord = texCoord * rgbFrame.10w + vec2(rgbFrame.x, 1.0 -rgbFrame.w - rgbFrame.y);

[0135] In this embodiment, after generating the target special effect video, the regional positions of the color channel image and the reduced transparency channel image in the target image frame can also be obtained, so as to obtain the sampling coordinates corresponding to the color channel image and the reduced transparency channel image by using the above regional positions, and then perform rendering based on the sampling coordinates. In this way, the accuracy of the target special effect video rendering can be ensured. In an exemplary embodiment, a rendering optimization method with a transparent effect is also provided. This method can significantly reduce the volume of the mp4 video file with an alpha channel and optimize the video processing algorithm to reduce the consumption of computing resources without affecting the image quality. The overall process can be as Figure 7 shown. Among them, the rendering optimization method can be implemented in the following two ways:

[0136] Solution 1: As Figure 8 shown, the color blocks of adjacent RGB channels share a difference-processed Alpha channel as the transparency value. That is to say, without losing the RGB color blocks, the bandwidth of the color blocks of the alpha channel is compressed.

[0137] For example: Pixel A and Pixel B will use A1 as the transparency value, Pixel B and Pixel C will use A2 as the transparency value, and so on for the others.

[0138] For all pixels, horizontal operations are performed, and the final MP4 preview is as Figure 4 shown.

[0139] Step 1:

[0140] Use the filter tool in the audio and video processing tool to separate the original MP4 into two parts (the left half is the RGB part, and the right half is the alpha part), which are defined as the RGB channel map and the alpha channel map

[0141] Step 2:

[0142] Reduce the width of the alpha channel in the right half by half (process it using the scaling function of the filter tool in the audio and video processing tool).

[0143] Step 3:

[0144] Combine the RGB channel map and the widened alpha channel to obtain the processed mp4.

[0145] Step 4:

[0146] Consumption of the processed MP4: According to the rendering area of the alpha channel in the entire MP4 screen, calculate the sampling coordinates of the OpenGL fragment shader so that the RGB pixels and Alpha correspond one by one, and finally render the MP4 with a transparent channel onto the screen.

[0147] Solution 2: On the basis of Solution 1, add the following steps:

[0148] 1. Add vertical compression and perform horizontal and vertical operations on all pixels.

[0149] 2. Compress the width and height of the alpha channel image by half. However, there is a drawback that there will be blank space in the lower right corner. Therefore, split the compressed alpha channel image into two parts and draw them in the upper right corner and the lower right corner respectively.

[0150] 3. Since image sampling may be affected by the RGB colors on the left half, flip the right half of the alpha channel horizontally.

[0151] The final MP4 preview obtained is as Figure 5 shown.

[0152] Step 1:

[0153] Use the filter tool of ffmpeg to separate the original MP4 into two parts (the left half is the RGB part and the right half is the alpha part), defined as the RGB channel image and the alpha channel image.

[0154] Step 2:

[0155] Reduce the width and height of the alpha channel in the right half by half (processed using the scaling function of the filter tool of ffmpeg).

[0156] Step 3:

[0157] Divide the scaled alpha channel image evenly into the left and right sides, defined as alpha channel - left and alpha channel - right.

[0158] Step 4:

[0159] Flip the alpha channel - right horizontally, defined as alpha channel - right - reversed - X.

[0160] Step 5:

[0161] Merge the RGB channel image, alpha channel - left, and alpha channel - right - reversed - X to obtain the processed MP4.

[0162] Step 6:

[0163] For the consumption of the processed mp4: according to the alpha channel in the rendering area of ​​the entire mp4 screen, the sampling coordinates of the OpenGL fragment shader are calculated, so that the RGB pixels and Alpha correspond one to one, and finally the mp4 with transparent channel is rendered to the screen.

[0164] Through this embodiment, the bandwidth of the input MP4 is optimized, and the size of the MP4 video file with an alpha channel can be significantly reduced without affecting the image quality, and the video processing algorithm is optimized to reduce the consumption of computing resources.

[0165] It should be understood that, although the various steps in the flowchart of the present disclosure are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0166] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0167] Figure 9 is a block diagram of a special effect video processing device according to an exemplary embodiment. Figure 9 The device includes a channel image extraction unit 901, a channel image reduction unit 902, a target image generation unit 903 and a special effect video generation unit 904.

[0168] The channel image extraction unit 901 is configured to obtain the color channel image and the transparency channel image contained in each video image frame in the original special effect video;

[0169] The channel image reduction unit 902 is configured to perform a reduction process on the transparency channel image and fill the reduced transparency channel image in the first image area;

[0170] The target image generation unit 903 is configured to perform the setting of filling the color channel image in the second image area, and merge the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate a target image frame corresponding to each video image frame;

[0171] The special effect video generation unit 904 is configured to perform the generation of a target special effect video by using each target image frame.

[0172] In an exemplary embodiment, the region height of the second image area is the same as the region height of the first image area; the channel image reduction unit 902 is further configured to perform the reduction process on the transparency channel image according to a preset reduction mode to obtain a reduced transparency channel image; fill and set the reduced transparency channel image in the first image area according to a filling method matching the reduction mode; the target image generation unit 903 is further configured to perform the horizontal splicing of the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate a target image frame.

[0173] In an exemplary embodiment, the channel image reduction unit 902 is further configured to perform, when the reduction mode is the first reduction mode, the horizontal compression of each pixel included in the transparency channel image according to a first compression ratio to obtain a reduced transparency channel image; fill and set the reduced transparency channel image in the first image area; wherein the region width of the first image area matches the image width of the reduced transparency channel image.

[0174] In an exemplary embodiment, the channel image reduction unit 902 is further configured to perform, when the reduction mode is the second reduction mode, the horizontal compression of each pixel included in the transparency channel image according to a second compression ratio, and perform the vertical compression of each pixel included in the horizontally compressed transparency channel image according to a preset compression ratio to obtain a reduced transparency channel image; perform the vertical segmentation of the reduced transparency channel image according to the preset compression ratio to obtain at least two sub-transparency channel images; arrange each sub-transparency channel image vertically in the first image area to fill and set each sub-transparency channel image in the first image area.

[0175] In an exemplary embodiment, the preset compression ratio is 50%; the channel image reduction unit 902 is further configured to longitudinally divide the reduced transparency channel image to obtain a first sub-transparency channel image and a second sub-transparency channel image; wherein the first sub-transparency channel image is the image corresponding to the left half region of the reduced transparency channel image, and the second sub-transparency channel image is the image corresponding to the right half region of the reduced transparency channel image; flip the second sub-transparency channel image horizontally, and arrange the first sub-transparency channel image and the horizontally flipped second sub-transparency channel image vertically in the first image region.

[0176] In an exemplary embodiment, the special effect video processing device further includes: a special effect video rendering unit configured to obtain the regional position information of the color channel image and the reduced transparency channel image in the target image frame; based on the regional position information, obtain the sampling coordinate information, and use the sampling coordinate information to perform rendering processing on the target special effect video.

[0177] In an exemplary embodiment, the special effect video rendering unit is further configured to obtain the preset vertex coordinates of the color channel image and the reduced transparency channel image in the target image frame, and obtain the image width and image height of the color channel image and the reduced transparency channel image in the target image frame; use the preset vertex coordinates, image width and image height as the regional position information.

[0178] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0179] Figure 10 FIG. is a block diagram of an electronic device 1000 for special effect video processing shown according to an exemplary embodiment. For example, the electronic device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0180] Referring to Figure 10 , the electronic device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0181] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above - mentioned methods. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0182] The memory 1004 is configured to store various types of data to support the operation of the electronic device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 can be implemented by any type of volatile or non - volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read - only memory (EEPROM), erasable programmable read - only memory (EPROM), programmable read - only memory (PROM), read - only memory (ROM), magnetic memory, flash memory, magnetic disks, optical disks, or graphene memory.

[0183] The power component 1006 provides power to various components of the electronic device 1000. The power component 1006 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the electronic device 1000.

[0184] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1008 includes a front - facing camera and / or a rear - facing camera. When the electronic device 1000 is in an operation mode, such as a shooting mode or a video mode, the front - facing camera and / or the rear - facing camera can receive external multimedia data. Each front - facing camera and rear - facing camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0185] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 1000 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 1004 or transmitted via the communication component 1016. In some embodiments, the audio component 1010 further includes a speaker for outputting audio signals.

[0186] The I / O interface 1012 provides an interface between the processing component 1002 and a peripheral interface module, and the peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0187] The sensor component 1014 includes one or more sensors for providing status assessments of various aspects of the electronic device 1000. For example, the sensor component 1014 can detect the on / off state of the electronic device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000. The sensor component 1014 can also detect a change in the position of the electronic device 1000 or an electronic device 1000 component, the presence or absence of user contact with the electronic device 1000, the orientation or acceleration / deceleration of the device 1000, and a change in the temperature of the electronic device 1000. The sensor component 1014 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 1014 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 1014 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0188] The communication component 1016 is configured to facilitate communication between the electronic device 1000 and other devices in a wired or wireless manner. The electronic device 1000 can access a wireless network based on communication standards, such as WiFi, a carrier network (such as 2G, 3G, 4G, or 5G), or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0189] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0190] In an exemplary embodiment, there is also provided a computer-readable storage medium including instructions, such as the memory 1004 including instructions, and the above instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method. For example, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0191] In an exemplary embodiment, there is also provided a computer program product including instructions, and the above instructions can be executed by the processor 1020 of the electronic device 1000 to complete the above method.

[0192] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation manners according to the description of the method embodiments. The specific implementation manners can refer to the description of the relevant method embodiments and will not be elaborated herein one by one.

[0193] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only to be regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0194] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A special effect video processing method, characterized in that, Including: Obtaining the color channel images and the transparency channel images included in each video image frame in the original special effect video; Performing a reduction process on the transparency channel image, and filling and setting the reduced transparency channel image in a first image area; Filling and setting the color channel image in a second image area, and merging the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate a target image frame corresponding to each video image frame; Generating a target special effect video by using each of the target image frames; 2. The method according to claim 1, wherein The regional height of the first image area is the same as the regional height of the second image area; the performing a reduction process on the transparency channel image, and filling and setting the reduced transparency channel image in the first image area includes: Performing a reduction process on the transparency channel image according to a preset reduction mode to obtain the reduced transparency channel image; Filling and setting the reduced transparency channel image in the first image area according to a filling method matching the reduction mode; The merging the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate a target image frame corresponding to each video image frame includes: Horizontally splicing the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate the target image frame; 3. The method according to claim 2, wherein The performing a reduction process on the transparency channel image according to a preset reduction mode to obtain the reduced transparency channel image includes: In the case where the reduction mode is a first reduction mode, horizontally compressing each pixel included in the transparency channel image according to a first compression ratio to obtain the reduced transparency channel image; The filling and setting the reduced transparency channel image in the first image area according to a filling method matching the reduction mode includes: Filling and setting the reduced transparency channel image in the first image area; wherein the regional width of the first image area matches the image width of the reduced transparency channel image; 4. The method according to claim 2, wherein The performing a reduction process on the transparency channel image according to a preset reduction mode to obtain the reduced transparency channel image includes: In the case where the reduction mode is a second reduction mode, horizontally compressing each pixel included in the transparency channel image according to a second compression ratio, and vertically compressing each pixel included in the horizontally compressed transparency channel image according to a preset compression ratio to obtain the reduced transparency channel image; The filling and setting the reduced transparency channel image in the first image area according to a filling method matching the reduction mode includes: Vertically splitting the reduced transparency channel image according to the preset compression ratio to obtain at least two sub-transparency channel images; Arrange the sub transparency channel images vertically in the first image area to fill and set the sub transparency channel images in the first image area.

5. The method according to claim 4, characterized in that, The preset compression ratio is 50%; the vertically splitting the reduced transparency channel image according to the preset compression ratio to obtain at least two sub transparency channel images includes: Vertically divide the reduced transparency channel image equally to obtain a first sub transparency channel image and a second sub transparency channel image; wherein the first sub transparency channel image is the image corresponding to the left half area of the reduced transparency channel image, and the second sub transparency channel image is the image corresponding to the right half area of the reduced transparency channel image; The arranging the sub transparency channel images vertically in the first image area includes: Flip the second sub transparency channel image horizontally, and arrange the first sub transparency channel image and the horizontally flipped second sub transparency channel image vertically in the first image area.

6. The method according to any one of claims 1 to 5, characterized in that, After generating the target special effect video by using each of the target image frames, it further includes: Obtain the regional position information of the color channel image and the reduced transparency channel image in the target image frame; Based on the regional position information, obtain the sampling coordinate information, and use the sampling coordinate information to perform rendering processing on the target special effect video.

7. The method according to claim 6, characterized in that The obtaining the regional position information of the color channel image and the reduced transparency channel image in the target image frame includes: Obtain the preset vertex coordinates of the color channel image and the reduced transparency channel image in the target image frame, and obtain the image width and image height of the color channel image and the reduced transparency channel image in the target image frame; Use the preset vertex coordinates, the image width and the image height as the regional position information.

8. An special effect video processing device, characterized in that It includes: A channel image extraction unit configured to execute obtaining the color channel image and the transparency channel image included in each video image frame of the original special effect video; A channel image reduction unit configured to execute reducing the transparency channel image and filling and setting the reduced transparency channel image in the first image area; A target image generation unit configured to execute filling and setting the color channel image in the second image area, and merging the color channel image included in the second image area and the reduced transparency channel image included in the first image area to generate a target image frame corresponding to each video image frame; A special effect video generation unit configured to execute generating a target special effect video by using each of the target image frames.

9. An electronic device, characterized in that, It includes: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the special effect video processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the special effect video processing method according to any one of claims 1 to 7.

11. A computer program product, comprising instructions, characterized in that, When the instructions are executed by a processor of an electronic device, the electronic device is enabled to execute the special effect video processing method according to any one of claims 1 to 7.

Images