Video blurring method and apparatus, storage medium, and electronic device

Abstract

A video blurring method, comprising: when in a video filming state, performing first blurring processing on a captured original image frame to obtain a first blurred frame with a first precision and first blurring data, wherein the first blurred frame is obtained on the basis of the first blurring data; and when in a non-video filming state, performing second blurring processing on the original image frame on the basis of the first blurring data to obtain a second blurred frame with a second precision, wherein the second precision is higher than the first precision.

Description

Video blurring method, device, storage medium, and electronic device

[0001] This application claims priority to the Chinese patent application filed on June 16, 2023, with application number 2023107218285 and invention name “A video blurring method, device, storage medium and electronic device”, the contents of which should be understood as incorporated into this application by reference. Technical Field

[0002] The embodiments of the present disclosure relate to, but are not limited to, video blurring technology, and in particular to a video blurring method, device, storage medium, and electronic device. Background Art

[0003] In order to avoid frame rate instability such as video frame drops, there are strict requirements on the running speed of video algorithms. Therefore, video algorithms are often lightweight, and it is difficult to obtain high-precision image processing effects by increasing the complexity of video algorithms.

[0004] Summary of the Invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0006] The technical solution disclosed in the present disclosure can achieve real-time blurring of video frames, reduce power consumption, ensure the stability of the frame rate during video shooting, and obtain more precise blurred frames through a more complex blurring algorithm after shooting is completed.

[0007] The present disclosure provides a method for blurring a video, including:

[0008] When in a video shooting state, performing a first blurring process on the shot original image frame to obtain a first blurring frame and first blurring data with a first precision; the first blurring frame is obtained based on the first blurring data;

[0009] When in a non-video shooting state, performing a second blurring process on the original image frame based on the first blurring data to obtain a second blurring frame with a second precision;

[0010] The second precision is higher than the first precision.

[0011] In an exemplary embodiment of the present disclosure, the first blurring data includes: a first blurring reference image and a first image frame to be blurred.

[0012] In an exemplary embodiment of the present disclosure, performing a first blurring process on the captured original image frame to obtain a first blurring frame and first blurring data with a first precision includes:

[0013] Downsampling the original image frame to obtain a first downsampled image frame;

[0014] Acquire a target area in the first down-sampled image frame and the first blur reference image corresponding to the target area;

[0015] Acquire the first image frame to be blurred according to the original image frame or the first down-sampled image frame;

[0016] Acquire the first defocused frame according to the first defocused reference image and the first image frame to be defocused;

[0017] The resolution of the original image frame is a first original resolution, the resolution of the first image frame to be blurred is a first blurred resolution, and the first blurred resolution is not lower than the resolution of the first downsampled image frame and not higher than the first original resolution.

[0018] In an exemplary embodiment of the present disclosure, the first blur reference image is a mask or a blur intensity map corresponding to the target area of ​​the first down-sampled image frame.

[0019] In an exemplary embodiment of the present disclosure, when the first defocused resolution is higher than a resolution of the first downsampled image frame and lower than the first original resolution, acquiring a first image frame to be defocused based on the original image frame or the first downsampled image frame includes:

[0020] down-sampling the original image frame to obtain the first image frame to be blurred; or

[0021] The first down-sampled image frame is up-sampled to obtain the first image frame to be blurred.

[0022] In an exemplary embodiment of the present disclosure, when the first defocused resolution is higher than a resolution of the first downsampled image frame and lower than the first original resolution, acquiring the first defocused frame according to the first defocused reference image and the first image frame to be defocused includes:

[0023] Upsampling the first defocused reference image with the first defocused resolution as a target to obtain a second defocused reference image;

[0024] blurring the first image frame to be blurred according to the second blurring reference image to obtain a first blurring result frame;

[0025] Upsampling the second defocused reference image and the first defocused result frame with the first original resolution as the target to obtain a first original defocused reference image and a first original defocused result frame;

[0026] The first original blurred reference image is then fused with the first original blurred result frame and the original image frame to obtain the first blurred frame.

[0027] In an exemplary embodiment of the present disclosure, when the first blurring resolution is equal to the resolution of the first downsampled image frame, acquiring the first image frame to be blurred according to the original image frame or the first downsampled image frame includes:

[0028] The first down-sampled image frame is used as the first image frame to be blurred.

[0029] In an exemplary embodiment of the present disclosure, when the first defocusing resolution is equal to the resolution of the first downsampled image frame, acquiring the first defocused frame according to the first defocusing reference image and the first image frame to be defocused includes:

[0030] blurring the first image frame to be blurred according to the first blur reference image to obtain a first blur result frame;

[0031] Upsampling the first defocused reference image and the first defocused result frame with the first original resolution as the target to obtain a first original defocused reference image and a first original defocused result frame;

[0032] The first original blurred reference image is then fused with the first original blurred result frame and the original image frame to obtain the first blurred frame.

[0033] In an exemplary embodiment of the present disclosure, when the first defocused resolution is equal to the first original resolution, acquiring the first image frame to be defocused according to the original image frame or the first downsampled image frame includes:

[0034] The original image frame is used as the first image frame to be blurred.

[0035] In an exemplary embodiment of the present disclosure, when the first defocused resolution is equal to the first original resolution, acquiring the first defocused frame according to the first defocused reference image and the first image frame to be defocused includes:

[0036] Upsampling the first defocused reference image with the first original resolution as the target to obtain a first original defocused reference image;

[0037] blurring the first image frame to be blurred according to the first original blur reference image to obtain a first original blur result frame;

[0038] The first original blurred reference image is then fused with the first original blurred result frame and the original image frame to obtain the first blurred frame.

[0039] In an exemplary embodiment of the present disclosure, performing a second blurring process on the original image frame based on the first blurring data to obtain a second blurring frame with a second precision includes:

[0040] Performing super-resolution processing on the first defocused reference image to obtain a super-resolution defocused reference image, wherein the super-resolution defocused reference image has a higher resolution than the first defocused reference image;

[0041] Acquire a second image frame to be blurred according to the original image frame or the first image frame to be blurred;

[0042] Acquire the second defocused frame according to the super-resolution defocused reference image and the second image frame to be defocused;

[0043] Among them, the resolution of the second image frame to be blurred is a second blurring resolution, the second blurring resolution is not lower than the first blurring resolution and not higher than the first original resolution, and the resolution of the super-resolution blurring reference image is not higher than the second blurring resolution.

[0044] In an exemplary embodiment of the present disclosure, when the second defocusing resolution is higher than the resolution of the super-resolution defocusing reference image and lower than the first original resolution, acquiring the second defocusing frame according to the super-resolution defocusing reference image and the second image frame to be defocused includes:

[0045] Upsampling the super-resolution defocused reference image with the second defocused resolution as a target to obtain a third defocused reference image;

[0046] blurring the second image frame to be blurred according to the third blurring reference image to obtain a second blurring result frame;

[0047] Upsampling the third defocused reference image and the second defocused result frame with the first original resolution as the target to obtain a second original defocused reference image and a second original defocused result frame;

[0048] The second original blurring result frame and the original image frame are fused according to the second original blurring reference image to obtain the second blurring frame.

[0049] In an exemplary embodiment of the present disclosure, when the second defocusing resolution is equal to the resolution of the super-resolution defocusing reference image, acquiring the second defocusing frame according to the super-resolution defocusing reference image and the second image frame to be defocused includes:

[0050] blurring the second image frame to be blurred according to the super-resolution blur reference image to obtain a second blur result frame;

[0051] Upsampling the super-resolution defocused reference image and the second defocused result frame with the first original resolution as the target to obtain a second original defocused reference image and a second original defocused result frame;

[0052] The second original blurring result frame and the original image frame are fused according to the second original blurring reference image to obtain the second blurring frame.

[0053] In an exemplary embodiment of the present disclosure, when the second defocused resolution is equal to the first original resolution, acquiring the second defocused frame according to the super-resolution defocused reference image and the second image frame to be defocused includes:

[0054] Upsampling the super-resolution blurred reference image with the first original resolution as the target to obtain a second original blurred reference image;

[0055] blurring the second image frame to be blurred according to the second original blur reference image to obtain a second original blur result frame;

[0056] The second original blurring result frame and the original image frame are fused according to the second original blurring reference image to obtain the second blurring frame.

[0057] The present disclosure provides a video blurring device, comprising:

[0058] a first blurring unit configured to, when in a video shooting state, perform a first blurring process on a shot original image frame to obtain a first blurring frame and first blurring data with a first precision; the first blurring frame is obtained based on the first blurring data;

[0059] a second blurring unit configured to, when in a non-video shooting state, perform a second blurring process on the original image frame based on the first blurring data to obtain a second blurring frame with a second precision;

[0060] The second precision is higher than the first precision.

[0061] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned video blurring methods are implemented.

[0062] The embodiment of the present disclosure further provides a terminal, comprising: a memory and a processor, wherein the memory stores a computer program;

[0063] The processor is configured to implement any of the above-mentioned video blurring methods when executing the computer program in the memory.

[0064] Through the above-mentioned embodiment of the present disclosure, real-time blurring of video frames can be achieved, power consumption can be reduced, and the stability of the frame rate during video shooting can be ensured. After shooting is completed, more complex blurring algorithms can be used to obtain more precise blurred frames.

[0065] Other features and advantages of the present disclosure will be described in the following description or understood through implementation of the present disclosure. Other advantages of the present disclosure can be realized and obtained through the solutions described in the description and the drawings.

[0066] Still other aspects will become apparent upon reading and understanding the accompanying drawings and detailed description.

[0067] Summary of the Figures

[0068] The accompanying drawings are used to provide an understanding of the technical solution of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solution of the present disclosure and do not constitute a limitation to the technical solution of the present disclosure.

[0069] FIG1 is a flow chart of a video blurring method provided by an embodiment of the present disclosure;

[0070] FIG2 is a flowchart of performing a first blurring process on a captured original image frame according to an embodiment of the present disclosure;

[0071] FIG3 is a flowchart of performing a second blurring process on an original image frame based on first blurring data according to an embodiment of the present disclosure;

[0072] FIG4 is a block diagram of a video blurring device provided by an embodiment of the present disclosure;

[0073] FIG5 is a block diagram of a terminal provided by an embodiment of the present disclosure.

[0074] Details

[0075] The present disclosure describes a plurality of embodiments, but the description is exemplary rather than restrictive, and it will be apparent to those skilled in the art that there may be more embodiments and implementations within the scope of the embodiments described in the present disclosure. Although many possible feature combinations are shown in the drawings and discussed in the embodiments, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with any other feature or element in any other embodiment, or may replace any other feature or element in any other embodiment.

[0076] The present disclosure includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The disclosed embodiments, features, and elements of the present disclosure may also be combined with any conventional features or elements to form a unique disclosure scheme defined by the claims. Any features or elements of any embodiment may also be combined with features or elements from other disclosed schemes to form another unique disclosure scheme defined by the claims. Therefore, it should be understood that any feature shown and / or discussed in this disclosure may be implemented individually or in any appropriate combination. Therefore, except for the limitations made according to the appended claims and their equivalents, the embodiments are not subject to other limitations. In addition, various modifications and changes may be made within the scope of protection of the appended claims.

[0077] In addition, when describing representative embodiments, the specification may have presented the method and / or process as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific order of the steps described herein, the method or process should not be limited to the steps in the specific order described. As will be understood by those skilled in the art, other orders of steps are also possible. Therefore, the specific order of the steps set forth in the specification should not be interpreted as a limitation on the claims. In addition, the claims to the method and / or process should not be limited to performing their steps in the order written, and those skilled in the art can readily understand that these orders can be changed and still remain within the spirit and scope of the disclosed embodiments.

[0078] The present disclosure provides a method for blurring a video, as shown in FIG1 , and the method includes:

[0079] S101: When in a video shooting state, perform a first blurring process on a shot original image frame to obtain a first blurring frame and first blurring data with a first precision, where the first blurring frame is obtained based on the first blurring data;

[0080] S102: When in a non-video shooting state, performing a second blurring process on the original image frame based on the first blurring data to obtain a second blurring frame with a second precision;

[0081] wherein the second precision is higher than the first precision;

[0082] The original image frame is subjected to blurring processing, so that the target area (i.e., the foreground area) in the original image frame remains clear, while the background area is blurred, thereby achieving an effect of highlighting the target area;

[0083] When in the video shooting state, in order to avoid frame rate instability such as video frame drops, a relatively lightweight first blurring process is performed on the original image frame to obtain a first blurring frame with lower precision;

[0084] When in a non-video shooting state, in order to improve image accuracy, based on the first blur data obtained in the first blur processing, a more complex second blur processing is performed on the original image frame to obtain a second blur frame with higher accuracy;

[0085] Through the method in this embodiment, real-time blurring of video frames can be achieved, power consumption can be reduced, and the stability of the frame rate during video shooting can be ensured. After shooting is completed, more complex blurring algorithms can be used to obtain more precise blurred frames.

[0086] Since the video blur algorithm is limited by the running speed, it generally runs at a resolution much smaller than the original video resolution. Therefore, video details, such as smaller light spots, cannot be rendered, and the blur boundaries are blurred, and the blur layering is also lost to a certain extent.

[0087] The present disclosure provides a method for performing a first blurring process on a captured original image frame, as shown in FIG2 , the method comprising:

[0088] S201, down-sampling the original image frame to obtain a first down-sampled image frame;

[0089] The process of downsampling the original image frame is a process of reducing the resolution of the original image frame, and a first downsampled image frame with a lower resolution is generated through the process. The resolution of the first downsampled image frame is lower than the first original resolution, and the resolution of the original image frame is the first original resolution. The number of pixels of the first downsampled image frame is smaller than the number of pixels of the original image frame, and the time consumed in performing corresponding calculations and processing on the first downsampled image frame is also correspondingly reduced.

[0090] In some embodiments, the downsampling method may be any one of the following methods: point cloud random downsampling, voxel downsampling, and sliding least squares sampling.

[0091] S202: Acquire a target area in the first down-sampled image frame and a first blur reference image corresponding to the target area;

[0092] The target area is the foreground area of ​​the first down-sampled image frame, and the other parts are the background area of ​​the first down-sampled image frame;

[0093] In this embodiment, the target region may be extracted by using a semantic segmentation method or a depth prediction method, or by using a semantic segmentation method and a depth prediction method;

[0094] In some embodiments, the target area is a portrait area, and the target is a portrait image. Using a portrait image as a target is only one implementation method provided by the embodiments of the present disclosure, and does not limit the technical solution of the present disclosure. Other target types are also applicable to the technical solution of the present disclosure, such as animals and plants, vehicles and other transportation vehicles, license plates, etc. For example, the target under the corresponding target type must meet at least one of the following conditions: having a definite shape; having a recognizable outline; and being able to use a corresponding detection algorithm to determine the area position to which it belongs in the image;

[0095] In some embodiments, the first defocused reference image may be a mask, which is a single-channel image that can be used to block all or part of the image to be processed during image processing. In this embodiment, the first defocused reference image may be a mask corresponding to the extracted target area, which blocks the target area during image processing.

[0096] In some embodiments, the mask is a binary image consisting of 0s and 1s, or in some embodiments, the mask can also be a multi-valued image; common multi-valued images include grayscale images and color images. Grayscale images usually have 256 possible gray levels, while color images can combine millions of possible colors through the three RGB (red, green, and blue) color channels;

[0097] In some embodiments, the first blur reference image can be a blur intensity map, which contains blur intensity information corresponding to background area pixels. It can play a role in distinguishing the foreground area and the background area and providing blur intensity information corresponding to the background area pixels in subsequent blur processing.

[0098] S203: Acquire a first image frame to be blurred according to the original image frame or the first down-sampled image frame;

[0099] The resolution of the original image frame is a first original resolution, the resolution of the first image frame to be blurred is a first blurring resolution, and the first blurring resolution is not lower than the resolution of the first down-sampled image frame and not higher than the first original resolution;

[0100] In some embodiments, the first original resolution is relatively high. Due to operating speed constraints, the blurring algorithm cannot be run at the first original resolution. The blurring algorithm needs to be run at a lower resolution. In this case, the first blurring resolution needs to be smaller than the first original resolution.

[0101] In some embodiments, the first original resolution is low, and the blurring algorithm may be run directly on the first original resolution. In this case, the first blurring resolution may be equal to the first original resolution.

[0102] Exemplarily, the original image frame or the first down-sampled image frame is sampled to obtain the first image frame to be blurred.

[0103] In an embodiment of the present disclosure, when the first defocused resolution is higher than the resolution of the first downsampled image frame and lower than the first original resolution, the first image frame to be defocused may be obtained by downsampling the original image frame or upsampling the first downsampled image frame;

[0104] When the first defocusing resolution is equal to the resolution of the first downsampled image frame, the first downsampled image frame may be used as the first image frame to be defocused;

[0105] When the first blurring resolution is equal to the first original resolution, the original image frame may be used as the first image frame to be blurred.

[0106] S204: Acquire the first defocused frame according to the first defocused reference image and the first image frame to be defocused;

[0107] When the first blurring resolution is higher than the resolution of the first downsampled image frame and lower than the first original resolution, step S204 may include:

[0108] S2051: Upsampling the first defocused reference image with the first defocused resolution as a target to obtain a second defocused reference image;

[0109] The first defocusing resolution is higher than the resolution of the first downsampled image frame, that is, the first defocusing resolution is higher than the resolution of the first defocusing reference image, and the first defocusing reference image needs to be upsampled to obtain a second defocusing reference image with a resolution of the first defocusing resolution;

[0110] S2052: blurring the first image frame to be blurred according to the second blurring reference image to obtain a first blurring result frame;

[0111] The resolution of the second defocused reference image is the same as the resolution of the first image frame to be defocused, that is, the first defocused resolution; a defocusing algorithm is run at the first defocused resolution to defocus the background area of ​​the first image frame to be defocused, thereby obtaining a first defocused result frame;

[0112] S2053: up-sampling the second defocused reference image and the first defocused result frame with the first original resolution as the target, to obtain a first original defocused reference image and a first original defocused result frame;

[0113] The first defocused resolution is lower than the first original resolution, that is, the resolutions of the second defocused reference image and the first defocused result frame are lower than the first original resolution, and the second defocused reference image and the first defocused result frame need to be upsampled to obtain a first original defocused reference image and a first original defocused result frame having a resolution of the first original resolution;

[0114] S2054: According to the first original blurred reference image, fuse the first original blurred result frame and the original image frame to obtain the first blurred frame.

[0115] When the first blurring resolution is equal to the resolution of the first downsampled image frame, step S204 may include:

[0116] S2061: blur the first image frame to be blurred according to the first blur reference image to obtain a first blur result frame;

[0117] The resolution of the first defocused reference image is the same as the resolution of the first image frame to be defocused, that is, both are the first defocused resolution; a defocusing algorithm is run at the first defocused resolution to defocus the background area of ​​the first image frame to be defocused, thereby obtaining the first defocused result frame;

[0118] S2062: Up-sampling the first defocused reference image and the first defocused result frame with the first original resolution as the target, to obtain a first original defocused reference image and a first original defocused result frame;

[0119] The first defocused resolution is lower than the first original resolution, that is, the resolutions of the first defocused reference image and the first defocused result frame are lower than the first original resolution, and the first defocused reference image and the first defocused result frame need to be upsampled to obtain a first original defocused reference image and a first original defocused result frame having a resolution of the first original resolution;

[0120] S2063: According to the first original blurred reference image, fuse the first original blurred result frame and the original image frame to obtain the first blurred frame.

[0121] When the first blurring resolution is equal to the first original resolution, step S204 may include:

[0122] S2071: Up-sampling the first defocused reference image with the first original resolution as the target to obtain a first original defocused reference image;

[0123] The first original resolution is higher than the resolution of the first defocused reference image, and the first defocused reference image needs to be upsampled to obtain a first original defocused reference image with a resolution of the first original resolution;

[0124] S2072: blur the first image frame to be blurred according to the first original blur reference image to obtain a first original blur result frame;

[0125] The resolution of the first original defocused reference image is the same as the resolution of the first image frame to be defocused, both being the first original resolution; a defocusing algorithm is run at the first original resolution to defocus the background area of ​​the first image frame to be defocused, thereby obtaining a first defocused result frame;

[0126] S2073: According to the first original blurred reference image, fuse the first original blurred result frame and the original image frame to obtain the first blurred frame.

[0127] In this embodiment, the first blurring data includes: the first blurring reference image, the first image frame to be blurred;

[0128] In some embodiments, the first blurring data further includes metadata, such as face information, focus information, etc.

[0129] In this embodiment, the first blurred data and the first blurred frame will be stored in a computer-readable storage medium, and the second blurred processing will be performed based on the first blurred data and the first blurred frame subsequently; it should be noted that, optionally, a compression algorithm can be used to compress the first blurred data and the first blurred frame. In some embodiments, compression algorithm H264 or H265 can be used for compression processing.

[0130] In this embodiment, the video frames are processed at a lower resolution, which greatly reduces the computational and power consumption overhead of obtaining the target area and its blurred reference image, ensuring that the power consumption of running the blurring algorithm is not too high, thereby avoiding the occurrence of frame rate instability such as frame drops.

[0131] The embodiment of the present disclosure provides a method for performing a second blurring process on the original image frame based on the first blurring data, as shown in FIG3 , the method comprising:

[0132] S301, performing super-resolution processing on the first defocused reference image to obtain a super-resolution defocused reference image;

[0133] The resolution of the super-resolution defocused reference image is higher than the resolution of the first defocused reference image;

[0134] In this embodiment, performing super-resolution processing on the first defocused reference image to obtain a super-resolution defocused reference image may include:

[0135] Inputting an image frame (e.g., the original image frame) having a higher resolution than the first defocused reference image and the first defocused reference image into a super-resolution module;

[0136] Performing super-resolution processing on the first defocused reference image using a super-resolution module to obtain a super-resolution defocused reference image;

[0137] Among them, super-resolution processing (SR), also known as super-resolution processing, can not only reconstruct a high-resolution defocused reference image, but also improve the accuracy of the defocused reference image boundary compared to the upsampling method, making the subsequent defocused image obtained by fusion based on the defocused reference image more accurate.

[0138] In this embodiment, the super-resolution module can be implemented by an edge-preserving filter, such as a guided filter or a bilateral solver, or by a neural network, such as a convolutional neural network (CNN) or a vision transformer (ViT).

[0139] S302: Acquire a second image frame to be blurred according to the original image frame or the first image frame to be blurred;

[0140] The resolution of the second image frame to be blurred is a second blurring resolution, the second blurring resolution is not lower than the first blurring resolution and not higher than the first original resolution, and the resolution of the super-resolution blurring reference image is not higher than the second blurring resolution;

[0141] In some embodiments, the first original resolution is relatively high. Due to operating speed constraints, the blurring algorithm cannot be run at the first original resolution. The blurring algorithm needs to be run at a lower resolution. In this case, the second blurring resolution needs to be smaller than the first original resolution.

[0142] In some embodiments, the first original resolution is low, and the blurring algorithm may be directly run on the first original resolution. In this case, the second blurring resolution may be equal to the first original resolution.

[0143] In the embodiment of the present disclosure, when the second defocused resolution is higher than the first defocused resolution and lower than the first original resolution, the second image frame to be defocused may be obtained by downsampling the original image frame or upsampling the first image frame to be defocused;

[0144] When the second defocusing resolution is equal to the first defocusing resolution, the first image frame to be defocused may be used as the second image frame to be defocused;

[0145] When the second blurred resolution is equal to the first original resolution, the original image frame may be used as the second image frame to be blurred.

[0146] S303: Acquire a second defocused frame according to the super-resolution defocused reference image and the second image frame to be defocused;

[0147] When the second blur resolution is higher than the resolution of the super-resolution blur reference image and lower than the first original resolution, step S303 includes:

[0148] S3041, upsampling the super-resolution defocused reference image with the second defocused resolution as a target to obtain a third defocused reference image;

[0149] The second defocusing resolution is higher than the resolution of the super-resolution defocusing reference image, and the super-resolution defocusing reference image needs to be upsampled to obtain a third defocusing reference image with a resolution of the second defocusing resolution;

[0150] S3042: blurring the second image frame to be blurred according to the third blurring reference image to obtain a second blurring result frame;

[0151] The resolution of the third defocused reference image is the same as the resolution of the second image frame to be defocused, that is, the second defocused resolution; a defocusing algorithm is run at the second defocused resolution to defocus the background area of ​​the second image frame to be defocused, thereby obtaining a defocused second defocused result frame;

[0152] S3043: up-sampling the third defocused reference image and the second defocused result frame with the first original resolution as the target to obtain a second original defocused reference image and a second original defocused result frame;

[0153] The second defocused resolution is lower than the first original resolution, that is, the resolutions of the third defocused reference image and the second defocused result frame are lower than the first original resolution, and the third defocused reference image and the second defocused result frame need to be upsampled to obtain a second original defocused reference image and a second original defocused result frame having the same resolution as the first original resolution;

[0154] S3044: According to the second original blurred reference image, fuse the second original blurred result frame and the original image frame to obtain a second blurred frame.

[0155] When the second blur resolution is equal to the resolution of the super-resolution blur reference image, step S303 includes:

[0156] S3051: blurring the second image frame to be blurred according to the super-resolution blur reference image to obtain a second blur result frame;

[0157] The resolution of the super-resolution defocused reference image is the same as the resolution of the second image frame to be defocused, both being the second defocused resolution; defocusing the background area of ​​the second image frame to be defocused at the second defocused resolution to obtain a defocused second defocused result frame;

[0158] S3052: up-sampling the super-resolution defocused reference image and the second defocused result frame with the first original resolution as the target, to obtain a second original defocused reference image and a second original defocused result frame;

[0159] The second defocused resolution is lower than the first original resolution, that is, the resolutions of the super-resolution defocused reference image and the second defocused result frame are lower than the first original resolution, and the super-resolution defocused reference image and the second defocused result frame need to be upsampled to obtain a second original defocused reference image and a second original defocused result frame having the same resolution as the first original resolution;

[0160] S3053: According to the second original blurred reference image, fuse the second original blurred result frame and the original image frame to obtain the second blurred frame.

[0161] When the second blurring resolution is equal to the first original resolution, step S303 includes:

[0162] S3061: Up-sampling the super-resolution blurred reference image with the first original resolution as the target to obtain a second original blurred reference image;

[0163] The first original resolution is higher than the resolution of the super-resolution blurred reference image, and the super-resolution blurred reference image needs to be upsampled to obtain a second original blurred reference image with a resolution of the first original resolution;

[0164] S3062: blur the second image frame to be blurred according to the second original blur reference image to obtain a second original blur result frame;

[0165] The resolution of the second original defocused reference image is the same as the resolution of the second image frame to be defocused, both being the first original resolution; defocusing the background area of ​​the second image frame to be defocused at the first original resolution to obtain a defocused second defocused result frame;

[0166] S3063: Fusing the second original blur result frame and the original image frame according to the second original blur reference image to obtain a second blur frame;

[0167] In this embodiment, by performing super-resolution processing on the first blurred reference image, not only can a high-resolution blurred reference image be reconstructed, but also the accuracy of the blurred reference image boundary can be improved, so that the blurred image obtained based on the fusion of the high-precision blurred reference image has higher accuracy, avoiding the blurred fusion boundary between the foreground area and the background area in the blurred video frame, making the boundary sharper, and the blurring effect more accurate, realistic, and natural.

[0168] The present disclosure provides a video blurring device 10, as shown in FIG4 , comprising:

[0169] The first defocusing unit 100 is configured to, when in a video shooting state, perform a first defocusing process on a shot original image frame to obtain a first defocused frame and first defocused data with a first precision; the first defocused frame is obtained based on the first defocused data;

[0170] The second blurring unit 200 is configured to, when in a non-video shooting state, perform a second blurring process on the original image frame based on the first blurring data to obtain a second blurring frame with a second precision;

[0171] The second precision is higher than the first precision.

[0172] An embodiment of the present disclosure also provides a non-volatile computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the video blurring method described in any of the previous embodiments are implemented.

[0173] The embodiment of the present disclosure further provides a terminal, as shown in FIG5 , comprising a memory 501 and a processor 502 , wherein the memory 501 stores a computer program;

[0174] The processor 502 is configured to implement the video blurring method as described in any of the previous embodiments when executing the computer program in the memory.

[0175] The serial numbers of the above-mentioned embodiments of the present disclosure are for description only and do not represent the advantages or disadvantages of the embodiments.

[0176] In the above embodiments of the present disclosure, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0177] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0178] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0179] In addition, the multiple functional units in each embodiment of the present disclosure may be integrated into a single processing unit, multiple units may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0180] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the contributing part or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, etc., which can store program codes.

[0181] The above description is merely an example implementation of the present disclosure. It should be noted that a person skilled in the art may make several improvements and modifications without departing from the principles of the present disclosure, and these improvements and modifications should also be considered within the scope of protection of the present disclosure.

Claims

1. A video blurring method, comprising: When in a video shooting state, performing a first blurring process on the shot original image frame to obtain a first blurring frame and first blurring data with a first precision; The first blur frame is obtained based on the first blur data; When in a non-video shooting state, performing a second blurring process on the original image frame based on the first blurring data to obtain a second blurring frame with a second precision; The second precision is higher than the first precision.

2. The video blurring method according to claim 1, wherein: The first blurring data includes: a first blurring reference image and a first image frame to be blurred.

3. The video blurring method according to claim 2, wherein: The performing a first blurring process on the captured original image frame to obtain a first blurring frame and first blurring data with a first accuracy includes: Down-sampling the original image frame to obtain a first down-sampled image frame; Acquire a target area in the first down-sampled image frame and the first blurred reference image corresponding to the target area; Acquire the first image frame to be blurred according to the original image frame or the first down-sampled image frame; Acquire the first blurred frame according to the first blurred reference image and the first image frame to be blurred; The resolution of the original image frame is a first original resolution, the resolution of the first image frame to be blurred is a first blurred resolution, and the first blurred resolution is not lower than the resolution of the first down-sampled image frame and not higher than the first original resolution.

4. The video blurring method according to claim 3, wherein: The first blur reference image is a mask or blur intensity map corresponding to the target area of ​​the first down-sampled image frame.

5. The video blurring method according to claim 3, wherein: When the first blurring resolution is higher than the resolution of the first down-sampled image frame and lower than the first original resolution, acquiring a first image frame to be blurred according to the original image frame or the first down-sampled image frame includes: Down-sampling the original image frame to obtain the first image frame to be blurred; or The first down-sampled image frame is up-sampled to obtain the first image frame to be blurred.

6. The video blurring method according to claim 3, wherein: When the first blur resolution is higher than the resolution of the first down-sampled image frame and lower than the first original resolution, acquiring the first blur frame according to the first blur reference image and the first image frame to be blurred includes: Upsampling the first defocused reference image with the first defocused resolution as a target to obtain a second defocused reference image; blurring the first image frame to be blurred according to the second blurring reference image to obtain a first blurring result frame; Upsampling the second blurred reference image and the first blurred result frame with the first original resolution as the target to obtain a first original blurred reference image and a first original blurred result frame; According to the first original blurred reference image, the first original blurred result frame and the original image frame are fused to obtain the first blurred frame.

7. The video blurring method according to claim 3, wherein: When the first blurring resolution is equal to the resolution of the first down-sampled image frame, acquiring a first image frame to be blurred according to the original image frame or the first down-sampled image frame includes: The first down-sampled image frame is used as the first image frame to be blurred.

8. The video blurring method according to claim 3, wherein: When the first blur resolution is equal to the resolution of the first down-sampled image frame, acquiring the first blur frame according to the first blur reference image and the first image frame to be blurred includes: blurring the first image frame to be blurred according to the first blurring reference image to obtain a first blurring result frame; Upsampling the first defocused reference image and the first defocused result frame with the first original resolution as the target to obtain a first original defocused reference image and a first original defocused result frame; According to the first original blurred reference image, the first original blurred result frame and the original image frame are fused to obtain the first blurred frame.

9. The video blurring method according to claim 3, wherein: When the first blurred resolution is equal to the first original resolution, acquiring a first image frame to be blurred according to the original image frame or the first down-sampled image frame includes: The original image frame is used as the first image frame to be blurred.

10. The video blurring method according to claim 3, wherein: When the first blur resolution is equal to the first original resolution, acquiring the first blur frame according to the first blur reference image and the first image frame to be blurred includes: Upsampling the first blurred reference image with the first original resolution as a target to obtain a first original blurred reference image; According to the first original blur reference image, blurring the first image frame to be blurred to obtain a first original blur result frame; According to the first original blurred reference image, the first original blurred result frame and the original image frame are fused to obtain the first blurred frame.

11. The video blurring method according to claim 3, wherein: The step of performing a second blurring process on the original image frame based on the first blurring data to obtain a second blurring frame with a second precision includes: Performing super-resolution processing on the first blurred reference image to obtain a super-resolution blurred reference image, wherein the super-resolution blurred reference image has a higher resolution than the first blurred reference image; Acquire a second image frame to be blurred according to the original image frame or the first image frame to be blurred; Acquire the second blurred frame according to the super-resolution blurred reference image and the second image frame to be blurred; Among them, the resolution of the second image frame to be blurred is a second blurring resolution, the second blurring resolution is not lower than the first blurring resolution and not higher than the first original resolution, and the resolution of the super-resolution blurring reference image is not higher than the second blurring resolution.

12. The video blurring method according to claim 11, wherein: When the second blur resolution is higher than the resolution of the super-resolution blur reference image and lower than the first original resolution, acquiring the second blur frame according to the super-resolution blur reference image and the second image frame to be blurred includes: Upsampling the super-resolution blurred reference image with the second blurred resolution as a target to obtain a third blurred reference image; blurring the second image frame to be blurred according to the third blurring reference image to obtain a second blurring result frame; The third blurred reference image and the second blurred result frame are up-scaled with the first original resolution as the target. Sampling to obtain a second original blurred reference image and a second original blurred result frame; According to the second original blurred reference image, the second original blurred result frame and the original image frame are fused to obtain the second blurred frame.

13. The video blurring method according to claim 11, wherein: When the second blur resolution is equal to the resolution of the super-resolution blur reference image, acquiring the second blur frame according to the super-resolution blur reference image and the second image frame to be blurred includes: blurring the second image frame to be blurred according to the super-resolution blur reference image to obtain a second blur result frame; Upsampling the super-resolution blurred reference image and the second blurred result frame with the first original resolution as the target to obtain a second original blurred reference image and a second original blurred result frame; According to the second original blurred reference image, the second original blurred result frame and the original image frame are fused to obtain the second blurred frame.

14. The video blurring method according to claim 11, wherein: When the second blur resolution is equal to the first original resolution, acquiring the second blur frame according to the super-resolution blur reference image and the second image frame to be blurred includes: Upsampling the super-resolution blurred reference image with the first original resolution as a target to obtain a second original blurred reference image; blurring the second image frame to be blurred according to the second original blurred reference image to obtain a second original blurred result frame; According to the second original blurred reference image, the second original blurred result frame and the original image frame are fused to obtain the second blurred frame.

15. A video blurring device, comprising: A first blurring unit is configured to, when in a video shooting state, perform a first blurring process on a shot original image frame to obtain a first blurring frame and first blurring data with a first precision; The first blur frame is obtained based on the first blur data; A second blurring unit is configured to, when in a non-video shooting state, perform a second blurring process on the original image frame based on the first blurring data to obtain a second blurring frame with a second precision; The second precision is higher than the first precision.

16. A non-transitory computer-readable storage medium, wherein a computer program is stored in the non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the video blurring method according to any one of claims 1 to 14 is implemented.

17. A terminal, comprising: A memory and a processor, wherein the memory stores a computer program; wherein, The processor is configured to implement the video blurring method according to any one of claims 1 to 14 when executing the computer program in the memory.

Images