Image blurring method, device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application provide a kind of image virtualization method, device, electronic equipment and computer readable storage medium, method includes: providing YUV preview image in Y channel image, UV channel image and corresponding depth image;Target Y channel image and UV channel image are carried out image blur processing, obtain Y' channel image and UV' channel image, target Y channel image is obtained by downsampling to Y channel image;According to target depth image, Y channel image, Y' channel image, UV channel image and UV' channel image determine YUV virtualization image, target depth image is obtained by size scaling to depth image.The embodiment of the present application is separately carried out downsampling to Y channel image and size scaling to depth image, can directly carry out virtualization processing to image data in YUV space, meanwhile, downsampling and size scaling also effectively reduce the data processing amount, guarantee the real-time performance of preview virtualization.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image blurring method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Existing devices with shooting capabilities usually have a preview bokeh function, which can preview and display the bokeh effect of the captured image, thereby enabling the capture of bokeh images that better meet the user's wishes.

[0003] In existing technologies, the images directly captured by the imaging device are often in the YUV color space. These are typically converted to smaller RGB images before blurring. However, these conversion operations involve significant computation and memory copying, increasing the blurring processing time and resulting in less than ideal real-time preview blurring. Summary of the Invention

[0004] This application provides an image blurring method, apparatus, electronic device, and computer-readable storage medium, aiming to solve the technical problem that the real-time performance of existing preview blurring technologies is not ideal.

[0005] On one hand, embodiments of this application provide an image blurring method, including:

[0006] Provides the Y channel image, UV channel image, and corresponding depth image in the YUV preview image;

[0007] Image blurring is performed on the target Y channel image and UV channel image to obtain the Y' channel image and UV' channel image. The target Y channel image is obtained by downsampling the Y channel image.

[0008] The YUV bokeh image is determined based on the target depth image, Y channel image, Y' channel image, UV channel image, and UV' channel image. The target depth image is obtained by scaling the depth image.

[0009] On the other hand, embodiments of this application also provide an image blurring device, including:

[0010] The module provides the Y channel image, UV channel image, and corresponding depth image in the YUV preview image;

[0011] The blur module is used to blur the target Y-channel image and UV-channel image to obtain the Y'-channel image and UV'-channel image. The target Y-channel image is obtained by downsampling the Y-channel image.

[0012] The blurring module is used to determine the YUV blurring image based on the target depth image, the Y channel image, the Y' channel image, the UV channel image, and the UV' channel image. The target depth image is obtained by scaling the depth image.

[0013] On the other hand, embodiments of this application also provide an electronic device, which includes a memory, a processor, and an image blurring program stored in the memory and executable on the processor. When the processor executes the image blurring program, it implements the steps in the image blurring method.

[0014] On the other hand, embodiments of this application also provide a computer-readable storage medium storing an image blurring program, which, when executed by a processor, implements the steps in the image blurring method.

[0015] This invention first downsamples the Y-channel image in the YUV preview image separately, then blurs the downsampled Y-channel image and the original UV-channel image to obtain the Y'-channel image and UV'-channel image. Finally, the scaled depth image, along with the Y-channel image, Y'-channel image, UV-channel image, and UV'-channel image, are used to determine the YUV blurred image. Compared to existing technologies, this invention, by downsampling the Y-channel image separately and scaling the depth image, allows for direct blurring of image data within the YUV space without data conversion. Simultaneously, downsampling and scaling effectively reduce data processing volume, ensuring real-time preview blurring. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a scene illustration of the image blurring method provided in the embodiments of this application;

[0018] Figure 2 This is a schematic flowchart of the first embodiment of the image blurring method provided in this application;

[0019] Figure 3 This is a schematic flowchart of the second embodiment of the image blurring method provided in this application.

[0020] Figure 4 This is a flowchart illustrating the third embodiment of the image blurring method provided in this application.

[0021] Figure 5 This is a schematic flowchart of the fourth embodiment of the image blurring method provided in this application.

[0022] Figure 6 This is a schematic flowchart of the fifth embodiment of the image blurring method provided in this application.

[0023] Figure 7 This is a flowchart of the sixth embodiment of the image blurring method provided in this application.

[0024] Figure 8 This is a flowchart of the seventh embodiment of the image blurring method provided in this application.

[0025] Figure 9 This is a schematic diagram of an embodiment of the image blurring device provided in this application.

[0026] Figure 10 This is a schematic diagram of an embodiment of the electronic device provided in this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of the present invention.

[0028] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to implement and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in the embodiments of this application.

[0029] This application provides an image blurring method, apparatus, electronic device, and computer-readable storage medium, which will be described in detail below.

[0030] The image blurring method in this embodiment of the invention is applied to an image blurring device, which is disposed in an electronic device. The electronic device includes a memory, a processor, and an image blurring program stored in the memory and executable on the processor. When the processor executes the image blurring program, it implements the steps of the image blurring method. The electronic device has a shooting function, such as a smartphone, digital camera, camcorder, etc. The image blurring device can be understood as a chip or processor integrated into the electronic device for executing a specific program.

[0031] like Figure 1 As shown, Figure 1 This is a schematic diagram of an image blurring scenario according to an embodiment of the present application. The image blurring scenario in this embodiment includes an image blurring device 100 and a shooting device 200. The shooting device 200 is mainly used to capture YUV images and depth images, while the image blurring device 100 runs a computer storage medium corresponding to the image blurring method to perform the image blurring steps.

[0032] In this embodiment of the invention, the image blurring device 100 is mainly used for: providing a Y-channel image, a UV-channel image, and a corresponding depth image in a YUV preview image; performing image blurring processing on the target Y-channel image and UV-channel image to obtain a Y'-channel image and a UV'-channel image, wherein the target Y-channel image is obtained by downsampling the Y-channel image; and determining a YUV blurred image based on the target depth image, the Y-channel image, the Y'-channel image, the UV-channel image, and the UV'-channel image, wherein the target depth image is obtained by scaling the depth image.

[0033] It should be noted that, Figure 1 The image blurring scenario shown is merely an example. The image blurring scenario described in the embodiments of the present invention is intended to more clearly illustrate the technical solutions of the embodiments of the present invention and does not constitute a limitation on the technical solutions provided in the embodiments of the present invention.

[0034] Based on the above-mentioned image blurring scenarios, an embodiment of the image blurring method is proposed.

[0035] like Figure 2 As shown, Figure 2 This is a flowchart illustrating the first embodiment of the image blurring method provided in this application. The image blurring method in this embodiment includes steps 201-203:

[0036] 201 provides the Y channel image, UV channel image, and corresponding depth image in the YUV preview image.

[0037] In this embodiment, after acquiring the YUV preview image, the capturing device sends it to the image blurring device via a data stream. Since the image data streams transmitted in electronic devices such as mobile phones and cameras are typically in NV12 or NV21 format, they are essentially YUV format image data streams, thus providing a YUV preview image. Further extraction of the channel data from the image data stream yields the corresponding channel image for the YUV preview image. For example, extracting only the Y channel data from the image data stream yields the Y channel image, while extracting the U and V channel data yields the corresponding UV channel image for the YUV preview image.

[0038] In this embodiment, the depth image corresponding to the YUV preview image can be obtained directly through a special lens, such as a TOF camera (i.e., a camera with TOF (Time of Flight) technology) directly capturing the YUV preview image and its depth image, or by processing the YUV preview image using existing depth estimation algorithms to obtain the depth image. There are many ways to obtain the depth image, which will not be elaborated upon here. It can be understood that the grayscale value of each pixel in the depth image is the depth value of the corresponding pixel in the YUV preview image.

[0039] 202. Perform image blurring on the target Y channel image and UV channel image to obtain the Y' channel image and UV' channel image. The target Y channel image is obtained by downsampling the Y channel image.

[0040] In this embodiment, since the amount of data transmitted by different channels in the YUV format image is different, the size of the Y channel image is generally 4 times that of the UV channel image. That is to say, the Y channel image needs to be downsampled by 2×2 in advance to obtain a target Y channel image with the same size as the UV channel image. This makes it easier to perform image blurring on the target Y channel image to obtain a Y' channel image with the same size as the UV' channel image.

[0041] In this embodiment, the Y channel image is downsampled to obtain the target Y channel image. On the one hand, this reduces the amount of computation in the blurring process. On the other hand, processing the Y channel image separately will not affect the UV channel image or the chromaticity of each pixel in the image, thus ensuring the blurring effect of the final image.

[0042] In this embodiment, there are many specific ways to implement image blurring, such as commonly used Gaussian blur, box blur, radial blur, etc., which will not be elaborated here. In general, some blurring parameters need to be set during the blurring process to achieve the corresponding blurring effect. As an optional solution in this application, such as... Figure 8As shown, a specific implementation method for blurring images is provided.

[0043] 203. Determine the YUV bokeh image based on the target depth image, Y channel image, Y' channel image, UV channel image, and UV' channel image. The target depth image is obtained by scaling the depth image.

[0044] In this embodiment, considering that the depth image is generally the same size as the Y channel image in the YUV preview image, that is, four times the size of the UV channel image, it is necessary to scale the depth image in advance to obtain a target depth image with the same size as the UV channel image. Specifically, the specific scaling method is not described in detail in this invention.

[0045] As an optional embodiment of this application, a method is provided to process the Y-channel image, Y'-channel image, UV-channel image, and UV'-channel image using a depth image to obtain a YUV blurred image. Please refer to the subsequent sections for details. Figure 3 And its explanations and descriptions.

[0046] This invention first downsamples the Y-channel image in the YUV preview image separately, then blurs the downsampled Y-channel image and the original UV-channel image to obtain the Y'-channel image and UV'-channel image. Finally, the scaled depth image, along with the Y-channel image, Y'-channel image, UV-channel image, and UV'-channel image, are used to determine the YUV blurred image. Compared to existing technologies, this invention, by downsampling the Y-channel image separately and scaling the depth image, allows for direct blurring of image data within the YUV space without data conversion. Simultaneously, downsampling and scaling effectively reduce data processing volume, ensuring real-time preview blurring.

[0047] like Figure 3 As shown, Figure 3 This is a schematic flowchart of a second embodiment of the image blurring method provided in this application.

[0048] In some embodiments of the present invention, a specific implementation scheme for determining a YUV bokeh image based on a target depth image, a Y channel image, a Y' channel image, a UV channel image, and a UV' channel image is proposed, specifically including steps 301-304:

[0049] 301. For each pixel in the target depth image, determine the corresponding fusion weight coefficient based on the pixel's grayscale value.

[0050] In this embodiment, since the target depth image is obtained by scaling up a depth image, the grayscale value of each pixel in the depth image can be understood as the depth value of the corresponding pixel in the YUV preview image. That is, for each pixel in the target depth image, the grayscale value of that pixel is related to the depth value of the corresponding pixel in the YUV preview image.

[0051] Since blurring typically refers to blurring the background and / or foreground regions of an image, excluding the main subject area, and the depth values ​​of each pixel can be used to determine which regions are the main subject, background, and foreground, an image blurring device can further utilize the grayscale values ​​of each pixel in the target depth image to determine whether each pixel needs blurring and the degree of blurring. This involves calculating the fusion weight coefficient corresponding to each pixel based on its grayscale value, which describes the degree of blurring for each pixel.

[0052] In this embodiment, the specific rules for calculating the fusion weight coefficient can be determined based on the actual blurring requirements. For example, as a feasible approach, when the grayscale value of a pixel is within a pre-defined range, the fusion weight coefficient corresponding to that pixel is determined to be 0, meaning no blurring occurs. Conversely, when the grayscale value of a pixel is outside the given range, the fusion weight coefficient corresponding to that pixel is determined to be 1, meaning complete blurring occurs, thereby achieving a background blurring effect. Alternatively, the grayscale value of the pixel can be normalized to achieve a foreground blurring effect; that is, the smaller the grayscale value of the pixel, the higher the degree of blurring. Of course, other rules for calculating the fusion weight coefficient will not be elaborated here.

[0053] As an optional solution in this application, since the focus of an image is often the key point of the shot, a method is provided to determine the fusion weight coefficients based on the grayscale value and focus depth value of each pixel. Please refer to the following sections for details. Figure 4 And its explanations and descriptions.

[0054] 302. Based on the fusion weight coefficients corresponding to each pixel, the Y channel image and the Y' channel image are fused to obtain the fused Y channel image.

[0055] It's understandable that since the Y-channel image is the original image data, while the Y'-channel image is the blurred image data, fusing the Y-channel and Y'-channel images according to a certain ratio yields a fused image of the Y-channel with a certain degree of blur. As mentioned earlier, the fusion weight coefficient can be understood as the degree of blurring for each pixel; that is, the fusion weight coefficient can be understood as the fusion ratio of each pixel in the Y-channel and Y'-channel images to achieve the corresponding blurring effect.

[0056] Typically, image fusion refers to combining the pixel values ​​of corresponding pixels in two images. Specifically, based on the fusion weight coefficients corresponding to each pixel, the steps for fusing the Y-channel image and the Y'-channel image to obtain the fused Y-channel image are detailed later. Figure 7 And its explanations and descriptions.

[0057] 303. Based on the fusion weight coefficients corresponding to each pixel, the UV channel image and the UV' channel image are fused to obtain the fused UV channel image.

[0058] In this embodiment, similar to step 302 above, the only difference is that the UV channel image and the UV' channel image are fused to obtain a fused image of the UV channel with a certain degree of blurring. The specific fusion rules are the same as those in step 302 above, and will not be repeated here.

[0059] 304. Combine the fused Y channel image and the fused UV channel image to obtain a YUV blurred image.

[0060] In this embodiment, the fused Y channel image can be understood as the Y channel data of the YUV bokeh image, and the fused UV channel image can be understood as the UV channel data of the YUV bokeh image. That is, the fused Y channel image and the fused UV channel image are used as Y channel data and UV channel data respectively to obtain the YUV bokeh image.

[0061] This application provides a specific implementation scheme for determining a YUV bokeh image based on a target depth image, a Y channel image, a Y' channel image, a UV channel image, and a UV' channel image. Specifically, it uses the grayscale value of each pixel in the target depth image to obtain the fusion weight coefficient corresponding to each pixel, and uses the fusion weight coefficient to fuse the Y channel image and the Y' channel image, as well as the UV channel image and the UV' channel image, respectively, to obtain the Y channel data and UV channel data of the YUV bokeh image, thereby obtaining the YUV bokeh image.

[0062] like Figure 4 As shown, Figure 4 This is a schematic flowchart of the third embodiment of the image blurring method provided in this application.

[0063] In some embodiments of this application, a specific implementation method is provided for determining the fusion weight coefficient corresponding to a pixel based on the grayscale value of the pixel, specifically including steps 401-403:

[0064] 401, Get the focus depth value corresponding to the YUV preview image.

[0065] In this embodiment, the focus depth value corresponding to the YUV preview image refers to the depth value of the focus point in the YUV preview image. The focus point can be determined directly during the process of capturing the YUV preview image by the imaging device, or it can be obtained from the YUV preview image based on a focus determination command received. The focus determination command is usually input by the user, which will not be elaborated upon here. After obtaining the focus point in the YUV preview image, the depth value corresponding to that focus point can be easily obtained using a depth image; this depth value is the focus depth value.

[0066] 402, calculate the difference between the gray value of a pixel and the focus depth value.

[0067] In this embodiment, the difference between the gray value of a pixel and the focal depth value can be understood as the depth difference between the pixel and the focal point in the image.

[0068] 403, calculate the fusion weight coefficient corresponding to the pixel based on the difference.

[0069] In this embodiment, the depth difference can be understood as indirectly reflecting the region where a pixel is located in the image. A negative depth difference means that the pixel's depth value is less than the focal depth value, indicating that the pixel may be a foreground region in the image. Conversely, a positive depth difference means that the pixel's depth value is greater than the focal depth value, indicating that the pixel may be a background region in the image. The smaller the absolute value of the depth difference, the smaller the distance between the pixel and the focal depth value, and the more likely it is to be a subject region in the image. Therefore, the fusion weight coefficient corresponding to each pixel can be further calculated based on the difference. The specific implementation method for calculating the corresponding fusion weight coefficient based on the difference can be found in the following sections. Figure 5 And its explanations and descriptions.

[0070] This embodiment provides a method for calculating the fusion weight coefficient using the grayscale value of pixels in the target depth image and the focal depth value of the YUV preview image. By calculating the difference between the grayscale value and the focal depth value of each pixel, the region where each pixel is located can be determined, thereby obtaining a fusion weight coefficient with better blurring effect.

[0071] like Figure 5 As shown, Figure 5 This is a schematic flowchart of the fourth embodiment of the image blurring method provided in this application.

[0072] In some embodiments of this application, a specific implementation process for calculating the fusion weight coefficient corresponding to a pixel based on the difference is provided, specifically including steps 501-502:

[0073] 501. Input the difference into the preset depth transformation formula to calculate the transformation depth value corresponding to the pixel.

[0074] In this embodiment, as described above, the difference between the grayscale value and the focal depth value of a pixel can indirectly reflect the region where the pixel is located in the image. As an optional embodiment of this application, the depth transformation formula can be:

[0075]

[0076] in, That is, pixel I in the depth map. i,j The corresponding transformation depth value, C is a coefficient, generally selected between 260 and 280, the purpose of which is to control the transformed depth value to not exceed the upper limit of 256, D i.j For pixel I in the depth map i,j The grayscale value, focus_depth is the focus depth value, D i.j -focus_depth is the pixel I i,j The corresponding difference.

[0077] As can be seen, after inputting the difference into the aforementioned depth transformation formula, the resulting transformed depth value for each pixel is negatively correlated with the absolute value of the difference. When the difference is 0, meaning the pixel's depth value is exactly the same as the focal depth value, the transformed depth value for that pixel reaches its maximum value. As the difference between the pixel's depth value and the focal depth value increases, the transformed depth value gradually decreases. In other words, by using the transformed depth value to obtain the fusion weight coefficient and achieving the fusion of the Y-channel and Y'-channel images, the blurring effect of pixels in the background and foreground regions with large differences between their depth values ​​and focal depth values ​​can be effectively improved.

[0078] The above depth transformation formula is merely one optional embodiment. Those skilled in the art can pre-set the corresponding depth transformation formula according to the actual blurring needs, which will not be elaborated upon here. Of course, in fact, different depth transformation formulas can also be pre-set for different blurring requirements, and after receiving the corresponding blurring instructions, the difference can be processed using the corresponding depth transformation formula. Please refer to the following sections for details. Figure 6 And its explanations and descriptions.

[0079] 502, normalize the transformation depth value to obtain the fusion weight coefficient corresponding to the pixel.

[0080] In this embodiment, the transform depth value corresponding to each pixel is normalized. Typically, this is done by dividing the transform depth value of each pixel by 256 to obtain the fusion weight coefficient for each pixel. However, in computer programs, since data is recorded in binary form, to simplify calculations, the transform depth value can be directly shifted right by 8 bits to obtain the fusion weight coefficient for each pixel.

[0081] Based on the foregoing description, the fusion weight coefficient can be understood as the fusion ratio of each pixel in the Y-channel image and the Y'-channel image (UV-channel image and UV'-channel image). The specific form of the fusion ratio can be various, but it can usually be described by a constant. For example, the ratio of one image can be set to 1, and the ratio of the other image can be described by this fusion ratio. Alternatively, the ratio of one image can be set as the fusion ratio, and the ratio of the other image can be described by (1 - fusion ratio). The fusion weight coefficient obtained by normalizing the transform depth value can be understood as the proportion of the original image, such as the Y-channel image and the UV-channel image, during the fusion process, while the remainder is the blurred image, that is, the proportion of the Y'-channel image and the UV'-channel image.

[0082] like Figure 6 As shown, Figure 6 This is a schematic flowchart of the fifth embodiment of the image blurring method provided in this application.

[0083] This embodiment provides an implementation method for obtaining the depth transformation formula based on the bokeh instruction, specifically including steps 601 to 603:

[0084] 601, Get the virtualization command.

[0085] In this embodiment, the blurring instructions are usually obtained by user input and mainly include background blurring, foreground blurring, region blurring, target blurring, etc., which will not be elaborated here.

[0086] 602. Query the preset database to obtain the depth transformation formula corresponding to the blurring command.

[0087] In this embodiment, the image blurring device stores the mapping relationship between depth transformation formulas and blurring instructions in the database in advance, so that after obtaining the blurring instruction input by the user, it can obtain the depth transformation formula corresponding to the blurring instruction.

[0088] Specifically, taking the depth transformation formula provided in step 501 above as an example, this formula can achieve a blurring effect on pixels in both the background and foreground regions. In other words, the depth transformation formula is pre-associated with foreground blurring in the database. Thus, when a user inputs a foreground blurring command, the depth transformation formula can be used to achieve a blurring effect on pixels in both the background and foreground regions. For example, when the blurring command is a background blurring command, the pre-stored depth transformation formula corresponding to the background blurring command is as follows:

[0089]

[0090] Among them, regarding C, D i.j The description of focus_depth is the same as above. That is, pixel I in the depth map. i，j The corresponding transformation depth value, C is a coefficient, D i，j For pixel I in the depth map i，j The depth value is defined as `focus_depth`, where `focus_depth` is the focus depth value. Additionally, `λ` is a scaling factor, preferably 0.1.

[0091] The technical solution provided in this embodiment, by pre-setting depth transformation formulas corresponding to different blurring commands, can obtain the corresponding depth transformation formulas after receiving the blurring command input by the user, and process the difference accordingly to obtain the corresponding transformation depth value. At this time, the obtained transformation depth value is used to fuse the image, thereby achieving the blurring effect corresponding to the blurring command.

[0092] like Figure 7 As shown, Figure 7 This is a schematic flowchart of the sixth embodiment of the image blurring method provided in this application.

[0093] In some embodiments of this application, a scheme is provided for fusing the Y-channel image and the Y'-channel image according to the fusion weight coefficients corresponding to each pixel to obtain a fused Y-channel image. Specifically, it includes steps 701-703:

[0094] 701. For each pixel in the target depth image, extract the pixel value of the corresponding pixel in the Y channel image and the Y' channel image.

[0095] In this embodiment, it should be noted that since the target depth image and the Y' channel image are obtained by downsampling or scaling the corresponding depth image and Y channel image in advance, that is, the size of the target depth image and the Y' channel image is the same as that of the UV channel image. That is, each pixel in the target depth image corresponds one-to-one with each pixel in the Y' channel image and the UV channel image. However, the Y channel image is 4 times the size of the target depth image and the Y' channel image. At this time, the coordinates of the corresponding pixel in the Y channel image are obtained by multiplying the coordinates of the pixel in the target depth image by 2, that is, shifting it left by 1 bit. In other words, for the pixel with coordinates (1, 1) in the target depth image, it is necessary to extract the pixel value of the pixel with coordinates (2, 2) in the Y channel image and the pixel value of the pixel with coordinates (1, 1) in the Y' channel image.

[0096] 702. Based on the fusion weight coefficients corresponding to the pixels, the pixel values ​​of the corresponding pixels in the Y channel image and the Y' channel image are weighted and summed to obtain the fused pixel value corresponding to the pixel.

[0097] In this embodiment, the aforementioned Figure 5 Taking the calculation method of the fusion weight coefficient shown as an example, combined with the description of steps 501 and 502, it can be seen that, through... Figure 5 The fusion weight coefficients calculated in the steps shown can be understood as the proportion of the original image during the fusion process. Specifically, taking a fusion weight coefficient of α as an example, the weight of the pixel value corresponding to the pixel in the Y channel image is α, and the weight of the pixel value corresponding to the pixel in the Y' channel image is (1-α). That is, by multiplying the pixel values ​​of the corresponding pixels in the Y channel image and the Y' channel image by their corresponding weights and summing them, the fused pixel value corresponding to the pixel can be obtained. Obviously, the fusion weight coefficient α is normalized and has a value range of 0 to 1, so the obtained fused pixel value will also be between 0 and 256.

[0098] Of course, if other methods for calculating the fusion weight coefficients are used, it is also feasible to use the corresponding rules to perform weighted summation of the pixel values ​​of corresponding pixels in the Y channel image and the Y' channel image, which will not be elaborated here.

[0099] 703. Generate a fused Y-channel image based on the fused pixel values ​​corresponding to each pixel.

[0100] In this embodiment, by combining the fused pixel values ​​corresponding to each pixel, a fused Y-channel image can be obtained. That is, the pixel value of each pixel in the fused Y-channel image is equivalent to the fused pixel value corresponding to that pixel. It is easy to understand that the generated fused Y-channel image has the same size as the target depth image. Typically, the obtained fused Y-channel image can be upsampled to obtain a fused Y-channel image of the original size. Upsampling is a conventional technique in this field and will not be elaborated upon here.

[0101] Of course, besides upsampling the fused Y-channel image after obtaining it to get the fused Y-channel image of the original size, other methods can also be used, for example...

[0102] 1) For each pixel in the Y channel image, extract the pixel value of that pixel, the pixel value of the corresponding pixel in the Y' channel image, and the fusion weight coefficient of the corresponding pixel in the target depth image;

[0103] 2) Based on the fusion weight coefficient corresponding to the pixel, the pixel values ​​of the corresponding pixels in the Y channel image and the Y' channel image are weighted and summed to obtain the fused pixel value corresponding to the pixel.

[0104] 3) Generate a fused Y-channel image based on the fused pixel values ​​corresponding to each pixel.

[0105] In this case, considering that the size of the Y-channel image is four times that of the Y'-channel image and the target depth image, the coordinates of the corresponding pixel in the Y'-channel image and the target depth image are half of the coordinates of the pixel in the Y-channel image. This usually requires rounding. For example, pixels with coordinates (1,1), (2,1), (1,2), and (2,2) in the Y-channel image have the corresponding pixel coordinates (1,1) in the Y-channel image.

[0106] It should be noted that while the above describes the method for fusing the Y-channel and Y'-channel images to obtain a fused Y-channel image, the method for fusing the UV-channel and UV'-channel images to obtain a fused UV-channel image can also be achieved through similar steps. Specifically, for each pixel in the target depth image, the pixel values ​​of the corresponding pixels in the UV-channel and UV'-channel images are extracted. Based on the fusion weight coefficients corresponding to the pixels, the pixel values ​​of the corresponding pixels in the UV-channel and UV'-channel images are weighted and summed to obtain the fused pixel value for each pixel. Finally, the fused UV-channel image is generated based on the fused pixel values ​​of each pixel.

[0107] However, it should be noted that since the UV channel image, UV' channel image and target depth image are the same size, that is, the pixels in each image have a one-to-one correspondence. In other words, the corresponding pixel can be obtained directly through coordinates without the need to transform the coordinates.

[0108] This application provides a specific implementation method for fusing the Y channel image and the Y' channel image according to the fusion weight coefficient corresponding to each pixel to obtain the fused Y channel image. By weighting and summing the pixel values ​​of each pixel in the original image and the pixel values ​​of each pixel in the blurred image, different weighting coefficients can achieve different degrees of blurring, thereby improving the blurring effect.

[0109] Specifically, for ease of understanding, the process of calculating the fused Y-channel image in the image blurring method provided in this application embodiment can be described by the following formula:

[0110] bokeh_Y i,j =(Y i,j *D i>>1,j>>1 +Y _i>>1,j>>1 *(256-D i>>1,j>>1 ))>>8

[0111] Among them, bokeh_Y i,j That is, the pixel value of the pixel at coordinates (i, j) in the fused Y channel image. i,j Y_ is the pixel value of the pixel at coordinates (i, j) in the Y-channel image. i,j Let D_ be the pixel value of the pixel at coordinates (i, j) in the Y' channel image. i,j Let be the gray value of the pixel at coordinates (i, j) in the target depth image. Essentially, it can be understood as the fusion weight coefficient corresponding to the pixel. Here, >> means right shift, that is, (i >> 1, j >> 1) is obtained by shifting i and j one position to the right in (i, j).

[0112] The formula for calculating the fused UV channel image is as follows:

[0113] bokeh_UV i,j =(UV) i,j *D i,j +UV _i,j *(256-D i,j ))>>8

[0114] Among them, bokeh_UV i,j This refers to the pixel value of the pixel at coordinates (i, j) in the fused UV channel image. The meanings of the other symbols can be found in the formulas provided in the process of calculating the fused Y channel image.

[0115] like Figure 8 As shown, Figure 8 This is a schematic flowchart of the seventh embodiment of the image blurring method provided in this application.

[0116] In some embodiments of this application, a method is provided for obtaining Y' channel images and UV' channel images by performing image blurring processing on channel images, specifically including steps 801-803:

[0117] 801, obtain the blur intensity parameter.

[0118] In this embodiment, the blurring intensity parameter is usually obtained by user input, and it can typically be represented by L.

[0119] 802. Query the preset database to obtain the fuzzy kernel size and fuzzy variance corresponding to the fuzzing intensity parameter.

[0120] In this embodiment, the image blurring device pre-stores the mapping relationship between blurring intensity parameters and parameters required during blurring processing. Thus, after obtaining the blurring intensity parameters input by the user, the corresponding parameters, such as the blur kernel size and blur variance, can be retrieved. As a feasible solution, the mapping relationship between the pre-stored blurring intensity parameters, blur kernel size, and blur variance in the database is shown in the following table:

[0121] bokeh intensity Gaussian blur kernel size Gaussian fuzzy kernel variance L≤14 3 1.0 14＜L≤28 3 2.0 28＜L≤42 3 3.0 42＜L≤60 3 4.5 60＜L≤80 3 5.0 80＜L≤90 5 1.0 90＜L≤100 5 1.2

[0122] The definitions of fuzzy kernel and fuzzy variance are common knowledge in the field of Gaussian fuzziness, and will not be elaborated here.

[0123] 803. Using the blur kernel size and blur variance as Gaussian blur parameters, Gaussian blur processing is performed on the target Y channel image and UV channel image to obtain the Y' channel image and UV' channel image.

[0124] In this embodiment, the blur kernel size and blur variance obtained from the aforementioned query are used as Gaussian blur parameters. Gaussian blurring of the target Y channel image yields the Y' channel image, and Gaussian blurring of the UV channel image yields the UV' channel image. The specific Gaussian blurring process is a conventional technique in the field of image processing and will not be elaborated upon here.

[0125] This application embodiment pre-defines the mapping relationship between blur intensity parameters, blur kernel size, and blur variance in the database. This allows the image blurring device to query the corresponding blur kernel size and blur variance after obtaining the blur intensity parameters input by the user. This facilitates the application of Gaussian blurring to the target Y channel image and UV channel image, thereby obtaining the Y' channel image and UV' channel image that meet the user's requirements.

[0126] like Figure 9 As shown, Figure 9 This is a schematic diagram of an embodiment of an image blurring device.

[0127] To better implement the image blurring method in the embodiments of this application, based on the image blurring method, the embodiments of this application also provide a structural schematic diagram of an image blurring device, as shown below. Figure 9 As shown, the image blurring device includes:

[0128] Module 901 is provided to provide the Y channel image, UV channel image and corresponding depth image in the YUV preview image;

[0129] The blurring module 902 is used to perform image blurring on the target Y channel image and UV channel image to obtain the Y' channel image and UV' channel image. The target Y channel image is obtained by downsampling the Y channel image.

[0130] The blurring module 903 is used to determine the YUV blurring image based on the target depth image, the Y channel image, the Y' channel image, the UV channel image, and the UV' channel image. The target depth image is obtained by scaling the depth image.

[0131] In some embodiments of this application, the above-mentioned virtualization module further includes:

[0132] The fusion weight coefficient determination submodule is used to determine the fusion weight coefficient corresponding to each pixel in the target depth image based on the pixel's gray value.

[0133] The Y-channel image fusion sub-module is used to fuse the Y-channel image and the Y'-channel image according to the fusion weight coefficients corresponding to each pixel to obtain the fused Y-channel image.

[0134] The UV channel image fusion sub-module is used to fuse the UV channel image and the UV' channel image according to the fusion weight coefficients corresponding to each pixel to obtain a fused UV channel image.

[0135] The combination submodule is used to combine the fused Y channel image and the fused UV channel image to obtain a YUV blurred image.

[0136] In some embodiments of this application, the above-mentioned sub-module for determining fusion weight coefficients includes:

[0137] The focus depth value acquisition unit is used to acquire the focus depth value corresponding to the YUV preview image;

[0138] The difference calculation unit is used to calculate the difference between the gray value and the focus depth value of a pixel;

[0139] The fusion weight coefficient calculation unit is used to calculate the fusion weight coefficient corresponding to the pixel based on the difference.

[0140] In some embodiments of this application, the above-mentioned fusion weight coefficient calculation unit includes:

[0141] The transformation depth value calculation sub-unit is used to input the difference into a preset depth transformation formula to calculate the transformation depth value corresponding to the pixel.

[0142] The normalization processing sub-unit is used to normalize the transformation depth values ​​to obtain the fusion weight coefficients corresponding to the pixels.

[0143] In some embodiments of this application, the above-mentioned fusion weight coefficient calculation unit further includes:

[0144] The virtualization instruction retrieves the next unit, which is used to retrieve the virtualization instruction;

[0145] The database query sub-unit is used to query the preset database to obtain the depth transformation formula corresponding to the bokeh instruction.

[0146] In some embodiments of this application, the above-mentioned Y-channel image fusion sub-module includes:

[0147] The pixel value extraction unit is used to extract the pixel value of the corresponding pixel in the Y channel image and the Y' channel image for each pixel in the target depth image;

[0148] The fused pixel value calculation unit is used to perform a weighted summation of the pixel values ​​of corresponding pixels in the Y channel image and the Y' channel image according to the fusion weight coefficients corresponding to the pixel points, so as to obtain the fused pixel value of the pixel points.

[0149] The Y-channel image generation unit is used to generate a fused Y-channel image based on the fused pixel values ​​corresponding to each pixel.

[0150] In some embodiments of this application, the aforementioned fuzzy module includes:

[0151] The submodule for obtaining blur intensity parameters is used to obtain blur intensity parameters;

[0152] The database query module is used to query a preset database to obtain the fuzzy kernel size and fuzzy variance corresponding to the fuzziness intensity parameter.

[0153] The Gaussian blur processing submodule is used to perform Gaussian blur processing on the target Y-channel image and UV-channel image using the blur kernel size and blur variance as Gaussian blur parameters, to obtain the Y'-channel image and UV'-channel image.

[0154] This invention also provides an electronic device, such as... Figure 10 As shown, Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device includes a memory, a processor, and an image blurring program stored in the memory and capable of running on the processor. When the processor executes the image blurring program, it implements the steps in the image blurring method of any embodiment.

[0155] Specifically, an electronic device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will understand that... Figure 10 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0156] The processor 1001 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 1002, and by calling data stored in the memory 1002, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1001.

[0157] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1002 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.

[0158] The electronic device also includes a power supply 1003 that supplies power to various components. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 1003 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0159] The electronic device may also include an input unit 1004, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0160] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 1001 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 1002 according to the following instructions, and the processor 1001 runs the applications stored in the memory 1002, thereby implementing the steps in any of the image blurring methods provided in the embodiments of the present invention.

[0161] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be accomplished by instructions, or by instructions controlling related hardware. These instructions can be stored in a storage medium and loaded and executed by a processor.

[0162] Therefore, embodiments of the present invention provide a computer storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. An image blurring program is stored on the computer-readable storage medium, and when the image blurring program is executed by a processor, it implements the steps of any of the image blurring methods provided in the embodiments of the present invention.

[0163] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0164] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0165] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0166] The above provides a detailed description of an image blurring method provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An image blurring method, characterized in that, include: Provides the Y channel image, UV channel image, and corresponding depth image in the YUV preview image; Image blurring is performed on the target Y channel image and the UV channel image to obtain the Y' channel image and the UV' channel image, wherein the target Y channel image is obtained by downsampling the Y channel image; A YUV bokeh image is determined based on the target depth image, the Y channel image, the Y' channel image, the UV channel image, and the UV' channel image, wherein the target depth image is obtained by scaling the depth image.

2. The method according to claim 1, characterized in that, The step of determining the YUV bokeh image based on the target depth image, the Y channel image, the Y' channel image, the UV channel image, and the UV' channel image includes: For each pixel in the target depth image, the fusion weight coefficient corresponding to the pixel is determined based on the gray value of the pixel. Based on the fusion weight coefficients corresponding to each pixel, the Y channel image and the Y' channel image are fused to obtain a fused Y channel image. Based on the fusion weight coefficients corresponding to each pixel, the UV channel image and the UV' channel image are fused to obtain a fused UV channel image. The fused Y channel image and the fused UV channel image are combined to obtain a YUV blurred image.

3. The method according to claim 2, characterized in that, The step of determining the fusion weight coefficient corresponding to the pixel based on the gray value of the pixel includes: Obtain the focus depth value corresponding to the YUV preview image; Calculate the difference between the grayscale value of the pixel and the focal depth value; The fusion weight coefficient corresponding to the pixel is calculated based on the difference.

4. The method according to claim 3, characterized in that, The step of calculating the fusion weight coefficient corresponding to the pixel based on the difference includes: The difference is input into a preset depth transformation formula to calculate the transformation depth value corresponding to the pixel. The transformation depth value is normalized to obtain the fusion weight coefficient corresponding to the pixel.

5. The method according to claim 4, characterized in that, Before inputting the difference into a preset depth transformation formula to calculate the transformation depth value corresponding to the pixel, the method further includes: Obtain the virtualization command; The preset database is queried to obtain the depth transformation formula corresponding to the bokeh instruction.

6. The method according to claim 2, characterized in that, The step of fusing the Y-channel image and the Y'-channel image according to the fusion weight coefficients corresponding to each pixel to obtain a fused Y-channel image includes: For each pixel in the target depth image, the pixel value of the corresponding pixel in the Y channel image and the Y' channel image is extracted; Based on the fusion weight coefficient corresponding to the pixel, the pixel values ​​of the corresponding pixels in the Y channel image and the Y' channel image are weighted and summed to obtain the fused pixel value corresponding to the pixel. A fused Y-channel image is generated based on the fused pixel values ​​corresponding to each pixel.

7. The method according to any one of claims 1 to 6, characterized in that, The step of performing image blurring processing on the target Y-channel image and the UV-channel image to obtain the Y'-channel image and the UV'-channel image includes: Obtain the blur intensity parameter; Query the preset database to obtain the fuzzy kernel size and fuzzy variance corresponding to the fuzziness intensity parameter; Using the blur kernel size and the blur variance as Gaussian blur parameters, Gaussian blur processing is performed on the target Y channel image and the UV channel image to obtain the Y' channel image and the UV' channel image.

8. An image blurring device, characterized in that, include: The module provides the Y channel image, UV channel image, and corresponding depth image in the YUV preview image; The blurring module is used to perform image blurring processing on the target Y channel image and the UV channel image to obtain a Y' channel image and a UV' channel image, wherein the target Y channel image is obtained by downsampling the Y channel image; The blurring module is used to determine a YUV blurring image based on the target depth image, the Y channel image, the Y' channel image, the UV channel image, and the UV' channel image, wherein the target depth image is obtained by scaling the depth image.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, and an image blurring program stored in the memory and executable on the processor. When the processor executes the image blurring program, it implements the steps of the image blurring method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image blurring program, which, when executed by a processor, implements the steps of the image blurring method as described in any one of claims 1 to 7.

Images