Image processing methods and apparatuses, electronic devices, storage media

By performing pixel value conversion on the bokeh and background layers and correcting the vignetting mask layer, the problem of reduced brightness in the bokeh area of ​​the blurred image was solved, achieving brightness preservation and natural boundary transition in the vignetting area, thus improving image quality.

CN117677968BActive Publication Date: 2026-11-17BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202280004247.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-20
Publication Date
2026-11-17
Estimated Expiration
2042-06-20

AI Technical Summary

Technical Problem

In existing technologies, when mobile terminal imaging devices process blurred images, the brightness and contrast of the spot area decrease, resulting in poor imaging effects. In particular, when adding vignetting effects, the brightness and contrast of the spot area are significantly reduced, affecting image quality.

Method used

By acquiring the vignetting layer and background layer of the image to be processed, pixel value conversion is performed to obtain the first and second coefficient layers. Based on these coefficient layers, the original vignetting mask layer is processed to correct the vignetting mask layer to preserve the high brightness and contrast of the vignetting area. Finally, the corrected vignetting mask layer is merged with the blurred image to form the target image.

Benefits of technology

When adding a vignetting effect, the brightness and contrast of the vignetting area are maintained to avoid significant attenuation, and the transition between the vignetting area and the background area is smoother and more natural, thus improving image quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117677968B_ABST
    Figure CN117677968B_ABST
Patent Text Reader

Abstract

An image processing method, apparatus, electronic device, and storage medium are disclosed. The image processing method includes acquiring a vignetting layer and a background layer of an image to be processed (S210); obtaining a first coefficient layer and a second coefficient layer based on the vignetting layer and the background layer (S220); processing an original vignetting mask layer based on the first coefficient layer and the second coefficient layer (S230); and fusing a blurred image of the image to be processed based on the processed target vignetting mask layer to obtain a target image (S240). When adding a vignetting effect to the blurred image, the bright pixels can retain good brightness and contrast, and the boundary transition between the vignetting area and the background area is smoother and more natural, improving image quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to an image processing method and apparatus, electronic device, and storage medium. Background Technology

[0002] Vignetting is a technical term in the field of photography, referring to the phenomenon where the corners of an image are darkened when photographing a scene with uniform brightness. In related technologies, for imaging devices such as mobile terminals, software algorithms can be used to add vignetting effects to captured images, thereby mimicking the imaging effect of professional cameras and making the captured images more layered and textured.

[0003] Vignette processing algorithms in related technologies are not effective in processing blurred images, especially for bright areas such as light spots. After vignetting, the brightness and contrast of the light spots decrease, resulting in poor image quality. Summary of the Invention

[0004] To improve image quality, this disclosure provides an image processing method and apparatus, an electronic device, and a storage medium.

[0005] In a first aspect, embodiments of this disclosure provide an image processing method, including:

[0006] Obtain the light spot layer and background layer corresponding to the image to be processed;

[0007] Pixel value conversion is performed on the light spot layer and the background layer respectively to obtain a first coefficient layer corresponding to the light spot layer and a second coefficient layer corresponding to the background layer;

[0008] Based on the first coefficient layer and the second coefficient layer, the original vignetting mask layer is processed to obtain the target vignetting mask layer; wherein, the pixel value of the spot area in the target vignetting mask layer is not less than the pixel value of the corresponding spot area in the original vignetting mask layer;

[0009] The blurred image of the image to be processed is fused based on the target vignetting mask layer to obtain the target image corresponding to the image to be processed.

[0010] In some embodiments, the step of converting the pixel values ​​of the light spot layer and the background layer respectively to obtain a first coefficient layer corresponding to the light spot layer and a second coefficient layer corresponding to the background layer includes:

[0011] The pixel values ​​of the valid pixels in the light spot layer are set to a first value, and the pixel values ​​of the invalid pixels in the light spot layer are set to a second value, thereby obtaining the first coefficient layer corresponding to the light spot layer;

[0012] The pixel values ​​of valid pixels in the background layer are set to a first value, and the pixel values ​​of invalid pixels in the background layer are set to a second value, thus obtaining the second coefficient layer corresponding to the background layer.

[0013] In some implementations, processing the original vignetting mask layer based on the first coefficient layer and the second coefficient layer to obtain the target vignetting mask layer includes:

[0014] The first coefficient layer and the second coefficient layer are filtered respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer;

[0015] Based on the first and second filter coefficient layers, the original vignetting mask layer is processed to obtain the target vignetting mask layer.

[0016] In some embodiments, processing the original vignetting mask layer based on the first filter coefficient layer and the second filter coefficient layer to obtain the target vignetting mask layer includes:

[0017] For any pixel in the original vignetting mask layer, the pixel weight corresponding to the pixel is determined based on the pixel value in the first filter coefficient layer and the pixel value in the second filter coefficient layer; wherein, the pixel weight of the pixel in the spot area is greater than the pixel weight of the pixel in the background area;

[0018] The target pixel value is determined based on the pixel weight and the original pixel value of the pixel in the original vignetting mask layer;

[0019] The target vignetting mask layer is obtained based on the target pixel value of each pixel.

[0020] In some embodiments, the method described in this disclosure further includes:

[0021] A bokeh layer is obtained by filtering the bokeh layer based on the first coefficient layer, and a background bokeh layer is obtained by filtering the background layer based on the second coefficient layer.

[0022] The first coefficient layer and the second coefficient layer are filtered respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer;

[0023] Based on the first filter coefficient layer and the second filter coefficient layer, the light spot blur layer and the background blur layer are fused to obtain the target blur layer;

[0024] The target blurred layer and the non-blurred layer of the image to be processed are fused together to obtain the blurred image of the image to be processed.

[0025] In some implementations, obtaining the spot layer and background layer corresponding to the image to be processed includes:

[0026] The image to be processed is segmented to obtain the region to be blurred and the region not blurred included in the image to be processed;

[0027] The region to be blurred is segmented to obtain the light spot layer and the background layer.

[0028] In some implementations, the spot layer includes the spot area of ​​the region to be blurred, and the background layer includes the region of the region to be blurred other than the spot area.

[0029] In a second aspect, embodiments of this disclosure provide an image processing apparatus, including:

[0030] The acquisition module is configured to acquire the spot layer and background layer corresponding to the image to be processed.

[0031] The coefficient layer module is configured to perform pixel value conversion on the light spot layer and the background layer respectively to obtain a first coefficient layer corresponding to the light spot layer and a second coefficient layer corresponding to the background layer;

[0032] The mask processing module is configured to process the original vignetting mask layer based on the first coefficient layer and the second coefficient layer to obtain a target vignetting mask layer; wherein the pixel value of the spot area in the target vignetting mask layer is not less than the pixel value of the corresponding spot area in the original vignetting mask layer.

[0033] The layer blending module is configured to blend the blurred image of the image to be processed based on the target vignetting mask layer to obtain the target image corresponding to the image to be processed.

[0034] In some implementations, the acquisition module is configured to:

[0035] The image to be processed is segmented to obtain the region to be blurred and the region not blurred included in the image to be processed;

[0036] The region to be blurred is segmented to obtain the light spot layer and the background layer.

[0037] In some implementations, the spot layer includes the spot area of ​​the region to be blurred, and the background layer includes the region of the region to be blurred other than the spot area.

[0038] In some implementations, the coefficient layer module is configured as follows:

[0039] The pixel values ​​of the valid pixels in the light spot layer are set to a first value, and the pixel values ​​of the invalid pixels in the light spot layer are set to a second value, thereby obtaining the first coefficient layer corresponding to the light spot layer;

[0040] The pixel values ​​of valid pixels in the background layer are set to a first value, and the pixel values ​​of invalid pixels in the background layer are set to a second value, thus obtaining the second coefficient layer corresponding to the background layer.

[0041] In some embodiments, the mask processing module is configured to:

[0042] The first coefficient layer and the second coefficient layer are filtered respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer;

[0043] Based on the first and second filter coefficient layers, the original vignetting mask layer is processed to obtain the target vignetting mask layer.

[0044] In some embodiments, the mask processing module is configured to:

[0045] For any pixel in the original vignetting mask layer, the pixel weight corresponding to the pixel is determined based on the pixel value in the first filter coefficient layer and the pixel value in the second filter coefficient layer; wherein, the pixel weight of the pixel in the spot area is greater than the pixel weight of the pixel in the background area;

[0046] The target pixel value is determined based on the pixel weight and the original pixel value of the pixel in the original vignetting mask layer;

[0047] The target vignetting mask layer is obtained based on the target pixel value of each pixel.

[0048] In some embodiments, the image processing apparatus of this disclosure further includes a filtering module, which is configured to:

[0049] A bokeh layer is obtained by filtering the bokeh layer based on the first coefficient layer, and a background bokeh layer is obtained by filtering the background layer based on the second coefficient layer.

[0050] The first coefficient layer and the second coefficient layer are filtered respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer;

[0051] Based on the first filter coefficient layer and the second filter coefficient layer, the light spot blur layer and the background blur layer are fused to obtain the target blur layer;

[0052] The target blurred layer and the non-blurred layer of the image to be processed are fused together to obtain the blurred image of the image to be processed.

[0053] In some implementations, the filtering module is configured to:

[0054] A first filtering process is performed on the background layer and the background coefficient layer to obtain the background blur layer and the first coefficient layer;

[0055] The light spot layer and the highlight coefficient layer are subjected to a second filtering process to obtain the highlight blur layer and the second coefficient layer; the first filtering process is different from the second filtering process.

[0056] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0057] processor; and

[0058] A memory storing computer instructions for causing the processor to perform the method according to any embodiment of the first aspect.

[0059] Fourthly, embodiments of this disclosure provide a storage medium storing computer instructions for causing a computer to perform the method described according to any embodiment of the first aspect.

[0060] The image processing method of this disclosure includes acquiring a vignetting layer and a background layer of an image to be processed; obtaining a first coefficient layer and a second coefficient layer based on the vignetting layer and the background layer; processing the original vignetting mask layer based on the first coefficient layer and the second coefficient layer to obtain a target vignetting mask layer; and performing a fusion processing on the blurred image of the image to be processed based on the target vignetting mask layer to obtain a target image. In this disclosure, the coefficient layer is used to correct the vignetting mask layer, so that when adding a vignetting effect to a blurred image, the bright pixels in the corner areas retain good brightness and contrast without significant attenuation, and the boundary transition between the vignetting area and the background area is smoother and more natural, improving image quality. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0062] Figure 1 This is a schematic diagram illustrating the principle of adding a vignetting effect to an image in related technologies.

[0063] Figure 2 This is a flowchart of an image processing method according to some embodiments of the present disclosure.

[0064] Figure 3 These are example images of the images to be processed according to some embodiments of this disclosure.

[0065] Figure 4 This is a flowchart of an image processing method according to some embodiments of the present disclosure.

[0066] Figure 5 This is a schematic diagram illustrating the principle of an image processing method according to some embodiments of this disclosure.

[0067] Figure 6 This is a schematic diagram illustrating the principle of an image processing method according to some embodiments of this disclosure.

[0068] Figure 7 This is a schematic diagram illustrating the principle of an image processing method according to some embodiments of this disclosure.

[0069] Figure 8 This is a flowchart of an image processing method according to some embodiments of the present disclosure.

[0070] Figure 9 This is a schematic diagram illustrating the principle of an image processing method according to some embodiments of this disclosure.

[0071] Figure 10 This is a schematic diagram illustrating the principle of an image processing method according to some embodiments of this disclosure.

[0072] Figure 11 This is a flowchart of an image processing method according to some embodiments of the present disclosure.

[0073] Figure 12 This is a flowchart of an image processing method according to some embodiments of the present disclosure.

[0074] Figure 13 This is a schematic diagram illustrating the principle of an image processing method according to some embodiments of this disclosure.

[0075] Figure 14This is a schematic diagram illustrating the principle of an image processing method according to some embodiments of this disclosure.

[0076] Figure 15 This is a schematic diagram illustrating the principle of an image processing method according to some embodiments of this disclosure.

[0077] Figure 16 This is a structural block diagram of an image processing apparatus according to some embodiments of the present disclosure.

[0078] Figure 17 This is a structural block diagram of an electronic device according to some embodiments of the present disclosure. Detailed Implementation

[0079] The technical solutions of this disclosure will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. Furthermore, the technical features involved in the different embodiments of this disclosure described below can be combined with each other as long as they do not conflict with each other.

[0080] Vignetting is a technical term in the field of photography, referring to the phenomenon where the corners of an image are darkened when photographing a scene with uniform brightness. In related technologies, for imaging devices such as mobile terminals, software algorithms can be used to add vignetting effects to captured images, thereby mimicking the imaging effect of professional cameras and making the captured images more layered and textured.

[0081] Background blurring refers to blurring the background of an image to make foreground elements, such as people or objects, stand out more in the overall image. In related technologies, most imaging algorithms do not add vignetting effects to blurred images, which can cause the image to lose depth and texture, resulting in poor image quality. Some imaging algorithms, after obtaining the blurred image, directly apply a fixed vignetting mask template to add a vignetting effect. However, this can lead to a decrease in brightness and contrast in the bright areas at the four corners of the image, reducing image quality.

[0082] For example, in one example, the original image is as follows: Figure 1 As shown in (a), the blurred image obtained after background blurring of the original image is as follows: Figure 1 As shown in (b). Through Figure 1 As can be seen in (b), there are many light spots in the upper left and upper right corners of the image. The image areas where these light spots are located are the light spot areas described in this disclosure.

[0083] Vignette masking Figure 1As shown in (c), the pixel value decreases from 255 to 0 from the image center towards the four corners. In related technologies, the vignetting mask is directly fused into... Figure 1 In the blurred image shown in (b), the following is obtained: Figure 1 The vignetting image shown in (d) is compared. Figure 1 As shown in (d) and (b), after fusing the vignetting mask, the brightness and contrast of the light spots in the upper left and upper right corners decrease significantly, resulting in a poorer light spot effect in the four corner areas and an unsatisfactory imaging effect. Furthermore, if the vignetting mask is overlaid on the background area through image segmentation, and then the light spot layer is overlaid on the vignetting image, the transition at the edges of the light spots becomes unnatural, further degrading the image quality.

[0084] Based on the deficiencies in the aforementioned related technologies, this disclosure provides an image processing method, apparatus, electronic device, and storage medium, aiming to improve image quality by ensuring the brightness and contrast of bright areas while achieving the effect of vignetting.

[0085] First, it is worth noting that this disclosure involves image processing, and the consent of all individuals appearing in this document has been obtained. Furthermore, in this embodiment, for the purpose of protecting personal privacy, all facial areas are blurred; however, in practical applications, blurring of facial areas is unnecessary. Those skilled in the art will understand this, and it will not be elaborated further here.

[0086] In a first aspect, the present disclosure provides an image processing method that can be applied to an electronic device. The electronic device in the present disclosure can be any type of device suitable for implementation, such as a smartphone, tablet computer, wearable device, personal computer, vehicle-mounted device, server, and cloud platform, etc. The present disclosure does not limit this.

[0087] like Figure 2 As shown, in some embodiments, the image processing method of this disclosure includes:

[0088] S210. Obtain the light spot layer and background layer corresponding to the image to be processed.

[0089] In this embodiment of the disclosure, the image to be processed refers to an image that needs to be blurred. It can be an image captured by the image acquisition device of an electronic device, such as a smartphone, which can capture the image to be processed by the camera of the smartphone; it can also be an image sent by other devices received by the electronic device, such as a smartphone, which can receive the image to be processed sent by an external device; it can also be an image downloaded by the electronic device through the network; this disclosure does not limit it.

[0090] It can be understood that the image to be processed includes a region to be blurred, which refers to the image area that needs to be blurred. This region can be the entire image area or only a portion of it. When the region to be blurred is the entire image area, it means that the entire image needs to be blurred.

[0091] When the area to be blurred is only a portion of the image to be processed, the layer to be processed also includes the non-blurred areas. The non-blurred areas refer to image areas that do not require blurring. The non-blurred areas and the area to be blurred together constitute the entire image area to be processed.

[0092] For example, in a portrait photography scenario, it is necessary to highlight the portrait area and fade the background area outside the portrait in the image. Thus, the portrait area in the image to be processed is the non-blurred area, and the other areas are the areas to be blurred.

[0093] In this embodiment of the disclosure, the area to be blurred includes a spot area and a background area other than the spot area. By separating the spot area and the background area into separate layers, the corresponding spot layer and background layer can be obtained.

[0094] For example Figure 3 In the scene image shown, the area of ​​the person in the foreground is the non-blurred area, and the other areas outside the person area are the areas to be blurred. The areas to be blurred include light spots formed by light or glass reflection, and the background area excluding the light spots. In this embodiment, image segmentation technology can be used to segment the areas to be blurred, separating the light spot area and the background area to form a light spot layer corresponding to the light spot area and a background layer corresponding to the background area.

[0095] In this embodiment of the disclosure, the image obtained after blurring the image to be processed is the blurred image, and the image obtained after fusing the blurred image with the target vignetting mask layer is the target image. The segmentation process of the spot layer and the background layer and the image blurring process are described in detail in the following embodiments of the disclosure, and will not be described in detail here.

[0096] S220. Perform pixel value conversion on the spot layer and the background layer respectively to obtain the first coefficient layer corresponding to the spot layer and the second coefficient layer corresponding to the background layer.

[0097] In this embodiment of the disclosure, after performing image segmentation on the region to be blurred to obtain a spot layer and a background layer, a corresponding first coefficient layer can be obtained based on the spot layer, and a corresponding second coefficient layer can be obtained based on the background layer.

[0098] The coefficient layer is used to assign appropriate coefficient values ​​to each pixel in the bokeh layer and the background layer, thereby enabling a smooth transition between the highlighted and background areas when blending the bokeh and background layers. For example, as described above... Figure 3 In the example, the high contrast of the bokeh area can be preserved, and the boundary between the bokeh layer and the background layer can have a smooth transition effect, resulting in a more natural imaging effect.

[0099] Furthermore, since the coefficient layer can reflect the blending weights of the bokeh layer and the background layer, that is, the first and second coefficient layers can be used to effectively quantify the similarity between each pixel in the blurred image and the bokeh region. Therefore, in this embodiment, the first and second coefficient layers can be used to... Figure 1 The vignetting mask layer shown in (c) is modified so that it blends with the blurred image, allowing the highlight areas to retain good brightness and contrast. This is described in detail below.

[0100] In some implementations, since the spot layer and background layer are partial image regions segmented from the image, they contain both valid and invalid pixels. In this embodiment, pixel value conversion can be performed on each layer based on its valid and invalid pixel values ​​to obtain the corresponding coefficient layer.

[0101] For example, taking a bokeh layer as an example, the valid pixel values ​​in the bokeh layer can be converted into a first value, and the invalid pixel values ​​in the bokeh layer can be converted into a second value. The resulting first coefficient layer is the image represented by the first and second values. The process of obtaining the second coefficient layer from the background layer is the same, which will be described in the following embodiments of this disclosure, and will not be detailed here.

[0102] S230. Based on the first coefficient layer and the second coefficient layer, process the original vignetting mask layer to obtain the target vignetting mask layer.

[0103] The original vignetting mask layer is Figure 1 The image shown in (c) has its grayscale value decreasing from 255 to 0 from the center towards the four corners. It can be seen that the original vignetting mask layer does not consider the content of the blurred image; that is, regardless of whether a pixel in the blurred image is a pixel in the vignetting area or a background pixel, it is treated as... Figure 1 The pixel values ​​of the original vignetting mask layer shown in (c) are superimposed and blended onto this pixel. This results in a significant loss of brightness and contrast when the bright pixels in the corner area are superimposed on the mask template pixel values.

[0104] Therefore, in this embodiment of the present disclosure, as described above, the similarity between each pixel in the blurred image and the light spot area can be quantified through the first coefficient layer and the second coefficient layer, which can effectively reflect the brightness level of each pixel in the blurred image. Thus, based on the first coefficient layer and the second coefficient layer, the original vignetting mask layer can be corrected to obtain the target vignetting mask layer.

[0105] For example, for pixels belonging to the vignetting region in the blurred image, the corresponding pixel value in the target vignetting mask layer also needs to be increased to be closer to 255. This way, after merging the target vignetting mask layer, the brightness and contrast of this pixel value will not be significantly lost, thus preserving the highlight effect. In other words, the pixel value of the vignetting region in the target vignetting mask layer is not less than the corresponding pixel value of the vignetting region in the original vignetting mask layer.

[0106] The principle has been explained above. In this step, the specific process of correcting the original vignetting mask layer to obtain the target vignetting mask layer will be described in the following implementation method, and will not be elaborated here.

[0107] S240. Based on the target vignetting mask layer, perform fusion processing on the blurred image of the image to be processed to obtain the target image corresponding to the image to be processed.

[0108] In this embodiment, the target vignetting mask layer is a vignetting mask that is specifically modified based on the bright pixels of the blurred image. After the target vignetting mask layer is fused with the blurred image, the pixels belonging to the spot area can still retain the bright effect without significant brightness and contrast attenuation.

[0109] In some implementations, traditional image fusion algorithms can be used to fuse the target vignetting mask layer and the blurred image. Those skilled in the art can undoubtedly understand and fully implement this by referring to the relevant technologies, and this disclosure will not elaborate further.

[0110] As can be understood, a blurred image refers to the image obtained after blurring the image to be processed, for example... Figure 1 As shown, Figure 1 Image (a) is the image to be processed. Figure 1 Image (b) is the blurred image obtained after blurring the background of image (a). In some embodiments of this disclosure, the image to be blurred can be processed using traditional image blurring methods. In other embodiments, the process of blurring the image to be blurred can also be performed based on the first coefficient layer and the second coefficient layer, which will be described in detail in the embodiments below.

[0111] As can be seen from the above, in this embodiment of the present disclosure, the coefficient layer is used to correct the vignetting mask layer, so that when adding a vignetting effect to the blurred image, the bright pixels in the corner area can retain good brightness and contrast without significant attenuation, and the boundary transition between the vignetting area and the background area is smoother and more natural, thus improving the image quality.

[0112] In the embodiments described below, unless otherwise specified, the following will be used as... Figure 3 The image shown is used as the image to be processed to illustrate the image processing method of this disclosure.

[0113] like Figure 4 As shown, in some embodiments, the image processing method of this disclosure, including the process of image segmentation of the image to be processed, includes:

[0114] S410. Perform image segmentation on the image to be processed to obtain the regions to be blurred and the regions not blurred in the image to be processed.

[0115] For example, in Figure 3 The image to be processed shown includes the area of ​​people and other areas besides people. The area of ​​people is the non-blurred area that does not need to be blurred, while the other areas besides people are the areas to be blurred.

[0116] In some implementations, the image to be processed can be segmented based on image segmentation algorithms in related technologies to obtain non-blurred regions and regions to be blurred. Image segmentation algorithms include, but are not limited to, traditional image segmentation algorithms, deep learning-based portrait segmentation algorithms, or physical depth-based image segmentation algorithms, etc. Those skilled in the art can understand and fully implement them, and this disclosure will not elaborate further.

[0117] In this example, for Figure 3 The non-blurred region obtained after image segmentation of the image to be processed is shown below. Figure 5 As shown in (a), the region to be blurred is as follows: Figure 5 As shown in (b).

[0118] S420. Divide the area to be blurred into layers to obtain a light spot layer and a background layer.

[0119] In some implementations, when performing image segmentation on the region to be blurred, the region can be segmented based on the brightness values ​​of each pixel in the region to be blurred, resulting in a spot layer and a background layer.

[0120] For example Figure 5The blurred area shown in (b) includes a light spot area formed by light or glass reflection and a background area other than the light spot area. The brightness value of each pixel in the light spot area is higher than the brightness value of each pixel in the background area.

[0121] Therefore, in this embodiment of the disclosure, an image segmentation method based on grayscale thresholding can be used, firstly... Figure 5 The area to be blurred, as shown in (b), is converted to a grayscale image. Then, pixels with values ​​greater than a preset threshold are identified as light spot region pixels. The area to be blurred is divided into light spot region and background region, resulting in corresponding light spot layer and background layer. In this example, the obtained light spot layer is as follows: Figure 6 As shown in (b), the background layer is as follows Figure 6 As shown in (a).

[0122] In the embodiments described below, this disclosure will use... Figure 6 Using the light spot layer and background layer shown as examples, the image processing method of this disclosure example will be explained.

[0123] In some embodiments, the image processing method of this disclosure, which obtains a first coefficient layer and a second coefficient layer, includes:

[0124] Set the pixel values ​​of the valid pixels in the bokeh layer to the first value, and set the pixel values ​​of the invalid pixels in the bokeh layer to the second value to obtain the first coefficient layer corresponding to the bokeh layer;

[0125] Set the pixel values ​​of valid pixels in the background layer to the first value, and set the pixel values ​​of invalid pixels in the background layer to the second value to obtain the second coefficient layer corresponding to the background layer.

[0126] Specifically, with Figure 6 Taking the background layer shown in (a) as an example, the pixels of the portrait area and the light spot area removed by image segmentation are all invalid pixels, while the pixels of the remaining background area are valid pixels. Therefore, the pixel values ​​of the valid pixels can be set to the first value, and the pixel values ​​of the invalid pixels can be set to the second value, thus obtaining the second coefficient layer corresponding to the background layer.

[0127] Similarly, with Figure 6 Taking the bokeh layer shown in (b) as an example, the pixels in the portrait area and background area removed by image segmentation are invalid pixels, while the pixels in the remaining bokeh area are valid pixels. Therefore, the pixel values ​​of the valid pixels can be set to the first value, and the pixel values ​​of the invalid pixels can be set to the second value, thus obtaining the first coefficient layer corresponding to the bokeh layer.

[0128] In some implementations, the first value can be set to 1, and the second value can be set to 0. Therefore, Figure 6The second coefficient layer corresponding to the background layer shown in (a) can be as follows: Figure 7 As shown in (a), Figure 6 The first coefficient layer corresponding to the light spot layer shown in (b) can be as follows: Figure 7 As shown in (b).

[0129] It is worth noting that the image processing method of this disclosure aims to add a vignetting effect to a blurred image of the image to be processed. Therefore, the method of this disclosure includes two processes: first, blurring the image to be processed to obtain a blurred image; and second, correcting the vignetting mask layer to obtain a target vignetting mask layer. After obtaining the blurred layer and the target vignetting mask layer, the two are blended to achieve the vignetting effect of the blurred layer of the image to be processed.

[0130] In some implementations, for example Figure 3 The image to be processed shown can be blurred using a traditional background blurring algorithm. For example, the blurred area can be divided into a spot area and a background area, and then the two areas can be filtered separately and fused to obtain a blurred area with a blurred effect. This blurred area can then be fused with the non-blurred area of ​​the portrait to obtain the final blurred image. Those skilled in the art can understand and fully implement this by referring to relevant technologies, and this disclosure will not elaborate further.

[0131] However, in the aforementioned traditional methods, a clear boundary appears between the light spot area and the background area in the blurred image, resulting in a harsh and unnatural transition of the light spot and a poor image blurring effect. Therefore, in some other embodiments of this disclosure, the light spot layer and the background layer can also be blurred based on the aforementioned first coefficient layer and second coefficient layer, resulting in a smoother transition between the light spot and the background boundary in the blurred image, thus improving the image blurring effect.

[0132] That is, in some embodiments of this disclosure, the blurring process of the image to be processed and the correction process of the original vignetting mask layer are both based on the first coefficient layer and the second coefficient layer, which will be described below.

[0133] First of all, Figure 3 The process of blurring the image shown to obtain a blurred image is explained.

[0134] like Figure 8 As shown, in some embodiments, the image processing method of this disclosure, in the process of blurring the image to be processed, includes:

[0135] S810. Filter the spot layer based on the first coefficient layer to obtain a spot blur layer, and filter the background layer based on the second coefficient layer to obtain a background blur layer.

[0136] S820. Filter the first coefficient layer and the second coefficient layer respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer.

[0137] In this embodiment, filtering processing is required for the spot layer, background layer, first coefficient layer, and second coefficient layer. In some embodiments, a first filtering process can be performed on the spot layer and the first coefficient layer to obtain a corresponding spot blur layer and a first filter coefficient layer. A second filtering process can be performed on the background layer and the second coefficient layer to obtain a corresponding background blur layer and a second filter coefficient layer.

[0138] Image filtering is the process of blurring or bokeh an image. In this embodiment, no specific filtering algorithm is limited. The first and second filtering processes can employ any suitable filtering algorithm, such as Gaussian filtering, mean filtering, median filtering, linear filtering, and other specific shape filtering. Furthermore, the first and second filtering processes can be the same filtering algorithm or different filtering algorithms, and this disclosure does not impose any restrictions on this.

[0139] It's worth noting that, taking the bokeh layer as an example, filtering the first coefficient layer makes the pixel values ​​in it smoother. Each pixel value in the first filtered coefficient layer can then be used as the coefficient value for that pixel in the corresponding bokeh layer. The same applies to the second coefficient layer. The resulting coefficient values ​​are used to subsequently blend the weight values ​​in the highlight blur layer and the background blur layer, thus making the transition between the highlight blur layer and the background blur layer smoother and more natural.

[0140] In some implementations, the filtering process for the spot layer and the background layer can be expressed by equation (1):

[0141] Result i =∑w i *mask i *PixelVal i (1)

[0142] The filtering process for the first coefficient layer and the second coefficient layer can be expressed by equation (2):

[0143] Result i =∑w i *mask i (2)

[0144] In equations (1) and (2), Result iw represents the filtered pixel value of the i-th pixel. i The mask represents the coefficients of the filter kernel. i PixelVal represents the pixel value of the i-th pixel in the coefficient layer. i This represents the pixel value of the i-th pixel in the original layer.

[0145] Using equations (1) and (2) above, we can... Figure 6 The background layer shown in (a) is filtered to obtain the following result: Figure 9 The background blur layer shown in (a) is represented as:

[0146] ∑w bi *Backmask i *Background i

[0147] Among them, w bi Backmask represents the coefficients of the filter kernel that performs the second filtering process. i This represents the second coefficient layer, Background. i This represents the background layer.

[0148] right Figure 6 The light spot layer shown in (b) is filtered to obtain the following result: Figure 9 The bokeh layer shown in (b) is represented as follows:

[0149] ∑w si *Spotmask i *Spot i

[0150] Among them, w si Spotmask represents the coefficients of the filter kernel that undergoes the first filtering process. i This represents the first coefficient layer, Spot. i This represents the light spot layer.

[0151] At the same time, Figure 7 The second coefficient layer shown in (a) is filtered to obtain the following result: Figure 9 The second filter coefficient layer shown in (c) is represented as: ∑w bi *Backmask i .right Figure 7 The first coefficient layer shown in (b) is filtered to obtain the following result: Figure 9 The first filter coefficient layer shown in (d) is represented as: ∑w si *Spotmask i .

[0152] It is understood that the first and second filtering processes described above can use the same filtering algorithm or different filtering algorithms. When the first and second filtering processes use the same filtering algorithm, w in the above formula... bi With w si The same is acceptable; this disclosure will not elaborate further.

[0153] S830. Based on the first and second filter coefficient layers, the light spot blur layer and the background blur layer are fused to obtain the target blur layer.

[0154] As described above, each pixel value in the first filter coefficient layer represents the weight coefficient corresponding to the pixel of the light spot layer during the bokeh layer blending. Similarly, each pixel value in the second filter coefficient layer represents the weight coefficient corresponding to the pixel of the background layer during the bokeh layer blending.

[0155] Therefore, after obtaining the bokeh layer and the background blur layer, all blur layers can be blended based on the first and second filter coefficient layers, as shown below:

[0156]

[0157] In equation (3) above, R i Indicates the target blurred layer, ∑w bi *Backmask i *Background i This indicates a background blur layer, ∑w si *Spotmask i *Spot i This represents a bokeh layer, ∑w bi *Backmask i This represents the second filter layer, ∑w si *Spotmask i This represents the first filter coefficient layer.

[0158] By using equation (3) to fuse the background blur layer and the light spot blur layer, the light spot area of ​​the area to be blurred can be merged with the background area to obtain the image of the complete blurred area, which is the target blur layer.

[0159] S840. The target blurred layer and the non-blurred layer of the image to be processed are fused to obtain the blurred image of the image to be processed.

[0160] After obtaining the target blurred layer, it can be image-fused with the corresponding non-blurred layer of the non-blurred area to obtain the target image. For example, in the example above, the target blurred layer can be fused with... Figure 5By merging the portrait layers shown in (a), the final target image can be obtained.

[0161] In one example, the above Figure 3 After the image to be processed using the image processing method of this disclosure is shown, the resulting target image can be as follows: Figure 10 As shown. See also Figure 10 As can be seen, the blurred image obtained using the image processing method of this disclosure retains the brightness and contrast of the bokeh area very well, making the entire portrait image more atmospheric and improving the image quality. Furthermore, the boundary between the highlight area of ​​the bokeh and the background area has a very smooth transition, resulting in a more natural bokeh effect.

[0162] As can be seen from the above, in this embodiment of the present disclosure, a weight coefficient is assigned to each pixel in the spot layer and the background layer through a coefficient layer, thereby making the boundary between the spot layer and the background layer have a better fusion effect and a smoother transition. While preserving the brightness and contrast of the spot layer itself, the transition of the fusion boundary between different layers is more natural, thus improving the imaging effect.

[0163] In obtaining Figure 10 After blurring the image as shown, a vignetting effect needs to be added to the blurring layer. The process of obtaining the target vignetting mask layer is explained below.

[0164] like Figure 11 As shown, in some embodiments, the image processing method of this disclosure, the process of obtaining a target vignetting mask layer, includes:

[0165] S1110. Filter the first coefficient layer and the second coefficient layer to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer.

[0166] S1120. Based on the first filter coefficient layer and the second filter coefficient layer, process the original vignetting mask layer to obtain the target vignetting mask layer.

[0167] In step S1110, the filtering process for the first coefficient layer and the second coefficient layer is described in detail in step S810 above for those skilled in the art.

[0168] In obtaining such Figure 9 The second filter coefficient layer shown in (c) and as shown in the figure Figure 9 After the first filter coefficient layer shown in (d), the results can be applied based on both. Figure 1 The original vignetting mask layer shown in (c) is corrected. The following section combines... Figure 12 Please provide an explanation.

[0169] like Figure 12 As shown, in some embodiments, the image processing method of this disclosure, the process of obtaining a target vignetting mask layer, includes:

[0170] S1121. For any pixel in the original vignetting mask layer, determine the pixel weight corresponding to the pixel based on the pixel value in the first filter coefficient layer and the pixel value in the second filter coefficient layer.

[0171] S1122. Determine the target pixel value based on the pixel weight and the original pixel value of the pixel in the original vignetting mask layer.

[0172] S1123. Obtain the target vignetting mask layer based on the target pixel value of each pixel.

[0173] It's understandable that the process of correcting the original vignetting mask layer is equivalent to processing it pixel by pixel. Figure 1 The process of adjusting the pixel values ​​in the original vignetting mask layer shown in (c) will be explained below using a single pixel i as an example.

[0174] As mentioned above, for pixel i, its pixel value in the second filter coefficient layer is w. bi *Backmask i Its pixel value in the first filter coefficient layer is w si *Spotmask i The pixel weight corresponding to this pixel is represented as:

[0175]

[0176] In equation (4) above, w 0i This represents the pixel weight of pixel i.

[0177] It is understandable that if the pixel value of pixel i is closer to 255, then the pixel value w in the first filter coefficient layer will be higher. si *Spotmask i The larger the pixel value w in the second filter coefficient layer, the higher the pixel value w. bi *Backmask i The smaller the value, the closer the pixel weight calculated according to equation (4) is to 1. Conversely, if the pixel value of pixel i is closer to 0, then the pixel value w in the first filter coefficient layer will be closer to 1. si *Spotmask i The smaller the value, the higher the pixel value w in the second filter coefficient layer. bi *Backmask i The larger the value, the closer the pixel weight calculated according to equation (4) is to 0. Therefore, each pixel can be accurately assigned a pixel weight value through equation (4).

[0178] After obtaining the pixel weight w of pixel i 0i Then, based on the pixel weight w 0i right Figure 1 The pixel value of pixel i in the original vignette mask layer shown in (c) is adjusted, as follows:

[0179] spotShadingMask i =w 0i *(255-shadingMaskOrig i )+shadingMaskOrig i (5)

[0180] In formula (5), spotShadingMask i w represents the target pixel value of pixel i. 0i ShadingMaskOrig represents the pixel weight of pixel i. i This represents the original pixel value of pixel i in the original vignetting mask layer.

[0181] The above describes the processing of the original vignette mask layer using one pixel i as an example. By performing the above processing on each pixel in the original vignette mask layer, the target pixel value of each pixel can be obtained. Based on the target pixel value of each pixel, the target vignette mask layer can be obtained.

[0182] For example Figure 3 In the example, for Figure 1 The target vignetting mask layer after correcting the original vignetting mask layer shown in (c) can be as follows: Figure 13 As shown. Comparison Figure 1 (c) and Figure 13 As shown, the target vignetting mask layer, after being corrected by the method disclosed in this paper, has a good effect on... Figure 10 The light spot areas in the blurred image shown have been corrected using pixel values.

[0183] In obtaining Figure 10 The blurred image shown, and Figure 13 After applying the target vignetting mask layer as shown, the blurred image can be blended based on the target vignetting mask layer, as represented as:

[0184] result i =val i *spotShadingMask i / 255 (6)

[0185] In equation (6), result iVal represents the pixel value of pixel i in the target image. i Indicates that pixel i is in Figure 10 The pixel values ​​in the blurred image shown, spotShadingMask i Indicates that pixel i is in Figure 13 The pixel values ​​in the target vignetting mask layer shown.

[0186] Using the formula (10) above, each pixel in the blurred image can be fused to obtain the target image. For example, in the example above, Figure 10 The blurred image shown is Figure 13 After the target vignetting mask layers are blended, the resulting target image is as shown. Figure 14 As shown. Comparison Figure 10 and Figure 14 It is evident that a vignetting effect has been added to the four corners of the image, and the brightness and contrast of the light spots in the corner areas have not decreased significantly.

[0187] As can be seen from the above, in this embodiment of the present disclosure, while ensuring that the brightness and contrast of the bright areas in the corners of the image are not reduced, a vignetting effect is added to the image to improve the image's layering and texture, thereby improving the imaging quality.

[0188] Additionally, it's worth noting that if the goal is simply to add a vignetting effect to the image without sacrificing the brightness and contrast of the light spots, then the light spot layer can be separated from the background layer and blended directly onto the background layer. Figure 1 The original vignetting mask layer is shown in (c), and then the bokeh layer is merged into the top layer. Since the bokeh layer is not involved in the fusion of the vignetting mask layer, it can maintain the original good brightness and contrast. However, in this approach, because the bokeh layer is directly merged into the top layer of the image, the boundary transition between the bokeh and the background is abrupt, resulting in an unnatural bokeh area.

[0189] In this embodiment, the pixel weights are calculated using the first and second filter coefficient layers after filtering, and the original vignetting mask layer is corrected, which can make the transition between the light spot and the background very smooth, making the light spot area more natural.

[0190] For example Figure 15 Image (a) shows the image effect obtained by directly merging the light spot layer onto the top layer of the image. Figure 15 Image (b) shows the image effect obtained using the image processing method of this disclosure. Comparing the two, it can be seen that the target image obtained by the method of this disclosure has a smoother and more natural transition between the light spot and the background, and the image quality is higher.

[0191] In some implementations, different filtering algorithms may be used when filtering the background layer, the second coefficient layer, the spot layer, and the first coefficient layer.

[0192] That is, in some embodiments, the image processing method of this disclosure includes:

[0193] A first filtering process is applied to the bokeh layer and the first coefficient layer to obtain a bokeh blur layer and a first filter coefficient layer; a second filtering process is applied to the background layer and the second coefficient layer to obtain a background blur layer and a second filter coefficient layer; wherein, the first filtering process is different from the second filtering process.

[0194] For example, in the example above, in the case of Figure 6 The background layer shown in (a) and Figure 7 When filtering the second coefficient layer shown in (a), Gaussian filtering can be used. And for... Figure 6 The light spot layer shown in (b) and Figure 7 When filtering the first coefficient layer shown in (b), an elliptical filter can be used to achieve a better spot effect.

[0195] Of course, those skilled in the art will understand that the first and second filtering methods are not limited to the examples above, but can be any other suitable filtering methods, and this disclosure does not limit them.

[0196] As can be seen from the above, in this embodiment of the disclosure, different filtering algorithms can be used to blur the bright areas and the background areas, thereby achieving better imaging results.

[0197] Secondly, this disclosure provides an image processing apparatus that can be applied to an electronic device. The electronic device in this disclosure can be any type of device suitable for implementation, such as a smartphone, tablet computer, wearable device, personal computer, vehicle-mounted device, server, and cloud platform. This disclosure does not limit the type of device.

[0198] like Figure 16 As shown, in some embodiments, the image processing apparatus of this disclosure includes:

[0199] The acquisition module 10 is configured to acquire the spot layer and background layer corresponding to the image to be processed;

[0200] The coefficient layer module 20 is configured to perform pixel value conversion on the spot layer and the background layer respectively, to obtain the first coefficient layer corresponding to the spot layer and the second coefficient layer corresponding to the background layer.

[0201] The mask processing module 30 is configured to process the original vignetting mask layer based on the first coefficient layer and the second coefficient layer to obtain the target vignetting mask layer; wherein the pixel value of the spot area in the target vignetting mask layer is not less than the pixel value of the corresponding spot area in the original vignetting mask layer.

[0202] The layer blending module 40 is configured to blend the blurred image of the image to be processed based on the target vignetting mask layer to obtain the target image corresponding to the image to be processed.

[0203] As can be seen from the above, in this embodiment of the present disclosure, the coefficient layer is used to correct the vignetting mask layer, so that when adding a vignetting effect to the blurred image, the bright pixels in the corner area can retain good brightness and contrast without significant attenuation, and the boundary transition between the vignetting area and the background area is smoother and more natural, thus improving the image quality.

[0204] In some implementations, the acquisition module 10 is configured to:

[0205] The image to be processed is segmented to obtain the regions to be blurred and the regions not blurred.

[0206] The blurred area is segmented to obtain a light spot layer and a background layer.

[0207] In some implementations, the spot layer includes the spot area of ​​the region to be blurred, and the background layer includes the region of the region to be blurred excluding the spot area.

[0208] In some implementations, the coefficient layer module 20 is configured as follows:

[0209] Set the pixel values ​​of the valid pixels in the bokeh layer to the first value, and set the pixel values ​​of the invalid pixels in the bokeh layer to the second value to obtain the first coefficient layer corresponding to the bokeh layer;

[0210] Set the pixel values ​​of valid pixels in the background layer to the first value, and set the pixel values ​​of invalid pixels in the background layer to the second value to obtain the second coefficient layer corresponding to the background layer.

[0211] In some implementations, the mask processing module 30 is configured to:

[0212] The first coefficient layer and the second coefficient layer are filtered respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer.

[0213] Based on the first and second filter coefficient layers, the original vignetting mask layer is processed to obtain the target vignetting mask layer.

[0214] In some implementations, the mask processing module 30 is configured to:

[0215] For any pixel in the original vignette mask layer, the pixel weight is determined based on the pixel values ​​in the first filter coefficient layer and the second filter coefficient layer; wherein, the pixel weight of the pixel in the spot area is greater than the pixel weight of the pixel in the background area.

[0216] The target pixel value is determined based on the pixel weight and the original pixel value of the pixel in the original vignette mask layer;

[0217] Based on the target pixel value of each pixel, a target vignetting mask layer is obtained.

[0218] In some embodiments, the image processing apparatus of this disclosure further includes a filtering module, which is configured to:

[0219] A bokeh layer is obtained by filtering the bokeh layer based on the first coefficient layer, and a background layer is obtained by filtering the background layer based on the second coefficient layer.

[0220] The first coefficient layer and the second coefficient layer are filtered respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer.

[0221] Based on the first and second filter coefficient layers, the spot blur layer and the background blur layer are fused to obtain the target blur layer;

[0222] The target blurred layer and the non-blurred layer of the image to be processed are blended to obtain the blurred image of the image to be processed.

[0223] In some implementations, the filtering module is configured as follows:

[0224] The background layer and the background coefficient layer are subjected to a first filtering process to obtain a background blur layer and a first coefficient layer;

[0225] A second filtering process is applied to the bokeh layer and the highlight coefficient layer to obtain a highlight blur layer and a second coefficient layer; the first filtering process is different from the second filtering process.

[0226] As described above, in this embodiment, while ensuring that the brightness and contrast of the highlighted areas in the corners of the image are not reduced, a vignetting effect is added to the image, improving the image's layering and texture, and thus enhancing the imaging quality. Furthermore, different filtering algorithms can be used to blur the light spot area and the background area, thereby achieving better imaging results.

[0227] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0228] processor; and

[0229] A memory storing computer instructions for causing the processor to perform the method according to any embodiment of the first aspect.

[0230] Fourthly, embodiments of this disclosure provide a storage medium storing computer instructions for causing a computer to perform the method described according to any embodiment of the first aspect.

[0231] Figure 17 The diagram shows a structural block diagram of an electronic device according to some embodiments of the present disclosure. The following is a description of the structure of the device in conjunction with the provided text. Figure 17 The principles related to the electronic devices and storage media of some embodiments of this disclosure will be explained.

[0232] Reference Figure 17 The electronic device 1800 may include one or more of the following components: a processing component 1802, a memory 1804, a power supply component 1806, a multimedia component 1808, an audio component 1810, an input / output (I / O) interface 1812, a sensor component 1816, and a communication component 1818.

[0233] Processing component 1802 typically controls the overall operation of electronic device 1800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1802 may include one or more processors 1820 to execute instructions. Furthermore, processing component 1802 may include one or more modules to facilitate interaction between processing component 1802 and other components. For example, processing component 1802 may include a multimedia module to facilitate interaction between multimedia component 1808 and processing component 1802. As another example, processing component 1802 may read executable instructions from memory to implement relevant functions of the electronic device.

[0234] Memory 1804 is configured to store various types of data to support the operation of electronic device 1800. Examples of this data include instructions for any application or method operating on electronic device 1800, contact data, phonebook data, messages, pictures, videos, etc. Memory 1804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0235] Power supply component 1806 provides power to various components of electronic device 1800. Power supply component 1806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1800.

[0236] The multimedia component 1808 includes a display screen that provides an output interface between the electronic device 1800 and the user. In some embodiments, the multimedia component 1808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 1800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera can receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0237] Audio component 1810 is configured to output and / or input audio signals. For example, audio component 1810 includes a microphone (MIC) configured to receive external audio signals when electronic device 1800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1804 or transmitted via communication component 1818. In some embodiments, audio component 1810 also includes a speaker for outputting audio signals.

[0238] I / O interface 1812 provides an interface between processing component 1802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0239] Sensor assembly 1816 includes one or more sensors for providing state assessments of various aspects of electronic device 1800. For example, sensor assembly 1816 may detect the on / off state of electronic device 1800, the relative positioning of components such as the display and keypad of electronic device 1800, changes in position of electronic device 1800 or a component of electronic device 1800, the presence or absence of user contact with electronic device 1800, the orientation or acceleration / deceleration of electronic device 1800, and temperature changes of electronic device 1800. Sensor assembly 1816 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1816 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1816 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0240] Communication component 1818 is configured to facilitate wired or wireless communication between electronic device 1800 and other devices. Electronic device 1800 can access wireless networks based on communication standards, such as Wi-Fi, 2G, 3G, 4G, 5G, or 6G, or combinations thereof. In one exemplary embodiment, communication component 1818 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1818 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0241] In an exemplary embodiment, the electronic device 1800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components.

[0242] Obviously, the above embodiments are merely examples for clear illustration and are not intended to limit the embodiments. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all embodiments here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this disclosure.

Claims

1. An image processing method, characterized by, include: Obtain the bokeh layer and background layer corresponding to the image to be processed; Pixel value conversion is performed on the light spot layer and the background layer respectively to obtain a first coefficient layer corresponding to the light spot layer and a second coefficient layer corresponding to the background layer; Based on the first coefficient layer and the second coefficient layer, the original vignetting mask layer is processed to obtain the target vignetting mask layer; wherein, the pixel value of the spot area in the target vignetting mask layer is not less than the pixel value of the corresponding spot area in the original vignetting mask layer; The blurred image of the image to be processed is fused based on the target vignetting mask layer to obtain the target image corresponding to the image to be processed; The step of converting the pixel values ​​of the light spot layer and the background layer respectively includes: setting the effective pixels to a first value and setting the invalid pixels to a second value; The process of processing the original vignetting mask layer based on the first coefficient layer and the second coefficient layer to obtain the target vignetting mask layer includes: Filter the first coefficient layer and the second coefficient layer to obtain the first filtered coefficient layer and the second filtered coefficient layer. Adjust the pixel values ​​of each pixel in the original vignetting mask layer based on the pixel values ​​in the first and second filter coefficient layers to obtain the target vignetting mask layer.

2. The method of claim 1, wherein, The step of converting the pixel values ​​of the light spot layer and the background layer respectively to obtain a first coefficient layer corresponding to the light spot layer and a second coefficient layer corresponding to the background layer includes: The pixel values ​​of the valid pixels in the light spot layer are set to a first value, and the pixel values ​​of the invalid pixels in the light spot layer are set to a second value, thereby obtaining the first coefficient layer corresponding to the light spot layer; The pixel values ​​of valid pixels in the background layer are set to a first value, and the pixel values ​​of invalid pixels in the background layer are set to a second value, thus obtaining the second coefficient layer corresponding to the background layer.

3. The method of claim 1, wherein, The step of processing the original vignetting mask layer based on the first coefficient layer and the second coefficient layer to obtain the target vignetting mask layer includes: The first coefficient layer and the second coefficient layer are filtered respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer; Based on the first and second filter coefficient layers, the original vignetting mask layer is processed to obtain the target vignetting mask layer.

4. The method of claim 3, wherein, The step of processing the original vignetting mask layer based on the first filter coefficient layer and the second filter coefficient layer to obtain the target vignetting mask layer includes: For any pixel in the original vignetting mask layer, the pixel weight corresponding to the pixel is determined based on the pixel value in the first filter coefficient layer and the pixel value in the second filter coefficient layer; wherein, the pixel weight of the pixel in the spot area is greater than the pixel weight of the pixel in the background area; The target pixel value is determined based on the pixel weight and the original pixel value of the pixel in the original vignetting mask layer; The target vignetting mask layer is obtained based on the target pixel value of each pixel.

5. The method of claim 1, wherein, Also includes: A bokeh layer is obtained by filtering the bokeh layer based on the first coefficient layer, and a background bokeh layer is obtained by filtering the background layer based on the second coefficient layer. The first coefficient layer and the second coefficient layer are filtered respectively to obtain the first filtered coefficient layer corresponding to the first coefficient layer and the second filtered coefficient layer corresponding to the second coefficient layer; Based on the first filter coefficient layer and the second filter coefficient layer, the light spot blur layer and the background blur layer are fused to obtain the target blur layer; The target blurred layer and the non-blurred layer of the image to be processed are fused together to obtain the blurred image of the image to be processed.

6. The method of claim 1, wherein, The process of obtaining the light spot layer and background layer corresponding to the image to be processed includes: The image to be processed is segmented to obtain the region to be blurred and the region not blurred included in the image to be processed; The region to be blurred is segmented to obtain the light spot layer and the background layer.

7. The method according to claim 6, characterized in that, The light spot layer includes the light spot area of ​​the area to be blurred, and the background layer includes the area of ​​the area to be blurred other than the light spot area.

8. An image processing apparatus characterized by comprising: include: The acquisition module is configured to acquire the spot layer and background layer corresponding to the image to be processed. The coefficient layer module is configured to perform pixel value conversion on the light spot layer and the background layer respectively to obtain a first coefficient layer corresponding to the light spot layer and a second coefficient layer corresponding to the background layer; The mask processing module is configured to process the original vignetting mask layer based on the first coefficient layer and the second coefficient layer to obtain a target vignetting mask layer; wherein the pixel value of the spot area in the target vignetting mask layer is not less than the pixel value of the corresponding spot area in the original vignetting mask layer. The layer blending module is configured to blend the blurred image of the image to be processed based on the target vignetting mask layer to obtain the target image corresponding to the image to be processed; The coefficient layer module is configured to set valid pixels to a first value and invalid pixels to a second value. The mask processing module is configured to: perform filtering processing on the first coefficient layer and the second coefficient layer to obtain a first filtered coefficient layer and a second filtered coefficient layer; adjust the pixel value of each pixel in the original vignette mask layer according to the pixel value in the first filtered coefficient layer and the pixel value in the second filtered coefficient layer to obtain the target vignette mask layer.

9. An electronic device, comprising: include: processor; and A memory storing computer instructions for causing the processor to perform the method according to any one of claims 1 to 7.

10. A storage medium, characterized by The computer contains computer instructions for causing the computer to perform the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Image processing method and apparatus

    CN106504220A

  • Image display method and device

    CN111741211A