Image blurring method, electronic equipment and computer readable storage medium

Abstract

The invention discloses an image blurring method, electronic equipment and a computer readable storage medium, and relates to the technical field of image processing, and the image blurring method comprises the steps: obtaining the pixel depth information of each pixel point in a to-be-blurred image, and determining the depth-of-field range parameter of a main body region in the to-be-blurred image; according to the pixel depth information and the field depth range parameter, a first filter coefficient corresponding to each pixel point in the image to be blurred is determined, and the farther the pixel point is from a focal plane of the main body area, the larger the first filter coefficient corresponding to the pixel point is; based on the first filter coefficient corresponding to each pixel point, pixel blurring processing is carried out on each pixel point in the to-be-blurred image, and the first filter coefficient is in positive correlation with the pixel blurring degree. Clear segmentation and reasonable coverage layers of the main body in the depth of field and front and back backgrounds outside the depth of field can be effectively ensured, and the definition of the main body in the depth of field is highlighted.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to image blurring methods, electronic devices, and computer-readable storage media. Background Technology

[0002] Background blurring refers to blurring the background of an image while clearly displaying the foreground. It's a widely used photographic technique that effectively highlights the subject and expresses visual aesthetics. Some professional cameras can directly capture images with a blurred background, but this requires a high-resolution lens with a large aperture and zoom capability. For cameras on electronic devices such as smartphones or tablets, due to limitations in size and thickness, their lens performance cannot match that of professional cameras, so they cannot directly capture images with a blurred background. Therefore, image processing software is needed to achieve background blurring.

[0003] Several digital image blurring techniques have been proposed, such as real-time rendering based on a digital image processor (ISP) system to achieve online image blurring. However, this digital image blurring method is prone to bleeding between the subject within the depth of field and the blurred background, resulting in insufficient sharpness of the subject's edges. It also typically ignores the rendering hierarchy relationship between the foreground and background at different distances from the shooting device in real optical imaging.

[0004] In other words, in related technologies, blurred images are prone to problems such as blurred boundaries between the subject and the foreground and background, and unnatural transitions, for example, color leakage and discontinuous blurring in the foreground and background. Summary of the Invention

[0005] The main purpose of this application is to provide an image blurring method, electronic device and computer-readable storage medium, which aims to solve the technical problem in related technologies that blurred images are prone to blurring of the boundaries between the subject and the foreground and background and unnatural transitions.

[0006] To achieve the above objectives, this application provides an image blurring method, comprising:

[0007] Obtain pixel depth information of each pixel in the image to be blurred, and determine the depth range parameters of the main area in the image to be blurred;

[0008] Based on the pixel depth information and the depth of field range parameters, the first filtering coefficient corresponding to each pixel in the image to be blurred is determined, wherein the farther the pixel is from the focal plane of the main body region, the larger the first filtering coefficient corresponding to the pixel is.

[0009] Based on the first filtering coefficient corresponding to each pixel, pixel blurring processing is performed on each pixel in the image to be blurred, wherein the first filtering coefficient is positively correlated with the degree of pixel blurring.

[0010] In addition, to achieve the above objectives, this application also provides an electronic device, which includes: a memory, a processor, and an image blurring program stored in the memory and executable on the processor, wherein the image blurring program, when executed by the processor, implements the steps of the image blurring method as described above.

[0011] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing an image blurring program, which, when executed by a processor, implements the steps of the image blurring method described above.

[0012] In addition, to achieve the above objectives, this application also provides a computer program product, which includes an image blurring program, and when the image blurring program is executed by a processor, it implements the steps of the image blurring method described above.

[0013] This application provides an image blurring method, an electronic device, and a computer-readable storage medium. The technical solution of this application involves acquiring pixel depth information of each pixel in the image to be blurred, and determining the depth-of-field range parameters of the main subject area in the image. Based on the pixel depth information and depth-of-field range parameters, a first filtering coefficient corresponding to each pixel in the image to be blurred is determined. The farther a pixel is from the focal plane of the main subject area, the larger its corresponding first filtering coefficient. Then, based on the first filtering coefficients corresponding to each pixel, pixel blurring processing is performed on each pixel in the image to be blurred. The first filtering coefficient is positively correlated with the degree of pixel blurring. Therefore, this application addresses the boundary sharpness issues of the main subject and the blurred background, as well as the coverage layer issues of the foreground and background, by proposing a depth-guided (Depth) method. The Guided image blurring scheme effectively ensures clear segmentation and reasonable coverage of the subject within the depth of field and the foreground and background outside the depth of field by utilizing relative depth information. This achieves the beneficial effect of highlighting the subject within the depth of field, ensuring a clear boundary between the subject within the depth of field and objects outside the depth of field (including foreground and background), especially when there is a relatively large depth of field difference between the subject within the depth of field and the foreground and background. This highlights the sharpness of the subject within the depth of field and solves the technical problem in related technologies where blurred images often result in blurred boundaries and unnatural transitions between the subject and the foreground and background. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating an embodiment of the image blurring method of this application.

[0017] Figure 2 This is a flowchart illustrating Embodiment 2 of the image blurring method of this application;

[0018] Figure 3 This is a flowchart illustrating Embodiment 3 of the image blurring method of this application;

[0019] Figure 4 A flowchart illustrating a specific embodiment of this application;

[0020] Figure 5 This is a schematic diagram of DC4 image blurring provided in a specific embodiment of this application;

[0021] Figure 6 This is a schematic diagram of DC16 image blurring provided in a specific embodiment of this application;

[0022] Figure 7 This is a schematic diagram of the first image blurring effect provided in a specific embodiment of this application;

[0023] Figure 8 This is a schematic diagram of the second image blurring effect provided in a specific embodiment of this application;

[0024] Figure 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the image blurring method in the embodiments of this application.

[0025] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0027] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0028] Currently, digital image blurring technology uses a consistent standard to blur images. This consistency fails to apply varying degrees of blurring based on depth when processing images with complex depth distributions, resulting in a lack of depth perception and leading to technical problems such as blurred boundaries between the subject and background, and unnatural transitions in the blurred image.

[0029] The main solution of this application embodiment is: to obtain pixel depth information of each pixel in the image to be blurred, and to determine the depth range parameter of the main body region in the image to be blurred; to determine the first filtering coefficient corresponding to each pixel in the image to be blurred according to the pixel depth information and the depth range parameter, wherein the farther the pixel is from the focal plane of the main body region, the larger the first filtering coefficient corresponding to the pixel; and to perform pixel blurring processing on each pixel in the image to be blurred based on the first filtering coefficient corresponding to each pixel, wherein the first filtering coefficient is positively correlated with the degree of pixel blurring.

[0030] This application proposes a depth-guided image blurring scheme to address the issues of boundary clarity between the subject and the blurred background in depth of field, as well as the coverage layer problem between the foreground and background. By utilizing relative depth information, it effectively ensures clear segmentation and reasonable coverage layering between the subject within the depth of field and the foreground and background outside the depth of field, achieving the beneficial effect of highlighting the subject in depth of field. This scheme ensures that the subject within the depth of field has a clear boundary with objects outside the depth of field (including foreground and background), especially when there is a relatively large depth of field difference between the subject within the depth of field and the foreground and background. This highlights the clarity of the subject within the depth of field and solves the technical problem in related technologies where blurred images often exhibit blurred boundaries and unnatural transitions between the subject and the background.

[0031] The execution subject of this application embodiment is an electronic device, which may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs (Televisions), desktop computers, etc., or any electronic device capable of performing the above functions. This application embodiment does not specifically limit this. The following uses an electronic device as the execution subject to describe the various embodiments of this application.

[0032] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0033] Example 1

[0034] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the image blurring method of this application.

[0035] In this embodiment, the image blurring method includes steps S100 to S300:

[0036] Step S100: Obtain pixel depth information of each pixel in the image to be blurred, and determine the depth range parameters of the main area in the image to be blurred.

[0037] As those skilled in the art will know, depth of field is an important concept in photography and filmmaking, referring to the distances in front of and behind the subject that remain in sharp focus during shooting. Simply put, it's the range of depth within a photograph or video, from the closest to the furthest point in a clear image. When shooting a scene, the camera will only set the focus at a specific distance, where the object at that focus is the sharpest. However, within a certain range in front of and behind this focus point, other objects will also appear relatively sharp; this range is called the depth of field. Depth of field can be very shallow, only a few centimeters, or very deep, reaching several meters or even infinity, depending on the equipment and technical parameters used during shooting.

[0038] In this embodiment, the depth-of-field range parameter is a parameter used to represent the depth of field. The object within this depth-of-field range parameter is the subject of the depth of field, that is, the main object in the image that the user wants to keep sharp. The area corresponding to this object in the image to be blurred is the subject area. This depth-of-field range parameter can be a depth interval, which can also be converted into a disparity interval. That is, the depth-of-field range parameter can also be a disparity interval.

[0039] It should be noted that, in this embodiment, the image to be blurred refers to the image that requires background blurring, including the main subject area that needs to be highlighted within the depth of field, and the foreground and background that need to be blurred outside the depth of field. The main subject area refers to the part of the image to be blurred that needs to remain sharp and not be blurred, i.e., the object or area that the user wants to emphasize. Pixel depth information refers to the depth information of a pixel, i.e., the distance information of a pixel in the image from the camera lens.

[0040] It's easy to understand that this depth-of-field range parameter reflects the depth range of the subject area, that is, the pixel depth range of each pixel in the subject area. In other words, this depth-of-field range parameter can be represented by the depth range of the subject area, or by the pixel depth range of each pixel in the subject area.

[0041] It is worth mentioning that, in this embodiment, pixel depth information can be represented by the disparity intensity value of the pixel in the disparity map, the depth value of the pixel in the depth map, or the layer information to which the pixel belongs in the layer map. In addition, the pixel depth information of each pixel can be determined by fusing at least two of the disparity map, depth map, and layer map.

[0042] That is, in the first feasible implementation, the step of obtaining the pixel depth information of each pixel in the image to be blurred in step S100 can be step A10: obtaining the disparity map of the image to be blurred, and determining the pixel depth information of each pixel in the image to be blurred based on the disparity intensity value of each pixel in the disparity map.

[0043] It should be noted that the parallax intensity value is used to characterize the magnitude of the positional shift of a point between images from different viewpoints. The greater the shift, the greater the parallax intensity value, which means that the point is closer to the camera lens of the electronic device.

[0044] This implementation method can accurately determine the depth information of each pixel in the image by using the parallax intensity value, thereby providing an accurate data basis for subsequent bokeh processing. At the same time, the parallax intensity value can better simulate the real optical bokeh effect, ensuring that areas at different depths have different degrees of bokeh, and achieving a more natural bokeh effect.

[0045] In a second feasible implementation, the step of obtaining the pixel depth information of each pixel in the image to be blurred in step S100 can also be step A20: obtaining the depth map of the image to be blurred, and determining the pixel depth information of each pixel in the image to be blurred based on the depth value of each pixel in the depth map.

[0046] This implementation provides the depth value of each pixel directly through the depth map, eliminating the need for additional calculations or estimations, thus simplifying the processing flow. Furthermore, the depth map typically has high accuracy and can provide detailed depth information, thereby ensuring a more natural blurring effect.

[0047] In a third feasible implementation, the step of obtaining the pixel depth information of each pixel in the image to be blurred in step S100 can also be step A30: obtaining a layered image of the image to be blurred, and determining the pixel depth information of each pixel in the image to be blurred based on the layered information of each pixel in the layered image.

[0048] It should be noted that a layered image is an image that categorizes different regions of an image according to depth levels. Layered information refers to the information in a layered image that indicates the depth level to which each pixel belongs.

[0049] In this embodiment, the layered map classifies the pixels in the image to be blurred according to the depth level, which can intuitively display different depth regions in the image to be blurred, making it easier to understand and debug. At the same time, the layered information can simplify the processing of depth information, and blurring of different depth regions can be achieved through simple layering.

[0050] Furthermore, in a fourth feasible implementation, the step of obtaining pixel depth information of each pixel in the image to be blurred in step S100 can also be step A40: based on at least two of the disparity map, depth map and layer map of the image to be blurred, the pixel depth information of each pixel in the image to be blurred is determined by fusing them.

[0051] This implementation method integrates multiple depth information sources to complement each other's shortcomings and improve the accuracy of depth information. Even if the depth information obtained by one method fails, other methods can still provide reliable depth information. Therefore, the most suitable combination can be selected according to the actual situation to flexibly respond to different application scenarios and enhance the robustness of the system.

[0052] By implementing any one of the above steps A10 to A40, this embodiment can accurately obtain the pixel depth information of each pixel in the image to be blurred, ensuring that the blurring process is more accurate and natural, and has high robustness and flexibility.

[0053] Furthermore, in this embodiment, the step S100 of determining the depth-of-field range parameter of the main subject region in the image to be blurred includes step A50 or step A60:

[0054] Step A50: Obtain the focus area in the image to be blurred, and determine the depth range parameters of the main subject area in the image to be blurred based on the depth information of each pixel in the focus area.

[0055] It's important to note that the focus area refers to the region in an image that is selected as the focus point by the autofocus function or manually by the user. It typically includes the main object or point of interest in the image. Simply put, the focus area is generally the subject area.

[0056] In this embodiment, the focus area in the image to be blurred can be obtained by manually selecting the focus area using the interactive interface of the relevant camera function in the electronic device. Alternatively, the focus area in the image to be blurred can be obtained by using the autofocus function provided by the electronic device to identify the main object area (i.e., the subject area) in the image to be blurred. Then, based on the pixel depth information of each pixel in the image to be blurred obtained in any of the implementation steps A10 to A40, the depth information (i.e., pixel depth information) of each pixel in the focus area can be obtained. Finally, based on the depth information of each pixel in the focus area, the depth range parameter of the subject area in the image to be blurred can be determined.

[0057] In this embodiment, the autofocus function can accurately identify the main object in the image, thereby determining the depth-of-field parameters. The user can manually select the focus area, allowing for flexible selection of key regions according to actual needs, thus improving processing flexibility. By utilizing the depth information of each pixel within the focus area, this embodiment can accurately determine the depth-of-field parameters, ensuring the sharpness of the main subject area after image blurring.

[0058] Step A60: Obtain the subject region corresponding to the input depth subject, and determine the depth range parameters of the subject region in the image to be blurred based on the depth information of each pixel within the subject region.

[0059] It should be noted that the depth-of-field subject refers to the main object or area in the image that the user wants to keep sharp; the corresponding area in the image to be blurred is the subject region. The parameters corresponding to the depth-of-field subject are used to determine the subject region from the image to be blurred.

[0060] In this embodiment, the user can manually input the parameters corresponding to the subject in the depth field through the interactive interface of the relevant camera function in the electronic device, and determine the subject area from the image to be blurred. Then, according to the pixel depth information of each pixel in the image to be blurred obtained in any of the implementation methods A10 to A40, the depth information of each pixel in the subject area is obtained, and then the depth range parameters of the subject area in the image to be blurred are determined according to the depth information of each pixel in the subject area.

[0061] In this embodiment, the subject area is determined by the user manually inputting parameters corresponding to the depth of field, allowing for precise identification of the parts that need to remain sharp. By utilizing the depth information of each pixel within the subject area, this embodiment can accurately determine the depth-of-field range parameters, ensuring the sharpness of the subject area after image blurring.

[0062] In addition, the depth-of-field range parameter can also be obtained directly by having the user manually input the depth-of-field range parameter.

[0063] Step S200: Based on pixel depth information and depth range parameters, determine the first filtering coefficient corresponding to each pixel in the image to be blurred, wherein the farther the pixel is from the focal plane of the main body region, the larger the first filtering coefficient corresponding to the pixel.

[0064] It should be noted that, in this embodiment, the focal plane of the main subject area refers to the plane on which the lens of the electronic device is aimed at the main subject area and focused most clearly during the shooting process. The first filter coefficient is a coefficient used to determine the degree of pixel blur. Under the same conditions, the larger the first filter coefficient, the higher the degree of pixel blur, that is, the first filter coefficient is positively correlated with the degree of pixel blur.

[0065] In this embodiment, for each pixel in the image to be blurred, its distance from the focal plane of the subject area is determined based on the relationship between its pixel depth information and the depth-of-field range parameter. Pixels closer to the focal plane are assigned a smaller first filter coefficient, while pixels farther from the focal plane are assigned a larger first filter coefficient. This ensures that the farther a pixel is from the focal plane in the image to be blurred, the larger its first filter coefficient. This ensures that when pixel blurring is performed on each pixel in the image to be blurred, pixels farther from the focal plane have a higher degree of blurring, resulting in pixels of different depths having different degrees of blurring. This enhances the sense of layering in the image blurring, achieves a more natural and smooth blurring effect, and avoids blurring of the boundaries between the foreground / background and the subject area.

[0066] It is easy to understand that for pixels within the main subject area, the first filtering coefficient can be set to 0, that is, no pixel blurring is applied to them, ensuring that the main subject area is clearly and prominently imaged.

[0067] Step S300: Based on the first filtering coefficient corresponding to each pixel, pixel blurring processing is performed on each pixel in the image to be blurred, wherein the first filtering coefficient is positively correlated with the degree of pixel blurring.

[0068] It should be noted that pixel blurring refers to blurring individual pixels, making them appear blurred. The degree of pixel blurring refers to the extent to which a pixel is blurred.

[0069] This embodiment applies the first filtering coefficient pixel by pixel to perform pixel blurring processing on each pixel in the image to be blurred, so that pixels at different depths have different degrees of pixel blurring, which enhances the sense of layering of the image blurring, makes the blurring effect more natural, ensures that the boundary between the subject area in the depth of field and the foreground and background outside the depth of field is clear, avoids smudging, and makes the final image blurring processing result more natural, beautiful and practical.

[0070] This embodiment obtains the pixel depth information of each pixel in the image to be blurred and determines the depth-of-field range parameter of the main body region in the image to be blurred. Based on the pixel depth information and depth-of-field range parameter, it determines the first filter coefficient corresponding to each pixel in the image to be blurred. The farther the pixel is from the focal plane of the main body region, the larger the first filter coefficient corresponding to the pixel. Then, based on the first filter coefficient corresponding to each pixel, pixel blurring processing is performed on each pixel in the image to be blurred. The first filter coefficient is positively correlated with the degree of pixel blurring. Thus, this embodiment proposes a depth-guided (Depth) method to address the boundary sharpness problem between the main body and the blurred background, as well as the coverage layer problem between the foreground and the background. The Guided image blurring scheme effectively ensures clear segmentation and reasonable coverage of the subject within the depth of field and the foreground and background outside the depth of field by utilizing relative depth information. This achieves the beneficial effect of highlighting the subject within the depth of field, ensuring a clear boundary between the subject within the depth of field and objects outside the depth of field (including foreground and background), especially when there is a relatively large depth of field difference between the subject within the depth of field and the foreground and background. This highlights the sharpness of the subject within the depth of field and solves the technical problem in related technologies where blurred images often result in blurred boundaries and unnatural transitions between the subject and the foreground and background.

[0071] Furthermore, in one feasible implementation, before the step of obtaining the pixel depth information of each pixel in the image to be blurred in step S100, step A70 is also included:

[0072] Step A70: Obtain the initial image, perform image preprocessing on the initial image, and obtain the image to be blurred.

[0073] Image preprocessing includes at least one of image smoothing, image noise reduction, upsampling, and downsampling.

[0074] It should be noted that, in this embodiment, the initial image refers to the original image without any processing. It can be obtained by capturing a picture through the lens of an electronic device or by loading it into the storage device of the electronic device.

[0075] Those skilled in the art will recognize that image smoothing is used to reduce image noise and make the image smoother, image denoising is used to remove random noise from the image and retain the main features of the image, upsampling is used to increase the resolution of the image and make the image clearer, and downsampling is used to reduce the resolution of the image and reduce processing time and resource consumption.

[0076] This implementation preprocesses the initial image to obtain a higher-quality image to be blurred, thus providing better conditions for subsequent depth information acquisition and blurring processing. This ensures that the subsequent blurring process is more accurate and natural, thereby achieving the desired blurring effect. The specific image preprocessing method(s) chosen depends on the characteristics of the initial image and the application scenario of the image blurring, and can be flexibly set by the user according to actual needs. This implementation does not impose specific restrictions on this.

[0077] Example 2

[0078] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the image blurring method of this application.

[0079] In this embodiment, the same or similar content as in the above embodiments can be referred to the above description, and will not be repeated hereafter.

[0080] In this embodiment, the first filtering coefficients corresponding to each pixel include: the first blurring coefficient of each pixel in the foreground region and the second blurring coefficient of each pixel in the background region;

[0081] Step S200, which involves determining the first filter coefficients corresponding to each pixel in the image to be blurred based on pixel depth information and depth-of-field range parameters, may include steps S210 to S220:

[0082] Step S210: Determine the foreground and background regions in the image to be blurred based on pixel depth information and depth range parameters.

[0083] In this embodiment, the foreground region refers to the area in the image to be blurred that is closer to the lens of the electronic device than the subject region, i.e., the region with a depth less than the depth-of-field range parameter, also known as the foreground background outside the depth of field. The background region refers to the area in the image to be blurred that is farther from the lens of the electronic device than the subject region, i.e., the region with a depth greater than the depth-of-field range parameter, also known as the background background outside the depth of field.

[0084] It should be noted that the blurring coefficient refers to the first filter coefficient, used to determine the degree of blurring of pixels. The first blurring coefficient specifically refers to the blurring coefficient of pixels within the foreground region, used to determine the degree of blurring within that region. The second blurring coefficient refers to the blurring coefficient of pixels within the background region, used to determine the degree of blurring within that region. For pixels within the foreground region, all other things being equal, a larger first blurring coefficient indicates a higher degree of blurring; that is, the first blurring coefficient is positively correlated with the degree of pixel blurring. For pixels within the background region, all other things being equal, a larger second blurring coefficient indicates a higher degree of blurring; that is, the second blurring coefficient is positively correlated with the degree of pixel blurring.

[0085] This embodiment, based on the pixel depth information of each pixel in the image to be blurred and the depth-of-field range parameters of the main body region, can determine which pixels in the image to be blurred are closer to the lens of the electronic device than the main body region, and which pixels are farther away from the lens of the electronic device than the main body region. Pixels that are closer to the lens of the electronic device than the main body region are designated as pixels in the foreground region, while pixels that are farther away from the lens of the electronic device than the main body region are designated as pixels in the background region. This determines the foreground and background regions in the image to be blurred, ensuring that the edges of the main body region are clear and without boundary blurring during subsequent blurring processing.

[0086] Step S220: Based on the first preset lookup table function, determine the first blur coefficient of each pixel in the foreground region, and based on the second preset lookup table function, determine the second blur coefficient of each pixel in the background region, wherein the first blur coefficient is positively correlated with the degree of pixel blur, and the second blur coefficient is positively correlated with the degree of pixel blur.

[0087] As those skilled in the art will know, a lookup function typically refers to a function in a programming or computational tool whose main purpose is to find and return the required data from a predefined data set (such as an array, list, table, etc.). A lookup function can map input data to another value, returning different preset results based on different input values. It can also perform interpolation calculations between known data points to estimate the value of unknown data points.

[0088] In this embodiment, both the first and second preset lookup table functions are pre-defined lookup table functions used to determine the blurring coefficient of a pixel based on its depth information (i.e., pixel depth information). Specifically, the first preset lookup table function determines the blurring coefficient (i.e., the first blurring coefficient) for pixels in the foreground region based on their depth information, while the second preset lookup table function determines the blurring coefficient (i.e., the second blurring coefficient) for pixels in the background region based on their depth information.

[0089] It should be noted that in this embodiment, the first preset lookup table function and the second preset lookup table function are different. That is, for pixels of the same depth, when they belong to the foreground region, the first blurring coefficient determined by the first preset lookup table function is different from the second blurring coefficient determined by the second preset lookup table function when they belong to the background region.

[0090] This embodiment sets a first preset lookup table function and a second preset lookup table function to apply different blurring standards to the foreground and background regions, thereby more precisely controlling the degree of blurring in different regions. This ensures a more natural transition between the foreground, subject, and background regions, avoiding abruptness, making the boundary between the subject region and the foreground or background region clearer, improving the overall visual effect and blurring layering of the blurred image, and better simulating the physical characteristics and visual perception requirements in natural optical imaging.

[0091] Furthermore, in a feasible implementation, step S300, which involves performing image blurring processing on each pixel in the image to be blurred based on the first filtering coefficients corresponding to each pixel, may include steps S310 to S330:

[0092] Step S310: Based on the first blurring coefficient, perform pixel blurring processing on each pixel in the foreground region to obtain the foreground region image after pixel blurring processing.

[0093] It should be noted that the foreground region image refers to the image corresponding to the foreground region in the image to be blurred.

[0094] In this embodiment, after determining the first blurring coefficient of each pixel in the foreground region through a lookup table function, the image corresponding to the foreground region, i.e. the foreground region image, is extracted from the image to be blurred. Then, a blurring algorithm is used to apply the corresponding first blurring coefficient to each pixel in the foreground region image to perform pixel blurring processing, thereby obtaining the foreground region image after pixel blurring processing.

[0095] Step S320, and, based on the second blurring coefficient, perform pixel blurring processing on each pixel in the background region to obtain the background region image after pixel blurring processing.

[0096] It should be noted that the background region image refers to the image corresponding to the foreground region in the image to be blurred.

[0097] In this embodiment, after determining the second blurring coefficient of each pixel in the background region by using a lookup table function, the image corresponding to the background region, i.e., the background region image, is extracted from the image to be blurred. Then, a blurring algorithm is used to apply the corresponding second blurring coefficient to each pixel in the background region image to perform pixel blurring processing, thereby obtaining the background region image after pixel blurring processing.

[0098] Step S330: Obtain the main region image of the main region in the image to be blurred, and fuse the main region image with the foreground region image and background region image after pixel blurring to obtain the target blurred image.

[0099] It should be noted that the subject region image refers to the image corresponding to the subject region in the image to be blurred, while the target blurred image refers to the image obtained after pixel blurring of each pixel in the image to be blurred.

[0100] In this embodiment, after obtaining the foreground region image and background region image after pixel blurring, the image corresponding to the main region, i.e. the main region image, is extracted from the image to be blurred. This image is then fused pixel by pixel with the foreground region image and background region image after pixel blurring through image fusion techniques such as masking layers and multi-channel synthesis to finally obtain the target blurred image.

[0101] It should be noted that in step S330, when fusing the main image with the pixel-blurred foreground and background images, the pixel-blurred foreground and background images can be initially fused together to ensure a more natural blurring effect in the foreground and background areas outside the depth of field, avoiding abrupt transitions between the foreground and background areas. Then, the initially fused image is finally fused with the main image to ensure the clarity and detail of the main area. Alternatively, the pixel-blurred background image can be initially fused with the main image to ensure the clarity and detail of the main area, prioritizing the visual effect of the main area, reducing blurring at the boundary between the main and background areas, ensuring a clear boundary between them, better guiding the viewer's attention to the main area, and avoiding visual interference from the foreground area. Then, the initially fused image is finally fused with the pixel-blurred foreground image. Alternatively, you can first perform a preliminary fusion of the foreground image after pixel blurring with the subject image to ensure the clarity and detail of the subject area, prioritizing the visual effect of the subject area and reducing the blurring of the boundary between the subject and foreground areas. This ensures a clear boundary between the subject and foreground areas and avoids visual breaks. Then, you can fuse the preliminarily fused image with the background image after pixel blurring. Alternatively, you can directly fuse all three together without any particular order.

[0102] It is worth mentioning that, in addition to the above-described implementation method, this embodiment can also directly perform pixel-by-pixel blurring processing after determining the first filtering coefficient of each pixel in the image to be blurred, and directly obtain the target blurred image.

[0103] Example 3

[0104] Please refer to Figure 3 , Figure 3This is a flowchart illustrating the image blurring method of this application in Embodiment 3.

[0105] In this embodiment, the same or similar content as in the above embodiments can be referred to the above description, and will not be repeated hereafter.

[0106] In this embodiment, step S300, which involves performing pixel blurring processing on each pixel in the image to be blurred based on the first filtering coefficients corresponding to each pixel, may further include steps S340 to S360:

[0107] Step S340: Take each pixel in the image to be blurred as the center pixel and determine the neighboring pixels corresponding to each center pixel.

[0108] As those skilled in the art will know, in image processing, when a certain pixel is selected as the core of the processing, that pixel is the center pixel, and the pixels around the center pixel are the neighboring pixels. The pixel block formed by the center pixel and its neighboring pixels can be a 3*3 pixel block or a 4*4 pixel block. This embodiment does not impose any specific limitations on this.

[0109] In this embodiment, the size of the pixel block can be set according to actual needs, and then the pixels in the image to be blurred are used as the center pixels to construct the pixel block, and the pixels outside the center pixel of the pixel block are determined as the neighboring pixels corresponding to the center pixel, thereby realizing step S340.

[0110] Step S350: Determine the second filtering coefficients corresponding to each pixel in the image to be blurred based on the pixel depth difference information between each center pixel and its corresponding neighboring pixels.

[0111] It should be noted that pixel depth difference information reflects the difference in pixel depth between a pixel and its neighboring pixels, that is, the depth relationship between a pixel and its neighboring pixels. This pixel depth difference information can be determined by comparing the pixel depth information of the center pixel with the pixel depth information of its neighboring pixels.

[0112] For example, when pixel depth information is represented by a depth value, the pixel depth difference information reflects the difference in depth values ​​between a pixel and its neighboring pixels, and is determined by the depth value of the pixel as the center pixel and the depth values ​​of its neighboring pixels.

[0113] In addition, it should be noted that the second filtering coefficient is similar to the first filtering coefficient, and is also used to determine the degree of blurring of the pixel. The difference is that the first filtering coefficient depends on the depth relationship between the pixel and the focal plane, while the second filtering coefficient depends on the depth relationship between the pixel and its neighboring pixels.

[0114] In this embodiment, the degree of pixel blurring depends on the magnitudes of its first and second filter coefficients; that is, the degree of pixel blurring is jointly determined by the first and second filter coefficients. For a pixel in the image to be blurred, when the second filter coefficient remains constant, a larger first filter coefficient results in a higher degree of pixel blurring, and vice versa. In other words, when the second filter coefficient remains constant, the first filter coefficient is positively correlated with the degree of pixel blurring; when the first filter coefficient remains constant, the second filter coefficient is positively correlated with the degree of pixel blurring.

[0115] For example, in one feasible implementation, step S350 may include steps S351 to S353:

[0116] Step S351: Based on the pixel depth difference information, compare the magnitude between the first depth value of the center pixel and the second depth value of its corresponding neighboring pixels.

[0117] In this embodiment, the first depth value refers to the depth value of the center pixel, which is determined by the pixel depth information of the center pixel. The second depth value is used to reflect the depth of the neighboring pixels corresponding to the center pixel.

[0118] In this embodiment, the depth value of each neighboring pixel can be determined by the pixel depth information of each neighboring pixel corresponding to the center pixel. Then, the average or weighted average of the depth values ​​of each neighboring pixel is used as the second depth value of the neighboring pixel corresponding to the center pixel. Alternatively, the neighboring pixels with depth values ​​less than the first depth value are taken as valid neighboring pixels, and the average or weighted average of the depth values ​​of each valid neighboring pixel is used as the second depth value of the neighboring pixel corresponding to the center pixel.

[0119] In this embodiment, step S340 takes each pixel in the image to be blurred as a center pixel and determines the neighboring pixels corresponding to each center pixel. First, it determines the depth value of each center pixel and its corresponding neighboring pixels by using the pixel depth information of each center pixel and its corresponding neighboring pixels. Then, it calculates the first depth value of each center pixel and the second depth value of its corresponding neighboring pixels. Next, it determines the pixel depth difference information between each center pixel and its corresponding neighboring pixels based on the first depth value of each center pixel and the second depth value of its corresponding neighboring pixels. Finally, based on the pixel depth difference information, it compares the magnitude between the first depth value of each center pixel and the second depth value of its corresponding neighboring pixels to determine the depth relationship between each center pixel and its corresponding neighboring pixels.

[0120] In step S352, if the first depth value is less than the second depth value, then based on the first depth value, the second filtering coefficients corresponding to each pixel in the image to be blurred are determined from the first preset lookup table function.

[0121] In step S353, if the first depth value is greater than or equal to the second depth value, then based on the first depth value and the second depth value, the second filter coefficients corresponding to each pixel in the image to be blurred are determined from the second preset lookup table function.

[0122] It should be noted that, in this embodiment, both the first and second preset lookup table functions are pre-defined lookup table functions used to determine the second filtering coefficient of the pixel serving as the center pixel. Specifically, the first preset lookup table function directly determines the second filtering coefficient of the pixel serving as the center pixel based on the first depth value of the center pixel when the first depth value of the center pixel is less than the second depth value of its corresponding neighboring pixels. The second preset lookup table function comprehensively determines the second filtering coefficient of the pixel serving as the center pixel based on both the first depth value of the center pixel and the second depth values ​​of its corresponding neighboring pixels when the first depth value of the center pixel is greater than or equal to the second depth value of its corresponding neighboring pixels.

[0123] It is easy to understand that in this embodiment, when the first depth value is less than the second depth value, the pixel that serves as the center pixel is located closer to the lens of the electronic device within its corresponding pixel block. In this case, based on the first depth value of the pixel, the second filtering coefficient of the pixel is directly determined from the first threshold lookup table function. This reduces the interference from neighboring pixels located farther from the lens of the electronic device, ensuring that neighboring pixels (pixels corresponding to objects farther from the lens of the electronic device) do not blur into the center pixel (pixels corresponding to objects closer to the lens of the electronic device), thus realistically simulating the bokeh effect of a DSLR camera. Conversely, when the first depth value is greater than or equal to the second depth value, the pixel that serves as the center pixel is located farther from the lens of the electronic device within its corresponding pixel block. In this case, based on the first depth value of the pixel and the second depth values ​​of its neighboring pixels, the second filtering coefficient of the pixel is comprehensively determined from the second threshold lookup table function. This ensures that the center pixel located farther from the lens of the electronic device can comprehensively consider the depth information of its surrounding neighboring pixels, avoiding excessive blurring that leads to loss of detail and maintaining the naturalness of the image.

[0124] This implementation uses a first preset lookup table function and a second preset lookup table function to apply different blurring standards to the center pixel within its corresponding pixel block, depending on its distance from the lens of the electronic device. This avoids blurring of objects far from the imaging device into objects close to it, thus realistically simulating the optical blurring effect of a DSLR camera while preventing excessive blurring that could lead to loss of detail. This ensures that pixel blurring is more natural and accurate, and that the main subject area is highlighted, achieving a high-quality background blurring effect.

[0125] Step S360: Based on the first and second filtering coefficients corresponding to each pixel, perform pixel blurring processing on each pixel in the image to be blurred.

[0126] Specifically, when the second filter coefficient remains unchanged, the first filter coefficient is positively correlated with the degree of pixel blurring; when the first filter coefficient remains unchanged, the second filter coefficient is positively correlated with the degree of pixel blurring.

[0127] In this embodiment, step S360 may involve using a fuzzing algorithm to apply the corresponding first filter coefficient to each pixel in the image to be blurred for preliminary pixel blurring, obtaining a preliminarily blurred image to be blurred. Then, the fuzzing algorithm is used to apply the corresponding second filter coefficient to each pixel in the preliminarily blurred image to be blurred for final pixel blurring, obtaining the final blurred target image. Alternatively, the second filter coefficient corresponding to each pixel may be applied first for preliminary pixel blurring, and then the first filter coefficient corresponding to each pixel may be applied first for final pixel blurring. Alternatively, the first and second filter coefficients may be combined to calculate the final filter coefficient for each pixel, and then pixel blurring may be performed pixel by pixel using this final filter coefficient.

[0128] Furthermore, it is worth mentioning that when applying the first filtering coefficient corresponding to each pixel for pixel blurring, the technical solution in Embodiment 2 above can be combined to apply the first blurring coefficient to the foreground region for pixel blurring and apply the second blurring coefficient to the background region for pixel blurring.

[0129] This implementation determines the depth relationship between each pixel and its neighboring pixels by using the pixel depth difference information between the center pixel and its corresponding neighboring pixels. Thus, different blurring standards are adopted for different distances of the center pixel from the lens of the electronic device in its corresponding pixel block. This avoids blurring objects far from the imaging device into objects close to the imaging device, so as to realistically simulate the optical blurring imaging effect of SLR cameras. At the same time, it avoids excessive blurring that leads to loss of details, thereby ensuring that the pixel blurring processing is more natural and accurate, and can highlight the subject area to achieve a high-quality background blurring effect.

[0130] To facilitate understanding of the technical concept or principle of the image blurring method of this application as described in the above embodiments, a specific embodiment is given below:

[0131] In this specific embodiment, an 8-bit RBG (Red-Blue-Green, a color space representation) image is used as the image to be blurred. A pixel with coordinates (i,j) is defined, and its RGB channel values ​​are (p... r,i,j ,p g,i,j ,p b,i,j Its disparity intensity value is D. i,j Where i = 0, 1, ..., h, j = 0, 1, ..., w, represent pixel spatial position indices, h represents the image height of the image to be blurred, w represents the image width of the image to be blurred, and p r,i,j This represents the pixel value (i.e., brightness value) of the pixel in the R channel, p g,i,j p represents the pixel value of that pixel in the G channel. b,i,j This indicates the pixel value of the pixel in the B channel.

[0132] Please refer to Figure 4 The process steps of this specific embodiment are as follows:

[0133] Step S1: Set the depth of field range and preprocess the disparity map (or depth map, or segmentation map) of the image to be blurred.

[0134] In this specific embodiment, the disparity map (or depth map, or segmentation map, where the segmentation map is also called the layer map) of the image to be blurred is first filtered. For example, a bilateral filter can be used to filter the disparity image to smooth the image and make up for the insufficient accuracy of the layer estimation.

[0135] In this specific embodiment, the depth of field range (i.e., the depth of field parameter) is set to (focal). s ,focal e ), where focal s The lower limit of the depth of field range, focal e This represents the upper limit of the depth-of-field range. This range indicates the depth-of-field range for which a sharp subject (i.e., the subject in the depth of field) is expected to be output. This range is the parallax range, or the range of parallax intensity values. It's easy to understand that (0, focal...) s (focal) corresponds to the background area. e ,255) corresponds to the foreground region, (focal s ,focal e The corresponding area is the main area.

[0136] In this specific embodiment, (focal) s ,focal e The depth of field (DNF) can be obtained in real-time through the preview box in the camera's interactive interface on the electronic device, or through the electronic device's autofocus function. That is, the focus area in the image to be blurred is acquired, and the depth-of-field parameters of the main subject area in the image to be blurred are determined based on the depth information of each pixel within the focus area; or, based on the input parameters corresponding to the subject, the main subject area in the image to be blurred is determined, and the depth-of-field parameters of the main subject area in the image to be blurred are determined based on the depth information of each pixel within the subject area.

[0137] In this specific embodiment, for a pixel with pixel coordinates (i,j) (hereinafter referred to as pixel (i,j)), the difference D between its disparity intensity value and the depth range can be calculated. diff,i,j The calculation formula is as follows:

[0138]

[0139] In this specific embodiment, the depth-of-field marker value FlagInFocal for pixel (i,j) can be calculated. i,j To determine whether a pixel is within the depth of field, the following formula is used:

[0140]

[0141] It's not hard to understand that when FlagInFocal i,j When FlagInFocal is 1, it indicates that the pixel is within the depth of field and located in the main subject area. i,j When the value is 0, it indicates that the pixel is outside the depth of field and is located in the foreground and background areas (including the foreground and background areas).

[0142] In this embodiment, the depth-of-field marker value FlagFront for pixel (i,j) can be calculated. i,j To determine whether a pixel is outside the depth of field, the following formula is used:

[0143]

[0144] It's not hard to understand when FlagFront i,j When FlagFront is 1, it indicates that the pixel is outside the depth of field and located in the foreground region. i,j When the value is 0, it indicates that the pixel is outside the depth of field and is located in the background area.

[0145] Using the two marker values ​​mentioned above, this specific embodiment can divide the pixels in the image to be blurred into pixels located within the depth of field and belonging to the main subject region, pixels located outside the depth of field and belonging to the foreground region, and pixels located outside the depth of field and belonging to the background region. This allows for the assignment of different first filter coefficients to pixels located in different regions. In other words, based on pixel depth information and depth of field parameters, the first filter coefficient corresponding to each pixel in the image to be blurred is determined. The farther a pixel is from the focal plane of the main subject region, the larger its corresponding first filter coefficient.

[0146] Step S2: Detect high-brightness light sources.

[0147] In this specific embodiment, after distinguishing the image corresponding to the subject region within the depth of field (i.e., the subject region image) and the images corresponding to the foreground and background regions outside the depth of field (including the foreground region image and the background region image) from the image to be blurred, high-brightness light sources will be detected and processed in the images corresponding to the foreground and background regions outside the depth of field.

[0148] It should be noted that in this specific embodiment, the image to be blurred is an RGB image. Therefore, the detection of high-brightness light sources is performed on each of the RGB three channels. However, when the image to be blurred is a YUV (a color space representation method, where Y is the luminance component and U and V are the chrominance components) image or an HSV (Hue-Saturation-Value, a color space representation method) image, the detection of high-brightness light sources is performed on either the Y channel or the V channel.

[0149] Taking the R channel as an example, the local brightness value (bright) of pixel (i,j) can be calculated. i,j The calculation formula is as follows:

[0150]

[0151] Where region represents the set of pixel coordinates of the neighboring pixels of pixel (i,j), and (m,n) represents the pixel coordinates of the neighboring pixels of pixel (i,j).

[0152] In this specific embodiment, certain conditions can be set to determine whether a pixel (i,j) is a point light source.

[0153] For example, five conditions, a to e, can be set to determine whether pixel (i,j) is a point light source:

[0154]

[0155] Where cond a represents condition a, cond b represents condition b, cond c represents condition c, cond d represents condition d, cond e represents condition e, scale1 represents the configurable multiplicative parameter threshold, and th1, th2 and th3 represent different configurable additive thresholds.

[0156] It should be noted that the above is only one feasible implementation method for determining point light sources. Users can add other conditions or delete some conditions according to actual needs to form new feasible implementation methods.

[0157] In this specific embodiment, if the above-mentioned five preset conditions are simultaneously met, then pixel (i,j) can be determined to be saturated in the current channel, i.e., it is determined to be a point light source. If pixel (i,j) is determined to be a point light source, then for p r,i,j Perform brightness enhancement to obtain the pixel value p of pixel (i,j) in the R channel after brightness enhancement. r,boost,i,j The calculation formula is as follows:

[0158] p r,boost,i,j =p r,i,j ×scale2;

[0159] Here, scale2 represents the configurable multiplicative threshold.

[0160] It is worth mentioning that, in order to facilitate the formation of light spots through blurring operations in subsequent steps, this specific embodiment can use a higher bit width to represent p. r,boost,i,j For example, if the image to be blurred is 8 bits, p r,boost,i,j The maximum value is 255. After increasing the brightness, p r,boost,i,j It can be represented by 10 bits, with a maximum value of 1023.

[0161] This specific embodiment can determine whether a pixel located outside the depth of field and belonging to the foreground and background is a point light source in the R, G, and B channels using the above method. If so, the brightness of that channel is increased, thereby forming light spots of different colors in the final target blurred image.

[0162] It should be noted that when a pixel is identified as a point light source in a certain channel, and the brightness of that channel is increased, the brightness of other channels can also be increased to some extent to prevent the formation of overly saturated light spots, or the brightness of other channels can be suppressed to form light spots with higher saturation. For example, for RGB 3 channels, if the R channel is identified as a point light source, the brightness of the other G and B channels can also be increased to prevent the formation of overly saturated red light spots, or the brightness of the other G and B channels can be suppressed to form red light spots with higher saturation.

[0163] Step S3: Fuse the enhanced point light source and the image to be blurred.

[0164] In this specific embodiment, after the brightness of the pixels belonging to point light sources outside the main area in the image to be blurred is increased, they also need to be fused with the image to be blurred to obtain a new image to be blurred.

[0165] Taking the R channel as an example, pixel (i,j) is the light source point on the R channel. After pixel (i,j) is brightened on the R channel and fused with the image to be blurred, the pixel value of pixel (i,j) on the R channel in the new image to be blurred is p. r,merge,i,j The fusion formula is as follows:

[0166] p r,merge,i,j =p r,i,j +α1×p r,boost,i,j ;

[0167] Where α1 represents the configurable fusion coefficient.

[0168] In this specific embodiment, p r,merge,i,j Similarly, the high bit width can be used to characterize it.

[0169] Step S4, foreground and background blurring scheme based on depth guidance.

[0170] In this specific embodiment, after fusing and enhancing the point light source, it is also necessary to perform blurring processing on the new image to be blurred based on depth guidance, so as to obtain the blurred image.

[0171] Taking the R channel as an example, first calculate the blur radius r of pixel (i,j). i,j , where r i,j It can be obtained through direct calculation, or by dividing the disparity map into different levels and then looking up the table according to the level.

[0172] In one example, r i,j The calculation formula is as follows:

[0173] r i,j =MAX((D diff,i,j / 255)×size kernel,max size kernel,max );

[0174] Where MAX(a,b) represents taking the larger value between a and b, and size kernel,max Indicates the configurable maximum allowed fuzzy radius.

[0175] In another example, the disparity map with disparity intensity values ​​in the range of (0, 255) can be divided into, for example, N = 5 levels, with different levels corresponding to different sizes of blur radius. In this example, the blur filter generated by the blur radius can be preset.

[0176] In this specific embodiment, the blur radius r of pixel point (i,j) is obtained. i,j After that, the corresponding blur filter can be obtained. For example, the radius of the blur filter can be set to r. i,j The filter coefficients of this fuzzy filter are constants.

[0177] In this specific embodiment, in the blurred image obtained after performing a blur operation (i.e., pixel blurring) on ​​the new image to be blurred based on depth guidance, the new pixel value of pixel (i,j) in the R channel is p. r,blur,i,j .

[0178] In one example, p r,blur,i,j The first calculation formula is as follows:

[0179]

[0180] Among them, f d f represents the first filter coefficient obtained from the disparity map. euclid f represents the second filter coefficient obtained from the fuzzy filter. s This represents the custom third filter coefficients, sum. weight f represents the sum of filter coefficient weights. d,m,n f is the first filter coefficient for pixel (m,n). euclid,m,n f is the second filter coefficient for pixel (m,n). s,m,n Let be the third filter coefficient for pixel (m,n).

[0181] In this example, f d,m,n The calculation formula is as follows:

[0182]

[0183] Among them, D m,n Let σ be the disparity intensity value of pixel (m,n). d This indicates configurable parameters.

[0184] It should be noted that, in order to ensure that objects far from the imaging device do not blur into objects close to the imaging device, and to realistically simulate the bokeh effect of SLR optical imaging, this specific embodiment calculates the first filter coefficient f of each neighboring pixel. d,m,n Afterwards, according to D m,n -D i,jThe positive or negative sign of the value is used to determine the depth relationship between the center pixel and its neighboring pixels. For D m,n -D i,j If the disparity intensity value of the center pixel (i,j) is ≥0, meaning it is less than or equal to the disparity intensity value of the neighboring pixel (m,n), then the neighboring pixel can be used to fuse the center pixel (i,j), which is the first filtering coefficient f of that neighboring pixel. d,m,n The value remains unchanged, while for D m,n -D i,j If the disparity intensity value of the center pixel (i,j) is less than 0, meaning the disparity intensity value of the neighboring pixel (m,n) is greater than that of the neighboring pixel, then the first filter coefficient f of the neighboring pixel is... d,m,n The value is set to 0, meaning that neighboring pixels whose depth value is less than the first depth value (the larger the disparity intensity value, the smaller the depth value) are considered valid neighboring pixels (i.e., neighboring pixels whose second filter coefficient value is not 0).

[0185]

[0186] In this example, f s,m,n The calculation formula is as follows:

[0187]

[0188] Among them, LUT s (.) represents a monotonically increasing function.

[0189] The above f s The physical meaning is: the farther the relative distance between the pixel and the focal plane, the greater the f. s The larger the value, the greater the first filter coefficient corresponding to the pixel. That is, the farther the pixel is from the focal plane of the main body region, the larger the first filter coefficient corresponding to that pixel.

[0190] The above f euclid,m,n It can be done at the fuzzy radius r i,j The internal configuration can be set to a constant or other custom function shape to create different shaped light spot effects.

[0191] In this example, the first filter coefficient is the same as the second filter coefficient, and the third filter coefficient is the same as the first filter coefficient. The above f d,m,n The calculation formula and corresponding steps are as follows: Based on the pixel depth difference information between each center pixel and its corresponding neighboring pixels, determine the second filter coefficient corresponding to each pixel in the image to be blurred. The above f s,m,n The calculation formula and corresponding steps are as follows: Based on the pixel depth information and depth range parameters, determine the first filtering coefficient corresponding to each pixel in the image to be blurred.

[0192] In another example, p r,blur,i,j The second calculation formula is as follows:

[0193] p r,blur,i,j =p r,merge,i,j ×f d,i,j ×f euclid,i,j ×f s,i,j ;

[0194] Among them, f d,i,j f is the first filter coefficient for pixel (i,j). euclid,i,j f is the second filter coefficient for pixel (i,j). s,i,j Let be the third filter coefficient for pixel (i,j).

[0195] In this example, f d,i,j =LUT d (D i,j LUT d (.) is a pre-defined lookup function that is monotonically decreasing.

[0196] In this example, the depth-of-field marker value FlagFront can be used. i,j Configure the first filter coefficients for pixels belonging to the foreground and background regions respectively.

[0197] Taking pixel (i,j) as an example, f dij The calculation formula is as follows:

[0198]

[0199] Among them, LUT d,front (.) represents the lookup function for the foreground region (i.e., the first preset lookup function), LUT d,back (.) is the lookup function for the background area (i.e., the second preset lookup function).

[0200] In this example, f euclid,i,j and f s,i,j The calculation method is similar to that in the previous example, and will not be repeated here. Please refer to the previous example for f. s,m,n The calculation formula, and f euclid,m,n The settings.

[0201] In this example, the product of the first filter coefficient and the second filter coefficient is the first filter coefficient. The product of the first filter coefficient and the second filter coefficient, determined by a lookup function for the foreground region, is the first blurring coefficient. The product of the first filter coefficient and the second filter coefficient, determined by a lookup function for the background region, is the second blurring coefficient. The corresponding steps in this example are: determining the foreground and background regions in the image to be blurred based on pixel depth information and depth-of-field parameters; determining the first blurring coefficient for each pixel in the foreground region based on the first preset lookup function; and determining the second blurring coefficient for each pixel in the background region based on the second preset lookup function.

[0202] It should be noted that this specific embodiment can also use a dynamic fusion weighting scheme, where the fusion coefficients can be obtained by, for example, depth and brightness.

[0203] Step S5 yields the output result.

[0204] In this specific embodiment, after performing a blurring operation (i.e. pixel blurring processing) on ​​the new image to be blurred to obtain a blurred image, the new pixel values ​​of each pixel in the blurred image need to be fused with the pixel values ​​in the image to be blurred in order to obtain the pixel values ​​on the final target blurred image.

[0205] Taking the fusion of pixel values ​​in the R channel with those in the image to be blurred as an example, in the R channel, the new pixel value of pixel (i,j) in the blurred image is p. r,blur,i,j The pixel value in the target blurred image is p r,out,i,j p r,out,i,j The calculation formula is as follows:

[0206] p r,out,i,j =p r,blur,i,j ×α2+p r,i,j ×(1-α2);

[0207] Where α2 represents a configurable coefficient.

[0208] In this specific embodiment, after obtaining the target blurred image, artificial noise can be added to the target blurred image, which is a common operation in the blurring process.

[0209] It should be noted that this specific embodiment supports multi-scale image blurring.

[0210] In one example, such as Figure 5 As shown, this specific embodiment can perform image blurring by downsampling by 4 times.

[0211] This example, from input to output, first takes the image to be blurred, its disparity map (or depth map, or segmentation map), and a 4x downsampled image obtained by downsampling the image to be blurred. Then, based on the disparity map (or depth map, or segmentation map), the image to be blurred is blurred to obtain a blurred image. Simultaneously, bright point light sources are detected from the 4x downsampled image and their brightness is enhanced to obtain enhanced point light sources. These enhanced point light sources are then fused into the 4x downsampled image to obtain an enhanced point light source map. Next, the blurred image is rendered into the enhanced point light source map to obtain a rendered image. This rendered image is then upsampled by 2x to obtain a 2x upsampled image. Finally, the 2x upsampled image is fused with the image to be blurred to obtain the target blurred image. Noise is then removed from this image, resulting in a high-quality blurred image output.

[0212] In another example, such as Figure 6 As shown, this specific embodiment can perform image blurring by downsampling by 16 times.

[0213] This example, from input to output, first takes the image to be blurred, its disparity map (or depth map, or segmentation map), and a 16x downsampled image obtained by downsampling the image to be blurred. Then, based on the disparity map (or depth map, or segmentation map), the image to be blurred is blurred to obtain a blurred image. Simultaneously, bright point light sources are detected from the 16x downsampled image and their brightness is enhanced to obtain enhanced point light sources. These enhanced point light sources are then merged into the 16x downsampled image to obtain an enhanced point light source map. Next, the blurred image is rendered into the enhanced point light source map to obtain a rendered image, which is then upsampled by 4x to obtain a 4x upsampled image. Finally, the 4x upsampled image is merged with the image to be blurred to obtain the target blurred image, and noise is removed to achieve a high-quality blurred image output.

[0214] The image blurring effect in this specific embodiment is as follows: Figure 7 and Figure 8 As shown.

[0215] exist Figure 7 In the image, the top left corner is the image to be blurred, the top right corner is the target blurred image obtained after blurring the top left corner image (i.e., pixel blurring), the bottom left corner is a flawed image where the edges between different objects in the background area are not sufficiently blurred when blurring the top left corner image, and the top right corner is a flawed image where the edges between the background area and the subject area are blurred when blurring the top left corner image.

[0216] This specific embodiment utilizes the first filter coefficient f dIt can preserve the edges of objects at similar depths, meaning that the subject within the depth of field and the background at different depths outside the depth of field are less likely to blur during the blurring process, thus solving the problem of boundary sharpness between the subject within the depth of field and objects outside the depth of field, achieving... Figure 7 The effect in the lower left corner is further enhanced by effectively combining the second filter coefficient f. s It can provide higher blur intensity to the background that is far from the depth of the subject, thereby achieving... Figure 7 The effect in the upper right corner maintains both the sharpness of the inner and outer edges of the depth of field and the blur intensity of the blurred background.

[0217] exist Figure 8 In the image, the top left corner is the image to be blurred; the top right corner is the image to be blurred by enhancing the brightness of the point light source during the image blurring process; the bottom left corner is the image to be blurred by enhancing the brightness of the point light source during the image blurring process; and the bottom right corner is the image to be blurred by enhancing the brightness of the point light source during the image blurring process.

[0218] It should be noted that the above examples are only for the purpose of assisting in understanding this application and do not constitute a limitation on the image blurring method of this application. Any simple transformations based on this technical concept are all within the protection scope of this application.

[0219] In addition, please refer to Figure 9 , Figure 9 This is a schematic diagram of the device structure of the hardware operating environment involved in the image blurring method in the embodiments of this application.

[0220] This application also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the image blurring method in the above embodiments.

[0221] The following is for reference. Figure 9The diagram illustrates a structural schematic of an electronic device suitable for implementing the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs (Televisions), desktop computers, or any electronic device capable of performing the above functions. Figure 9 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0222] like Figure 9 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. While electronic devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0223] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0224] The electronic device provided in this application, employing the image blurring method described in the above embodiments, can solve the technical problem in related technologies where blurred images easily exhibit blurred boundaries between the subject and the foreground / background, resulting in unnatural transitions. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the image blurring method provided in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the methods of the above embodiments, and will not be elaborated upon here.

[0225] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0226] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the above claims.

[0227] In addition, this application also provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the steps of the image blurring method in the above embodiments.

[0228] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0229] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0230] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to: acquire pixel depth information of each pixel in the image to be blurred, and determine the depth-of-field range parameter of the main subject region in the image to be blurred; determine a first filtering coefficient corresponding to each pixel in the image to be blurred based on the pixel depth information and the depth-of-field range parameter, wherein the farther the pixel is from the focal plane of the main subject region, the larger the first filtering coefficient corresponding to the pixel; and perform pixel blurring processing on each pixel in the image to be blurred based on the first filtering coefficient corresponding to each pixel, wherein the first filtering coefficient is positively correlated with the degree of pixel blurring.

[0231] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0232] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0233] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0234] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for performing the steps of the above-described image blurring method, which can solve the technical problem in related technologies that blurred images easily exhibit blurred boundaries between the subject and the foreground / background, and unnatural transitions. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the image blurring method provided in the above embodiments, and will not be repeated here.

[0235] Furthermore, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image blurring method as described in the above embodiments.

[0236] The computer program product provided in this application can solve the technical problem in related technologies that blurred images easily exhibit blurred boundaries and unnatural transitions between the subject and the foreground / background. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the image blurring method provided in the above embodiments, and will not be repeated here.

[0237] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An image blurring method, comprising: Obtain pixel depth information of each pixel in the image to be blurred, and determine the depth range parameters of the main area in the image to be blurred; Based on the pixel depth information and the depth of field range parameters, the first filtering coefficient corresponding to each pixel in the image to be blurred is determined, wherein the farther the pixel is from the focal plane of the main body region, the larger the first filtering coefficient corresponding to the pixel is. Based on the first filtering coefficient corresponding to each pixel, pixel blurring processing is performed on each pixel in the image to be blurred, wherein the first filtering coefficient is positively correlated with the degree of pixel blurring.

2. The image blurring method as described in claim 1, characterized in that, Based on the first filtering coefficients corresponding to each pixel, pixel blurring processing is performed on each pixel in the image to be blurred, including: Each pixel in the image to be blurred is taken as the center pixel, and the neighboring pixels corresponding to each center pixel are determined. Based on the pixel depth difference information between each center pixel and its corresponding neighboring pixels, the second filtering coefficients corresponding to each pixel in the image to be blurred are determined. Based on the first and second filtering coefficients corresponding to each pixel, pixel blurring processing is performed on each pixel in the image to be blurred. Wherein, when the second filter coefficient remains unchanged, the first filter coefficient is positively correlated with the degree of pixel blurring; when the first filter coefficient remains unchanged, the second filter coefficient is positively correlated with the degree of pixel blurring.

3. The image blurring method as described in claim 2, characterized in that, Based on the pixel depth difference information between each center pixel and its corresponding neighboring pixels, the second filtering coefficients corresponding to each pixel in the image to be blurred are determined, including: Based on the pixel depth difference information, the magnitude of the first depth value of the center pixel and the second depth value of its corresponding neighboring pixels are compared. If the first depth value is less than the second depth value, then based on the first depth value and the second depth value, the second filtering coefficients corresponding to each pixel in the image to be blurred are determined from the first preset lookup table function. If the first depth value is greater than or equal to the second depth value, then based on the first depth value, the second filtering coefficients corresponding to each pixel in the image to be blurred are determined from the second preset lookup table function.

4. The image blurring method as described in claim 1, characterized in that, The first filtering coefficients corresponding to each pixel include: the first blurring coefficient of each pixel in the foreground region, and the second blurring coefficient of each pixel in the background region; Based on the pixel depth information and the depth-of-field range parameters, the first filtering coefficients corresponding to each pixel in the image to be blurred are determined, including: Based on the pixel depth information and the depth-of-field range parameters, the foreground region and background region in the image to be blurred are determined; Based on a first preset lookup table function, a first blur coefficient for each pixel in the foreground region is determined, and based on a second preset lookup table function, a second blur coefficient for each pixel in the background region is determined. The first blur coefficient is positively correlated with the degree of pixel blur, and the second blur coefficient is positively correlated with the degree of pixel blur.

5. The image blurring method as described in claim 4, characterized in that, Based on the first filtering coefficients corresponding to each pixel, image blurring processing is performed on each pixel in the image to be blurred, including: Based on the first blurring coefficient, pixel blurring processing is performed on each pixel in the foreground region to obtain a pixel-blurred foreground region image; and, Based on the second blurring coefficient, pixel blurring processing is performed on each pixel in the background region to obtain a background region image after pixel blurring processing; Obtain the main region image of the main area in the image to be blurred, and fuse the main region image with the foreground region image and the background region image after pixel blurring to obtain the target blurred image.

6. The image blurring method as described in claim 1, characterized in that, Before obtaining the pixel depth information of each pixel in the image to be blurred, the following steps are included: Acquire an initial image, perform image preprocessing on the initial image to obtain the image to be blurred; The image preprocessing includes at least one of image smoothing, image noise reduction, upsampling, and downsampling.

7. The image blurring method according to any one of claims 1 to 6, characterized in that, Determining the depth-of-field range parameters of the subject region in the image to be blurred includes: Obtain the focus area in the image to be blurred, and determine the depth-of-field range parameters of the main subject area in the image to be blurred based on the depth information of each pixel within the focus area; or, Based on the parameters corresponding to the input depth subject, the subject region in the image to be blurred is determined, and based on the depth information of each pixel in the subject region, the depth range parameters of the subject region in the image to be blurred are determined.

8. An electronic device, characterized in that, include: The image blurring program, stored in the memory and executable on the processor, wherein when executed by the processor, the image blurring program implements the steps of the image blurring method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image blurring program, which, when executed by a processor, implements the steps of the image blurring method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes an image blurring program, which, when executed by a processor, implements the steps of the image blurring method as described in any one of claims 1 to 7.

Images