An image processing method and device, electronic equipment and storage medium

Abstract

The application discloses an image processing method and device, electronic equipment and a storage medium. The method comprises the following steps: extracting a region image with any pixel point as a reference from a to-be-processed image as an input image when low-frequency extraction is performed on a corresponding pixel point; obtaining a guide image when low-frequency extraction is performed on the corresponding pixel point based on the input image of the pixel point, and extracting an edge image of the input image; analyzing the input image, the guide image and the edge image of the pixel point based on a guided filter to obtain a low-frequency extraction coefficient of the corresponding pixel point; obtaining an output image of the corresponding pixel point after low-frequency extraction based on the low-frequency extraction coefficient, the guide image and the edge image of the pixel point; and fusing the output images of all the pixel points in the to-be-processed image to obtain a final output image of the to-be-processed image after low-frequency extraction. In the foregoing manner, the application can improve the extraction capability of low-frequency information of an image edge.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] Edge-preserving image smoothing is a basic problem of image processing and many image-related algorithms, and a large number of methods have been used in image smoothing. For example, a guided filter algorithm, a linear filter algorithm, a bilateral filter algorithm and the like, wherein the guided filter algorithm is used for global processing of an image, and the linear filter algorithm and the bilateral filter algorithm are used for local processing of an image.

[0003] When the guided filter algorithm is used for global processing of an image, low-frequency information of an edge cannot be extracted well, especially when a guided image and an input image are the same image, at this time, an output image obtained by using the guided filter algorithm in an edge region is the input image, that is, the guided filter algorithm cannot extract low-frequency information of an edge region of an image. SUMMARY

[0004] The present application mainly solves the technical problem of providing an image processing method and device, electronic equipment and storage medium, which can improve the extraction capability of low-frequency information of an image edge.

[0005] To solve the above technical problem, the present application provides an image processing method in a first aspect, which comprises the following steps: extracting a region image with any pixel point as a reference from a to-be-processed image as an input image for performing low-frequency extraction of a corresponding pixel point; obtaining a guided image for performing low-frequency extraction of the corresponding pixel point based on the input image of the pixel point, and extracting an edge image of the input image; analyzing the input image, the guided image and the edge image of the pixel point based on a guided filter to obtain a low-frequency extraction coefficient of the corresponding pixel point; obtaining an output image of the corresponding pixel point after low-frequency extraction based on the low-frequency extraction coefficient of the pixel point, the guided image and the edge image; and fusing the output images of all the pixel points in the to-be-processed image to obtain a final output image of the to-be-processed image after low-frequency extraction.

[0006] To solve the above technical problems, the second aspect of the present application provides an image processing device, which comprises an acquisition module, a first determination module, an extraction coefficient determination module, a second determination module and a fusion module, wherein the acquisition module is configured to extract a region image with any pixel point as a reference from a to-be-processed image as an input image when performing low-frequency extraction on the corresponding pixel point; the first determination module is configured to obtain a guide image when performing low-frequency extraction on the corresponding pixel point based on the input image of the pixel point, and extract an edge image of the input image; the extraction coefficient determination module is configured to analyze the input image, the guide image and the edge image of the pixel point based on a guided filter to obtain a low-frequency extraction coefficient of the corresponding pixel point; the second determination module is configured to obtain an output image of the corresponding pixel point after low-frequency extraction based on the low-frequency extraction coefficient, the guide image and the edge image of the pixel point; and the fusion module is configured to fuse the output images of the pixel points in the to-be-processed image to obtain a final output image of the to-be-processed image after low-frequency extraction.

[0007] To solve the above technical problems, the third aspect of the present application provides an electronic device, which comprises a memory and a processor coupled with each other, and the memory stores program instructions; the processor is configured to execute the program instructions stored in the memory to implement the method provided in the first aspect.

[0008] To solve the above technical problems, the fourth aspect of the present application provides a computer readable storage medium for storing program instructions, which can be executed to implement the method provided in the first aspect.

[0009] The present application has the following beneficial effects: different from the prior art, after extracting a region image with any pixel point as a reference from a to-be-processed image as an input image when performing low-frequency extraction on the corresponding pixel point, obtaining a guide image based on the input image, and extracting an edge image of the input image, analyzing the input image, the guide image and the edge image of the pixel point based on a guided filter to obtain a low-frequency extraction coefficient of the corresponding pixel point, obtaining an output image of the corresponding pixel point after low-frequency extraction based on the low-frequency extraction coefficient, the guide image and the edge image of the pixel point, and finally fusing the output images of the pixel points in the to-be-processed image to obtain a final output image of the to-be-processed image after low-frequency extraction, the edge image is introduced, and when the pixel point is in an edge region of the image, the output image of the pixel point can obtain low-frequency information of the edge region based on the edge image, that is, the extraction capability of the low-frequency information of the image edge can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a flowchart of an embodiment of the image processing method provided by the present application;

[0011] Figure 2is a schematic diagram of an embodiment of the input image and the guide image provided by the present application;

[0012] Figure 3 is a schematic diagram of a framework of an embodiment of the image processing device provided by the present application;

[0013] Figure 4 is a schematic diagram of a framework structure of an embodiment of the electronic device provided by the present application;

[0014] Figure 5 is a schematic diagram of a framework of an embodiment of the computer readable storage medium provided by the present application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0016] It should be noted that the description of "first", "second", etc. in the embodiments of the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features with "first" and "second" can explicitly or implicitly include at least one of the features.

[0017] In this document, the term "embodiment" means that the specific features, structures or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean that it refers to the same embodiment, nor is it independent or alternative to other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0018] Please refer to Figure 1 and Figure 2 , Figure 1 is a schematic diagram of a flow of an embodiment of the image processing method provided by the present application, Figure 2 is a schematic diagram of an embodiment of the input image and the guide image provided by the present application; the method comprises:

[0019] S11: Extract a region image with any pixel point as a reference from a to-be-processed image as an input image when performing low-frequency extraction on a corresponding pixel point.

[0020] In an embodiment, the to-be-processed image can be an image that needs to be smoothed. Any pixel point (such as a pixel point at the center of the to-be-processed image) is selected from the to-be-processed image, and a region image with the selected pixel point as a reference is extracted from the to-be-processed image.Figure 2 the pixel point k in the input image, and the input image corresponding to the pixel point is taken as the input image (e.g., I in Figure 2 the image q) of the low-frequency extraction.

[0021] S12: Based on the input image of the pixel point, a guide image is obtained when the low-frequency extraction is performed on the corresponding pixel point, and an edge image of the input image is extracted.

[0022] When the image is filtered by using the guided filter algorithm, a guide image needs to be introduced to filter the input image, so that the output image is as similar as possible to the input image, but the texture part of the output image is similar to the guide image. In an embodiment, the guide image can be an image with the same size as the input image, but the pixel values of the respective pixel points are different. In another embodiment, the guide image can be an image identical to the input image, i.e., the guide image and the input image are the same image. Specifically, after determining the input image corresponding to the pixel point performing the low-frequency extraction, the input image can be selected and taken as the guide image (e.g., I in Figure 2 ) of the corresponding pixel point.

[0023] The edge image can be obtained based on the input image, and the edge image can be an image obtained by performing noise reduction processing on the input image and retaining the edge information of the input image. In a specific embodiment, the input image can be smoothed by using the bilateral filter algorithm to obtain the edge image. It can be understood that in other embodiments, other ways capable of retaining the edges of the image can also be used, such as mean shift filtering, which is not limited here.

[0024] S13: Based on the guided filtering, the input image of the pixel point, the guide image, and the edge image are analyzed to obtain the low-frequency extraction coefficient of the corresponding pixel point.

[0025] In an embodiment, the existing guided filter algorithm can be used to calculate the low-frequency extraction coefficient, and the existing guided filter algorithm is shown in the following formula.

[0026]

[0027] wherein a k and b k are the low-frequency extraction coefficients, I i is the guide image corresponding to the i-th pixel point, p i is the input image corresponding to the i-th pixel point, and ε is a regularization coefficient, and ω k represents an image region with the pixel point k as the center and r as the radius in the input image. By respectively taking the partial derivatives of a k and b k in the above formula and setting the derivatives to zero, a kand b k The calculation results are as follows.

[0028]

[0029]

[0030] wherein, is the mean value of each pixel point in the image region with k as the center and r as the radius in the guide image, is the mean value of each pixel point in the image region with k as the center and r as the radius in the input image, σ k is the variance of each pixel point in the image region with k as the center and r as the radius in the guide image, and ω represents the number of pixel points in the image region with k as the center and r as the radius in the input image.

[0031] When the input image and the guide image are the same image, i.e. I i = p i , then Further, make The existing guide filtering algorithm considers that the guide image and the output image are a local linear model in a two-dimensional window, and thus assumes that the output image is a linear transformation q k of the guide image in the window w i = a k I i + b k , When the pixel point k is in the edge region of the image, σ k tends to 1, so that a k also tends to 1, b k tends to 0, so that q i tends to I i , i.e. the output image of the edge region is approximately equal to the input image, and thus the image low-frequency information of the edge region cannot be effectively extracted. When the pixel point k is not in the edge region of the image, σ k changes little, for example, when the pixel point k is in a flat region, σ k tends to 0, so that a k also tends to 0, b k tends to , so that q i tends to , i.e. the output image of the flat region is approximately equal to the pixel mean value of the guide image.

[0032] As can be seen from the above, existing guided filtering algorithms cannot effectively extract low-frequency information from image edge regions, thus failing to enhance the texture of edge regions during image enhancement. Furthermore, the larger the filter radius, the less significant the texture enhancement near the edges. Therefore, this application improves upon existing guided filtering algorithms.

[0033] In another embodiment, this application constructs a low-frequency signal energy function based on existing guided filtering algorithms, using low-frequency extraction coefficients as unknowns and characterizing the relationship between the guiding image, edge image, and output image. Based on the low-frequency signal energy function, and analyzing the guiding and edge images of each pixel, the low-frequency extraction coefficients for the corresponding pixels are obtained. Finally, the linear relationship between the guiding image, edge image, and output image is determined based on the low-frequency extraction coefficients. The low-frequency extraction coefficients in the low-frequency signal energy function can be only a... k .

[0034] In one specific implementation, constructing the low-frequency signal energy function includes the following six steps: Step 1, obtaining the low-frequency extraction coefficient a of the corresponding pixel. k With the pixel I of the guide image i The product of these two parameters is used as the first parameter, i.e., the first parameter is a. k I i And obtain the pixel variance σ of the guide image. k The first regularization coefficient ε1 and the second regularization coefficient ε2 are used to adjust the filtering intensity within the first pixel range, and the second regularization coefficient ε2 is used to adjust the filtering intensity within the second pixel range, which is smaller than the first pixel range. Both the first and second pixel ranges are defined by the filtering radius r; the smaller the filtering radius r, the smaller the pixel range. Step two: Based on the pixel variance σ of the guiding image... k The second regularization coefficient ε2 determines the marginal coefficient α. k Among them, the marginal coefficient α k This represents the probability that a pixel belongs to an image edge. Specifically, it can be obtained as the square of the pixel variance of the guiding image. The sum of the second regularization coefficient ε2 is used as the tenth parameter; based on the square of the pixel variance of the guiding image. The ratio of the marginal coefficient α to the tenth parameter determines the marginal coefficient α. k ,Right now In other implementations, edge coefficient α can also be generated using other methods such as statistical gradient information or texture information. k Step 3, based on the edge coefficient α k Edge image, low-frequency extraction coefficient a k and the average pixel value of the guide image Determine the second parameter. Specifically, obtain the preset value and the low-frequency extraction coefficient 'a'.k a first difference value, the first difference value being 1-a k , and a preset value and the edge coefficient a k a second difference value, the second difference value being 1-a k ; multiplying the edge coefficient a k and a pixel T i of the edge image, as a sixth parameter, i.e., the sixth parameter being a k T i , and multiplying the second difference value and a pixel mean value of the guide image, as a seventh parameter, i.e., the seventh parameter being summing the sixth parameter and the seventh parameter, to obtain an eighth parameter, i.e., the eighth parameter being multiplying the first difference value and the eighth parameter, as a second parameter, i.e., the second parameter being wherein the preset value can be 1. In step four, summing the first parameter and the second parameter and a square difference value k of the pixel of the input image when performing the low-frequency extraction on the corresponding pixel, as a third parameter of the pixel, i.e., the third parameter being In step five, summing the third parameters of all the pixels, as a fourth parameter, and multiplying a square of a first regularization coefficient e1 and a low-frequency extraction coefficient a k , as a fifth parameter. In step six, obtaining a low-frequency signal energy function based on a sum of the fourth parameter and the fifth parameter. The low-frequency signal energy function obtained by the above manner is shown in the following formula.

[0035]

[0036] wherein the low-frequency extraction coefficient a k , I i is a guide image corresponding to the i-th pixel, a k is an edge coefficient, T i is an edge image corresponding to the i-th pixel, is a mean value of all the pixels in an image region with a radius of r in the guide image and with the i-th pixel as the center, p i is an input image corresponding to the i-th pixel, and e1 is a first regularization coefficient.

[0037] In another specific embodiment, the low-frequency signal energy function further includes a texture parameter, the texture parameter representing texture information of the input image. Since the edge image is obtained by smoothing the input image, the pixel of the edge image can be equal to the difference between the pixel of the guide image and the texture parameter. Therefore, the sixth parameter in the above embodiment is the product of the difference between the pixel of the guide image and the texture parameter and the edge coefficient, i.e., the sixth parameter being a k (I i -Texi ). Therefore, the low-frequency signal energy function can also be expressed by the following formula.

[0038]

[0039] In other specific embodiments, the low-frequency signal energy function further comprises a texture reservation parameter, which is used to adjust the reservation degree of strong texture and / or weak texture in the output image, specifically, the pixel I i and the product of the texture reservation parameter β and the texture parameter T exi is subtracted, to obtain a ninth parameter, i.e., the ninth parameter is I i - βT exi . The product of the ninth parameter and the edge coefficient α k is taken as a sixth parameter, so that the low-frequency signal energy function obtained is shown in the following formula.

[0040]

[0041] The greater the texture reservation parameter is, the more texture information is reserved.

[0042] The constructed low-frequency signal energy function is obtained, and based on the low-frequency signal energy function, the guide image and the edge image of the pixel point, the low-frequency extraction coefficient corresponding to the pixel point can be obtained. Specifically, taking the low-frequency signal energy function as an example, when the input image and the guide image are the same image, the low-frequency signal energy function is converted to By deriving the low-frequency extraction coefficient a k and making the derived equation equal to 0, the following equation can be obtained: For the convenience of analysis, let s k , then

[0043] S14: Based on the low-frequency extraction coefficient of the pixel point, the guide image and the edge image, the output image after low-frequency extraction of the corresponding pixel point is obtained.

[0044] The edge image of the pixel point and the reference image representing the overall situation of the guide image of the corresponding pixel point are fused once based on the edge coefficient to obtain a fused image of the corresponding pixel point. Specifically, based on the edge coefficient, a first weight for performing once fusion on the edge image of the corresponding pixel point is determined, and based on the first weight, a second weight negatively correlated with the first weight is determined. The edge image and the reference image are weighted based on the first weight and the second weight respectively to obtain the fused image. In an embodiment, the first weight can be the edge coefficient, and the second weight can be the difference between a preset value and the edge coefficient. The preset value can be 1. At this time, a first product between the first weight and a pixel of the edge image and a second product between the second weight and a pixel of the reference image are obtained, and the first product and the second product are summed to obtain a pixel of the fused image. In another embodiment, the first weight can also be the difference between a preset value and the edge coefficient, and the second weight can be the edge coefficient. The first product between the second weight and a pixel of the edge image and the second product between the first weight and a pixel of the reference image are summed to obtain a pixel of the fused image. The pixel of the reference image representing the overall situation of the guide image of the corresponding pixel point can be a pixel mean of each pixel point in an image region with a radius of r centered on the i th pixel point in the guide image.

[0045] The guide image and the fused image of the pixel point are fused twice based on the low-frequency extraction coefficient to obtain an output image of the corresponding pixel point after low-frequency extraction. Specifically, based on the low-frequency extraction coefficient of the pixel point, a third weight for performing twice fusion on the guide image of the corresponding pixel point is determined, and based on the third weight, a fourth weight negatively correlated with the third weight is determined. The guide image and the fused image are weighted based on the third weight and the fourth weight respectively to obtain the output image. In an embodiment, the low-frequency extraction coefficient can be taken as the third weight, and the difference between a preset value and the low-frequency extraction coefficient can be taken as the fourth weight. The preset value can be 1 or other values, which can be set by a user. A third product between the third weight and a pixel of the guide image and a fourth product between the fourth weight and a pixel of the fused image are summed to obtain the output image of the corresponding pixel point after low-frequency extraction. Understandably, in another embodiment, the low-frequency extraction coefficient can also be taken as the fourth weight, and the difference between a preset value and the low-frequency extraction coefficient can be taken as the third weight. At this time, a product between the fourth weight and a pixel of the guide image is taken as the third product, and a product between the fourth weight and a pixel of the fused image is taken as the fourth product.

[0046] The output image of the corresponding pixel point after low-frequency extraction obtained through the above manner can be represented by the following formula.

[0047]

[0048] wherein q iis the output image corresponding to the i-th pixel point, I i is the guide image corresponding to the i-th pixel point, T i is the edge image corresponding to the i-th pixel point, E is the mean value of each pixel point in the image region with radius r centered at k in the guide image, a k is the low-frequency extraction coefficient, a k is the edge coefficient.

[0049] S15: fusing the output images of each pixel point in the to-be-processed image to obtain the final output image of the to-be-processed image after low-frequency extraction.

[0050] After the output images of each pixel point are determined, the output images of each pixel point can be fused to obtain the final output image of the to-be-processed image after low-frequency extraction. In an embodiment, the output images of each pixel point can be spliced to obtain the final output image. It can be understood that other fusion methods can also be used, which are not limited herein.

[0051] In the above manner, when the guide image and the input image are the same image, and the i-th pixel point is near the edge of the image, the edge coefficient a k tends to 1, the variance σ k of each pixel point in the image region with radius r centered at k in the guide image increases, so that the edge coefficient a k tends to 1, and the low-frequency extraction coefficient a When there is strong texture near the edge region, S k increases, so that the low-frequency extraction coefficient a k tends to 1, and at this time the output image is approximately equal to the input image, and the output image will retain the strong texture information of the edge of the output image as much as possible; when there is weak texture near the edge region, S k decreases, so that the low-frequency extraction coefficient a k tends to 0, and at this time the output image is approximately equal to the edge image, and the output image will discard the weak texture information in the input image as much as possible. Further, when the output image is subsequently enhanced, the weak texture near the edge region can be enhanced more than the strong texture, reducing the case of over-enhancement of the strong texture.

[0052] When the guide image and the input image are the same image, and the i-th pixel point is in a flat region of the image, the variance σ k of each pixel point in the image region with radius r centered at k in the guide image decreases, so that the edge coefficient a k tends to 0, the low-frequency extraction coefficient a k tends to 0, and the output image tends to the mean value of each pixel point in the image region with radius r centered at k in the guide image.

[0053] The above method extracts a region image with any pixel point as a reference from the image to be processed as an input image for performing low-frequency extraction on the corresponding pixel point, obtains a guide image based on the input image, and extracts an edge image of the input image, analyzes the input image, the guide image and the edge image of the pixel point based on the guide filter to obtain a low-frequency extraction coefficient of the corresponding pixel point, obtains an output image of the corresponding pixel point after low-frequency extraction based on the low-frequency extraction coefficient of the pixel point, the guide image and the edge image, and finally fuses the output images of the pixel points in the image to be processed to obtain the final output image of the image to be processed after low-frequency extraction. By introducing the edge image, when the pixel point is in the edge region of the image, the output image of the pixel point can obtain the low-frequency information of the edge region based on the edge image, thereby improving the extraction capability of the low-frequency information of the image edge.

[0054] Referring to Figure 3 , Figure 3 is a schematic diagram of an embodiment of the image processing device provided by the present application. The image processing device 30 comprises an acquisition module 31, a first determination module 32, an extraction coefficient determination module 33, a second determination module 34 and a fusion module 35. The acquisition module 31 is configured to extract a region image with any pixel point as a reference from the image to be processed as an input image for performing low-frequency extraction on the corresponding pixel point. The first determination module 32 is configured to obtain a guide image for performing low-frequency extraction on the corresponding pixel point based on the input image of the pixel point, and extract an edge image of the input image. The extraction coefficient determination module 33 is configured to analyze the input image, the guide image and the edge image of the pixel point based on the guide filter to obtain a low-frequency extraction coefficient of the corresponding pixel point. The second determination module 34 is configured to obtain an output image of the corresponding pixel point after low-frequency extraction based on the low-frequency extraction coefficient of the pixel point, the guide image and the edge image. The fusion module 35 is configured to fuse the output images of the pixel points in the image to be processed to obtain the final output image of the image to be processed after low-frequency extraction.

[0055] By introducing the edge image, when the pixel point is in the edge region, the output image corresponding to the pixel point can extract the low-frequency information of the image edge based on the edge image.

[0056] In an embodiment, obtaining a guide image for performing low-frequency extraction on the corresponding pixel point based on the input image of the pixel point comprises: selecting the input image of the pixel point as the guide image of the corresponding pixel point.

[0057] Analyzing the input image, the guide image and the edge image of the pixel point based on the guide filter to obtain a low-frequency extraction coefficient of the corresponding pixel point comprises: analyzing the guide image and the edge image of the pixel point based on the guide filter to obtain a low-frequency extraction coefficient of the corresponding pixel point.

[0058] When the guiding image and the input image are the same image, the output image obtained by the existing guided filter algorithm in the edge region of the image is the input image, so the low-frequency information of the edge region cannot be extracted. However, in the above manner, the output image obtained in the edge region of the image is the edge image, so the low-frequency information of the edge region can be effectively extracted.

[0059] In an embodiment, based on the guided filter, the guiding image and the edge image of a pixel point are analyzed to obtain a low-frequency extraction coefficient of the corresponding pixel point, including: based on the guided filter, a low-frequency signal energy function is constructed, taking the low-frequency extraction coefficient as an unknown quantity and representing the relationship between the guiding image, the edge image and the output image; based on the low-frequency signal energy function, and the guiding image and the edge image of the pixel point, the low-frequency extraction coefficient of the corresponding pixel point is obtained.

[0060] In an embodiment, based on the guided filter, a low-frequency signal energy function is constructed, taking the low-frequency extraction coefficient as an unknown quantity and representing the relationship between the guiding image, the edge image and the output image, including: the product of the low-frequency extraction coefficient of the corresponding pixel point and the pixel of the guiding image is obtained as a first parameter, and the pixel variance of the guiding image, a first regularization coefficient and a second regularization coefficient are obtained, wherein the first regularization coefficient is used to adjust the filtering strength in the first pixel range, and the second regularization coefficient is used to adjust the filtering strength in the second pixel range, and the second pixel range is smaller than the first pixel range; based on the pixel variance of the guiding image and the second regularization coefficient, an edge coefficient is determined; wherein the edge coefficient represents the probability that the pixel point belongs to the edge of the image; based on the edge coefficient, the edge image, the low-frequency extraction coefficient and the pixel mean value of the guiding image, a second parameter is determined; the square of the difference between the pixel of the input image when the low-frequency extraction is performed on the corresponding pixel point and the sum of the first parameter and the second parameter is taken as a third parameter of the pixel point; the sum of the third parameters of each pixel point is taken as a fourth parameter, and the product of the first regularization coefficient and the square of the low-frequency extraction coefficient is taken as a fifth parameter; based on the sum of the fourth parameter and the fifth parameter, the low-frequency signal energy function is obtained.

[0061] In an embodiment, based on the edge coefficient, the edge image, the low-frequency extraction coefficient and the pixel mean value of the guiding image, the second parameter is determined, including: a first difference value between a preset value and the low-frequency extraction coefficient is obtained, and a second difference value between the preset value and the edge coefficient is obtained; the product of the edge coefficient and the pixel of the edge image is taken as a sixth parameter, and the product of the second difference value and the pixel mean value of the guiding image is taken as a seventh parameter; the sum of the sixth parameter and the seventh parameter is obtained to obtain an eighth parameter; the product of the first difference value and the eighth parameter is taken as the second parameter.

[0062] In an embodiment, the low-frequency signal energy function is constructed based on the guided filtering, the low-frequency extraction coefficient is taken as an unknown quantity, and the low-frequency signal energy function represents a relationship among the guide image, the edge image, and the output image, and the method further comprises: obtaining a texture parameter; wherein the texture parameter represents texture information of the input image, and a pixel of the edge image is equal to a difference between a pixel of the guide image and the texture parameter.

[0063] The product of the edge coefficient and the pixel of the edge image is taken as the sixth parameter, and the method comprises: taking the product of the difference between the pixel of the guide image and the texture parameter and the edge coefficient as the sixth parameter.

[0064] In an embodiment, the low-frequency signal energy function is constructed based on the guided filtering, the low-frequency extraction coefficient is taken as an unknown quantity, and the low-frequency signal energy function represents a relationship among the guide image, the edge image, and the output image, and the method further comprises: obtaining a texture preservation parameter, the texture preservation parameter being used to adjust a texture preservation degree in the output image.

[0065] The product of the difference between the pixel of the guide image and the texture parameter and the edge coefficient is taken as the sixth parameter, and the method comprises: obtaining a ninth parameter by subtracting the product of the pixel of the guide image and the texture preservation parameter and the texture parameter; and taking the product of the ninth parameter and the edge coefficient as the sixth parameter.

[0066] In an embodiment, the edge coefficient is determined based on the pixel variance of the guide image and a second regularization coefficient, and the method comprises: obtaining a tenth parameter by summing the square of the pixel variance of the guide image and the second regularization coefficient; and determining the edge coefficient based on a ratio of the square of the pixel variance of the guide image to the tenth parameter.

[0067] In an embodiment, the output image of the corresponding pixel point after low-frequency extraction is obtained based on the low-frequency extraction coefficient of the pixel point, the guide image, and the edge image, and the method comprises: performing one-time fusion on the edge image of the pixel point based on the edge coefficient and a reference image representing an overall situation of the guide image of the corresponding pixel point to obtain a fusion image of the corresponding pixel point; and performing two-time fusion on the guide image and the fusion image of the pixel point based on the low-frequency extraction coefficient to obtain the output image of the corresponding pixel point after low-frequency extraction.

[0068] In an embodiment, the fusion image of the corresponding pixel point is obtained by performing one-time fusion on the edge image of the pixel point based on the edge coefficient and a reference image representing an overall situation of the guide image of the corresponding pixel point, and the method comprises: determining a first weight for performing one-time fusion on the edge image of the corresponding pixel point based on the edge coefficient, and determining a second weight negatively correlated with the first weight based on the first weight; and weighting the edge image and the reference image based on the first weight and the second weight respectively to obtain the fusion image.

[0069] In an embodiment, the guided image and the fused image of the pixel point are secondarily fused based on the low-frequency extraction coefficient to obtain an output image of the pixel point after low-frequency extraction, including: determining a third weight when the guided image of the pixel point is secondarily fused based on the low-frequency extraction coefficient of the pixel point, and determining a fourth weight negatively correlated with the third weight based on the third weight; weighting the guided image and the fused image based on the third weight and the fourth weight respectively to obtain the output image.

[0070] Referring to Figure 4 , Figure 4 is a frame structure schematic diagram of an embodiment of the electronic device provided in the present application.

[0071] The electronic device 40 includes a memory 41 and a processor 42 coupled with each other. The memory 41 stores program instructions. The processor 42 is configured to execute the program instructions stored in the memory 41 to implement the steps of any of the image processing method embodiments described above. In a specific implementation scenario, the electronic device 40 can include but is not limited to a microcomputer, a server, and in addition, the electronic device 40 can also include a notebook computer, a tablet computer and other mobile devices, which are not limited herein.

[0072] Specifically, the processor 42 is configured to control itself and the memory 41 to implement the steps of any of the image processing method embodiments described above. The processor 42 can also be referred to as a CPU (Central Processing Unit). The processor 42 can be an integrated circuit chip with processing capability. The processor 42 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 42 can be implemented by an integrated circuit chip.

[0073] Referring to Figure 5 , Figure 5 is a frame structure schematic diagram of an embodiment of the computer readable storage medium provided in the present application.

[0074] The computer readable storage medium 50 stores program instructions 51. The program instructions 51 are executed by the processor to implement the steps of any of the image processing method embodiments described above.

[0075] The computer readable storage medium 50 can specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, which can store a computer program, or can be a server storing the computer program, which can send the stored computer program to other devices for running, or can run the stored computer program by itself.

[0076] The above description of various embodiments tends to emphasize the differences between various embodiments, and the same or similar parts can be referred to each other, and will not be described herein for the sake of brevity.

[0077] In several embodiments provided in the present application, it should be understood that the disclosed method and device can be implemented in other ways. For example, the above-described device implementation is only schematic, for example, the division of the module or unit is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0078] The unit described as a separate component can be or can not be physically separated, and the component displayed as a unit can be or can not be a physical unit, that is, can be located in one place, or can be distributed to a plurality of network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the present embodiment scheme.

[0079] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0080] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0081] If the technical solutions of the present application involve personal information, the product applying the technical solutions of the present application has been explicitly informed of the personal information processing rules before processing the personal information and has obtained the personal independent consent. If the technical solutions of the present application involve sensitive personal information, the product applying the technical solutions of the present application has obtained the personal independent consent before processing the sensitive personal information and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as a camera, an explicit and prominent mark is set to inform that the personal information collection range has been entered and the personal information will be collected. If the individual voluntarily enters the collection range, it is regarded as consent to collect the personal information. Or, on the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the individual to upload the personal information by himself / herself. The personal information processing rules can include the personal information processor, the processing purpose, the processing method, and the type of processed personal information.

[0082] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation based on the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that, include: Extract the region image based on any pixel from the image to be processed, and use it as the input image when performing low-frequency extraction on the corresponding pixel; Based on the input image of the pixel, a guide image is obtained when performing low-frequency extraction on the corresponding pixel, and the edge image of the input image is extracted; Based on guided filtering, a low-frequency signal energy function is constructed, which uses the low-frequency extraction coefficients as unknowns and characterizes the relationship between the guided image, edge image and output image. Based on the low-frequency signal energy function, and the guiding image and edge image of the pixel, the low-frequency extraction coefficients corresponding to the pixel are obtained. Based on the low-frequency extraction coefficients of the pixel, the guiding image, and the edge image, the output image corresponding to the pixel after low-frequency extraction is obtained. The output images of each pixel in the image to be processed are fused to obtain the final output image of the image to be processed after low-frequency extraction. The steps for constructing the low-frequency signal energy function include: The product of the low-frequency extraction coefficient corresponding to the pixel and the pixel of the guide image is obtained as the first parameter. The pixel variance, the first regularization coefficient, and the second regularization coefficient of the guide image are also obtained. The first regularization coefficient is used to adjust the filtering intensity within the first pixel range, and the second regularization coefficient is used to adjust the filtering intensity within the second pixel range. The second pixel range is smaller than the first pixel range. An edge coefficient is determined based on the pixel variance of the guiding image and the second regularization coefficient; wherein the edge coefficient represents the probability that the pixel belongs to the edge of the image; Obtain a first difference between a preset value and the low-frequency extraction coefficient, and a second difference between the preset value and the edge coefficient; The product of the edge coefficient and the pixels of the edge image is used as the sixth parameter, and the product of the second difference and the pixel mean of the guide image is used as the seventh parameter. Summing the sixth and seventh parameters yields the eighth parameter; The product of the first difference and the eighth parameter is used as the second parameter; The third parameter of the pixel is the square of the difference between the sum of the first parameter and the second parameter and the pixel of the input image when performing low-frequency extraction on the corresponding pixel. The sum of the third parameters of each pixel is obtained as the fourth parameter, and the product of the square of the first regularization coefficient and the low-frequency extraction coefficient is obtained as the fifth parameter. The low-frequency signal energy function is obtained based on the sum of the fourth and fifth parameters.

2. The method according to claim 1, characterized in that, The process of obtaining a guide image for performing low-frequency extraction on the input image based on the pixels includes: The input image of the pixel is selected as the guide image for the corresponding pixel.

3. The method according to claim 1, characterized in that, The step of constructing a low-frequency signal energy function based on the guided filtering, with the low-frequency extraction coefficients as unknowns and characterizing the relationship between the guided image, edge image, and output image, further includes: Obtain texture parameters; wherein the texture parameters represent the texture information of the input image, and the pixels of the edge image are equal to the difference between the pixels of the guide image and the texture parameters; The step of using the product of the edge coefficient and the pixels of the edge image as the sixth parameter includes: The sixth parameter is the product of the difference between the pixels of the guiding image and the texture parameter and the edge coefficient.

4. The method according to claim 3, characterized in that, The step of constructing a low-frequency signal energy function based on the guided filtering, with the low-frequency extraction coefficients as unknowns and characterizing the relationship between the guided image, edge image, and output image, further includes: Obtain texture preservation parameters, which are used to adjust the degree of texture preservation in the output image; The step of using the product of the difference between the pixels of the guiding image and the texture parameter and the edge coefficient as the sixth parameter includes: The ninth parameter is obtained by subtracting the product of the pixels of the guiding image and the texture preservation parameter and the texture parameter; The product of the ninth parameter and the edge coefficient is used as the sixth parameter.

5. The method according to claim 1, characterized in that, The step of determining the edge coefficient based on the pixel variance of the guiding image and the second regularization coefficient includes: The sum of the square of the pixel variance of the guide image and the second regularization coefficient is obtained as the tenth parameter; The edge coefficient is determined based on the ratio of the square of the pixel variance of the guiding image to the tenth parameter.

6. The method according to claim 1 or 2, characterized in that, The process of obtaining the output image corresponding to the pixel after low-frequency extraction based on the low-frequency extraction coefficients, the guiding image, and the edge image includes: The edge image of the pixel and the reference image representing the overall situation of the guiding image corresponding to the pixel are fused together to obtain the fused image corresponding to the pixel. The guiding image and the fused image of the pixel are fused a second time based on the low-frequency extraction coefficient to obtain the output image of the corresponding pixel after low-frequency extraction.

7. The method according to claim 6, characterized in that, The process of fusing the edge image of the pixel based on the edge coefficient and the reference image representing the overall situation of the guiding image corresponding to the pixel to obtain the fused image corresponding to the pixel includes: Based on the edge coefficient, a first weight is determined when the edge image corresponding to the pixel is fused once, and based on the first weight, a second weight that is negatively correlated with the first weight is determined; The edge image and the reference image are weighted based on the first weight and the second weight, respectively, to obtain the fused image.

8. The method according to claim 6, characterized in that, The step of performing a secondary fusion of the guiding image and the fused image of the pixel based on the low-frequency extraction coefficient to obtain the output image of the corresponding pixel after low-frequency extraction includes: Based on the low-frequency extraction coefficients of the pixel, a third weight is determined when the guiding image corresponding to the pixel performs the secondary fusion, and based on the third weight, a fourth weight that is negatively correlated with the third weight is determined. The guiding image and the fused image are weighted based on the third weight and the fourth weight, respectively, to obtain the output image.

9. An image processing apparatus, characterized in that, include: The acquisition module is used to extract a region image based on any pixel from the image to be processed, and use it as the input image when performing low-frequency extraction for the corresponding pixel. The first determining module is used to obtain a guiding image for performing low-frequency extraction on the corresponding pixel based on the input image of the pixel, and to extract the edge image of the input image; The extraction coefficient determination module is used to construct a low-frequency signal energy function based on guided filtering, which uses the low-frequency extraction coefficient as an unknown quantity and characterizes the relationship between the guiding image, edge image and output image; and to analyze the low-frequency signal energy function, the guiding image and edge image of the pixel to obtain the low-frequency extraction coefficient corresponding to the pixel. The second determining module is used to obtain the output image corresponding to the pixel after low-frequency extraction based on the low-frequency extraction coefficient of the pixel, the guiding image and the edge image; The fusion module is used to fuse the output images based on each pixel in the image to be processed to obtain the final output image of the image to be processed after low-frequency extraction. The steps for constructing the low-frequency signal energy function include: The product of the low-frequency extraction coefficient corresponding to the pixel and the pixel of the guide image is obtained as the first parameter. The pixel variance, the first regularization coefficient, and the second regularization coefficient of the guide image are also obtained. The first regularization coefficient is used to adjust the filtering intensity within the first pixel range, and the second regularization coefficient is used to adjust the filtering intensity within the second pixel range. The second pixel range is smaller than the first pixel range. An edge coefficient is determined based on the pixel variance of the guiding image and the second regularization coefficient; wherein the edge coefficient represents the probability that the pixel belongs to the edge of the image; Obtain a first difference between a preset value and the low-frequency extraction coefficient, and a second difference between the preset value and the edge coefficient; The product of the edge coefficient and the pixels of the edge image is used as the sixth parameter, and the product of the second difference and the pixel mean of the guide image is used as the seventh parameter. Summing the sixth and seventh parameters yields the eighth parameter; The product of the first difference and the eighth parameter is used as the second parameter; The third parameter of the pixel is the square of the difference between the sum of the first parameter and the second parameter and the pixel of the input image when performing low-frequency extraction on the corresponding pixel. The sum of the third parameters of each pixel is obtained as the fourth parameter, and the product of the square of the first regularization coefficient and the low-frequency extraction coefficient is obtained as the fifth parameter. The low-frequency signal energy function is obtained based on the sum of the fourth and fifth parameters.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor that are coupled to each other. The memory stores program instructions; The processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed to implement the method as described in any one of claims 1-8.

Images