Image enhancement method and device, electronic equipment and storage medium

Abstract

The invention provides an image enhancement method and device, electronic equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: carrying out the guided image filtering of a to-be-processed image, and obtaining a smooth layer and a detail layer of the to-be-processed image; determining a target adaptive enhancement control gain based on a linear model coefficient in a guided image filtering process, and enhancing the detail layer by using the target adaptive enhancement control gain to obtain an enhanced detail layer, the value of the linear model coefficient at the edge pixel point being greater than the value of the linear model coefficient at the smooth region pixel point; and fusing the smooth layer and the enhanced detail layer to obtain a detail enhanced image of the to-be-processed image. According to the technical scheme provided by the invention, the amplification of the noise in the smooth region can be suppressed while the detail information of the image is enhanced, and adaptive image detail enhancement for suppressing noise amplification is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image enhancement method, device, electronic equipment and storage medium. Background Art

[0002] Detail enhancement is a technical link in the image enhancement process to enhance the detail texture in the image. The visual effect of the image can be improved by enhancing the detail texture. In the detail enhancement process, the image to be processed can be decomposed into a smooth layer and a detail layer by performing edge-preserving smoothing filtering on the image to be processed. The detail layer is enhanced and then superimposed with the smooth layer to obtain a detail-enhanced image. This method will enhance the detail information and noise contained in the detail layer equally, and cannot distinguish between the smooth area noise and the detail area texture. If the image details are to be enhanced, the noise will also be amplified in the same proportion; if the noise amplification is to be suppressed, it will lead to the loss of image details and cause the image details to be blurred. Therefore, in the detail enhancement process, how to suppress the amplification of the smooth area noise while enhancing the image detail information is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides an image enhancement method, device, electronic device and storage medium, which are used to solve the problem in the prior art of how to suppress the amplification of noise in smooth areas while enhancing image detail information, and realize adaptive image detail enhancement with suppressed noise amplification.

[0004] The present invention provides an image enhancement method, comprising:

[0005] Performing guided image filtering on the image to be processed to obtain a smoothing layer and a detail layer of the image to be processed;

[0006] Determining a target adaptive enhancement control gain based on a linear model coefficient in a guided image filtering process, and enhancing the detail layer using the target adaptive enhancement control gain to obtain an enhanced detail layer; wherein the value of the linear model coefficient at an edge pixel point is greater than the value at a smooth area pixel point;

[0007] The smoothing layer and the enhanced detail layer are merged to obtain a detail-enhanced image of the image to be processed.

[0008] According to an image enhancement method provided by the present invention, the guided image filtering is performed on the image to be processed to obtain the smoothing layer and the detail layer of the image to be processed, comprising:

[0009] Determining an edge perception weight based on an image grayscale range, a total number of pixels, and a second pixel variance under a second filtering window of the guide image;

[0010] Determining the linear model coefficients based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge-aware weights;

[0011] Determining a linear model bias based on a pixel average of the image to be processed under the first filtering window, a pixel average of the guide image under the first filtering window, and the linear model coefficient;

[0012] Filtering the image to be processed by using the linear model coefficient and the linear model bias to obtain the smoothing layer;

[0013] The image to be processed is subtracted from the smooth layer to obtain the detail layer.

[0014] According to an image enhancement method provided by the present invention, the linear model coefficient is determined based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge perception weight, including:

[0015] Determining local linear model coefficients based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge perception weight;

[0016] For each pixel point in the image to be processed, determining the overlapping filtering window where the pixel point is located;

[0017] Based on the guiding kernel weight of each overlapping filter window, the local linear model coefficients of each overlapping filter window are weighted averaged to obtain the linear model coefficient; the guiding kernel weight represents the weight of the edge direction.

[0018] According to an image enhancement method provided by the present invention, determining a weighted linear model bias based on a pixel average of the image to be processed under the first filtering window, a pixel average of the guide image under the first filtering window, and the linear model coefficient includes:

[0019] Determining a linear model bias based on a pixel average of the image to be processed under the first filtering window, a pixel average of the guide image under the first filtering window, and the linear model coefficient;

[0020] For each pixel point in the image to be processed, determining the overlapping filtering window where the pixel point is located;

[0021] Based on the guiding kernel weights of the overlapping filter windows, the linear model biases of the overlapping filter windows are weighted averaged to obtain a weighted linear model bias; the guiding kernel weights represent the weights of the edge directions.

[0022] According to an image enhancement method provided by the present invention, the step of determining a target adaptive enhancement control gain based on a linear model coefficient in a guided image filtering process comprises:

[0023] Obtain the maximum linear model coefficient and the minimum linear model coefficient in the process of guiding image filtering;

[0024] Determining a first adjustment coefficient and a second adjustment coefficient based on the maximum linear model coefficient, the minimum linear model coefficient, a preset maximum detail gain, and a preset minimum detail gain;

[0025] The linear model coefficient in the guided filtering process is adjusted based on the first adjustment coefficient and the second adjustment coefficient to obtain the target adaptive enhanced control gain.

[0026] According to an image enhancement method provided by the present invention, after fusing the smoothing layer and the enhanced detail layer to obtain the detail enhanced image of the image to be processed, the method further includes:

[0027] Perform pixel normalization processing on the detail-enhanced image to obtain a target detail-enhanced image of the image to be processed.

[0028] According to an image enhancement method provided by the present invention, before the guided image filtering is performed on the image to be processed, the method further includes:

[0029] An original image to be processed is acquired, and at least one of bad pixel detection, two-dimensional noise reduction and contrast enhancement is performed on the original image to be processed to obtain the image to be processed.

[0030] The present invention also provides an image enhancement device, comprising:

[0031] A filtering layering module, used for performing guided image filtering on the image to be processed to obtain a smoothing layer and a detail layer of the image to be processed;

[0032] A detail enhancement module, used to determine a target adaptive enhancement control gain based on a linear model coefficient in a guided image filtering process, and enhance the detail layer using the target adaptive enhancement control gain to obtain an enhanced detail layer; wherein the value of the linear model coefficient at an edge pixel point is greater than the value at a smooth area pixel point;

[0033] The layer fusion module is used to fuse the smooth layer and the enhanced detail layer to obtain a detail enhanced image of the image to be processed.

[0034] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described image enhancement methods when executing the computer program.

[0035] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image enhancement method described in any one of the above is implemented.

[0036] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image enhancement method described in any one of the above is implemented.

[0037] The image enhancement method, device, electronic device and storage medium provided by the present invention can first perform guided image filtering on the image to be processed to obtain the smooth layer and detail layer of the image to be processed; then determine the target adaptive enhancement control gain based on the linear model coefficient in the guided image filtering process, and use the target adaptive enhancement control gain to enhance the detail layer to obtain the enhanced detail layer; then fuse the smooth layer and the enhanced detail layer to obtain the detail enhanced image of the image to be processed, thereby realizing the detail enhancement of the image to be processed. Among them, the value of the linear model coefficient of the guided image filtering at the edge pixel point is greater than the value at the smooth area pixel point, which improves the edge protection performance, so that the noise masking effect at the edge of the image is stronger than the noise masking effect at the smooth part of the image, so that when the smooth layer and the detail layer are separated, there will be no obvious edge blurring phenomenon in the smooth layer; further, the target adaptive enhancement control gain determined by the linear model coefficient can more accurately enhance the image edge pixel points when performing detail layer enhancement, avoid amplifying the noise in the smooth area of ​​the image, thereby suppressing the amplification of the noise in the smooth area while enhancing the image detail information, and realizing adaptive image detail enhancement that suppresses noise amplification. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 is a schematic diagram of a flow chart of an image enhancement method provided by an embodiment of the present invention;

[0040] Figure 2 is one of the schematic diagrams of overlapping filter windows of the same pixel point in an embodiment of the present invention;

[0041] Figure 3 2 is a schematic diagram of overlapping filter windows of the same pixel point in an embodiment of the present invention;

[0042] Figure 4 3 is a schematic diagram of overlapping filter windows of the same pixel point in an embodiment of the present invention;

[0043] Figure 5 4 is a schematic diagram of overlapping filter windows of the same pixel point in an embodiment of the present invention;

[0044] Figure 6 Schematic diagram 5 of overlapping filter windows of the same pixel in an embodiment of the present invention;

[0045] Figure 7 6 is a schematic diagram of overlapping filter windows of the same pixel in an embodiment of the present invention;

[0046] Figure 8 7 is a schematic diagram of overlapping filter windows of the same pixel point in an embodiment of the present invention;

[0047] Fig. 9 8 is a schematic diagram of overlapping filter windows of the same pixel point in an embodiment of the present invention;

[0048] Fig.10 Schematic diagram 9 of overlapping filter windows of the same pixel in an embodiment of the present invention;

[0049] Fig.11 is a schematic structural diagram of an image enhancement device provided by an embodiment of the present invention;

[0050] Fig.12 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] It should be noted that the serial numbers assigned to the objects described in the present invention, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.

[0053] In the process of detail enhancement, the image to be processed can be decomposed into a smooth layer and a detail layer by edge-preserving smoothing filtering, and the detail layer is enhanced and then superimposed with the smooth layer to obtain a detail-enhanced image. However, this method will enhance the detail information and noise in the detail layer to the same extent, resulting in noise amplification in the image.

[0054] To address this problem, considering that the human visual system is more sensitive to noise in smooth areas of an image, but has a higher tolerance for noise in high-frequency areas of an image, this noise masking (NM) feature can be quantified as a noise masking function, which is proportional to the severity of the numerical change in the local area where the current pixel is located. Based on this, an adaptive detail enhancement method based on noise masking can be used to enhance the details of the image. An adaptive gain control (AGC) function that enhances the detail layer is designed based on the noise masking feature, which can enhance the detail information of the image while limiting the amplification of noise in smooth areas. Therefore, how to design the AGC function is the core of detail enhancement.

[0055] Guided Image Filtering (GIF) is a local linear model based on the local linear assumption to obtain the filtered output image q with respect to the guided image I, which can be expressed by the following formula (1):

[0056]

[0057] Among them, q i represents the filter output of the i-th pixel, are the linear model coefficients, is the linear model bias, I i is the i-th pixel point of the guide image, and the guide image may be the image to be processed.

[0058] The output of the image to be processed after GIF filtering is a smooth layer. The detail layer can be obtained by subtracting the smooth layer from the image to be processed. For the detail layer, the linear model coefficients in the GIF local linear model can be used. An adaptive gain control (AGC) function G for enhancing the detail layer is constructed as a reference, and the AGC function G is positively correlated with the noise mask function. The constructed AGC function can be expressed as:

[0059]

[0060] Among them, G i It represents the AGC gain of the i-th pixel, and α and β are the adjustment coefficients.

[0061] Afterwards, the detail layer can be enhanced using the AGC function, and the detail-enhanced image can be obtained using the following formula (3):

[0062] O=BP+G·D (3)

[0063] Among them, O represents the detail enhanced image, BP represents the enhanced smoothing layer, G represents the AGC gain, and D represents the detail layer.

[0064] In this way, when using GIF to enhance the details of the processed image, the noise mask characteristics can be considered to limit the noise amplification in the smooth area while enhancing the image detail information. However, GIF is a linear filter algorithm. When performing local detail enhancement, its linear model coefficient cannot distinguish whether each pixel in the local image is the noise in the smooth area or the texture in the detail area. If the image details are to be significantly enhanced, the noise will also be amplified in the same proportion; if the noise amplification is to be suppressed, it will lead to the loss of image details, resulting in blurred image details and even halo artifacts.

[0065] Based on this, an embodiment of the present invention provides an image enhancement method, which obtains a smooth layer and a detail layer of an image to be processed by guided image filtering, determines a target adaptive enhancement control gain based on a linear model coefficient in the filtering process, and uses the target adaptive enhancement control gain to enhance the detail layer, and then fuses the enhanced detail layer and the smooth layer to obtain a detail enhanced image. The value of the linear model coefficient in the filtering process at the edge pixel point is greater than the value at the smooth area pixel point, which can better distinguish the image details and the smooth area noise in the process of guided image filtering of the image, improve the edge protection performance, and construct the AGC gain based on the noise mask characteristic to better distinguish the image details and the noise in the smooth area, so as to achieve adaptive image detail enhancement that suppresses noise amplification.

[0066] Combine the following Figure 1-Figure 10 The image enhancement method of the present invention is described. The image enhancement method can be applied to electronic devices such as terminal devices or servers. The terminal devices may include mobile phones, computers, vehicle-mounted devices, tablet computers, wearable devices, smart home devices, etc. The server may include an independent server, a cluster server or a cloud server, etc. The image enhancement method can also be applied to an image enhancement device provided in an electronic device such as a terminal device or a server, and the image enhancement device can be implemented by software, hardware or a combination of both.

[0067] Figure 1 The schematic diagram of the process of the image enhancement method provided by the embodiment of the present invention is exemplarily shown. Figure 1 As shown, the image enhancement method may include the following steps 110 to 130.

[0068] Step 110: performing guided image filtering on the image to be processed to obtain a smoothing layer and a detail layer of the image to be processed.

[0069] Guided image filtering is a filtering method based on the local linear assumption, that is, the filtered output image is locally linearly correlated with the guide image. Given the output image to be processed and the guide image, after guided image filtering, the filtered output image is similar to the content of the image to be processed and the texture of the guide image. The principle of guided image filtering can also be expressed as:

[0070]

[0071] Among them, q i represents the filter output, ω k is the set of pixels in a square filter window with pixel k as the center and r as the radius; i Represents the guide image, linear model coefficient a k and the linear model bias b k By solving the loss function E(a k ,b k ) is obtained. k ,b k ) can be expressed as:

[0072]

[0073] Among them, ε is a regularization factor common to all filter windows, which is used to punish excessive a k ;p i Represents the i-th pixel in the image to be filtered.

[0074] Detail enhancement is used to enhance the detail texture of the image, so that the linear model of guided image filtering can have a larger a at the edge texture k At the same time, in order to avoid amplifying the noise in the smooth area, different penalty coefficients can be used for edge texture and smooth area. Based on this, in an exemplary embodiment, the regularization factor can be adaptively adjusted using the edge-aware weight, and a is determined using the loss function shown in the following formula (6): k and b k :

[0075]

[0076] Among them, I (k) is the edge-aware weight.

[0077] In this way, the edge in the local area can be identified by edge perception weights, and the regularization factor ε can be adjusted adaptively to improve the edge protection performance and reduce the occurrence of halo artifacts. The value of the linear model coefficient determined in this way is greater at the edge pixel point than at the smooth area pixel point.

[0078] For any pixel i in the image to be processed, |ω|=(2r+1) 2 overlapping square filter windows. For example, Figure 2 to Figure 10 This is a schematic diagram of overlapping filter windows for the same pixel, combined with Figure 2 to Figure 10 , taking the window radius r = 1 as an example, the pixel i is located in 9 overlapping filtering windows. In each overlapping filtering window, different local linear model coefficients and filtering outputs can be calculated. Then the filtering result q i It can be a i The arithmetic mean of all overlapping filter windows. Based on this, formula (4) can be transformed into:

[0079]

[0080] Since a pixel point is located in multiple overlapping square overlapping filter windows, the method of calculating the arithmetic mean using formula (7) uses the same weight for each overlapping filter window, and ignores the edge direction of the image area where the current pixel point is located, which will cause the image edge to be blurred. In view of this, in an exemplary embodiment, the weighted average of each overlapping filter window can be used as the filter output. Specifically, a guiding kernel weight can be introduced for each overlapping filter window, that is, the weight w of each overlapping filter window is determined. ik Based on this, formula (7) can be transformed into:

[0081]

[0082] in,

[0083] For example, the guiding kernel weight w ik It can be determined based on the edge information in the overlapping filter window, for example, it can be determined based on the gradient of the pixel point in the overlapping filter window, using the guided kernel weight w ik It can effectively capture the edge direction information, that is, the guiding kernel weight w ik The weight of the edge direction can be represented.

[0084] In this way, by introducing the guide kernel weights to each overlapping filter window, the guided image filter can effectively collect edge direction information, further improving the edge protection performance. Greater than Further improved edge protection.

[0085] In the embodiment of the present invention, As the linear model coefficients for guiding image filtering, that is, the linear model coefficients are determined based on edge-aware weights and guided kernel weights, and can describe the NM characteristics of the image.

[0086] Exemplarily, the image to be processed for guided image filtering may be an image obtained by preprocessing the input original image to be processed. Specifically, before the image to be processed is subjected to guided image filtering, the original image to be processed may be obtained, and at least one of bad pixel detection, two-dimensional noise reduction, and contrast enhancement may be performed on the original image to be processed to obtain the image to be processed.

[0087] Exemplarily, in the embodiment of the present invention, the image to be processed may be represented by floating-point data, and the value range may be [0, 1].

[0088] Step 120: Determine a target adaptive enhancement control gain based on the linear model coefficients in the guided image filtering process, and use the target adaptive enhancement control gain to enhance the detail layer to obtain an enhanced detail layer.

[0089] The linear model coefficients in the image filtering process are guided so that the values ​​at the edge pixels are greater than the values ​​at the smooth area pixels.

[0090] After obtaining the detail layer of the image to be processed, the linear model coefficients in the guided image filtering process can be used to quantitatively describe the NM characteristics of the image to be processed, and the target adaptive enhancement control gain can be determined accordingly. For example, the principle of formula (2) can be combined to replace the linear model coefficients in the guided image filtering process with Construct the AGC function G, which can be expressed as:

[0091]

[0092] Among them, α is the first adjustment coefficient, and β is the second adjustment coefficient. In this way, the target adaptive enhancement control gain G of each pixel point can be determined according to formula (9): i .

[0093] Exemplarily, α and β may be preset values, or may be determined based on a maximum linear model coefficient and a minimum linear model coefficient in a guided image filtering process, and a preset maximum detail gain and a preset minimum detail gain.

[0094] After obtaining the target adaptive enhancement control gain, the detail layer D can be enhanced pixel by pixel by the following formula (10) to obtain the enhanced detail layer DP:

[0095] DP=G.*D (10)

[0096] Step 130: Fusing the smoothing layer and the enhanced detail layer to obtain a detail-enhanced image of the image to be processed.

[0097] After obtaining the enhanced detail layer, the enhanced detail layer DP can be fused with the smooth layer B by the following formula (11) to obtain the detail enhanced image O of the image to be processed.

[0098] O=B+DP (11)

[0099] The image enhancement method provided by the present invention can first perform guided image filtering on the image to be processed to obtain the smooth layer and detail layer of the image to be processed; then determine the target adaptive enhancement control gain based on the linear model coefficient in the guided image filtering process, and use the target adaptive enhancement control gain to enhance the detail layer to obtain an enhanced detail layer; then fuse the smooth layer and the enhanced detail layer to obtain a detail enhanced image of the image to be processed, thereby achieving detail enhancement of the image to be processed. Among them, the value of the linear model coefficient of the guided image filtering at the edge pixel point is greater than the value at the smooth area pixel point, which improves the edge protection performance, so that the noise masking effect at the edge of the image is stronger than the noise masking effect at the smooth part of the image, so that when the smooth layer and the detail layer are separated, no obvious edge blurring phenomenon will appear in the smooth layer; further, the target adaptive enhancement control gain determined by the linear model coefficient can more accurately enhance the image edge pixel points when performing detail layer enhancement, avoid amplifying the image smooth area noise, thereby enhancing the image detail information while suppressing the amplification of the smooth area noise, thereby achieving adaptive image detail enhancement that suppresses noise amplification.

[0100] based on Figure 1 The image enhancement method of the corresponding embodiment, in an exemplary embodiment, performs guided image filtering on the image to be processed to obtain a smooth layer and a detail layer of the image to be processed, which may include: determining an edge perception weight based on the image grayscale range, the total number of pixels and the second pixel variance of the guide image under the second filtering window; determining a linear model coefficient based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window and the edge perception weight; determining a linear model bias based on the pixel average of the image to be processed under the first filtering window, the pixel average of the guide image under the first filtering window and the linear model coefficient; filtering the image to be processed using the linear model coefficient and the linear model bias to obtain a smooth layer; and subtracting the image to be processed from the smooth layer to obtain a detail layer.

[0101] Among them, the filtering radius of the second filtering window is the filtering radius used when determining the edge perception weight, for example, the value can be 3; the filtering radius of the first filtering window is the filtering radius used in guiding image filtering, for example, the value range can be [2, 5].

[0102] Exemplarily, the edge perception weight is less than 1 at the smooth area pixel point and greater than 1 at the edge pixel point.

[0103] In an exemplary embodiment, determining linear model coefficients based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge perception weights may include: determining local linear model coefficients based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge perception weights; for each pixel point in the image to be processed, determining the overlapping filtering window in which the pixel point is located; based on the guide kernel weights of each overlapping filtering window, weighted averaging the local linear model coefficients of each overlapping filtering window to obtain the linear model coefficients; wherein the guide kernel weight represents the weight of the edge direction.

[0104] In an exemplary embodiment, determining the linear model bias based on the pixel average of the image to be processed under the first filter window, the pixel average of the guide image under the first filter window, and the linear model coefficient may include: determining the local linear model bias based on the pixel average of the image to be processed under the first filter window, the pixel average of the guide image under the first filter window, and the linear model coefficient; for each pixel point in the image to be processed, determining the overlapping filter window where the pixel point is located; based on the guiding kernel weights of each overlapping filter window, performing weighted averaging on the local linear model biases of each overlapping filter window to obtain the linear model bias; wherein the guiding kernel weight represents the weight of the edge direction.

[0105] Specifically, the edge perception weight can be determined based on the image grayscale range, the total number of pixels and the second pixel variance under the second filtering window of the guide image using the following formula (12):

[0106]

[0107] Among them, I (k) represents the edge perception weight of pixel k, represents the second pixel variance; λ = (0.001 × L)^2, is an adjustment parameter and is a positive number, L is the image grayscale range of the guide image; N is the total number of pixels of the guide image.

[0108] The second pixel variance of the guided image under the second filter window It can be based on the pixel average of the guidance image under the second filter window and the autocorrelation coefficient Specifically, the second pixel variance can be determined using the following formula (13):

[0109]

[0110] in, represents the pixel average of the guide image within the filter window with a radius of r2, I represents the guide image, r2 represents the filter radius of the second filter window, and f mean Indicates the average value.

[0111] The first pixel variance var of the guide image under the first filter window I It can be based on the pixel average mean of the guide image under the first filter window I And the autocorrelation coefficient corr under the second filter window I Specifically, the first pixel variance var can be determined using the following formula (14): I :

[0112] var I =corr I -mean I .*mean I (14)

[0113] Among them, mean I =f mean (I, r), represents the pixel average of the guided image I within the filter window with a radius of r, r represents the filter radius of the first filter window, and f mean Indicates average value.

[0114] The covariance cov between the guide image and the image to be processed Ip The pixel average mean of the guide image I under the first filter window can be I , the pixel average mean of the image P to be processed under the first filter window p , and the correlation coefficient between the guide image and the image to be processed under the first filter window. Specifically, the covariance cov can be determined using the following formula (15): Ip :

[0115] cov Ip =corr Ip -mean I .*mean p (15)

[0116] Among them, corr Ip =f mean(I.*p,r), mean p =f mean (p,r).

[0117] For example, when obtaining the edge-aware weight Ψ I , covariance cov Ip and the first pixel variance var I , the pixel average mean of the image P to be processed under the first filter window p And the pixel average mean of the guide image under the first filter window I Afterwards, the local linear model coefficient a and the local linear model bias b can be determined using the following formulas (16) and (17) respectively:

[0118]

[0119] b=mean p -a.*mean I (17)

[0120] Among them, ε is a regularization factor. It can be understood that both a and b are in the form of matrices, a represents the local linear model coefficients of all pixels in the image to be processed, and b represents the local linear model bias of all pixels in the image to be processed.

[0121] Furthermore, based on the determined local linear model coefficient a, for each pixel of the image to be processed, the guiding kernel weight w corresponding to the pixel can be ik (h), calculate the weighted average of the local linear model coefficients to obtain the linear model coefficients Specifically, It can be expressed as:

[0122]

[0123] Among them, h is the smoothing parameter, which can be taken as 2.4, for example; ω k It is a set of pixels in a square filter window with pixel k as the center and r as the radius.

[0124] Similarly, based on the determined local linear model bias b, for each pixel of the image to be processed, the guiding kernel weight w corresponding to the pixel can be calculated. ik (h) Calculate the weighted average of the local linear model bias to obtain the linear model bias Specifically, It can be expressed as:

[0125]

[0126] Among them, h is the smoothing parameter, which can be taken as 2.4, for example; ω k It is a set of pixels in a square filter window with pixel k as the center and r as the radius.

[0127] The guiding image is required in the guiding image filtering process. The guiding image can be the image to be processed that needs to be filtered, or it can be an image different from the image to be processed. In the embodiment of the present invention, the guiding image can be taken as the image to be processed. The linear model coefficients are obtained and weighted linear model bias Afterwards, the linear model coefficients corresponding to each pixel can be used and weighted linear model bias The pixel point is filtered to obtain a smoothing layer of the image to be processed. Exemplarily, the image to be processed can be filtered based on the following formula (20) to obtain a filtering output result q:

[0128]

[0129] The image to be processed and the guide image are both set to I. The filter output result q can be set to the smoothing layer B, and then the detail layer D=IB can be calculated.

[0130] Based on the image enhancement methods of the above embodiments, it can be understood that the linear model coefficients and linear model bias Guided filtering of the processed image can be regarded as a weighted guided filtering with a guided kernel. According to the above linear model coefficients Determination process, when the image to be processed and the guide image are both I, combined with the above formula (16), the local linear model coefficient a can be rewritten as:

[0131]

[0132] Correspondingly, for a pixel point k within a certain filtering window, the local linear model coefficient a can be expressed as:

[0133]

[0134] in, Represents the pixel variance of pixel k under the first filter window. Edge perception weight Ψ I (k) is less than 1 in smooth areas of the image and greater than 1 at the edge of the image. Therefore, if pixel k is located at the edge of the image, then a k Compared with GIF, it will be larger, which effectively improves the edge protection performance.

[0135] Based on the image enhancement methods of the above embodiments, it can be understood that after obtaining the local linear model coefficient a and the local linear model bias b, the embodiment of the present invention can use a weighted average based on the guide kernel weight to solve the problem that each pixel point exists in multiple overlapping filter windows. Replace the arithmetic mean of weighted guided image filtering Since the guiding kernel weight introduces edge direction information, the edge pixels of the image are Greater than In this way, the edge protection performance of the weighted guided filter with the guided kernel is further improved compared with the weighted guided filter.

[0136] It can be understood that the linear model coefficient increases significantly at the edge of the image and decreases significantly at the smooth part of the image, which is positively correlated with the NM characteristics of the image. That is, the noise masking effect is strong at the edge pixels, while the noise masking effect is weak at the smooth area pixels.

[0137] Based on the image enhancement method of each of the above embodiments, in an exemplary embodiment, after obtaining the linear model coefficient, the linear model coefficient can be used to quantify the NM characteristics of the image, and the AGC function G can be constructed based on this, and the detail layer of the processed image can be enhanced using G. Specifically, determining the target adaptive enhancement control gain based on the linear model coefficient in the guided image filtering process may include: obtaining the maximum linear model coefficient and the minimum linear model coefficient in the guided image filtering process; determining the first adjustment coefficient and the second adjustment coefficient based on the maximum linear model coefficient, the minimum linear model coefficient, and the preset maximum detail gain and the preset minimum detail gain; adjusting the linear model coefficient in the guided filtering process based on the first adjustment coefficient and the second adjustment coefficient to obtain the target adaptive enhancement control gain.

[0138] Specifically, in combination with formula (9), the first adjustment coefficient α and the second adjustment coefficient β determine the effect of detail enhancement. Optionally, a preset maximum detail gain G may be preset. max and preset minimum detail gain G min , based on G max and G min Determine α and β. When G i =G max ;when When G i =G min .in, is the maximum linear model coefficient, is the minimum linear model coefficient. Based on this, α and β can be expressed as:

[0139]

[0140]

[0141] For example, G min The value can be 1, G max The value range of can be [1.25, 2].

[0142] After obtaining the AGC function G, the detail layer can be enhanced pixel by pixel based on the principle of formula (10) to obtain an enhanced detail layer DP. Furthermore, the enhanced detail layer can be fused with the smooth layer based on the principle of formula (11) to obtain a detail enhanced image O.

[0143] Based on the image enhancement methods of the above-mentioned embodiments, in an exemplary embodiment, since illegal values ​​less than 0 or greater than 1 may exist in the detail enhanced image O obtained after the enhanced detail layer and the smooth layer are fused, after obtaining the detail enhanced image O, the detail enhanced image O may be further subjected to pixel normalization processing to obtain a target detail enhanced image of the image to be processed. At this time, the obtained target detail enhanced image eliminates illegal values ​​in the image relative to the detail enhanced image before the pixel normalization processing.

[0144] Specifically, the following formulas (25) and (26) may be used in sequence to perform pixel normalization processing on the detail enhanced image O to obtain the target detail enhanced image O″.

[0145] O′=O-min(O) (25)

[0146]

[0147] Wherein, min(O) is the minimum grayscale value in the detail enhanced image O, O′ is the processed image obtained by subtracting the minimum grayscale value from the grayscale value of each pixel in the detail enhanced image O; max(O′) is the maximum grayscale value in the processed image O′.

[0148] The image enhancement method provided by the embodiment of the present invention can use edge-aware weights to adaptively adjust the regularization factor in the loss function so that the linear model coefficient a at the edge of the image is k Compared with the linear model coefficient in the guided image filter, the edge protection performance is improved. In addition, for the problem that the same pixel point is in multiple overlapping filter windows, the weighted average of the linear model calculation results in each overlapping filter window is used. Replace the arithmetic mean The introduced guiding kernel weight can effectively collect the edge direction information of the image, so that Further improve the edge protection performance. In this way, the linear model coefficient As the linear model coefficients of the linear model when guiding image filtering, when separating the smooth layer and the detail layer of the image to be processed, there will be no obvious edge blurring in the smooth layer. When quantizing the NM characteristics and generating the AGC function based on them, compared with the existing GIF, edge-aware weights and guided kernel weights are introduced to make the linear model coefficients It rises rapidly at the edge of the image, which can better distinguish the edge of the image from the noise in the smooth area. Therefore, the linear model coefficient The NM characteristics of the image to be processed can be better described, and the AGC function constructed in this way can more accurately enhance the edge area of ​​the image and avoid amplifying the noise in the smooth area.

[0149] The image enhancement method provided by the embodiment of the present invention can adaptively enhance the details of the image to be processed based on the weighted guided image filtering with the guided kernel, and can effectively enhance the image detail information while suppressing the amplification of the noise in the smooth area. Moreover, the linear model coefficients introduced with the edge-aware weights and the guided kernel weights can be used to quantify the expression of the noise mask characteristics, and the AGC function can be constructed to enhance the detail layer, which can accurately describe the high and low frequency properties of the image to be processed and effectively distinguish the detail texture and the noise in the smooth area of ​​the image to be processed.

[0150] The image enhancement device provided by the present invention is described below. The image enhancement device described below and the image enhancement method described above can be referred to each other.

[0151] Fig.11 The schematic diagram of the structure of the image enhancement device provided by the embodiment of the present invention is exemplarily shown. Fig.11 As shown, the image enhancement device may include: a filtering layering module 1110, which is used to perform guided image filtering on the image to be processed to obtain a smooth layer and a detail layer of the image to be processed; a detail enhancement module 1120, which is used to determine a target adaptive enhancement control gain based on a linear model coefficient in the guided image filtering process, and use the target adaptive enhancement control gain to enhance the detail layer to obtain an enhanced detail layer, wherein the value of the linear model coefficient at the edge pixel point is greater than the value at the smooth area pixel point; a layer fusion module 1130, which is used to fuse the smooth layer and the enhanced detail layer to obtain a detail enhanced image of the image to be processed.

[0152] In an exemplary embodiment, the filtering layering module 1110 includes: a first determination unit, which is used to determine the edge perception weight based on the image grayscale range, the total number of pixels and the second pixel variance of the guide image under the second filtering window; a second determination unit, which is used to determine the linear model coefficient based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window and the edge perception weight; a third determination unit, which is used to determine the linear model bias based on the pixel average of the image to be processed under the first filtering window, the pixel average of the guide image under the first filtering window and the linear model coefficient; a filtering unit, which is used to filter the image to be processed using the linear model coefficient and the linear model bias to obtain a smoothing layer; and a processing unit, which is used to subtract the image to be processed from the smoothing layer to obtain a detail layer.

[0153] In an exemplary embodiment, the second determination unit is specifically used to: determine the local linear model coefficients based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge perception weight; for each pixel point in the image to be processed, determine the overlapping filtering window where the pixel point is located; based on the guide kernel weights of each overlapping filtering window, perform weighted averaging on the local linear model coefficients of each overlapping filtering window to obtain the linear model coefficients; wherein the guide kernel weight represents the weight of the edge direction.

[0154] In an example embodiment, the third determination unit is specifically used to: determine the local linear model bias based on the pixel average of the image to be processed under the first filtering window, the pixel average of the guide image under the first filtering window and the linear model coefficient; for each pixel point in the image to be processed, determine the overlapping filtering window where the pixel point is located; based on the guide kernel weights of each overlapping filtering window, perform weighted averaging on the local linear model biases of each overlapping filtering window to obtain a weighted linear model bias; the guide kernel weight represents the weight of the edge direction.

[0155] In an exemplary embodiment, the detail enhancement module 1120 includes: an acquisition unit for acquiring the maximum linear model coefficient and the minimum linear model coefficient in the guided image filtering process; a fourth determination unit for determining the first adjustment coefficient and the second adjustment coefficient based on the maximum linear model coefficient, the minimum linear model coefficient, and the preset maximum detail gain and the preset minimum detail gain; an adjustment unit for adjusting the linear model coefficients in the guided filtering process based on the first adjustment coefficient and the second adjustment coefficient to obtain a target adaptive enhancement control gain.

[0156] In an exemplary embodiment, the image enhancement device may further include: a normalization processing module, configured to perform pixel normalization processing on the detail-enhanced image to obtain a target detail-enhanced image of the image to be processed.

[0157] In an exemplary embodiment, the image enhancement device may further include: a preprocessing module, which is used to obtain an original image to be processed, and perform at least one of bad pixel detection, two-dimensional noise reduction and contrast enhancement on the original image to be processed to obtain an image to be processed.

[0158] Fig.12 An example of a structural diagram of an electronic device is shown in FIG. Fig.12 As shown, the electronic device may include: a processor 1210, a communication interface 1220, a memory 1230 and a communication bus 1240, wherein the processor 1210, the communication interface 1220 and the memory 1230 communicate with each other through the communication bus 1240. The processor 1210 may call the logic instructions in the memory 1230 to execute the image enhancement method provided by any of the above method embodiments.

[0159] In addition, the logic instructions in the above-mentioned memory 1230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0160] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image enhancement method provided by any of the above method embodiments.

[0161] On the other hand, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the image enhancement method provided by any of the above method embodiments.

[0162] By way of example, computer readable storage media include non-transitory computer readable storage media.

[0163] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image enhancement method, characterized in that: include: Performing guided image filtering on the image to be processed to obtain a smoothing layer and a detail layer of the image to be processed; Determining a target adaptive enhancement control gain based on a linear model coefficient in a guided image filtering process, and enhancing the detail layer using the target adaptive enhancement control gain to obtain an enhanced detail layer; wherein the value of the linear model coefficient at an edge pixel point is greater than the value at a smooth area pixel point; The smoothing layer and the enhanced detail layer are merged to obtain a detail-enhanced image of the image to be processed.

2. The image enhancement method according to claim 1, characterized in that: The step of performing guided image filtering on the image to be processed to obtain a smoothing layer and a detail layer of the image to be processed includes: Determining an edge perception weight based on an image grayscale range, a total number of pixels, and a second pixel variance under a second filtering window of the guide image; Determining the linear model coefficients based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge-aware weights; Determining a linear model bias based on a pixel average of the image to be processed under the first filtering window, a pixel average of the guide image under the first filtering window, and the linear model coefficient; Filtering the image to be processed by using the linear model coefficient and the linear model bias to obtain the smoothing layer; The image to be processed is subtracted from the smooth layer to obtain the detail layer.

3. The image enhancement method according to claim 2, characterized in that: The determining of the linear model coefficient based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge perception weight comprises: Determining local linear model coefficients based on the covariance between the guide image and the image to be processed, the first pixel variance of the guide image under the first filtering window, and the edge perception weight; For each pixel point in the image to be processed, determining the overlapping filtering window where the pixel point is located; Based on the guiding kernel weight of each overlapping filter window, the local linear model coefficients of each overlapping filter window are weighted averaged to obtain the linear model coefficient; the guiding kernel weight represents the weight of the edge direction.

4. The image enhancement method according to claim 2, characterized in that: The determining the linear model bias based on the pixel average of the image to be processed under the first filtering window, the pixel average of the guide image under the first filtering window, and the linear model coefficient comprises: Determine a local linear model bias based on a pixel average of the image to be processed under the first filtering window, a pixel average of the guide image under the first filtering window, and the linear model coefficient; For each pixel point in the image to be processed, determining the overlapping filtering window where the pixel point is located; Based on the guide kernel weights of the overlapping filter windows, the local linear model biases of the overlapping filter windows are weighted averaged to obtain the linear model bias; the guide kernel weights represent the weights of the edge directions.

5. The image enhancement method according to any one of claims 1 to 4, characterized in that: The step of determining the target adaptive enhancement control gain based on the linear model coefficient in the guided image filtering process includes: Obtain the maximum linear model coefficient and the minimum linear model coefficient in the process of guiding image filtering; Determining a first adjustment coefficient and a second adjustment coefficient based on the maximum linear model coefficient, the minimum linear model coefficient, a preset maximum detail gain, and a preset minimum detail gain; The linear model coefficient in the guided filtering process is adjusted based on the first adjustment coefficient and the second adjustment coefficient to obtain the target adaptive enhanced control gain.

6. The image enhancement method according to any one of claims 1 to 4, characterized in that: After fusing the smooth layer and the enhanced detail layer to obtain the detail enhanced image of the image to be processed, the method further includes: Perform pixel normalization processing on the detail-enhanced image to obtain a target detail-enhanced image of the image to be processed.

7. The image enhancement method according to any one of claims 1 to 4, characterized in that: Before performing guided image filtering on the image to be processed, the method further includes: An original image to be processed is acquired, and at least one of bad pixel detection, two-dimensional noise reduction and contrast enhancement is performed on the original image to be processed to obtain the image to be processed.

8. An image enhancement device, characterized in that: include: A filtering layering module, used for performing guided image filtering on the image to be processed to obtain a smoothing layer and a detail layer of the image to be processed; A detail enhancement module, used to determine a target adaptive enhancement control gain based on a linear model coefficient in a guided image filtering process, and enhance the detail layer using the target adaptive enhancement control gain to obtain an enhanced detail layer; wherein the value of the linear model coefficient at an edge pixel point is greater than the value at a smooth area pixel point; The layer fusion module is used to fuse the smooth layer and the enhanced detail layer to obtain a detail enhanced image of the image to be processed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the image enhancement method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image enhancement method according to any one of claims 1 to 7 is implemented.

Images