A high dynamic infrared image detail enhancement method based on local edge-preserving filtering

Abstract

The application relates to a high-dynamic infrared image detail enhancement method based on a local edge-preserving filtering method, which comprises the following steps: inputting a high-dynamic infrared image; decomposing the high-dynamic infrared image into a smooth layer and multiple detail layers through a local edge-preserving filtering method; adjusting the image dynamic range of the smooth layer through a Gamma correction method to enhance the low-frequency structure of the smooth layer; removing artificial marks of each detail layer and stretching the detail layers; combining the smooth layer and all the detail layers to obtain an initial enhancement picture; adjusting the brightness of the initial enhancement picture to high, medium and low respectively, and fusing the initial enhancement pictures with the brightness of high, medium and low to obtain a final output detail enhancement image. The enhancement image obtained by the application has fewer artificial marks, clear local details of the image, and has good application value.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically a high dynamic range infrared image detail enhancement method based on local edge-preserving filtering. Background Technology

[0002] Infrared imaging technology has wide applications in various fields. Infrared radiation is essentially electromagnetic waves with wavelengths of 760nm-1mm, located between the visible light and millimeter-wave bands. It reflects the thermal radiation characteristics of objects; the higher the temperature, the greater the energy of the radiation. Infrared imaging systems use special materials to sense the infrared radiation of objects, converting the received infrared waves into electrical signals, and then converting the electrical signals into image signals. Infrared imaging systems can operate in all weather conditions, using radiation differences to distinguish between real and false targets. They have strong anti-interference capabilities and can display information related to thermal motion in the real world, thus expanding the spectral range perceptible to the human eye, and therefore have gained widespread application. However, due to limitations in imaging principles and manufacturing processes, infrared images also have some shortcomings, such as: concentrated overall grayscale distribution, low contrast, and poor layering; compared to visible light images, infrared images have weaker texture detail information and lower spatial resolution; and the system is susceptible to various noise interferences. These shortcomings seriously affect the infrared imaging system's perception of the environment and reduce the reliability and accuracy of target detection and identification. Although many image enhancement methods have been proposed in recent years to improve the quality of infrared images, they still have some limitations, such as insufficient detail enhancement, the generation of artificial artifacts, high computational cost, and slow convergence speed. These problems have not yet been effectively solved in infrared imaging systems. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a high dynamic range infrared image detail enhancement method based on local edge-preserving filtering, which addresses the above-mentioned shortcomings. This invention uses local edge-preserving filtering to adaptively estimate the local details of the image. The algorithm adjusts the contrast of the image from both global and local perspectives and stretches the detail layer using the principle of human vision. The enhanced image obtained by this method has fewer artificial traces and clear local details, and has good application value.

[0004] To solve the above technical problems, the present invention adopts the following technical solution:

[0005] A high dynamic range infrared image detail enhancement method based on local edge-preserving filtering includes the following steps:

[0006] Step S1: Input a high dynamic range infrared image;

[0007] Step S2: Decompose the high dynamic range infrared image into a smoothing layer and multiple detail layers using a local edge-preserving filtering method;

[0008] Step S3, adjusting the image dynamic range of the smoothing layer by Gamma correction method to enhance the low frequency structure of the smoothing layer;

[0009] Step S4, removing the artificial marks of each detail layer and stretching the detail layer;

[0010] Step S5, combining the smoothing layer and all the detail layers to obtain an initial enhanced picture;

[0011] Step S6, adjusting the brightness of the initial enhanced picture to high, medium and low respectively, and fusing the initial enhanced pictures with high, medium and low brightness to obtain a final output detail enhanced image.

[0012] Further, the step 2 comprises the following steps:

[0013] Step S21, presetting a plurality of discrete Gamma values;

[0014] Step S22, calculating the entropy of each Gamma corrected smoothing layer, and selecting the Gamma corrected smoothing layer with the maximum entropy as the adjustment result;

[0015] The formula is as follows:

[0016]

[0017]

[0018] Wherein, γ0 is the correction index of the Gamma corrected smoothing layer with the maximum entropy, I SE is the Gamma corrected smoothing layer with the maximum entropy.

[0019] Further, the method for removing the artificial marks of the detail layer in the step S4 comprises the following steps:

[0020] Step S31, comparing the X-axis gradient of the detail layer with the X-axis gradient of the original picture, when the X-axis gradient of the detail layer is greater than β times of the X-axis gradient of the original picture, the X-axis gradient of the original picture is output; if the X-axis gradient of the detail layer is less than or equal to β times of the X-axis gradient of the original picture, the X-axis gradient of the detail layer is output;

[0021] The formula is as follows:

[0022]

[0023] Wherein, D' x is the output X-axis gradient of the detail layer, I x is the X-axis gradient of the original picture, and D X is the X-axis gradient of the detail layer.

[0024] Step S32, compare the gradient of the detail layer Y axis with the gradient of the original drawing Y axis, when the gradient of the detail layer Y axis is greater than the gradient of the original drawing Y axis by β times, then output the Y axis gradient of the original drawing; if the gradient of the detail layer Y axis is less than or equal to the gradient of the original drawing Y axis by β times, then output the Y axis gradient of the detail layer;

[0025] The formula is as follows:

[0026]

[0027] Wherein, D' y is the output Y axis gradient of the detail layer, I y is the Y axis gradient of the original drawing, D y is the gradient of the detail layer Y axis.

[0028] Compared with the prior art, the present application has the following advantages after adopting the above technical scheme:

[0029] The present application uses the local edge-preserving filtering method to divide the image into a smooth layer and multiple detail layers, adaptively estimates the local details of the image, and adjusts the detail layer and the smooth layer to enhance the global contrast and the local contrast of the image. The local edge-preserving filtering method is used because it can adaptively smooth the local area of the image, estimate the details of the local area, and does not need to set a uniform threshold for estimation, and has good adaptability to the smoothing of different scene images.

[0030] For the smooth layer (low-frequency layer), the present application uses the maximum entropy Gamma correction method to adjust the global contrast, which can enhance the low-frequency structure of the smooth layer.

[0031] For the detail layer, the present application uses the principle of human eye visual characteristics to adjust the detail layer, so that the result is more consistent with the subjective vision of the human eye, and uses the different brightness level image fusion method to further adjust and enhance the brightness and contrast of the result. The enhanced image obtained by this method has fewer artificial traces, the local details of the image are clear, and the visual effect is natural.

[0032] Finally, the brightness of the initial enhanced result is adjusted to a higher, moderate, and lower level, and the fusion method is used to adjust the brightness and further show the details of the high-brightness area and the low-dark area in the image. The fusion output is used as the final infrared image detail enhancement result, and the different brightness level image fusion method is used to further adjust and enhance the brightness and contrast of the result.

[0033] The present application will be described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1Fig. 1 is a general flowchart of the present application;

[0035] Figure 2 Fig. 5 is a graph of the Gamma correction curves for different parameters;

[0036] Figure 3 Fig. 6 is a raw incandescent 16-bit infrared image (a) and its histogram (b);

[0037] Figure 4 Fig. 7 is a graph of the experimental results of the 16-bit incandescent infrared image dynamic range compression method;

[0038] Figure 5 Fig. 8 is a graph of the edge-preserving filtering results;

[0039] Figure 6 Fig. 9 is a graph of the experimental results of the fused brightness adjustment;

[0040] Figure 7 Fig. 10 is a graph of the overall process results of the algorithm of the present application. DETAILED DESCRIPTION

[0041] The principles and features of the present application will be described below in conjunction with the accompanying drawings, in which the examples are presented only to explain the present application and not to limit the scope of the present application.

[0042] The principles and features of the present application will be described below in conjunction with the accompanying drawings, in which the examples are presented only to explain the present application and not to limit the scope of the present application.

[0043] In order to facilitate the understanding of the present application, the present application will be described more fully below in conjunction with the accompanying drawings. The embodiments of the present application are shown in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application.

[0045] It will be understood that the terms spatially relative terms such as "beneath", "below", "lower", "above", "upper" and the like, can be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientations depicted in the figures. For example, if a device is inverted or rotated by 90 degrees, the descriptions of a "lower" element or feature as being "above" or "below" another element or feature is intended to encompass both an orientation of the device as originally depicted and an orientation of the device after it has been inverted or rotated by 90 degrees. Thus, the exemplary terms "below" and "above" can encompass both an orientation of above and below. The device can also be oriented in other ways (e.g., rotated 90 degrees or at other orientations), and the spatially relative terms used herein are to be interpreted accordingly.

[0046] As used herein, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. It will be further understood that the terms "comprises", "comprising", "includes" and / or "including", or the like, when used herein, specify the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

[0047] As shown in the figure, a high dynamic infrared image detail enhancement method based on local edge-preserving filtering, characterized by comprising the following steps: Figure 1

[0048] Step S1, input a high dynamic infrared image.

[0049] Step S2, decompose the high dynamic infrared image into a smooth layer and multiple detail layers by a local edge-preserving filtering method.

[0050] Wherein, step S2 is executed and comprises the following steps.

[0051] Step S21, preset multiple discrete Gamma values.

[0052] Step S22, calculate the entropy of each Gamma-corrected smooth layer, and select the Gamma-corrected smooth layer with the maximum entropy as the adjustment result.

[0053] The formula is as follows:

[0054]

[0055]

[0056] Wherein, γ0 is the correction index when the Gamma-corrected smooth layer has the maximum entropy, I SE ​The smooth layer after Gamma correction has the maximum entropy.

[0057] Step S3, adjusting the image dynamic range of the smooth layer by Gamma correction method, enhancing the low frequency structure of the smooth layer.

[0058] The method for removing the artificial trace of the detail layer comprises the following steps.

[0059] Step S31, comparing the gradient of the X axis of the detail layer with the gradient of the X axis of the original image, when the gradient of the X axis of the detail layer is greater than β times of the gradient of the X axis of the original image, outputting the X axis gradient of the original image, and when the gradient of the X axis of the detail layer is less than or equal to β times of the gradient of the X axis of the original image, outputting the X axis gradient of the detail layer.

[0060] The formula is as follows:

[0061]

[0062] D' = D - β * I x I is the X axis gradient of the original image, D is the X axis gradient of the detail layer, and D' is the output X axis gradient of the detail layer. x I is the X axis gradient of the original image, D is the X axis gradient of the detail layer, and D' is the output X axis gradient of the detail layer. X I is the X axis gradient of the original image, D is the X axis gradient of the detail layer, and D' is the output X axis gradient of the detail layer.

[0063] Step S32, comparing the gradient of the Y axis of the detail layer with the gradient of the Y axis of the original image, when the gradient of the Y axis of the detail layer is greater than β times of the gradient of the Y axis of the original image, outputting the Y axis gradient of the original image, and when the gradient of the Y axis of the detail layer is less than or equal to β times of the gradient of the Y axis of the original image, outputting the Y axis gradient of the detail layer.

[0064] The formula is as follows:

[0065]

[0066] D' = D - β * I y I is the y axis gradient of the original image, D is the y axis gradient of the detail layer, and D' is the output y axis gradient of the detail layer. y I is the y axis gradient of the original image, D is the y axis gradient of the detail layer, and D' is the output y axis gradient of the detail layer. y I is the y axis gradient of the original image, D is the y axis gradient of the detail layer, and D' is the output y axis gradient of the detail layer.

[0067] Step S4, removing the artificial trace of each detail layer, and stretching the detail layer.

[0068] Step S5, combining the smooth layer and all the detail layers to obtain an initial enhanced picture.

[0069] Step S6, adjusting the brightness of the initial enhanced picture to high, medium and low respectively, and fusing the initial enhanced pictures with high, medium and low brightness to obtain a final output detail enhanced image.

[0070] Example 1:

[0071] The present application is divided into three steps: global contrast adjustment of smooth layer based on entropy guidance, detail layer enhancement based on visual principle, and brightness adjustment of enhancement result.

[0072] (1) Global contrast adjustment of smooth layer based on entropy guidance

[0073] Firstly, a high dynamic infrared image is smoothed by using local edge-preserving filter to obtain a smooth layer and a detail layer. The smooth layer and the detail layer need to be adjusted to enhance the contrast of the image. The smooth layer has a dynamic range similar to the original image, so it is necessary to compress the dynamic range of the smooth layer. The present application uses Gamma correction method to adjust the dynamic range of the image. Gamma correction changes the mapping relationship between input and output through the index γ, as shown in the formula (1). When γ>1, the brightness of the output result is darker than the initial input image, and the greater the value, the greater the degree of stretching of the higher gray scale. When γ<1, the brightness of the output result is brighter than the initial input image, and the smaller the value, the greater the degree of stretching of the lower gray scale. Figure 2

[0074] When applying Gamma correction to adjust an image, the brightness and darkness of the image are not known in advance, and the value of Gamma usually needs to be determined manually, which limits the automation of the application of Gamma correction. Entropy is an important concept in information theory, which measures the distribution of random variables, and is also widely used in image detail enhancement methods. The present application proposes to use entropy theory to guide the selection of Gamma correction parameters, and to take the Gamma correction result with the maximum entropy as the output result of the smooth layer. Since it is relatively complex to accurately solve the Gamma value with the maximum entropy, a number of discrete Gamma values are pre-set, the entropy of each Gamma corrected image is calculated, and the Gamma corrected image with the maximum entropy is selected as the adjustment result, as shown in the formula (2).

[0075] Figure 3 And Figure 4 are the experimental results of the Gamma correction algorithm based on maximum entropy proposed in the present application, Figure 3 (a)

[0076]

[0077]

[0078] is the original 16-bit incandescent lamp infrared image, Figure 3 (b) is its histogram. From Figure 3 (b), it can be seen that the original gray scale distribution is concentrated around 10000, and we cannot see the content in Figure 3 (a), Figure 4 shows the maximum entropy Gamma correction result. From the result Figure 3 and Figure 4 ​The result can be seen, not only can see the lamp holder, lamp cover part, the outline of the background also appears, this is due to the Gamma correction to the background part of the gray difference is also enlarged.

[0079] (2) Detail layer enhancement based on visual principle

[0080] After adjusting the overall contrast of the image of the smooth layer, the local contrast of the image also needs to be enhanced. The local contrast of the image is mainly reflected in the detail layer. The method based on layered enhancement is easy to produce halo at the edge of the enhanced result. This is because the edge-preserving filtering result in the ideal case should be consistent with the original image at the edge, but in the actual edge-preserving filtering result, the filtering result only approximates the edge of the original image, and still remains some errors, as shown in Figure 5 (a) is a step image, which is regarded as an ideal edge, (b) is an edge-preserving filtering result, and (c) is a corresponding detail layer. As can be seen from (c), there are still some errors near the edge. When the detail layer is enlarged, these errors also become larger, and when they are fused with the smooth layer, these errors will form a halo at the edge. In order to prevent the enhanced result from appearing halo at the edge, a suppression mechanism based on gradient information is proposed to modify the gradient of the x-axis detail layer as follows:

[0081]

[0082] The gradient of the detail layer is compared with the original image, and when the gradient of the detail layer is greater than the gradient of the original image by β times, it is changed to the gradient of the original image. The gradient of the y-axis of the detail layer is also modified similarly, and the modified detail layer is calculated.

[0083] (3) Brightness adjustment of enhanced result

[0084] After calculating the global enhanced smooth layer and the local enhanced detail layer, they are combined to obtain the comprehensive enhanced result, so as to obtain an image that simultaneously achieves global contrast and local contrast enhancement.

[0085]

[0086] Wherein, η is the weighting factor of the detail layer. When the resolution of the image is large, the image can be multi-scale decomposed by using the local edge-preserving filter, and decomposed into multiple detail layers. After stretching each detail layer, the enhanced result is outputted by fusing the stretched detail layer with the base layer image. Through the above method, an image with improved overall and local contrast can be obtained. However, the output enhanced image sometimes appears overall light or overall dark, so the brightness needs to be adjusted. A feasible solution is to adjust the average brightness of the image to a certain level. However, when the target brightness is small, the details of the original bright area can be shown, but the details of the dark area cannot be seen. Conversely, when the target brightness is large, the details of the original dark area can be shown, but the details of the bright area cannot be seen. To solve this problem, this paper borrows the idea of multi-illumination high dynamic image fusion, and proposes to adjust the image to multiple brightness levels, and then fuse the multiple results to obtain the final enhanced result. After such processing, the overall brightness of the image is balanced, and the details of the bright and dark areas are also enhanced. The brightness adjustment experiment is shown in Figure 6 As can be seen, the details of each part of the fused image are enhanced and shown, Figure 7 The result image of the overall process of the algorithm.

[0087] The above is an example of the best embodiment of the present application, wherein the parts not described in detail are the common knowledge of those skilled in the art. The protection scope of the present application is subject to the content of the claims, and any equivalent transformation based on the technical inspiration of the present application is also within the protection scope of the present application.

Claims

1. A high dynamic infrared image detail enhancement method based on local edge-preserving filtering, characterized in that, The method comprises the following steps: Step S1, inputting a high dynamic infrared image; Step S2, decomposing the high dynamic infrared image into a smooth layer and multiple detail layers by a local edge-preserving filtering method; Step S3, adjusting the image dynamic range of the smooth layer by a Gamma correction method, and enhancing the low-frequency structure of the smooth layer; Step S4, removing artificial traces of each detail layer, and stretching the detail layers; Step S5, combining the smooth layer and all the detail layers to obtain an initial enhanced picture.

2. The local edge-preserving filter based high dynamic infrared image detail enhancement method according to claim 1, wherein, The step 2 comprises the following steps: Step S21, presetting multiple discrete Gamma values; Step S22, calculating the entropy of each Gamma-corrected smooth layer, and selecting the Gamma-corrected smooth layer with the maximum entropy as the adjustment result.

3. The local edge-preserving filter based high dynamic infrared image detail enhancement method according to claim 2, wherein, The formula of the step S22 is: where γ0 is the correction exponent at which the Gamma-corrected smooth layer has the maximum entropy, I SE is the Gamma-corrected smooth layer having the maximum entropy.

4. The method of claim 1, wherein the method is performed by a processor. The method for removing artificial traces of the detail layer in the step S4 comprises comparing the gradient of each orientation of each detail layer with the gradient of the original image, and when the gradient of the detail layer is greater than the gradient of the original image by β times, the gradient of the original image is output; otherwise, the gradient of the detail layer is output.

5. The method of claim 4, wherein the method is performed by a processor. The compared gradient orientations include the X axis and the Y axis.

6. The method of claim 5, wherein the method is performed by a processor. The method for removing artificial traces of the detail layer in the step S4 comprises the following steps: Step S31, comparing the X-axis gradient of the detail layer with the X-axis gradient of the original image, and when the X-axis gradient of the detail layer is greater than the X-axis gradient of the original image by β times, the X-axis gradient of the original image is output; otherwise, the X-axis gradient of the detail layer is output; Step S32, comparing the Y-axis gradient of the detail layer with the Y-axis gradient of the original image, and when the Y-axis gradient of the detail layer is greater than the Y-axis gradient of the original image by β times, the Y-axis gradient of the original image is output; otherwise, the Y-axis gradient of the detail layer is output.

7. The method of claim 6, wherein the method is performed by a processor. The formula of the step S31 is: where D' is the X-axis gradient of the output detail layer, I x is the X-axis gradient of the output detail layer, I x is the X-axis gradient of the original image, D X is the X-axis gradient of the detail layer.

8. The method of claim 6, wherein the method is a local edge-preserving filter based high dynamic infrared image detail enhancement method. The formula of the step S32 is: Among them, D' y For the y-axis gradient of the output detail layer, I y D is the y-axis gradient of the original image. y This is the gradient of the y-axis of the detail layer.

9. The local edge-preserving filter based high dynamic infrared image detail enhancement method of claim 1, wherein, The method further comprises the following step:

10. The method of claim 9, wherein the method is performed by a processor. Step S6, adjusting the brightness of the initial enhanced image to enhance the details of the high-brightness area and the low-dark area in the initial enhanced image, to obtain a detail-enhanced image. The method for adjusting the brightness of the initial enhanced image to enhance the details of the high-brightness area and the low-dark area in the initial enhanced image in the step S6 specifically comprises adjusting the brightness of the initial enhanced picture to high, medium and low respectively, and fusing the initial enhanced pictures with the brightness of high, medium and low.

Images