Infrared image gray super-resolution display method

Abstract

The invention relates to an infrared image gray super-resolution display method, and belongs to the technical field of infrared imaging. According to the method, the infrared image is divided into the base layer and the detail layer through the filter, gray dynamic compression mapping and large dynamic range gray subdivision enhancement are performed on the base layer, adaptive detail enhancement is performed on the image of the detail layer, and gray fusion super-resolution display is performed on the image processed by the base layer and the detail layer, so that the imaging quality of the infrared image is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of infrared imaging, and in particular relates to a grayscale super-resolution display method for infrared images. Background Art

[0002] At present, the infrared focal plane imaging system has the problem of insufficient grayscale resolution when observing targets, especially when observing weak targets (such as invisible distant vehicles and unclear long-distance four-bar target line pairs) and high dynamic range scenes (such as high-temperature targets are oversaturated and cannot be seen clearly). Therefore, it is necessary to carry out research on grayscale super-resolution technology based on weak targets and high dynamic range scenes. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] The technical problem to be solved by the present invention is: to address the problems of "not being able to see clearly", "not being able to see far" and "not being able to see details" of infrared images, to carry out research on grayscale super-resolution technology for infrared images with a large dynamic range, to provide an infrared image grayscale super-resolution display method, and to improve the quality of infrared images.

[0005] (II) Technical solution

[0006] In order to solve the above technical problems, the present invention provides an infrared image grayscale super-resolution display method, wherein the original 14-bits / 16-bit infrared image is filtered by a guide filter and a deep filter respectively, and an output image I_Base1 is obtained by the guide filter, and an output image I_Base2 is obtained by the deep filter, and a detail image I_Detail is obtained by subtracting I_Base2 from I_Base1;

[0007] The output image I_Base2 is the base layer image. The base layer image I_Base2 is processed by a large dynamic range grayscale subdivision enhancement algorithm and a grayscale dynamic compression mapping algorithm. The image I_Base2_AVHE is obtained by the large dynamic range grayscale subdivision enhancement algorithm, and the image I_Base2_CPMAP is obtained by the grayscale dynamic compression mapping algorithm. The image I_Base2_AVHE and I_Base2_CPMAP are fused in grayscale to obtain the image I_Base_8bits.

[0008] The detail image I_Detail is subjected to an adaptive detail enhancement algorithm to obtain an image I_Detail';

[0009] The images I_Base_8bits and I_Detail' are fused by grayscale to obtain the final enhanced image I_Out_8bits.

[0010] Preferably, the deep filter is a deep detail extraction filter; the deep detail extraction filter draws on the idea of ​​the global filter, and changes the regularization factor λ from a fixed value to an adaptive variable determined by the relationship between local and global variances. The new cost function E is shown in formula (2):

[0011]

[0012] in,

[0013] G(p) is the guide image, which is the original input image or the image after the original input image is smoothed, G(p') is the entire neighborhood image, X(p) is the original input image, a p' and b p' is the preset neighborhood linear coefficient, ε is a fixed parameter used to adjust the weight size, is the neighborhood full-image variance, is the guide image variance, and N represents the total number of pixels in the original image.

[0014] Preferably, the deep filter is an anti-grayscale inversion and anti-halation deep filter.

[0015] Preferably, the large dynamic range grayscale subdivision enhancement algorithm is designed as follows:

[0016] Assume that the original 14-bit / 16-bit infrared image is X, which has L image gray levels. The histogram corresponding to gray level l is shown in formula (4):

[0017] H(l)=n l ,l=0,1,···,L-1 (4)

[0018] Where n l is the total number of pixels with gray value l in the original image, and L is the total number of gray levels in the original image. Through the histogram equalization algorithm and the desired number of gray levels R of the enhanced image, the image enhancement algorithm based on histogram equalization is derived, as shown in formula (5):

[0019]

[0020] Where T(l) is the mapping function, which represents the mapping from the input infrared image with gray level l to the image y with gray level R l ; The contrast enhancement ratio of adjacent gray levels l and l-1 of the original image after mapping is expressed by formula (6):

[0021]

[0022] In the formula, δ lProportional to H(l), that is, after histogram equalization, the grayscale with a larger distribution probability in the original image has a stronger image contrast with its adjacent grayscale; conversely, the contrast between the grayscale with a smaller distribution probability and its adjacent grayscale will be compressed;

[0023] Further, the projection histogram is expressed as formula (7), and the calculated value of formula (4) is replaced by the calculated value of formula (7);

[0024]

[0025] T is the preset pixel threshold, which is usually 40;

[0026] After obtaining the average statistical result H(l), the image after large dynamic range compression is obtained by formula (6), and then the final enhanced image is obtained by the CLAHE algorithm, that is, the image obtained by the large dynamic range grayscale subdivision enhancement algorithm.

[0027] Preferably, the grayscale dynamic compression mapping algorithm is designed as follows: by counting the grayscale distribution range of the region of interest, the compression ratio of the region of interest is improved, thereby achieving improvement in contrast and grayscale resolution of the region of interest.

[0028] Preferably, the adaptive detail image enhancement algorithm is designed as follows:

[0029] The image detail layer can be obtained by subtracting the image obtained by the deep filter from the image obtained by the guided filter. For the guided filter, the matrix obtained by the guided filter is It represents a lot of detail information and noise information of the original image; for deep filters, the matrix obtained by the deep filters Represents the detail information of the original image; therefore, The mask model Mask_P is designed to measure the details and noise level of the image, as shown in formula (8):

[0030]

[0031] The image I_Detail' obtained by the adaptive detail image enhancement algorithm is an image obtained by performing mask processing on the detail layer image obtained by the two-stage guided filter.

[0032] The present invention also provides a system for implementing the method.

[0033] The invention also provides an application of the method in the field of infrared imaging technology.

[0034] The invention also provides an application of the system in the field of infrared imaging technology.

[0035] (III) Beneficial effects

[0036] The effect of the present invention is that: the infrared image is divided into a base layer and a detail layer through a filter, the base layer is subjected to grayscale dynamic compression mapping and large dynamic range grayscale subdivision enhancement, the detail layer image is subjected to adaptive detail enhancement, and then the processed images of the base layer and the detail layer are subjected to grayscale fusion super-resolution display, thereby improving the imaging quality of the infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is the principle diagram of the method of the present invention;

[0038] Figure 2 The enhancement effects of different filters on the flower scene, (a) is the original image, (b) bilateral filtering effect, (c) guided filtering effect, (d) weighted guided filtering effect;

[0039] Figure 3 The enhancement effects of different filters on the sky scene, (a) is the original image, (b) is the guided filtering effect, and (c) is the weighted guided filtering effect;

[0040] Figure 4 For a large dynamic range scene, (a) is a large dynamic range scene, (b) is the algorithm of this solution, (c) is the histogram equalization algorithm, and (d) is the algorithm of this solution;

[0041] Figure 5 Comparison before and after enhancement of the region of interest, (a) is before enhancement of the region of interest, (b) is after enhancement of the region of interest;

[0042] Figure 6 is the detail layer image;

[0043] Figure 7 is the mask image, a) Image (k = 3, ε1 = 25), b) Image (k=3, ε1=2500), c) Mask_p;

[0044] Figure 8 Comparison of detail layer images before and after mask processing, a) before mask processing, b) after mask processing;

[0045] Fig. 9 Comparison of images before and after grayscale super-resolution, (a11) is before grayscale super-resolution, (b11) is after grayscale super-resolution, (a12) is before grayscale super-resolution, (b12) is after grayscale super-resolution, (a2) is before grayscale super-resolution, (b2) is after grayscale super-resolution, (a3) ​​is before grayscale super-resolution, (b3) is after grayscale super-resolution, (a4) is before grayscale super-resolution, (b4) is after grayscale super-resolution. DETAILED DESCRIPTION

[0046] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0047] The present invention aims at the problem of insufficient grayscale resolution when the infrared focal plane imaging system observes targets, especially small and weak targets (such as vehicles at a long distance cannot be seen, and four-bar target lines at a long distance cannot be seen clearly) and high dynamic range scenes (such as high temperature targets are oversaturated and cannot be seen clearly), and carries out research on grayscale super-resolution technology based on small and weak targets and high dynamic scenes. The present invention focuses on breakthroughs in key technologies for deep extraction of details, key technologies for grayscale segmentation enhancement in a large dynamic range, and key technologies for image grayscale fusion super-resolution display, and improves the human eye recognition imaging performance of infrared images of small and weak targets and high dynamic scenes and the MRTD test index capability of the system.

[0048] The present invention provides a method for displaying infrared image grayscale super-resolution. The technical solution for implementing the present invention is as follows:

[0049] (1) Deep extraction of infrared image details;

[0050] (2) Large dynamic range grayscale segmentation enhancement;

[0051] (3) Grayscale dynamic compression mapping;

[0052] (4) Adaptive detail image enhancement;

[0053] (5) Grayscale fusion super-resolution display.

[0054] like Figure 1 As shown, the specific implementation steps of this method are as follows:

[0055] The original 14-bit / 16-bit infrared image is filtered by a guided filter and a deep filter respectively. Assume that the output image I_Base1 is obtained by the guided filter, and the output image I_Base2 is obtained by the deep filter. The detail image I_Detail is obtained by subtracting I_Base2 from I_Base1.

[0056] The output image I_Base2 is the base layer image. The base layer image I_Base2 is processed by a large dynamic range grayscale subdivision enhancement algorithm and a grayscale dynamic compression mapping algorithm, wherein the image I_Base2_AVHE is obtained by the large dynamic range grayscale subdivision enhancement algorithm, and the image I_Base2_CPMAP is obtained by the grayscale dynamic compression mapping algorithm; I_Base2_AVHE and I_Base2_CPMAP are fused by grayscale to obtain the image I_Base_8bits.

[0057] The detail image I_Detail is processed by the adaptive detail enhancement algorithm to obtain the image I_Detail' 。

[0058] The images I_Base_8bits and I_Detail' are fused by grayscale to obtain the final enhanced image I_Out_8bits.

[0059] Combine the following Figure 1 The infrared image grayscale super-resolution display method of the present invention is further described as follows:

[0060] (1) Filter design

[0061] The filter used in the existing image enhancement algorithm is the guided filter. As a local edge-preserving smoothing filter, the guided filter has the advantages of flexible use, high computational efficiency, and no gradient inversion at the edge. However, it also has the problems of insufficient edge-preserving ability and weak ability to extract image details, which is specifically manifested in the appearance of halo artifacts at the edge of the image and the problem that the image details are not prominent enough. The reason is that the guided filter is a local optimization filter, and the regularization factor λ in its cost function E is fixed in all local windows, as shown in formula (1).

[0062]

[0063] This results in that when using guided filters for image enhancement, while details are enhanced, halos appear near the edges of the image, reducing image quality. On the other hand, in the existing usage of guided filters, there is a lack of guiding map input, resulting in limited ability of guided filters to extract image details. Therefore, the present invention designs a deep filter that is resistant to grayscale inversion and halos, namely a deep detail extraction filter, which can effectively improve image quality and detail presentation.

[0064] The deep detail extraction filter borrows the idea of ​​the global filter and changes the regularization factor λ from a fixed value to an adaptive variable determined by the relationship between local and global variances. The new cost function E is shown in formula (2):

[0065]

[0066] in,

[0067] G(p) is the guide image, which is the original input image or the image after the original input image is smoothed, G(p') is the entire neighborhood image, X(p) is the original input image, a p' and b p' is the preset neighborhood linear coefficient, ε is a fixed parameter used to adjust the weight size, is the neighborhood full-image variance, To guide the image variance, N represents the total number of pixels in the original image. At the edge, the adaptive regularization factor is smaller than the original λ, so it has a better edge preservation effect; while in the flat area, the adaptive regularization factor is larger than the original λ, so it has a better smoothing effect, can more effectively extract image details, and achieve the purpose of image enhancement. Figure 2 This is the enhancement effect diagram using different filters. It can be seen that at the arrow, the bilateral filter has a gradient reversal, the guided filter has a halo, and the weighted guided filter has no negative effect and has the best enhancement effect. Figure 3 It can be seen that there is no negative effect of using weighted guided filtering, and the enhanced image is more realistic.

[0068] (2) Large dynamic range grayscale subdivision enhancement algorithm

[0069] Assume that the original 14-bit / 16-bit infrared image is X, which has L image gray levels, and its histogram corresponding to gray level l is shown in formula (4).

[0070] H(l)=n l ,l=0,1,···,L-1 (4)

[0071] Where n l is the total number of pixels with gray value l in the original image, and L is the total number of gray levels in the original image. Through the histogram equalization algorithm and the desired gray level number R of the enhanced image (generally 256, i.e. 8-bit image), the image enhancement algorithm based on histogram equalization can be derived, as shown in formula (5).

[0072]

[0073] Where T(l) is a mapping function, which represents the mapping from an input infrared image with grayscale level (i.e. grayscale value, also known as grayscale level) l to an image y with grayscale level R l , N represents the total number of pixels of the original image. After the adjacent gray levels l and l-1 of the original image are mapped, the contrast enhancement ratio (of the original image) can be expressed by formula (6):

[0074]

[0075] In the formula, δ lProportional to H(l), that is, after histogram equalization, the grayscale with a larger distribution probability in the original image has a stronger image contrast with its adjacent grayscale; conversely, the contrast between the grayscale with a smaller distribution probability and its adjacent grayscale will be compressed. In actual images, pixels located in the background and uniform areas usually have grayscales with a larger distribution probability, while pixels located in the detail area usually correspond to grayscales with a smaller distribution probability. After histogram equalization, the noise in the background and flat areas of the image will be over-amplified, while the pixels in the detail area cannot be effectively enhanced, resulting in the histogram equalization not being able to achieve a good image enhancement effect.

[0076] To further enhance the contrast of the image detail area and avoid excessive amplification of the noise in the background and flat area, this scheme adopts the idea of ​​average histogram enhancement. The range of the output image is obtained by averaging each effective gray level of the input image. No matter how many pixels occupy the effective gray level, it is only calculated once. The so-called effective gray level means that a certain gray level is occupied by enough (T) pixels. Therefore, the projection histogram is expressed as formula (7), and the calculated value of formula (7) is used to replace the calculated value of formula (4);

[0077]

[0078] T is the preset pixel threshold, which is usually 40;

[0079] After obtaining the average statistical result H(l), the image after large dynamic range compression is obtained by formula (6), and then the final enhanced image is obtained by the CLAHE algorithm, that is, the image obtained by the large dynamic range grayscale subdivision enhancement algorithm. The image effects after compression by different compression algorithms are as follows: Figure 4 shown.

[0080] (3) Grayscale dynamic compression mapping algorithm

[0081] In order to improve the grayscale detail resolution of the image region of interest, the present invention proposes a grayscale dynamic compression mapping algorithm based on the region of interest. By statistically analyzing the grayscale distribution range of the region of interest, the compression ratio of the region of interest is improved, and the contrast and grayscale resolution of the region of interest are improved. Figure 5 shown.

[0082] (4) Adaptive detail image enhancement algorithm

[0083] Human vision experiments have shown that noise in flat areas has a greater stimulation on the human visual system, while noise in complex texture areas is not easily perceived by the human eye. Therefore, when enhancing image details, the ability to magnify details in flat areas should be minimized. At the same time, because the details in the flat part are small, there is no need to over-magnify them; while in areas with more details, the details can be magnified as much as possible, which can not only obtain a clearer visual effect, but also will not feel that the noise is over-magnified. In the actual processing process, the gradient coefficient is used as the adaptive gain control factor. A large gradient coefficient indicates that the image is more uniform, and a small gradient coefficient indicates that the image details are richer, thereby adaptively adjusting the magnification of details in different areas.

[0084] The image detail layer can be obtained by subtracting the image obtained by the deep filter from the image obtained by the guided filter, such as Figure 6 shown.

[0085] Depend on Figure 6 It can be seen that the detail layer image obtained by the two-stage guided filter contains a lot of noise information, and the image noise must be suppressed to obtain a better image enhancement effect.

[0086] By analyzing formula (7), we can know that for the guided filter, the matrix obtained by the guided filter is (Linear coefficients of guided filtering) represents a large amount of detail information and noise information of the original image; for deep filters, the matrix obtained by the deep filter (Linear coefficients of deep filtering) mainly represent the details of the original image. The mask model Mask_P is designed to measure the details and noise level of the image, as shown in formula (8).

[0087]

[0088] And Mask_P image such as Figure 7 shown.

[0089] The detail layer images before and after processing by the mask model Mask_P are as follows Figure 8 The image I_Detail' obtained by the adaptive detail enhancement algorithm is the image obtained by performing mask processing on the detail layer image obtained by the two-stage guided filter.

[0090] Depend on Figure 8 It can be seen that after mask processing, the noise of the detail layer image is effectively suppressed.

[0091] (5) Grayscale fusion super-resolution display

[0092] The grayscale fusion of the base layer image and the detail layer image is used to obtain the image I_Out_8bits, and the final super-resolution display effect is obtained, such as Fig. 9 As shown. The comparison of the target before and after magnification is as follows Fig. 9 As shown in (a12) and (b12).

[0093] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for displaying infrared image grayscale super-resolution, characterized in that: The original 14-bit / 16-bit infrared image is filtered by the guided filter and the deep filter respectively. The output image I_Base1 is obtained by the guided filter, and the output image I_Base2 is obtained by the deep filter. The detail image I_Detail is obtained by subtracting I_Base2 from I_Base1. The output image I_Base2 is the base layer image. The base layer image I_Base2 is processed by a large dynamic range grayscale subdivision enhancement algorithm and a grayscale dynamic compression mapping algorithm. The image I_Base2_AVHE is obtained by the large dynamic range grayscale subdivision enhancement algorithm, and the image I_Base2_CPMAP is obtained by the grayscale dynamic compression mapping algorithm. The image I_Base2_AVHE and I_Base2_CPMAP are fused in grayscale to obtain the image I_Base_8bits. The detail image I_Detail is subjected to an adaptive detail enhancement algorithm to obtain an image I_Detail'; The images I_Base_8bits and I_Detail' are fused by grayscale to obtain the final enhanced image I_Out_8bits.

2. The method according to claim 1, characterized in that The deep filter is a deep detail extraction filter; the deep detail extraction filter draws on the idea of ​​the global filter and changes the regularization factor λ from a fixed value to an adaptive variable determined by the relationship between local and global variances. The new cost function E is shown in formula (2): in, G(p) is the guide image, which is the original input image or the image after the original input image is smoothed, G(p') is the entire neighborhood image, X(p) is the original input image, a p' and b p' is the preset neighborhood linear coefficient, ε is a fixed parameter used to adjust the weight size, is the neighborhood full-image variance, is the guide image variance, and N represents the total number of pixels in the original image.

3. The method according to claim 2, characterized in that The deep filter is a deep filter that resists grayscale inversion and halo.

4. The method according to claim 2, characterized in that The large dynamic range grayscale subdivision enhancement algorithm is designed as follows: Assume that the original 14-bit / 16-bit infrared image is X, which has L image gray levels. The histogram corresponding to gray level l is shown in formula (4): H(l)=n l ,l=0,1,···,L-1 (4) Where n l is the total number of pixels with gray value l in the original image, and L is the total number of gray levels in the original image. Through the histogram equalization algorithm and the desired number of gray levels R of the enhanced image, the image enhancement algorithm based on histogram equalization is derived, as shown in formula (5): Where T(l) is the mapping function, which represents the mapping from the input infrared image with gray level l to the image y with gray level R l ; The contrast enhancement ratio of adjacent gray levels l and l-1 of the original image after mapping is expressed by formula (6): In the formula, δ l Proportional to H(l), that is, after histogram equalization, the grayscale with a larger distribution probability in the original image has a stronger image contrast with its adjacent grayscale; On the contrary, the contrast between the gray level with a smaller distribution probability and its adjacent gray level will be compressed; Further, the projection histogram is expressed as formula (7), and the calculated value of formula (4) is replaced by the calculated value of formula (7); T is the preset pixel threshold, which is usually 40; After obtaining the average statistical result H(l), the image after large dynamic range compression is obtained by formula (6), and then the final enhanced image is obtained by the CLAHE algorithm, that is, the image obtained by the large dynamic range grayscale subdivision enhancement algorithm.

5. The method according to claim 1, characterized in that The grayscale dynamic compression mapping algorithm is designed as follows: by counting the grayscale distribution range of the region of interest, the compression ratio of the region of interest is improved, thereby achieving improvement in contrast and grayscale resolution of the region of interest.

6. The method according to claim 4, characterized in that The adaptive detail image enhancement algorithm is designed as follows: The image detail layer can be obtained by subtracting the image obtained by the deep filter from the image obtained by the guided filter. For the guided filter, the matrix obtained by the guided filter is It represents a lot of detail information and noise information of the original image; for deep filters, the matrix obtained by the deep filters Represents the detail information of the original image; therefore, The mask model Mask_P is designed to measure the details and noise level of the image, as shown in formula (8): The image I_Detail' obtained by the adaptive detail image enhancement algorithm is an image obtained by performing mask processing on the detail layer image obtained by the two-stage guided filter.

7. The method according to claim 6, characterized in that T is 40.

8. A system for implementing the method according to any one of claims 1 to 7.

9. Application of the method according to any one of claims 1 to 7 in the field of infrared imaging technology.

10. Application of the system as claimed in claim 8 in the field of infrared imaging technology.

Images