Dual-band infrared image fusion method based on edge window guided filtering and laplacian pyramid

Abstract

The application discloses a dual-band infrared image fusion method based on edge window guided filtering and Laplacian pyramid, and relates to the technical field of multi-source image fusion.The application respectively carries out Laplace transformation on each image participating in fusion, obtains a Gaussian pyramid and a Laplacian pyramid of the image, then adopts a weighted average method to carry out image fusion on the top layer of the Laplacian pyramid, uses the maximum absolute value rule to obtain an initial weight map of fusion for the remaining layers of the Laplacian pyramid, and optimizes the initial weight map of fusion by using edge window guided filtering to obtain a fusion weight map, carries out fusion through the fusion weight, then carries out image reconstruction on the fused image, and obtains a final fused image.The dual-band infrared image fusion method provided by the application contains less noise and retains more edge information and detail features of the source images.

Description

Technical Field

[0001] This application relates to the field of multi-source image fusion technology, and in particular to a dual-band infrared image fusion method based on side-window guided filtering and Laplacian pyramid. Background Technology

[0002] Infrared imaging technology boasts advantages such as long detection range, strong environmental adaptability, and high recognition capability. As a passive imaging technology, it utilizes the infrared radiation emitted or reflected by objects to create an image. Therefore, it does not require an external light source, is unaffected by strong light interference, and can operate in all weather conditions. Infrared radiation has strong penetrating power in harsh natural conditions such as smoke, rain, snow, and fog, and can detect camouflaged targets. The infrared radiation band can be subdivided into three different bands: short-wave infrared (1-2.5μm), mid-wave infrared (3-5μm), and long-wave infrared (8-14μm). Short-wave infrared imaging offers higher contrast and clearer representation of target details, while long-wave infrared primarily reflects the thermal information of objects.

[0003] Single-band sensors acquire limited image information, failing to meet the requirements of use in harsh weather and complex scenarios. Multi-band video image sensors are used for image fusion to increase the image information across different spectral bands of the captured scene, thereby gaining a more comprehensive understanding of the scene. Currently, most image fusion research focuses on multi-source image fusion using visible light and long-wave infrared. However, short-wave infrared imaging offers superior environmental adaptability and detection capabilities compared to visible light imaging. Short-wave infrared imaging technology has an inherent advantage in night vision compared to visible light imaging, enabling image acquisition in dark environments. Furthermore, short-wave infrared has strong atmospheric penetration, providing excellent fog-penetrating imaging capabilities. By using short-wave infrared instead of visible light for multi-source image fusion, long-wave infrared can be used to detect hot targets against a cold background, while short-wave infrared can be used to identify targets, thereby improving environmental adaptability, enhancing the understanding of the captured scene, providing a more comprehensive description of object characteristics, and improving target detection capabilities.

[0004] The most common method for image fusion is based on multi-scale transform. This involves decomposing multi-source images at multiple scales, comprehensively utilizing multi-level information, processing the images at different scales according to certain fusion rules, and finally reconstructing the final fused image to obtain a more comprehensive and accurate fusion result. Classic multi-scale transforms include Laplacian pyramid transform, discrete wavelet transform, and dual-tree complex wavelet. These multi-scale transforms can extract salient features from source images and play an important role in multi-source image fusion. However, typical methods, such as pyramid transform fusion algorithms, may over-smooth the high-frequency components, leading to loss or blurring of details. Wavelet transform cannot accurately represent boundary features in images, causing information loss. Therefore, these fusion methods suffer from drawbacks such as blurred details and susceptibility to halo and artifact phenomena in image fusion. Summary of the Invention

[0005] This application provides a dual-band infrared image fusion method based on side-window guided filtering and Laplacian pyramid, which can be used to solve the technical problems of blurred details and halo and artifact phenomena in existing fusion methods.

[0006] This application provides a dual-band infrared image fusion method based on side-window guided filtering and Laplacian pyramid, the method including:

[0007] Step 1: Perform N Gaussian filtering and interleaved downsampling operations on the short-wave infrared image and the long-wave infrared image respectively to obtain the N+1 layer Gaussian pyramid of the short-wave infrared image and the N+1 layer Gaussian pyramid of the long-wave infrared image. The (l+1)th layer of the Gaussian image pyramid is the result of Gaussian filtering and interleaved downsampling of the l-th layer image.

[0008] Step 2: Decompose the short-wave infrared Gaussian pyramid and the long-wave infrared Gaussian pyramid obtained in Step 1 using the Laplace pyramid decomposition method to obtain the N+1 layer Laplace pyramids for short-wave infrared and the N+1 layer Laplace pyramids for long-wave infrared.

[0009] Step 3: For the high-frequency images of layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in Step 2, the initial weight maps of the short-wave infrared and long-wave infrared high-frequency images are obtained according to the maximum absolute value rule.

[0010] Step 4: Perform side-window guided filtering on the initial weight map of the high-frequency image obtained in Step 3. Use the image of the corresponding level of the Gaussian pyramid obtained in Step 1 as the guide map to filter and obtain the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images.

[0011] Step 5: Using the high-frequency images of layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in Step 2 and Step 4, and the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images, the fusion weight is used to obtain the fusion image of layers 0 to N-1 by weighted summation.

[0012] Step 6: For the Nth layer low-frequency image of the short-wave infrared and long-wave infrared Laplacian pyramid, the weighted average method is used to fuse the images to obtain the Nth layer fused image, which is then combined with the 0 to N-1 layer fused images obtained in Step 5 to obtain the N+1 layer fused image of the Laplacian pyramid.

[0013] Step 7: The final fused image is obtained by recursively reconstructing the Laplacian pyramid of the fused image obtained in Step 6 layer by layer according to the Laplacian pyramid reconstruction method.

[0014] Furthermore, the short-wave infrared Gaussian pyramid and the long-wave infrared Gaussian pyramid obtained in step 1 are decomposed using the Laplace pyramid decomposition method to obtain N+1 layers of Laplace pyramids for short-wave infrared and N+1 layers of Laplace pyramids for long-wave infrared.

[0015]

[0016] Among them G l This represents the l-th layer of the Gaussian pyramid image G0. This represents the image of the (l+1)th level Gaussian pyramid of G0 after being upsampled and magnified by one level using Gaussian filtering. LP l This represents the image of the lth layer of the Laplace pyramid.

[0017] Further, in step 3, the initial weight maps of the short-wave infrared and long-wave infrared high-frequency images from layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in step 2 are obtained according to the maximum absolute value rule; the specific method is as follows:

[0018]

[0019]

[0020] in This represents the initial weight map of the l-th layer in the shortwave infrared spectrum. This represents the initial weight map of the l-th layer in the long-wave infrared spectrum. This represents the image of the first layer of the Laplacian pyramid in shortwave infrared. This represents the image of the first layer of the Laplace pyramid in long-wave infrared.

[0021] Furthermore, in step 4, the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images is determined by the following method:

[0022]

[0023]

[0024] Where SWGF represents the side window guided filtering operation. This represents the fusion weight map of the l-th layer of shortwave infrared. This represents the fusion weight map of the l-th layer in the long-wave infrared spectrum. This represents the image of the l-th layer of the shortwave infrared Gaussian pyramid. This represents the image of the l-th layer of the long-wave infrared Gaussian pyramid. This represents the initial weight map of the l-th layer in the shortwave infrared spectrum. denoted as the initial weight map of the l-th layer in long-wave infrared, r represents the size radius of the side-window guided filter, and ε represents the constraint parameters of the side-window guided filter.

[0025] Further, using the high-frequency images of layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in steps 2 and 4, and the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images, a fused image of layers 0 to N-1 is obtained by weighted summation of the fusion weights, including:

[0026]

[0027] Among them LF l This represents the l-th layer image of the merged Laplace's pyramid. This represents the image of the first layer of the Laplacian pyramid in shortwave infrared. This represents the image of the first layer of the Laplacian pyramid in long-wave infrared. This represents the fusion weight map of the l-th layer of shortwave infrared. This represents the fusion weight map of the l-th layer in long-wave infrared.

[0028] Furthermore, the top-level image of the Nth layer weighted average fusion in step 6 is determined by the following method:

[0029]

[0030] Among them LF N This represents the topmost image of the Nth layer of the merged Laplace's Pyramid. This represents the top layer of the Nth level of the Laplacian pyramid in shortwave infrared. This represents the top layer of the Nth layer of the Laplace pyramid in long-wave infrared.

[0031] Furthermore, in step 7, the image reconstruction method is as follows:

[0032]

[0033] Among them, GF N The Nth layer of the Gaussian pyramid representing the fused image, LF N This represents the Nth layer image of the merged Laplacian pyramid, GF l The image represents the l-th layer of the Gaussian pyramid of the fused image, LF. l This represents the l-th layer image of the merged Laplace's pyramid. For the Gaussian pyramid layer 1+1 image of the fused image, GF l+1 After being amplified by an upsampled Gaussian filter and then compared with GF... l Images of the same size, GF0 is the final fused image.

[0034] This application, based on the Laplacian pyramid decomposition system, introduces a side-window guided filter to optimize the fusion weights of the high-frequency images obtained from the source image decomposition. This significantly improves the fusion effect of dual-band infrared images, enhances the protection of image edge information and detail features, and avoids the generation of halos and artifacts. In this invention, as long as short-wave infrared and long-wave infrared images of the same scene are input, effective multi-scale fusion can be performed to obtain a high-quality fused image. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating the fusion method provided in the embodiments of this application.

[0036] Figure 2 This is a flowchart illustrating the overall process of three-layer pyramid decomposition and fusion of the fusion method provided in this application embodiment.

[0037] Figure 3 This is a shortwave infrared image provided in the embodiments of this application.

[0038] Figure 4 This is a long-wave infrared image provided in the embodiments of this application.

[0039] Figure 5 This application provides an embodiment of the method for... Figure 3 and Figure 4 The fused image obtained by fusion. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0041] Step 1: For Figure 3 Shortwave infrared images and Figure 4The long-wave infrared image is subjected to N Gaussian filtering and interlaced downsampling operations to obtain the N+1 layer Gaussian pyramid of short-wave infrared and the N+1 layer Gaussian pyramid of long-wave infrared. The (l+1)th layer of the Gaussian image pyramid is obtained by Gaussian filtering and interlaced downsampling of the first layer image. Its number of rows and columns are both 1 / 2 of the first layer image.

[0042] Step 2: Decompose the short-wave infrared Gaussian pyramid and the long-wave infrared Gaussian pyramid obtained in Step 1 using the Laplace pyramid decomposition method to obtain the N+1 layer Laplace pyramids for short-wave infrared and long-wave infrared. The calculation formula is as follows:

[0043]

[0044] Among them G l This represents the first layer of the Gaussian pyramid image G0. This represents the image of the (l+1)th level Gaussian pyramid of G0 after being upsampled and magnified by one level using Gaussian filtering. LP l This represents the first layer of the Pyramid of Laplace;

[0045] Step 3: For the high-frequency images of layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in Step 2, the initial weight map of the short-wave infrared and long-wave infrared high-frequency images is obtained according to the maximum absolute value rule, as shown in the following formula:

[0046]

[0047]

[0048] in This represents the initial weight map of the first layer of shortwave infrared. This represents the initial weight map of the first layer of long-wave infrared. This represents the first layer of the shortwave infrared Laplacian pyramid image. This represents the first layer of the Laplace pyramid in long-wave infrared.

[0049] Step 4: Apply guided side-window filtering to the initial weight map of the high-frequency image obtained in Step 3. Use the image corresponding to the level of the Gaussian pyramid obtained in Step 1 as the guide map. The filtering side-window radius is 2, the complete filtering window size is 5×5, and the constraint parameter is 0.001. The filtered image is a fused weight map of the short-wave infrared and long-wave infrared high-frequency images. The formula is as follows:

[0050]

[0051]

[0052] SWGF represents the side window guided filtering operation. This represents the first layer fusion weight map of shortwave infrared. This represents the first layer fusion weight map of long-wave infrared. This represents the first layer of the shortwave infrared Gaussian pyramid image. This represents the first layer of the long-wave infrared Gaussian pyramid image. This represents the initial weight map of the first layer of shortwave infrared. This represents the initial weight map of the first layer of long-wave infrared, r represents the size radius of the side window guided filter, and ε represents the constraint parameters of the side window guided filter;

[0053] Step 5: Using the high-frequency images of layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in Steps 2 and 4, and the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images, the fused image of layers 0 to N-1 is obtained by weighted summation of the fusion weights, as shown in the following formula:

[0054]

[0055] Among them LF l The first layer of the merged Laplace's Pyramid image. This represents the first layer of the shortwave infrared Laplacian pyramid image. This represents the first layer of the long-wave infrared Laplacian pyramid image. This represents the first layer fusion weight map of shortwave infrared. This represents the first layer fusion weight map for long-wave infrared.

[0056] Step 6: For the Nth layer low-frequency image of the short-wave infrared and long-wave infrared Laplacian pyramids, a weighted average method is used to fuse the images to obtain the Nth layer fused image. This fused image is then combined with the fused images of layers 0 to N-1 obtained in Step 5 to obtain the N+1 layer fused Laplacian pyramid. The calculation formula for the Nth layer weighted average fusion is as follows:

[0057]

[0058] Among them LF N This represents the topmost image of the Nth layer of the merged Laplace's Pyramid. This represents the top layer of the Nth level of the Laplacian pyramid in shortwave infrared. This represents the top layer of the Nth layer of the Laplace pyramid in long-wave infrared.

[0059] Step 7: Reconstruct the final fused image by recursively applying the Laplacian pyramid of the fused image obtained in Step 6, layer by layer, using the Laplacian pyramid reconstruction method. The image reconstruction formula is as follows:

[0060]

[0061] Among them, GF N The Nth layer of the Gaussian pyramid representing the fused image, LF N This represents the Nth layer image of the merged Laplacian pyramid, GF l This represents the first layer of the Gaussian pyramid of the fused image, LF. l The first layer of the merged Laplace's Pyramid image. For the Gaussian pyramid layer 1+1 image of the fused image, GF l+1 After being amplified by an upsampled Gaussian filter and then compared with GF... l For images of the same size, GF0 represents the final fused image, i.e. Figure 5 .

[0062] The embodiments described above do not constitute a limitation on the scope of protection of this application.

Claims

1. A dual-band infrared image fusion method based on side-window guided filtering and Laplacian pyramid, characterized in that, The method includes: Step 1: Perform N Gaussian filtering and interleaved downsampling operations on the short-wave infrared image and the long-wave infrared image respectively to obtain the N+1 layer Gaussian pyramid of the short-wave infrared image and the N+1 layer Gaussian pyramid of the long-wave infrared image. The (l+1)th layer of the Gaussian image pyramid is the result of Gaussian filtering and interleaved downsampling of the l-th layer image. Step 2: Decompose the short-wave infrared Gaussian pyramid and the long-wave infrared Gaussian pyramid obtained in Step 1 using the Laplace pyramid decomposition method to obtain the N+1 layer Laplace pyramids for short-wave infrared and the N+1 layer Laplace pyramids for long-wave infrared. Step 3: For the high-frequency images of layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in Step 2, the initial weight maps of the short-wave infrared and long-wave infrared high-frequency images are obtained according to the maximum absolute value rule. Step 4: Perform side-window guided filtering on the initial weight map of the high-frequency image obtained in Step 3. Use the image of the corresponding level of the Gaussian pyramid obtained in Step 1 as the guide map to filter and obtain the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images. Step 5: Using the high-frequency images of layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in Step 2 and Step 4, and the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images, the fusion weight is used to obtain the fusion image of layers 0 to N-1 by weighted summation. Step 6: For the Nth layer low-frequency image of the short-wave infrared and long-wave infrared Laplacian pyramid, the weighted average method is used to fuse the images to obtain the Nth layer fused image, which is then combined with the 0 to N-1 layer fused images obtained in Step 5 to obtain the N+1 layer fused image of the Laplacian pyramid. Step 7: The final fused image is obtained by recursively reconstructing the Laplacian pyramid of the fused image obtained in Step 6 layer by layer according to the Laplacian pyramid reconstruction method.

2. The method according to claim 1, characterized in that, The short-wave infrared Gaussian pyramid and the long-wave infrared Gaussian pyramid obtained in step 1 are decomposed using the Laplace pyramid decomposition method to obtain N+1 layers of Laplace pyramids for short-wave infrared and N+1 layers of Laplace pyramids for long-wave infrared. Among them G l This represents the l-th layer of the Gaussian pyramid image G0. This represents the image of the (l+1)th level Gaussian pyramid of G0 after being upsampled and magnified by one level using Gaussian filtering. LP l This represents the image of the lth layer of the Laplace pyramid.

3. The method according to claim 1, characterized in that, Step 3: For the high-frequency images of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in Step 2, from layers 0 to N-1, the initial weight map of the short-wave infrared and long-wave infrared high-frequency images is obtained according to the maximum absolute value rule. include: in This represents the initial weight map of the l-th layer in the shortwave infrared spectrum. This represents the initial weight map of the l-th layer in the long-wave infrared spectrum. This represents the image of the first layer of the Laplacian pyramid in shortwave infrared. This represents the image of the first layer of the Laplace pyramid in long-wave infrared.

4. The method according to claim 1, characterized in that, In step 4, the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images is determined by the following method: SWGF represents the side window guided filtering operation. This represents the fusion weight map of the l-th layer of shortwave infrared. This represents the fusion weight map of the l-th layer in long-wave infrared. This represents the image of the l-th layer of the shortwave infrared Gaussian pyramid. This represents the image of the l-th layer of the long-wave infrared Gaussian pyramid. This represents the initial weight map of the l-th layer in the shortwave infrared spectrum. denoted as the initial weight map of the l-th layer in long-wave infrared, r represents the size radius of the side-window guided filter, and ε represents the constraint parameters of the side-window guided filter.

5. The method according to claim 1, characterized in that, Using the high-frequency images of layers 0 to N-1 of the short-wave infrared Laplacian pyramid and the long-wave infrared Laplacian pyramid obtained in steps 2 and 4, and the fusion weight map of the short-wave infrared and long-wave infrared high-frequency images, the fused image of layers 0 to N-1 is obtained by weighted summation of the fusion weights, including: Among them LF l This represents the l-th layer image of the merged Laplace's pyramid. This represents the image of the first layer of the Laplacian pyramid in shortwave infrared. This represents the image of the first layer of the Laplacian pyramid in long-wave infrared. This represents the fusion weight map of the l-th layer of shortwave infrared. This represents the fusion weight map of the l-th layer in long-wave infrared.

6. The method according to claim 1, characterized in that, In step 6, the top layer image of the Nth layer weighted average fusion is determined by the following method: Among them LF N This represents the topmost image of the Nth layer of the merged Laplace's Pyramid. This represents the top layer of the Nth level of the Laplacian pyramid in shortwave infrared. This represents the top layer of the Nth layer of the Laplace pyramid in long-wave infrared.

7. The method according to claim 1, characterized in that, In step 7, the image reconstruction method is as follows: Among them, GF N The Nth layer of the Gaussian pyramid representing the fused image, LF N This represents the Nth layer image of the merged Laplacian pyramid, GF l The image represents the l-th layer of the Gaussian pyramid of the fused image, LF. l This represents the l-th layer image of the merged Laplace's pyramid. For the Gaussian pyramid layer 1+1 image of the fused image, GF l+1 After being amplified by an upsampled Gaussian filter and then compared with GF... l Images of the same size, GF0 is the final fused image.

Images