An infrared image and visible light image fusion method

Abstract

The application discloses an infrared image and visible light image fusion method, which utilizes wavelet transform to efficiently obtain multi-scale expression of the image to be fused, and combines with accurate feature description capability of a reversible neural network to obtain a better quality fused image; compared with early traditional image fusion methods based on manual design (such as wavelet transform, curved surface model and sparse representation), parameters of the reversible neural network of the application are automatically obtained through end-to-end learning to avoid complicated feature selection engineering design and have stronger adaptability; compared with deep learning methods based on common convolutional neural networks, the reversible structure of the reversible neural network of the application can accurately represent internal structural information of the image, that is, the application can obtain infrared and visible light fused images rich in details, improve consistency of image details and global semantics, and obtain better quality fused images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method for fusing infrared and visible light images. Background Technology

[0002] Image fusion technology is an important branch of information fusion and is one of the current research hotspots in the fields of computer vision and pattern recognition. Early image fusion methods, such as those based on wavelet transform, surface models, and sparse representation, were all based on manual design. That is, the above image fusion methods require tedious feature selection engineering design and have poor adaptability.

[0003] In recent years, with the development of deep learning, various convolutional neural networks have been introduced into image fusion tasks. Although fusing features extracted from ordinary convolutional neural networks can yield better feature representations, the irreversible structure of ordinary convolutional neural networks limits their ability to accurately model the intrinsic structure of images, thus affecting the final image fusion quality.

[0004] Therefore, how to provide a method for fusing infrared and visible light images with higher image quality and greater adaptability is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method for fusing infrared and visible light images, which results in a fused image with higher quality and greater adaptability.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for fusing infrared and visible light images includes the following steps: S1: Perform wavelet transform on the infrared image to be fused to obtain the initial high-frequency infrared component and the initial low-frequency infrared component; The visible light images to be fused are subjected to wavelet transform to obtain the initial high-frequency components and the initial low-frequency components of visible light; S2: Input the initial infrared high-frequency component and the initial infrared low-frequency component into the reversible neural network in the forward direction to obtain the infrared reconstructed high-frequency component and the infrared reconstructed low-frequency component. The initial visible light high-frequency component and the initial visible light low-frequency component are input into the reversible neural network to obtain the visible light reconstructed high-frequency component and the visible light reconstructed low-frequency component. S3: Perform maximum value fusion on the infrared reconstructed high-frequency component and the visible light reconstructed high-frequency component to obtain the initial fused high-frequency component; The infrared reconstructed low-frequency component and the visible light reconstructed low-frequency component are averaged and fused to obtain an initial fused low-frequency component. S4: The initial fused high-frequency component and the initial fused low-frequency component are inversely input into the reversible neural network to obtain the enhanced fused high-frequency component and the enhanced fused low-frequency component; S5: Perform inverse wavelet transform on the enhanced fusion high-frequency component and the enhanced fusion low-frequency component to obtain the final fused image.

[0008] In S2, the reversible neural network consists of N layers, where N is greater than or equal to 2. Each layer performs the following operations: infrared high frequency components Input to the first depth transformation module Obtain intermediate quantity ; intermediate quantity With infrared low-frequency components Add them together to obtain the low-frequency infrared component. ; Infrared low-frequency components Input to the second depth transformation module Obtain intermediate quantity ; intermediate quantity With infrared high frequency components Input to the exponential linear unit to obtain intermediate quantities ; Infrared low-frequency components Input to the third depth transformation module Obtain intermediate quantity ; intermediate quantity With intermediate quantity Add them together to obtain the infrared high-frequency component. ; in, ; Infrared high frequency components The initial infrared high-frequency component; Infrared low-frequency component The initial infrared low-frequency component; Infrared high frequency components The high-frequency components of the infrared image are reconstructed. Infrared low-frequency component The low-frequency component is reconstructed from the infrared signal.

[0009] Each layer also performs the following operations: High frequency components of visible light Input to the first depth transformation module Obtain intermediate quantity ; intermediate quantity With the low-frequency component of visible light Add them together to obtain the low-frequency component of visible light. ; Low-frequency components of visible light Input to the second depth transformation module Obtain intermediate quantity ; intermediate quantity With visible light high frequency components Input to the exponential linear unit to obtain intermediate quantities ; Low-frequency components of visible light Input to the third depth transformation module Obtain intermediate quantity ; intermediate quantity With intermediate quantity Adding them together yields the high-frequency components of visible light. ; in, ; High-frequency components of visible light The initial visible light high-frequency component; Low-frequency components of visible light This refers to the initial low-frequency component of visible light; High-frequency components of visible light Reconstruct the high-frequency components of the visible light; Low-frequency components of visible light To reconstruct the low-frequency components of the visible light.

[0010] Preferably, the calculation formula for the initial fused high-frequency components includes: ; in, Represents the high-frequency components of visible light reconstruction. The value of the element in the i-th row and j-th column; Indicates the high-frequency components of infrared reconstruction The value of the element in the i-th row and j-th column; Indicates the initial fusion high-frequency components The value of the element in the i-th row and j-th column.

[0011] Preferably, the calculation formula for the initial fused low-frequency component includes: ; in, Represents the low-frequency components of visible light reconstruction. The value of the element in the i-th row and j-th column; Indicates the low-frequency component of infrared reconstruction The value of the element in the i-th row and j-th column; Indicates the initial fusion of low-frequency components The value of the element in the i-th row and j-th column.

[0012] Preferably, each layer of the reversible neural network in S4 performs the following operations: Will integrate low-frequency components Input to the second depth transformation module Obtain intermediate quantity ; intermediate amount Input to the exponential linear unit to obtain intermediate quantities ; Will integrate low-frequency components Input to the third depth transformation module Obtain intermediate quantity ; Will integrate high-frequency components Subtract the intermediate amount Obtain intermediate quantity ; intermediate amount Divide by the intermediate amount To obtain fused high-frequency components ; Will integrate high-frequency components Input to the first depth transformation module Obtain intermediate quantity ; Will integrate low-frequency components Subtract the intermediate amount Obtain fused low-frequency components ; in, ; Fusion of high frequency components The initial fused high-frequency components ; Integrating low-frequency components The initial fused low-frequency component ; Fusion of high frequency components For the enhanced fusion high-frequency components; Integrating low-frequency components This refers to the enhanced fusion of low-frequency components.

[0013] Preferably, the wavelet transform expression in S1 is: ; in, Indicates wavelet transform operation; This indicates either an infrared image or a visible light image to be fused. , These represent tensor shape adjustment operations for dimension separation and dimension merging, respectively; This indicates a dimension swapping operation, used to swap two separated dimensions; C represents the number of input channels. This represents a grouped convolution with C groups.

[0014] Preferably, the inverse wavelet transform expression in S5 is:

[0015] in, Indicates the inverse wavelet transform operation; Indicates a channel splicing operation; , These represent tensor shape adjustment operations for dimension separation and dimension merging, respectively; This indicates a dimension swapping operation, used to swap two separated dimensions; C represents the number of input channels. This represents a grouped deconvolution with C groups; This indicates the enhanced fusion of high-frequency components. This refers to the enhanced fusion of low-frequency components.

[0016] Preferably, the first depth transformation module Second Depth Transformation Module and the third depth transformation module This refers to the bottleneck residual block in MobileNet V1; the bottleneck residual block is composed of a 1x1 convolutional layer, a 3x3 depth-separable convolutional layer, and a 1x1 convolutional layer connected in series.

[0017] Preferably, the second depth transformation module and the third depth transformation module These are bottleneck residuals with the same structure but different weights.

[0018] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method for fusing infrared and visible light images. This method utilizes wavelet transform to efficiently obtain multi-scale representations of the images to be fused, and combines this with the accurate feature description capabilities of reversible neural networks, thereby achieving the following beneficial technical effects: 1) Compared with early traditional image fusion methods based on manual design (such as wavelet transform, surface model, and sparse representation), the reversible neural network parameters of this invention automatically obtain image representations through end-to-end learning, avoiding cumbersome feature selection engineering design and having stronger adaptability.

[0019] 2) Compared with deep learning methods based on ordinary convolutional neural networks, the reversible structure of the reversible neural network of this invention can accurately represent the intrinsic structural information of the image. That is, this invention can obtain infrared and visible light fusion images with rich details, improve the consistency between image details and global semantics, and obtain fusion images of better quality. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the overall method of the present invention; Figure 2 This is a schematic diagram illustrating the acquisition of infrared reconstruction high-frequency components, infrared reconstruction low-frequency components, visible light reconstruction high-frequency components, and visible light reconstruction low-frequency components according to the present invention. Figure 3 This is a schematic diagram illustrating the acquisition of enhanced fusion low-frequency components and enhanced fusion high-frequency components according to the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1-3 As shown in the figure, an embodiment of the present invention discloses a method for fusing infrared images and visible light images, including the following steps: S1: Perform wavelet transform on the infrared image to be fused to obtain the initial high-frequency infrared component and the initial low-frequency infrared component; The visible light images to be fused are subjected to wavelet transform to obtain the initial high-frequency components and the initial low-frequency components of visible light; Furthermore, the wavelet transform expression is: ; in, Indicates wavelet transform operation; This indicates either an infrared image or a visible light image to be fused. , These represent tensor shape adjustment operations for dimension separation and dimension merging, respectively; This indicates a dimension swapping operation, used to swap two separated dimensions; C represents the number of input channels. This represents a grouped convolution with C groups (i.e., the number of input channels).

[0024] Note: H W The infrared image to be fused in C (H) W The visible light image to be fused (C) becomes 1 / 2H after wavelet transform. 1 / 2W 4C, where C low-resolution channels constitute 1 / 2H 1 / 2W The initial infrared low-frequency component of C (1 / 2H) 1 / 2W The initial visible light low-frequency component of C); the remaining 3C channels consist of high-frequency components in different directions, forming 1 / 2H. 1 / 2W The initial infrared high-frequency component of 3C (1 / 2H) 1 / 2W The initial visible light high-frequency component of 3C); where H represents height, W represents width, and C represents the number of channels.

[0025] S2: Input the initial infrared high-frequency component and the initial infrared low-frequency component into the reversible neural network in the forward direction to obtain the infrared reconstructed high-frequency component and the infrared reconstructed low-frequency component. The initial visible light high-frequency component and the initial visible light low-frequency component are input into the reversible neural network to obtain the visible light reconstructed high-frequency component and the visible light reconstructed low-frequency component. Furthermore, the reversible neural network comprises N layers, where N is greater than or equal to 2, and each layer performs the following operations: 1 / 2H 1 / 2W Infrared high-frequency components of 3C products Input to the first depth transformation module , obtain 1 / 2H 1 / 2W intermediate values ​​of C ; 1 / 2H 1 / 2W intermediate values ​​of C With 1 / 2H 1 / 2W The infrared low-frequency component of C Add them together to get 1 / 2H 1 / 2W The infrared low-frequency component of C ; 1 / 2H 1 / 2W The infrared low-frequency component of C Input to the second depth transformation module , obtain 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W 3C intermediate quantity With 1 / 2H 1 / 2W Infrared high-frequency components of 3C products Inputting into the exponential linear unit yields 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W The infrared low-frequency component of C Input to the third depth transformation module , obtain 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W 3C intermediate quantity With 1 / 2H 1 / 2W 3C intermediate quantity Add them together to get 1 / 2H 1 / 2W Infrared high-frequency components of 3C products ; in, ; Infrared high frequency components The initial infrared high-frequency component; Infrared low-frequency component The initial infrared low-frequency component; Infrared high frequency components The high-frequency components of the infrared image are reconstructed. Infrared low-frequency component The low-frequency component is reconstructed from the infrared signal.

[0026] In summary: Infrared low-frequency component The calculation formula is: ; Infrared high frequency components The calculation formula is: ; Furthermore, each layer also performs the following operations: 1 / 2H 1 / 2W High-frequency visible light components of 3C products Input to the first depth transformation module , obtain 1 / 2H 1 / 2W intermediate values ​​of C ; 1 / 2H 1 / 2W intermediate values ​​of C With 1 / 2H 1 / 2W The visible light low-frequency component of C Add them together to get 1 / 2H 1 / 2W The visible light low-frequency component of C ; 1 / 2H 1 / 2W The visible light low-frequency component of C Input to the second depth transformation module , obtain 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W 3C intermediate quantity With 1 / 2H 1 / 2W High-frequency visible light components of 3C products Inputting into the exponential linear unit yields 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W The visible light low-frequency component of C Input to the third depth transformation module , obtain 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W 3C intermediate quantity With 1 / 2H 1 / 2W 3C intermediate quantity Adding them together gives 1 / 2H 1 / 2W High-frequency visible light components of 3C products ; in, ; High-frequency components of visible light The initial visible light high-frequency component; Low-frequency components of visible light This refers to the initial low-frequency component of visible light; High-frequency components of visible light Reconstruct the high-frequency components of the visible light; Low-frequency components of visible light To reconstruct the low-frequency components of the visible light.

[0027] In summary: Low-frequency components of visible light The calculation formula is: ; High-frequency components of visible light The calculation formula is: ; S3: Perform maximum value fusion on the infrared reconstructed high-frequency component and the visible light reconstructed high-frequency component to obtain the initial fused high-frequency component; Furthermore, the calculation formula for the initial fused high-frequency components includes: ; in, Represents the high-frequency components of visible light reconstruction. The value of the element in the i-th row and j-th column; Indicates the high-frequency components of infrared reconstruction The value of the element in the i-th row and j-th column; Indicates the initial fusion high-frequency components The value of the element in the i-th row and j-th column.

[0028] The infrared reconstructed low-frequency component and the visible light reconstructed low-frequency component are averaged and fused to obtain an initial fused low-frequency component. Furthermore, the calculation formula for the initial fused low-frequency component includes: ; in, Represents the low-frequency components of visible light reconstruction. The value of the element in the i-th row and j-th column; Indicates the low-frequency component of infrared reconstruction The value of the element in the i-th row and j-th column; Indicates the initial fusion of low-frequency components The value of the element in the i-th row and j-th column.

[0029] S4: The initial fused high-frequency component and the initial fused low-frequency component are inversely input into the reversible neural network to obtain the enhanced fused high-frequency component and the enhanced fused low-frequency component; Furthermore, each layer of the reversible neural network performs the following operations: 1 / 2H 1 / 2W C's fusion of low-frequency components Input to the second depth transformation module , obtain 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W 3C intermediate quantity Inputting into the exponential linear unit yields 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W C's fusion of low-frequency components Input to the third depth transformation module , obtain 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W 3C high-frequency component Subtract 1 / 2H 1 / 2W 3C intermediate quantity Obtain 1 / 2H 1 / 2W 3C intermediate quantity ; 1 / 2H 1 / 2W 3C intermediate quantity Divide by 1 / 2H 1 / 2W 3C intermediate quantity , obtain 1 / 2H 1 / 2W 3C high-frequency component ; 1 / 2H 1 / 2W 3C high-frequency component Input to the first depth transformation module , obtain 1 / 2H 1 / 2W intermediate values ​​of C ; 1 / 2H 1 / 2W C's fusion of low-frequency components Subtract 1 / 2H 1 / 2W intermediate values ​​of C , obtain 1 / 2H 1 / 2W C's fusion of low-frequency components ; in, ; Fusion of high frequency components The initial fused high-frequency components ; Integrating low-frequency components The initial fused low-frequency component ; Fusion of high frequency components For the enhanced fusion high-frequency components; Integrating low-frequency components This refers to the enhanced fusion of low-frequency components.

[0030] In summary: The fused high-frequency components The calculation formula is: ; The fusion of low-frequency components The calculation formula is: .

[0031] S5: Perform inverse wavelet transform on the enhanced fusion high-frequency component and the enhanced fusion low-frequency component to obtain the final fused image.

[0032] Furthermore, the inverse wavelet transform expression is:

[0033] in, Indicates the inverse wavelet transform operation; Indicates a channel splicing operation; , These represent tensor shape adjustment operations for dimension separation and dimension merging, respectively; This indicates a dimension swapping operation, used to swap two separated dimensions; C represents the number of input channels. This represents a grouped deconvolution with a number of groups C (i.e., the number of input channels); This indicates the enhanced fusion of high-frequency components. This refers to the enhanced fusion of low-frequency components.

[0034] Furthermore, the first depth transformation module Second Depth Transformation Module and the third depth transformation module This refers to the bottleneck residual block in MobileNet V1; the bottleneck residual block is composed of a 1x1 convolutional layer, a 3x3 depth-separable convolutional layer, and a 1x1 convolutional layer connected in series.

[0035] Furthermore, the first depth transformation module The number of output channels of the 1x1 convolutional layer is C, and the second depth transformation module... and the third depth transformation module The output channels of the 1x1 convolutional layers are all 3C.

[0036] Furthermore, the second depth transformation module and the third depth transformation module These are bottleneck residuals with the same structure but different weights.

[0037] Furthermore, the total loss function of this invention is: ; In the above formula, Indicates the image fusion loss. Indicates the frequency correlation loss in infrared and visible light; image fusion loss. Image gradient texture loss and image content intensity loss Sum of; Image gradient texture loss This represents the gradient texture loss of the final fused image, the infrared image to be fused, and the visible light image to be fused; image content intensity loss. This represents the content intensity loss of the final fused image, the infrared image to be fused, and the visible light image to be fused.

[0038] Furthermore, image content intensity loss and image gradient texture loss The calculation formulas are as follows: ; ; In the formula, F represents the final fused image, I represents the infrared image to be fused, and represents the visible light image to be fused. This represents the Sobel gradient image of the final fused image; and These represent the Sobel gradient images of the infrared image to be fused and the visible light image to be fused, respectively. Furthermore, the infrared-visible light frequency correlation loss The calculation formula is: ; in, This represents high-frequency correlation loss; This represents low-frequency correlation loss; This indicates correlation calculation; , , These represent the high-frequency components of infrared reconstruction, the high-frequency components of visible light reconstruction, the low-frequency components of infrared reconstruction, and the low-frequency components of visible light reconstruction, respectively. This represents the numerical stability constant (to prevent negative or zero numbers in the denominator; the default value is 1.01).

[0039] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0040] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for fusing infrared and visible light images, characterized in that, Includes the following steps: S1: Perform wavelet transform on the infrared image to be fused to obtain the initial high-frequency infrared component and the initial low-frequency infrared component; The visible light images to be fused are subjected to wavelet transform to obtain the initial high-frequency components and the initial low-frequency components of visible light; S2: Input the initial infrared high-frequency component and the initial infrared low-frequency component into the reversible neural network in the forward direction to obtain the infrared reconstructed high-frequency component and the infrared reconstructed low-frequency component. The initial visible light high-frequency component and the initial visible light low-frequency component are input into the reversible neural network to obtain the visible light reconstructed high-frequency component and the visible light reconstructed low-frequency component. S3: Perform maximum value fusion on the infrared reconstructed high-frequency component and the visible light reconstructed high-frequency component to obtain the initial fused high-frequency component; The infrared reconstructed low-frequency component and the visible light reconstructed low-frequency component are averaged and fused to obtain an initial fused low-frequency component. S4: The initial fused high-frequency component and the initial fused low-frequency component are inversely input into the reversible neural network to obtain the enhanced fused high-frequency component and the enhanced fused low-frequency component; S5: Perform inverse wavelet transform on the enhanced fusion high-frequency component and the enhanced fusion low-frequency component to obtain the final fused image; In S2, the reversible neural network consists of N layers, where N is greater than or equal to 2. Each layer performs the following operations: infrared high frequency components Input to the first depth transformation module Obtain intermediate quantity ; intermediate quantity With infrared low-frequency components Add them together to obtain the low-frequency infrared component. ; Infrared low-frequency components Input to the second depth transformation module Obtain intermediate quantity ; intermediate quantity With infrared high frequency components Input to the exponential linear unit to obtain intermediate quantities ; Infrared low-frequency components Input to the third depth transformation module Obtain intermediate quantity ; intermediate quantity With intermediate quantity Add them together to obtain the infrared high-frequency component. ; in, ; Infrared high frequency components The initial infrared high-frequency component; Infrared low-frequency component The initial infrared low-frequency component; Infrared high frequency components The high-frequency components of the infrared image are reconstructed. Infrared low-frequency component The low-frequency component is reconstructed from the infrared signal; Each layer also performs the following operations: High frequency components of visible light Input to the first depth transformation module Obtain intermediate quantity ; intermediate quantity With the low-frequency component of visible light Add them together to obtain the low-frequency component of visible light. ; Low-frequency components of visible light Input to the second depth transformation module Obtain intermediate quantity ; intermediate quantity With visible light high frequency components Input to the exponential linear unit to obtain intermediate quantities ; Low-frequency components of visible light Input to the third depth transformation module Obtain intermediate quantity ; intermediate quantity With intermediate quantity Adding them together yields the high-frequency components of visible light. ; in, ; High-frequency components of visible light The initial visible light high-frequency component; Low-frequency components of visible light This refers to the initial low-frequency component of visible light; High-frequency components of visible light Reconstruct the high-frequency components of the visible light; Low-frequency components of visible light To reconstruct the low-frequency components of the visible light.

2. The method for fusing infrared and visible light images according to claim 1, characterized in that, The calculation formula for the initial fused high-frequency components includes: ； in, Represents the high-frequency components of visible light reconstruction. The value of the element in the i-th row and j-th column; Indicates the high-frequency components of infrared reconstruction The value of the element in the i-th row and j-th column; Indicates the initial fusion high-frequency components The value of the element in the i-th row and j-th column.

3. The method for fusing infrared and visible light images according to claim 2, characterized in that, The calculation formula for the initial fused low-frequency component includes: ； in, Represents the low-frequency components of visible light reconstruction. The value of the element in the i-th row and j-th column; Indicates the low-frequency component of infrared reconstruction The value of the element in the i-th row and j-th column; Indicates the initial fusion of low-frequency components The value of the element in the i-th row and j-th column.

4. The method for fusing infrared and visible light images according to claim 3, characterized in that, Each layer of the reversible neural network described in S4 performs the following operation: Will integrate low-frequency components Input to the second depth transformation module Obtain intermediate quantity ; intermediate amount Input to the exponential linear unit to obtain intermediate quantities ; Will integrate low-frequency components Input to the third depth transformation module Obtain intermediate quantity ; Will integrate high-frequency components Subtract the intermediate amount Obtain intermediate quantity ; intermediate amount Divide by the intermediate amount To obtain fused high-frequency components ; Will integrate high-frequency components Input to the first depth transformation module Obtain intermediate quantity ; Will integrate low-frequency components Subtract the intermediate amount Obtain fused low-frequency components ; in, ; Fusion of high frequency components The initial fused high-frequency components ; Integrating low-frequency components The initial fused low-frequency component ; Fusion of high frequency components For the enhanced fusion high-frequency components; Integrating low-frequency components This refers to the enhanced fusion of low-frequency components.

5. The method for fusing infrared and visible light images according to claim 1, characterized in that, The wavelet transform expression in S1 is: ； in, Indicates wavelet transform operation; This indicates either an infrared image or a visible light image to be fused. , These represent tensor shape adjustment operations for dimension separation and dimension merging, respectively; This indicates a dimension swapping operation, used to swap two separated dimensions; C represents the number of input channels. This represents a grouped convolution with C groups.

6. The method for fusing infrared and visible light images according to claim 1, characterized in that, The inverse wavelet transform expression in S5 is: ； in, Indicates the inverse wavelet transform operation; Indicates a channel splicing operation; , These represent tensor shape adjustment operations for dimension separation and dimension merging, respectively; This indicates a dimension swapping operation, used to swap two separated dimensions; C represents the number of input channels. This represents a grouped deconvolution with C groups; This indicates the enhanced fusion of high-frequency components. This refers to the enhanced fusion of low-frequency components.

7. A method for fusing infrared and visible light images according to any one of claims 2-4, characterized in that, First Depth Transformation Module Second Depth Transformation Module and the third depth transformation module This refers to the bottleneck residual block in MobileNet V1; the bottleneck residual block is composed of a 1x1 convolutional layer, a 3x3 depth-separable convolutional layer, and a 1x1 convolutional layer connected in series.

8. A method for fusing infrared and visible light images according to any one of claims 2-4, characterized in that, Second Depth Transformation Module and the third depth transformation module These are bottleneck residuals with the same structure but different weights.

Images