Infrared and visible light image fusion method, device, system and storage medium

Abstract

The present invention discloses a method, device, system, and storage medium for fusion of infrared and visible light images, belonging to the field of computer vision technology. The method comprises: inputting infrared and visible light images into a pre-built encoder network for feature extraction to obtain features at various scales of the infrared and visible light images; inputting the features at various scales of the infrared and visible light images into a pre-built fusion network for feature fusion to obtain fused features at various scales; and inputting the fused features at various scales into a pre-built decoder network for reconstruction to obtain a fused image of the infrared and visible light images. The scales include the base layer, background layer, and detail layer of the infrared and visible light images. This method can perform deep decomposition of the infrared and visible light images, enhancing the quality of the fused image.

Description

Technical Field

[0001] The present invention relates to a method, device, system and storage medium for fusing infrared and visible light images, and belongs to the technical field of computer vision. Background Art

[0002] Infrared sensors detect infrared radiation generated by or reflected from a target to reveal its outline. Grayscale variations are caused by changes in the object's radiation. Their images can overcome some visual impairments, detecting hidden thermal targets in the background, are unaffected by lighting conditions and weather, and operate 24 / 7. However, these images suffer from blurred backgrounds, low resolution, low signal-to-noise ratio, low contrast, and limited ability to depict texture detail.

[0003] Visible light sensors capture a description of an object based on the intensity of the spectral information reflected from its surface, which is fed back to the camera's photoreceptor. These images offer high resolution, rich spectral information, and high spatial resolution, clearly capturing scene details under certain conditions. However, visible light sensor imaging is susceptible to natural conditions such as illumination and weather, severely impacting image quality at night or in low visibility.

[0004] Because infrared and visible light images are complementary to each other, fusion technology can exploit the advantageous features of each source image and fully integrate them. The resulting fused image contains detailed scene and target information, enabling accurate detection of hidden targets around the clock, even in adverse weather conditions.

[0005] Currently, image fusion algorithms based on multiscale decomposition are widely used in the field of image fusion, such as the Laplace pyramid transform, discrete wavelet transform, non-subsampled contourlet transform, and non-subsampled shearlet transform. The most challenging problem with existing multiscale transform algorithms is designing appropriate filters to fully decompose the image's background and detail features. However, most filters are manually designed by researchers, and their feature extraction performance relies on the ability to manually extract features. This makes it difficult to find a universal feature extraction method, and it is not possible to effectively extract and separate infrared image features. Consequently, it is impossible to analyze and design appropriate fusion rules based on the different characteristics of infrared and visible light images.

[0006] Convolutional Neural Networks (CNNs) offer significant advantages in feature extraction, providing more information than traditional manual feature extraction methods. The multi-scale transform extension combines multi-scale transforms with CNNs, leveraging the advantages of CNN feature extraction to effectively address the difficulty in extracting features found in traditional multi-scale transform methods. However, the design of fusion rules within CNNs has been overly simplistic, failing to highlight the distinct advantages of infrared and visible light images. This results in the fused image losing some detail and texture, making it unclear and even difficult to perceive directly. Summary of the Invention

[0007] The purpose of the present invention is to provide a method, device, system and storage medium for fusion of infrared and visible light images, which can perform deep decomposition of infrared images and visible light images and enhance the quality of fused images.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] In a first aspect, the present invention provides a method for fusing infrared and visible light images, comprising:

[0010] Input the infrared image and visible light image into the pre-built encoder network for feature extraction to obtain the features of each scale of the infrared image and visible light image;

[0011] Inputting the scale features of the infrared image and the visible light image into a pre-built fusion network for feature fusion to obtain fusion features of each scale;

[0012] The fused features at each scale are input into the pre-built decoder network for reconstruction to obtain a fused image of the infrared image and the visible light image;

[0013] The scale includes a base layer, a background layer, and a detail layer of infrared images and visible light images.

[0014] In combination with the first aspect, further, the encoder network includes several connected convolutional layers, each of the convolutional layers includes a connected 3×3 convolution function, a padding function, a batch normalization function, and an activation function, and the loss function of the encoder network is:

[0015]

[0016] Among them, L1 is the loss function of the encoder network, Φ is the hyperbolic tangent function, is the background layer feature of visible light image and infrared image, is the detail layer feature of the visible light image and infrared image, and α1 is the first weight coefficient.

[0017] In combination with the first aspect, further, the decoder network includes several connected convolutional layers, each of the convolutional layers includes a connected 3×3 convolution function, a padding function, a batch normalization function, and an activation function, and the loss function of the decoder network is:

[0018]

[0019] Where L2 is the loss function of the decoder network, α2, α3, α4 are the second, third, and fourth weight coefficients, and I is the infrared image input to the encoder. is the infrared prediction image output after encoder reconstruction, V is the visible light image input to the encoder, is the visible light prediction image output after encoder reconstruction, λ is the fifth weight coefficient, λ=50, is the gradient operator, and SSIM(·) is the structural similarity index function.

[0020] In combination with the first aspect, further, the fusion network fuses the scale features of the infrared image and the visible light image according to the fusion weights corresponding to the scale features of the infrared image obtained by calculation to obtain fusion features of each scale.

[0021] In combination with the first aspect, further, the calculation formula of the fusion weight is:

[0022]

[0023] Among them, W0 is the fusion weight corresponding to the detail layer feature of the infrared image, W1 is the fusion weight corresponding to the background layer feature of the infrared image, W2 is the fusion weight corresponding to the base layer feature of the infrared image, g σ1 、g σ2 are the first and second Gaussian filter operators, σ1 and σ2 are the first and second standard deviations of the Gaussian function, * is the convolution symbol, and C is the saliency of the detail layer features of the infrared image. S τ (·) is the nonlinear transformation function from [0,1] to [0,1], τ is S τ (·) is the speed from 0 to 1, P is the normalization function, R is the detail layer distance between the infrared image and the visible light image, is the detail layer feature of the infrared image, It is the detail layer feature of the visible light image.

[0024] In combination with the first aspect, further, inputting the fusion features of each scale into the pre-built decoder network for reconstruction, and obtaining a fusion image of the infrared image and the visible light image includes:

[0025] The background layer fusion features and detail layer fusion features are input into the pre-built decoder network for reconstruction to obtain the fusion image of the infrared image and the visible light image;

[0026] Adding the base layer fusion features to the input of the convolutional layer of the pre-built decoder network to preserve the pixel intensity and gradient information of the infrared image and the visible light image and accelerate the convergence speed of the decoder network;

[0027] When the base layer fusion feature is added to the input of the convolutional layer of the pre-built decoder network, the base layer fusion feature is cascaded with the output of the corresponding convolutional layer.

[0028] In a second aspect, the present invention provides an infrared and visible light image fusion device, comprising:

[0029] Feature extraction module: used to input infrared images and visible light images into the pre-built encoder network for feature extraction, and obtain the various scale features of infrared images and visible light images;

[0030] Feature fusion module: used to input the scale features of the infrared image and the visible light image into a pre-built fusion network for feature fusion to obtain fusion features of each scale;

[0031] Image fusion module: used to input the fusion features of each scale into the pre-built decoder network for reconstruction to obtain the fusion image of infrared image and visible light image;

[0032] The scale includes a base layer, a background layer, and a detail layer of infrared images and visible light images.

[0033] In a third aspect, the present invention provides an infrared and visible light image fusion system, comprising a processor and a storage medium;

[0034] The storage medium is used to store instructions;

[0035] The processor is configured to operate according to the instructions to execute the steps of the method according to any one of the first aspects.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The infrared and visible light image fusion method provided by the present invention combines a multi-scale decomposition method with an autoencoder network, and utilizes the advantages of feature extraction of the autoencoder network. Compared with the traditional multi-scale decomposition method of manual feature extraction, it can obtain more information and perform in-depth decomposition of infrared images and visible light images. At the same time, the infrared and visible light image fusion method provided by the present invention can adaptively calculate the fusion weights corresponding to the various scale features of the infrared image, and highlight the dominant features of the infrared image and the visible light image in the fused image. The infrared and visible light image fusion method provided by the present invention can not only fully preserve the thermal radiation information in the infrared image, but also effectively reproduce the texture details in the visible light image, with better visual effects and evaluation indicators, and enhance the quality of the fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a diagram of the infrared and visible light image fusion framework provided by an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of the encoder network and decoder network structure for inputting infrared images for training according to an embodiment of the present invention;

[0041] Figure 3 Schematic diagram of the encoder network and decoder network structure for training with input visible light images provided by an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of the structure and parameters of each convolutional layer provided by an embodiment of the present invention;

[0043] Figure 5 Schematic diagram of the network structure for the infrared and visible light image fusion application provided by an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of a specific infrared and visible light image fusion application process provided by an embodiment of the present invention;

[0045] Figure 7 Schematic diagram of the results of random testing on the TNO image fusion datasets provided by an embodiment of the present invention;

[0046] Figure 8 1 is a schematic diagram of the results of a random test application on the NIR Scene Datasets dataset provided by an embodiment of the present invention;

[0047] Figure 9 3 is a schematic diagram of the results of random testing on the FLIR image fusion datasets provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The technical solution of the present application will be further described in detail below in conjunction with specific implementation methods.

[0049] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. The embodiments of the present application and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0050] Example 1:

[0051] Figure 1 This is a flow chart of a method for fusion of infrared and visible light images provided in this embodiment. This flow chart only shows the logical sequence of the method in this embodiment. Different methods can be used without conflict. Figure 1 The steps shown or described are accomplished in the order shown.

[0052] The infrared and visible light image fusion method provided in this embodiment can be applied to a terminal and can be executed by an infrared and visible light image fusion device. The device can be implemented by software and / or hardware and can be integrated into a terminal, such as any tablet computer or computer device with communication function. Figure 1 The method of this embodiment specifically includes the following steps:

[0053] Step 1: Input the infrared image and visible light image into the pre-built encoder network for feature extraction to obtain the features of each scale of the infrared image and visible light image;

[0054] In this embodiment, the scale includes a base layer, a background layer, and a detail layer of the infrared image and the visible light image.

[0055] The encoder network consists of several connected convolutional layers. Each convolutional layer includes a connected 3×3 convolution function, a padding function, a batch normalization function, and an activation function. The loss function of the encoder network is:

[0056]

[0057] Among them, L1 is the loss function of the encoder network, Φ is the hyperbolic tangent function, is the background layer feature of visible light image and infrared image, is the detail layer feature of the visible light image and infrared image, and α1 is the first weight coefficient.

[0058] In this embodiment, the encoder network's loss function reduces the distance between the background layers of the infrared and visible light images and increases the distance between the detail layers of the infrared and visible light images. Background layer features are common to both infrared and visible light images, while detail layer features are different between them. Therefore, the encoder network's loss function enables the encoder network to obtain better background and detail layer features.

[0059] In this embodiment, the encoder network includes five convolutional layers: CONV1, CONV2, CONV3, CONV4, and CONV5. The infrared image and the visible light image are input to the encoder network, and the base layer features of the infrared image are obtained through CONV2. and the base layer features of visible light images Get the base layer features of the infrared image through CONV3 and the base layer features of visible light images Obtain the background layer features of the infrared image through CONV4 and background layer features of visible light images Obtain the detail layer features of the infrared image through CONV5 and detail layer features of visible light images

[0060] Step 2: Input the scale features of the infrared image and the visible light image into the pre-built fusion network for feature fusion to obtain the fusion features of each scale;

[0061] In this embodiment, the fusion network fuses the scale features of the infrared image and the visible light image based on the fusion weights corresponding to the scale features of the infrared image obtained by calculation to obtain fusion features of each scale.

[0062] The calculation formula of fusion weight is:

[0063]

[0064] Among them, W0 is the fusion weight corresponding to the detail layer feature of the infrared image, W1 is the fusion weight corresponding to the background layer feature of the infrared image, and W2 is the fusion weight corresponding to the base layer feature of the infrared image. are the first and second Gaussian filter operators, σ1 and σ2 are the first and second standard deviations of the Gaussian function, σ2 = 2 × σ1, * is the convolution symbol, and C is the saliency of the detail layer features of the infrared image. S τ (·) is the nonlinear transformation function from [0,1] to [0,1], τ is S τ (·) is the speed from 0 to 1, P is the normalization function, R is the detail layer distance between the infrared image and the visible light image, is the detail layer feature of the infrared image, It is the detail layer feature of the visible light image.

[0065] In this embodiment, the scale features of the infrared image and the visible light image are fused according to the fusion weights corresponding to the scale features of the infrared image obtained by calculation. The scale fusion features obtained include: base layer fusion features and Background layer fusion features and detail layer fusion features

[0066] in,

[0067] Step 3: Input the fusion features of each scale into the pre-built decoder network for reconstruction to obtain the fusion image of the infrared image and the visible light image.

[0068] In this embodiment, the decoder network includes several connected convolutional layers, each of which includes a connected 3×3 convolution function, a padding function, a batch normalization function, and an activation function. The loss function of the decoder network is:

[0069]

[0070] Where L2 is the loss function of the decoder network, α2, α3, α4 are the second, third, and fourth weight coefficients, and I is the infrared image input to the encoder. is the infrared prediction image output after encoder reconstruction, V is the visible light image input to the encoder, is the visible light prediction image output after encoder reconstruction, λ is the fifth weight coefficient, is the gradient operator, and SSIM(·) is the structural similarity index function.

[0071] In this embodiment, the decoder network ensures through its loss function that the fused image obtained after reconstruction maintains the pixel intensity, structural similarity and gradient information of the infrared image and visible light image before reconstruction, and ensures the texture consistency of the fused image obtained after reconstruction and the infrared image and visible light image before reconstruction.

[0072] In this embodiment, the decoder network includes four convolutional layers: CONV6, CONV7, CONV8 and CONV9.

[0073] Inputting the fusion features of each scale into the pre-built decoder network for reconstruction, obtaining the fusion image of the infrared image and the visible light image includes the following steps:

[0074] Step 1: Input the background layer fusion features and detail layer fusion features into the pre-built decoder network for reconstruction to obtain the fusion image of the infrared image and the visible light image;

[0075] Step 2: Add the base layer fusion features to the input of the convolutional layer of the pre-built decoder network to preserve the pixel intensity and gradient information of the infrared image and visible light image, and accelerate the convergence of the decoder network.

[0076] In this embodiment, when the base layer fusion features are added to the input of the convolutional layer of the pre-built decoder network, the base layer fusion features are cascaded with the output of the corresponding convolutional layer. Add to the input of CONV7 and Cascaded with the output of CONV7, Add to the input of CONV8 and Cascaded with the output of CONV8.

[0077] This embodiment also provides a training part of the infrared and visible light image fusion method, which includes the following steps:

[0078] S1: Name the pre-acquired infrared image and visible light image in the format of IR1.bmp and VIS1.bmp, representing the infrared image and visible light image respectively, as the training data set;

[0079] S2: According to Figure 2 and Figure 3 The network structure shown, and Figure 4 The structure and parameters of each convolutional layer shown in the figure are used to construct the network, and the loss function L of the overall network is obtained. total =L1+L2, α1=0.5, α2=α3=α4=10. In the training part, since the fusion network is not composed of convolutional neural networks, it does not participate in the training and the fusion network is closed;

[0080] S3: Follow Figure 2 and Figure 3 The network structure shown in the figure inputs infrared images and visible light images into the encoder network respectively to start training;

[0081] S4: After the encoder network and decoder network training is completed, the parameters of the entire network are saved to a file.

[0082] In this embodiment, after the encoder network and the decoder network are trained, the file that stores the parameters of the entire network is read, and the entire network is set to evaluation mode. The parameters of the entire network are fixed, and the fusion network is opened. Figure 5As shown. The infrared image and visible light image to be fused are input into the encoder network to obtain the scale features of the infrared image and visible light image, as shown Figure 6 As shown in Figure 2, the features at each scale of the infrared image and visible light image are input into the fusion network, the fusion weights are calculated, and the features at each scale are fused to obtain the fused features at each scale. The fused features at each scale are simultaneously input into the decoder network for reconstruction to obtain the fused image of the infrared image and visible light image.

[0083] like Figure 7 、 Figure 8 、 Figure 9 As shown in FIG, a schematic diagram of the results of random testing of the infrared and visible light image fusion method provided in this embodiment on three public datasets, namely, TNOimage fusion datasets, NIR Scene Datasets, and FLIR image fusion datasets, is shown. All of them can fuse infrared images and visible light images well, and can highlight the thermal radiation information of the infrared image and retain the texture detail information and gradient information of the visible light image in the fused image.

[0084] The infrared and visible light image fusion method provided in this embodiment combines the multi-scale decomposition method with the autoencoder network, and utilizes the advantages of the autoencoder network feature extraction. Compared with the traditional multi-scale decomposition method of manual feature extraction, it can obtain more information and perform in-depth decomposition of infrared images and visible light images. At the same time, the infrared and visible light image fusion method provided by the present invention can adaptively calculate the fusion weights corresponding to the various scale features of the infrared image, and highlight the advantageous features of the infrared image and the visible light image in the fused image. The infrared and visible light image fusion method provided in this embodiment can not only fully retain the thermal radiation information in the infrared image, but also effectively reproduce the texture details in the visible light image, thereby enhancing the quality of the fused image. Compared with the existing infrared and visible light image fusion method, it has better visual effects and evaluation indicators.

[0085] Example 2:

[0086] This embodiment provides an infrared and visible light image fusion device, including:

[0087] Feature extraction module: used to input infrared images and visible light images into the pre-built encoder network for feature extraction, and obtain the various scale features of infrared images and visible light images;

[0088] Feature fusion module: used to input the scale features of infrared images and visible light images into the pre-built fusion network for feature fusion to obtain fusion features of each scale;

[0089] Image fusion module: used to input the fusion features of each scale into the pre-built decoder network for reconstruction to obtain the fusion image of infrared image and visible light image;

[0090] The scales include the base layer, background layer, and detail layer of infrared images and visible light images.

[0091] The infrared and visible light image fusion device provided in the embodiment of the present application can execute the infrared and visible light image fusion method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0092] Example 3:

[0093] This embodiment provides a system, including a processor and a storage medium;

[0094] The storage medium is used to store instructions;

[0095] The processor is configured to operate according to the instructions to execute the steps of the method in Example 1.

[0096] Example 4:

[0097] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method in Embodiment 1 are implemented.

[0098] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0102] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for fusing infrared and visible light images, characterized in that: include: Input the infrared image and visible light image into the pre-built encoder network for feature extraction to obtain the features of each scale of the infrared image and visible light image; Inputting the scale features of the infrared image and the visible light image into a pre-built fusion network for feature fusion to obtain fusion features of each scale; The fused features at each scale are input into the pre-built decoder network for reconstruction to obtain a fused image of the infrared image and the visible light image; Wherein, the scale includes a base layer, a background layer, and a detail layer of the infrared image and the visible light image; Input the fusion features of each scale into the pre-built decoder network for reconstruction, and obtain the fusion image of the infrared image and the visible light image including: The background layer fusion features and detail layer fusion features are input into the pre-built decoder network for reconstruction to obtain the fusion image of the infrared image and the visible light image; Adding the base layer fusion features to the input of the convolutional layer of the pre-built decoder network to preserve the pixel intensity and gradient information of the infrared image and the visible light image and accelerate the convergence speed of the decoder network; When the base layer fusion feature is added to the input of the convolutional layer of the pre-built decoder network, the base layer fusion feature is cascaded with the output of the corresponding convolutional layer.

2. The infrared and visible light image fusion method according to claim 1, characterized in that: The encoder network includes several connected convolutional layers, each of which includes a connected 3×3 convolution function, a padding function, a batch normalization function, and an activation function. The loss function of the encoder network is: ； in, is the loss function of the encoder network, is the hyperbolic tangent function, 、 is the background layer feature of visible light image and infrared image, 、 is the detail layer feature of visible light image and infrared image, is the first weight coefficient.

3. The infrared and visible light image fusion method according to claim 1, characterized in that: The decoder network includes several connected convolutional layers, each of which includes a connected 3×3 convolution function, a padding function, a batch normalization function, and an activation function. The loss function of the decoder network is: ； in, is the loss function of the decoder network, 、 、 are the second, third and fourth weight coefficients, is the infrared image input to the encoder, is the infrared prediction image output after reconstruction by the encoder, is the visible light image input to the encoder, is the visible light prediction image output after encoder reconstruction, is the fifth weight coefficient, is the gradient operator, is the structural similarity index function.

4. The infrared and visible light image fusion method according to claim 1, characterized in that: The fusion network fuses the scale features of the infrared image and the visible light image according to the fusion weights corresponding to the scale features of the infrared image obtained by calculation, so as to obtain fusion features of each scale.

5. The infrared and visible light image fusion method according to claim 4, characterized in that: The calculation formula of the fusion weight is: ； in, is the fusion weight corresponding to the detail layer features of the infrared image, is the fusion weight corresponding to the background layer features of the infrared image, is the fusion weight corresponding to the base layer feature of the infrared image, 、 are the first and second Gaussian filter operators, 、 is the first and second standard deviation of the Gaussian function, is the convolution symbol, is the saliency strength of the detail layer features of the infrared image, , For Map to Nonlinear transformation processing function, for The speed from 0 to 1, is the normalization function, , is the detail layer distance between the infrared image and the visible light image, , is the detail layer feature of the infrared image, It is the detail layer feature of the visible light image.

6. An infrared and visible light image fusion device, characterized in that: include: Feature extraction module: used to input infrared images and visible light images into the pre-built encoder network for feature extraction, and obtain the various scale features of infrared images and visible light images; Feature fusion module: used to input the scale features of the infrared image and the visible light image into a pre-built fusion network for feature fusion to obtain fusion features of each scale; Image fusion module: used to input the fusion features of each scale into the pre-built decoder network for reconstruction to obtain the fusion image of infrared image and visible light image; Wherein, the scale includes a base layer, a background layer, and a detail layer of the infrared image and the visible light image; Input the fusion features of each scale into the pre-built decoder network for reconstruction, and obtain the fusion image of the infrared image and the visible light image including: The background layer fusion features and detail layer fusion features are input into the pre-built decoder network for reconstruction to obtain the fusion image of the infrared image and the visible light image; Adding the base layer fusion features to the input of the convolutional layer of the pre-built decoder network to preserve the pixel intensity and gradient information of the infrared image and the visible light image and accelerate the convergence speed of the decoder network; When the base layer fusion feature is added to the input of the convolutional layer of the pre-built decoder network, the base layer fusion feature is cascaded with the output of the corresponding convolutional layer.

7. An infrared and visible light image fusion system, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images