Infrared and visible image fusion method and system based on rank-one decomposition network

Abstract

The application provides an infrared and visible light image fusion method and system based on a rank-one decomposition network, and the method comprises the following steps: acquiring an infrared image and a visible light image; stacking the infrared image and the visible light image in a channel dimension; performing step-by-step rank-one decomposition processing on the stacked image sequence to obtain a multi-stage rank-one component, a residual error of the infrared image and a residual error of the visible light image; respectively fusing the multi-stage rank-one component and the residual errors to obtain a fusion component C containing all rank-one component information f and a fusion component E containing two residual error information f ; adding the fusion components C f and E f to obtain a desired fusion image I f . The infrared image and the visible light image are placed in the rank-one decomposition network for step-by-step processing, and the rank-one component and the residual error are extracted, so that the rank-one component can be enhanced and the residual error can be smoothed, which is beneficial to the application of the image processing technology in the later stage and improves the utilization rate of the image self-similarity characteristics.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to a method and system for fusion of infrared and visible light images based on rank-one decomposition networks. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In this era of information overload, sensor technology is rapidly evolving. The emergence of complex applications demands comprehensive information about specific scenes to enhance understanding of various conditions. Since sensors of the same type only acquire information from one aspect and cannot provide all the necessary details, image fusion technology plays an increasingly important role in modern applications and computer vision. Infrared and visible light image fusion is a significant branch of multi-source image fusion. Infrared and visible light images derive from different modalities, providing scene information from different perspectives. Based on the principle of reflected light imaging, visible light images contain rich texture information; based on the principle of thermal radiation imaging, infrared images contain prominent targets. Furthermore, infrared and visible images possess almost all the inherent characteristics of objects. Ultimately, infrared and visible images have complementary properties, resulting in robust and information-rich fused images. Visible images typically have high spatial resolution and considerable detail and contrast. Therefore, they are well-suited for human visual perception. However, these images are easily affected by adverse conditions such as poor lighting, fog, and other effects of inclement weather. Meanwhile, infrared images, which depict the thermal radiation of objects, are resistant to these interferences but generally have lower resolution and poorer texture.

[0004] Infrared and visible light image fusion aims to generate a fused image containing salient targets and rich background textures by combining infrared and visible light images. Infrared and visible light image fusion methods can be broadly categorized into four types: multi-scale transformation-based methods, sparse representation-based methods, subspace clustering-based methods, and deep learning-based methods. 1) Transformation-based image fusion methods typically project the source image into the transform domain using appropriate transformation rules, representing the source image with different coefficients in the transform domain. Then, a fusion strategy based on activity level measurement is used to merge the coefficients, and finally, an inverse transformation is performed on the merged coefficients to obtain the fused image. 2) Sparse representation-based methods fully utilize the sparse representation of the image, performing sparse processing on the source image. First, the source image is localized, i.e., divided into blocks according to a certain block structure. The sparse coefficients are solved using a sparse decomposition algorithm under an overcomplete dictionary, and the fused image is obtained using a small number of non-zero elements of the image. 3) Spatial domain-based methods directly manipulate the pixel values ​​of the image. The simplest fusion strategies are the maximum value method and the weighted average method, which are computationally simple and fast. 5) With the development of deep learning in recent years, numerous methods for fusing infrared and visible light images using various convolutional neural networks have emerged and demonstrated excellent performance. Convolutional neural networks are data-driven and can effectively learn various image features through training data to reconstruct images, such as SDNet, IFCNN, and U2Fusion. However, existing methods still suffer from insufficient mining of background texture information, resulting in unsatisfactory fusion effects. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the present invention provides an infrared and visible light image fusion method and system based on rank-one decomposition network to simultaneously acquire salient target and rich background texture information.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0007] The first aspect of this invention provides a method for fusion of infrared and visible light images based on a rank-one decomposition network, comprising:

[0008] Acquire infrared and visible light images;

[0009] The infrared and visible light images are stacked in the channel dimension;

[0010] The stacked image sequence is subjected to a step-by-step rank-one decomposition to obtain multi-level rank-one components, the residual of the infrared image, and the residual of the visible light image.

[0011] The multi-level rank-one components, the residuals of the infrared image, and the residuals of the visible light image are fused separately to obtain a fused component C containing information from all rank-one components. f And the fusion component E containing two residual information f ;

[0012] The fusion component C f and E f The desired fused image I is obtained by adding elements together. f .

[0013] A second aspect of the present invention provides an infrared and visible light image fusion system based on a rank-one decomposition network, comprising: an image acquisition module configured to acquire infrared images and visible light images;

[0014] The image stacking module is configured to stack the infrared image and the visible light image in the channel dimension.

[0015] The rank-one decomposition module is configured to perform rank-one decomposition on the stacked image sequence step by step to obtain multi-level rank-one components, the residual of the infrared image, and the residual of the visible light image.

[0016] The fusion component generation module is configured to fuse the multi-level rank-one components, the residuals of the infrared image, and the residuals of the visible light image, respectively, to obtain a fusion component C containing information from all rank-one components. f And the fusion component E containing two residual information f ;

[0017] The fused image generation module is configured to: merge the fused components C f and E f The desired fused image I is obtained by adding elements together. f .

[0018] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps of an infrared and visible light image fusion method based on a rank-one decomposition network as described in the first aspect of the present invention.

[0019] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of an infrared and visible light image fusion method based on a rank-one decomposition network as described in the first aspect of the present invention.

[0020] The above one or more technical solutions have the following beneficial effects:

[0021] (1) The present invention places infrared images and visible light images into a rank-1 decomposition network for stepwise processing, extracting rank-1 components and residuals. This allows the present invention to enhance the rank-1 components and smooth the residuals respectively, which is beneficial to the application of subsequent image processing techniques and improves the utilization rate of image self-similarity features.

[0022] (2) In order to better fuse the residuals, the residuals are input into the residual fusion module. First, the attention map is obtained through the spatial attention module. The attention map can better fuse the salient targets in the infrared image and the background texture information in the visible light image, so that the fused image contains more detailed information.

[0023] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0025] Figure 1 The flowchart shows the infrared and visible light image fusion method based on rank-one decomposition network in the first embodiment.

[0026] Figure 2 This is a structural diagram of the rank-one decomposition module in the first embodiment.

[0027] Figure 3 This is a structural diagram of the rank-one component fusion module in the first embodiment.

[0028] Figure 4 (a) is a structural diagram of the spatial attention module in the first embodiment.

[0029] Figure 4 (b) is a structural diagram of the residual fusion module in the first embodiment.

[0030] Figure 5 (a) Figure 5 (b) Figure 5 (c) are the infrared image, the visible light image, and the fused image of the first embodiment, respectively.

[0031] In the diagram, Apool(R) represents average pooling in the rowdirection;

[0032] Apool(C) represents average pooling along the column direction;

[0033] ⊙ represents matrix multiplication;

[0034] This represents the element-wise multiplication of matrices.

[0035] Indicates concatenation;

[0036] ⊕ represents matrix addition;

[0037] This represents matrix subtraction. Detailed Implementation

[0038] Example 1

[0039] like Figure 1 As shown, this embodiment discloses an infrared and visible light image fusion method based on a rank-one decomposition network, including:

[0040] Step 1: Input Image

[0041] The dataset of this invention uses the TNO dataset containing multispectral nighttime images of different scenes, cropped to the corresponding size as needed. Infrared images IRI1∈R 120×120×1 and visible light image VISII2∈R 120×120×1 Then, the two are stacked along the channel dimension to obtain Z1∈R. 120×120×2 .

[0042] Step 2, Feature Extraction

[0043] VISI contains rich texture details but is easily affected by lighting conditions. IRI can preserve thermal radiation information under different conditions but cannot present visual details. In order to make full use of the information contained in IRI and VISI, this invention obtains a rank-one matrix through rank-one decomposition, and represents the image by the sum of multiple rank-one matrices and the residuals.

[0044] Specifically, Z1∈R is obtained by stacking IRI and VISI. 120×120×2 The rank-one decomposition module is used to extract structural self-similarity information, resulting in the rank-one component C1∈R. 120×120×1 Subtracting the rank-one component C1 from both the infrared image (IRI) and the visible light image (VISI) yields A1∈R. 120×120×1 B1∈R 120×120×1 Then, A1 and B1 are stacked along the channel dimension using concatenation to obtain Z2∈R.120×120×2 The obtained Z2 is input into the second rank-one decomposition module to obtain the rank-one component C2∈R. 120 ×120×1 ...and so on, obtaining R rank-one components C1, C2, C3, ... C through R rank-one decomposition modules. R ∈R 120×120×1 ;

[0045] Among them, such as Figure 2 As shown, the specific steps to obtain the rank-one matrix components through rank-one decomposition include:

[0046] First, IRI and VISI are stacked in the channel dimension. IRI and VISI each have 1 channel. After stacking, the number of channels increases to 2. The stacked image sequence is then fed into two convolutional layers with 3×3 kernels, stride of 1, 32 output channels, and ReLU activation function. The output features of the two convolutional layers are summed. The summation result is then stacked with the output features of the first convolutional layer in the channel dimension. The stacked image sequence is then fused through a convolutional layer with 1×1 kernel, stride of 1, and 64 output channels.

[0047] Next, the features are processed through two branches to obtain row and column vectors respectively. In each branch, the input features are multiplied element-wise with the output features obtained by passing the input features through a 3×3 convolutional layer with a stride of 1 and a sigmoid activation function. Then, the output features are passed through two 3×3 convolutional layers with a stride of 1, the first with 64 output channels and the second with 1 output channel, thus each branch obtains an intermediate feature. Finally, the first branch uses an average pooling layer to average the rows of the intermediate features to obtain the column vector U. r The second branch uses an average pooling layer to average the columns of the intermediate features to obtain the row vector V. r T Finally, multiplying the resulting column vectors by the row vectors yields a matrix U with rank one. r V r T .

[0048] Let R rank-1 components C1, C2, C3, ..., C R Summing yields C sum ∈R 120×120×1 The sum of C, which is the infrared image IRI minus the rank-one component. sum The residuals E1∈R of IRI are obtained. 120×120×1 The sum of the visible light image VISI minus the rank component C sum The residual E2∈R of VISI is obtained. 120×120×1 .

[0049] Step 3: Feature Fusion

[0050] Step 301: Extract the R rank-one components C1, C2, C3, ... C R ∈R 120×120×1 Simultaneously, the input is fed into the rank-one component fusion module for feature fusion, resulting in C containing information from all rank-one components. f ∈R 120×120×1 ;

[0051] like Figure 3 As shown, this invention fuses all rank-one components obtained from the rank-one decomposition module using a rank-one component fusion module. First, all rank-one components are stacked along the channel dimension. Then, they are sequentially passed through three convolutional layers with 3×3 kernels, a stride of 1, 32 output channels, and a Leaky ReLU activation function. The outputs of the first and second convolutional layers are added together as the input to the third convolutional layer. After the outputs of the second and third convolutional layers are added together, they are sequentially passed through a convolutional layer with 3×3 kernels, a stride of 1, and 16 output channels, a Leaky ReLU activation function, and a convolutional layer with 3×3 kernels, a stride of 1, and 1 output channel. Finally, the fused feature C is... f Output.

[0052] Step 302: Input the two residual blocks E1 and E2 into the residual fusion module, obtain the spatial attention map through the spatial attention module, and fuse E1 and E2 using the attention map to obtain E containing the information of the two residual blocks. f ∈R 120 ×120×1 ;

[0053] like Figure 4 As shown in (a), this invention obtains the spatial attention map of the residuals of two source images through spatial attention blocks. First, the two residuals are stacked together in the channel dimension, increasing the number of channels from 1 to 2. Then, they are passed through two convolutional layers with 3×3 kernels, a stride of 1, and 32 output channels. Finally, the spatial attention map of the residuals is obtained through the Sigmoid activation function.

[0054] like Figure 4As shown in (b), to simultaneously obtain information from the two residuals, we fuse the residuals using a residual fusion network. Two residuals are input into a spatial attention block to obtain a spatial attention map. Each residual block is passed through a convolutional layer with a 3×3 kernel, a stride of 1, and 32 output channels. The output features are then multiplied element-wise with the spatial attention map. The multiplied features are stacked along the channel dimension and then sequentially passed through a convolutional layer with a 3×3 kernel, a stride of 1, and 32 output channels, a Leaky ReLU activation function, another convolutional layer with a 3×3 kernel, a stride of 1, and 32 output channels, and a third convolutional layer with a 3×3 kernel, a stride of 1, and 1 output channel, resulting in a fused residual containing information from both residual images.

[0055] Step 4: Image Reconstruction

[0056] The obtained fusion feature C f ∈R 120×120×1 and E f ∈R 120×120×1 Adding them together yields the fused image;

[0057] In this embodiment, the image fusion training model based on the rank-one decomposition network uses L1Loss and MSELoss loss functions.

[0058] The reconstruction loss consists of L1 and L2:

[0059] L reconstitution =L1+L2

[0060] L1 and L2 use MSE loss:

[0061]

[0062]

[0063] The texture loss function uses L1Loss:

[0064]

[0065] The gradient loss function uses L1Loss:

[0066]

[0067] The fusion loss is composed of the texture loss L texture and gradient loss L gradient constitute:

[0068] L f =L texture +1.2*L gradient

[0069] The overall loss function consists of reconstruction loss and fusion loss:

[0070] L total =L reconstitution +L f

[0071] Where I1∈R 120×120×1 The input infrared image is IRI, I2∈R 120×120×1 The input visible light images are VISI, C1, C2, C3, ... C R ∈R 120×120×1 E1 and E2 are the rank-one components obtained from the rank-one decomposition module, respectively, and the residuals obtained by subtracting the rank-one components from IRI and VISI. f ∈R 120×120×1 The image is a fused image obtained from IRI and VISI using an infrared and visible light image fusion network based on rank-one decomposition. This indicates that the gradient with respect to x is calculated.

[0072] The effects of this invention can be further illustrated by the following simulation.

[0073] 1. Simulation environment:

[0074] PyCharm Community Edition 2022.1.2 x64, NVIDIA GEFORCE RTX3090, Ubuntu18.04.

[0075] 2. Simulation content:

[0076] Simulation 1: The dataset used in this invention is the TNO dataset, which contains multispectral nighttime images of different scenes. Infrared and visible light images are fused together, and the results are as follows: Figure 5 As shown, where:

[0077] Figure 5 (a) is a visible light image, with a size of 620×450×1.

[0078] Figure 5 (b) is an infrared image, with a size of 620×450×1.

[0079] Figure 5 (c) For the purpose of using the present invention Figure 5 (a) and Figure 5 (b) The fused image obtained after fusion, with a size of 620×450×1.

[0080] The comparison shows that, Figure 5 (c) thermal radiation information compared to Figure 5(a) There is a significant improvement; the edges and textures of prominent targets are clearer, compared to Figure 5 (b) In comparison, Figure 5 (c) has richer background texture information, thus demonstrating that the present invention can... Figure 5 (a) and Figure 5 (b) Achieve better integration.

[0081] Simulation 2: To demonstrate the effectiveness of the present invention, the method of the present invention and other existing deep neural network methods, DIFNet and SDNet, were compared. Figure 5 (a) and Figure 5 (b) The images are fused, and the fusion results are evaluated using objective metrics, as follows:

[0082] 1) Peak Signal-to-Noise Ratio (PSNR) can be used to evaluate the spatial quality of the fused reconstruction in bands. The minimum PSNR value is 0. The larger the PSNR, the smaller the difference between the two images, indicating that the fused result is more similar to the reference image.

[0083] 2) Mean Square Error (MSE) reflects the degree of difference between variables. It is an objective evaluation index of image quality based on pixel error. It is used to measure the difference between the fused image and the ideal reference image. The smaller the MSE, the better the quality of the fused image.

[0084] 3) Mutual Information (MI) measures the similarity between two images, i.e., how much information the fused image retains from the original images. Higher MI indicates that the fused image retains more information from the source images, resulting in better quality.

[0085] 4) Fusion performance based on noise assessment N abf This represents the ratio of noise added manually to the fused image during the fusion process; a higher value is better.

[0086] 5) Discrete Cosine Feature Mutual Information (FMI) dct The mutual information of discrete cosine and wavelet features is calculated; the larger the value, the more mutual information there is.

[0087] 6) Wavelet Feature Mutual Information (FMI) w Calculate the mutual information between discrete cosine and wavelet features; the larger the value, the better.

[0088] The integration results of the present invention and the prior art were evaluated from an objective evaluation perspective based on the above evaluation indicators, and the results are shown in Table 1.

[0089] Table 1 Objective evaluation of the fusion results of various methods

[0090]

[0091]

[0092] As shown in Table 1, the peak signal-to-noise ratio (PSNR), mutual information (MI), and discrete cosine characteristic mutual information (FMI) of this invention are... dct Wavelet Feature Mutual Information (FMI) w Both are greater than the evaluation values ​​of existing technologies, including mean squared error (MSE) and fusion performance N based on noise assessment. abf All of these values ​​are lower than the evaluation values ​​of the prior art, and all of these evaluation values ​​are better than the evaluation values ​​of the prior art. It can be seen from this that most of the objective evaluation indicators of the present invention are better than the objective evaluation indicators of the prior art.

[0093] Example 2

[0094] This embodiment discloses an infrared and visible light image fusion system based on a rank-one decomposition network, including: an image acquisition module configured to acquire infrared images and visible light images;

[0095] The image stacking module is configured to stack the infrared image and the visible light image in the channel dimension.

[0096] The rank-one decomposition module is configured to perform rank-one decomposition on the stacked image sequence step by step to obtain multi-level rank-one components, the residual of the infrared image, and the residual of the visible light image.

[0097] The fusion component generation module is configured to fuse the multi-level rank-one components, the residuals of the infrared image, and the residuals of the visible light image, respectively, to obtain a fusion component C containing information from all rank-one components. f And the fusion component E containing two residual information f ;

[0098] The fused image generation module is configured to: combine the fused components c f and E f The desired fused image I is obtained by adding elements together. f .

[0099] Example 3

[0100] The purpose of this embodiment is to provide a computer-readable storage medium.

[0101] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of an infrared and visible light image fusion method based on a rank-one decomposition network as described in Embodiment 1 of this disclosure.

[0102] Example 4

[0103] The purpose of this embodiment is to provide an electronic device.

[0104] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in an infrared and visible light image fusion method based on a rank-one decomposition network as described in Embodiment 1 of this disclosure.

[0105] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0106] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0107] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for fusing infrared and visible light images based on a rank-one decomposition network, characterized in that, include: Acquire infrared and visible light images; The infrared and visible light images are stacked in the channel dimension; The stacked image sequence is subjected to a step-by-step rank-one decomposition to obtain multi-level rank-one components, the residual of the infrared image, and the residual of the visible light image. The step of performing a step-by-step rank-one decomposition on the stacked image sequence to obtain multi-level rank-one components includes: decomposing the stacked image sequence... The input is fed into the first-level rank-one decomposition module to obtain the rank-one components. Subtract the rank-one component from the infrared image get Subtract the rank-one component from the visible light image get ; Will and Image sequences are obtained by stacking along the channel dimension. Image sequence The input is fed into the second-level rank-one decomposition module to obtain the rank-one components. Infrared image minus rank component get Visible light image minus rank component get ; The result and Image sequences are stacked along the channel dimension to obtain Repeat the iteration R times to obtain R rank-1 components. ; The multi-level rank-one components, the residuals of the infrared image, and the residuals of the visible light image are fused separately to obtain a fused component containing information from all rank-one components. and the fusion component containing two residual information The method for calculating the residuals of the infrared image and the visible light image includes: summing the acquired R rank-one components to obtain... ; Subtract the sum of R rank-one components from the infrared image. Obtain the residual of the infrared image ; Subtract the sum of R rank-one components from the visible light image. Obtain the residual of the visible light image ; The residuals of the infrared image and the visible light image are fused to obtain a fused component containing information from both residuals. The specific steps include: Two residuals and The input is fed into the residual fusion module, where a spatial attention map is obtained through the spatial attention module. The attention map is then compared with... and The components are fused to obtain a fused component containing information from both residuals. ; fusion components and The desired fused image is obtained by adding elements together. .

2. The infrared and visible light image fusion method based on rank-one decomposition network as described in claim 1, characterized in that, The specific steps for inputting the stacked image sequence into the rank-one decomposition module to obtain the rank-one component include: inputting the stacked image sequence into two convolutional layers in sequence; The outputs of the two convolutional layers are summed, and the summation result is stacked with the output of the first convolutional layer in the channel dimension. The stacked image sequence is then fused with features through a convolutional layer. The image sequence after feature fusion is processed through two branches to obtain row vectors and column vectors respectively; Multiplying the resulting row vector and column vector yields the rank-1 component.

3. The infrared and visible light image fusion method based on a rank-one decomposition network as described in claim 1, characterized in that, The process of obtaining the spatial attention map through the spatial attention module includes: Two residuals and The layers are stacked together along the channel dimension, then passed through two convolutional layers in sequence, and finally the spatial attention map of the residual is obtained by passing through the Sigmoid activation function.

4. The infrared and visible light image fusion method based on a rank-one decomposition network as described in claim 1, characterized in that, The attention map and and The integration includes: residual and Feature extraction is performed using convolutional layers; The output features of the two residuals are multiplied element-wise with the spatial attention map, and the two multiplied features are stacked in the channel dimension. The stacked features are input into the convolutional layer to obtain a fused component containing information from the two residual images.

5. An infrared and visible light image fusion system based on a rank-one decomposition network, characterized in that: include: The image acquisition module is configured to acquire infrared images and visible light images; The image stacking module is configured to stack the infrared image and the visible light image in the channel dimension. The rank-one decomposition module is configured to perform rank-one decomposition on the stacked image sequence step by step to obtain multi-level rank-one components, the residual of the infrared image, and the residual of the visible light image. The step of performing a step-by-step rank-one decomposition on the stacked image sequence to obtain multi-level rank-one components includes: decomposing the stacked image sequence... The input is fed into the first-level rank-one decomposition module to obtain the rank-one components. Subtract the rank-one component from the infrared image get Subtract the rank-one component from the visible light image get ; Will and Image sequences are obtained by stacking along the channel dimension. Image sequence The input is fed into the second-level rank-one decomposition module to obtain the rank-one components. Infrared image minus rank component get Visible light image minus rank component get ; The result and Image sequences are stacked along the channel dimension to obtain Repeat the iteration R times to obtain R rank-1 components. ; The fusion component generation module is configured to fuse the multi-level rank-one components, the residuals of the infrared image, and the residuals of the visible light image, respectively, to obtain a fusion component containing information from all rank-one components. and the fusion component containing two residual information ; The method for calculating the residuals of the infrared image and the visible light image includes: summing the acquired R rank-one components to obtain... ; Subtract the sum of R rank-one components from the infrared image. Obtain the residual of the infrared image ; Subtract the sum of R rank-one components from the visible light image. Obtain the residual of the visible light image ; The residuals of the infrared image and the visible light image are fused to obtain a fused component containing information from both residuals. The specific steps include: Two residuals and The input is fed into the residual fusion module, where a spatial attention map is obtained through the spatial attention module. The attention map is then compared with... and The components are fused to obtain a fused component containing information from both residuals. ; The fused image generation module is configured to: combine the fused components and The desired fused image is obtained by adding elements together. .

6. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by a processor, the program implements the steps in the infrared and visible light image fusion method based on a rank-one decomposition network as described in any one of claims 1-4.

7. An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the infrared and visible light image fusion method based on a rank-one decomposition network as described in any one of claims 1-4.

Images