Lightweight fusion method for infrared and visible images based on spatial and frequency domain information

Abstract

The application discloses an infrared and visible light image lightweight fusion method based on space domain and frequency domain information, comprising: taking infrared and visible light images as inputs, after processing through various modules of the model, a fusion image with significant thermal targets and rich background texture is generated. The method of the application is mainly divided into two branches of space domain and frequency domain, wherein the space domain branch refines the original infrared and visible light images to improve the semantic representation of images of different modalities, and the frequency domain branch mainly performs frequency domain fusion of images. Finally, the outputs of the two branches are spliced together and input into a fusion module composed of multiple convolution layers, and the final fusion image is output by the module. The application is based on a parallel processing structure to fully extract effective information of two modalities in the frequency domain and the space domain, so as to generate a good quality fusion image, the overall process is simple, has excellent reasoning efficiency, can be used as a preprocessing module of a downstream visual task and improves the performance of the downstream visual task.

Description

Technical Field

[0001] This invention relates to the field of image fusion technology, and in particular to a lightweight fusion method for infrared and visible light images that integrates spatial and frequency domain information. Background Technology

[0002] Image fusion extracts valuable information from images acquired by different sensors to create images with richer details. Currently, image fusion has wide applications in many fields, such as remote sensing, medical imaging, and autonomous driving. Infrared and visible image fusion is a typical image fusion task. Infrared images contain thermal radiation information; even in low light conditions, thermal targets stand out due to their high contrast. However, infrared images are not sensitive to objects with weak thermal radiation, thus lacking texture detail. Visible images possess rich texture detail but are highly susceptible to weather and lighting conditions, leading to the loss of target location and structural information in low-visibility scenarios, which is detrimental to some downstream vision tasks. Infrared and visible image fusion leverages the imaging characteristics of both modalities, mining complementary information between infrared and visible images to generate fused images with rich texture detail and contrast information, thereby improving visual perception for both humans and machines.

[0003] The main challenge in infrared and visible image fusion lies in extracting valuable information from both modalities and using this information to generate a fused image. Early traditional methods primarily included sparse representation methods, multi-scale transformation methods, subspace methods, and hybrid models. While these traditional methods can generate decent fused images in some cases, the complexity of the fusion rules and the limitations of manual feature extraction restrict the representation of infrared and visible features, thus reducing fusion performance. With the success of deep learning methods in numerous computer vision tasks, some infrared and visible image fusion algorithms utilize convolutional neural networks (CNNs), generative adversarial networks (GANs), or Transformers to obtain fused images. Although these deep learning-based algorithms can produce satisfactory fused images, they primarily focus on spatial domain fusion, neglecting information in the frequency domain. Furthermore, some methods have highly complex network structures, resulting in slow inference speeds and difficulty meeting the real-time requirements of practical scenarios.

[0004] Most image fusion methods focus only on image features in the spatial domain, neglecting features in the frequency domain. However, the features of these two domains are distinct. In the spatial domain, objects and backgrounds are more easily observed. In the frequency domain, edges and textures can be better identified by searching for high-frequency information and repetitive patterns. Therefore, if a model can fuse information from different domains, it will have a more comprehensive understanding of the image. Existing frequency-domain fusion methods either only capture frequency domain information without integrating spatial domain information, or generate the final fusion result through sequential processing of frequency and spatial domains. The limitation of this sequential structure is that the network may easily overlook valuable information from one domain while focusing on effective information from another.

[0005] Therefore, how to provide a lightweight fusion method for infrared and visible light images that has the ability to learn hidden representations from both the frequency domain and the spatial domain is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In response to the aforementioned research status, this invention provides a lightweight fusion method for infrared and visible light images based on spatial and frequency domain information. Based on a parallel processing structure, it fully extracts effective information from both modes in the frequency and spatial domains, thereby generating a high-quality fused image. This solves the problem that most image fusion methods cannot effectively combine frequency domain information and are difficult to meet the timeliness requirements in practical applications.

[0007] This invention provides a lightweight fusion method for infrared and visible light images based on spatial and frequency domain information, comprising the following steps:

[0008] Source image pair acquisition steps:

[0009] Acquire source image pairs of infrared and visible light images of the same target;

[0010] Both the infrared image and the visible light image are input into the spatial domain branch and the frequency domain branch, and the two branches process the source image pairs in parallel.

[0011] Spatial domain branching processing steps:

[0012] Feature extraction is performed on the source image pairs to obtain infrared feature maps and visible light feature maps;

[0013] The infrared feature map and the visible light feature map are respectively input into two three-branch structures. The three-branch structures are used to extract gradient information of the current source image, suppress redundancy information, and extract complementary information of another source image, respectively, to obtain the three-branch structure processing results of the two modes.

[0014] The three-branch structure processing results of the two modes are added element by element to obtain the output of the spatial domain branches of the two modes;

[0015] Frequency domain branching processing steps:

[0016] The source image pair is transformed from the spatial domain to the frequency domain by using Fast Fourier Transform to obtain a frequency domain image pair;

[0017] Calculate the amplitude and phase for each frequency domain image pair;

[0018] The amplitudes of the two modes are concatenated along the channel dimension and then fused to obtain the amplitude fusion result; the phases of the two modes are concatenated along the channel dimension and then fused to obtain the phase fusion result.

[0019] By using inverse Fourier transform, the amplitude fusion result and the phase fusion result are converted back to the spatial domain to obtain the spatial domain output;

[0020] After enhancing the spatial domain output, the enhancement result is refined and reconstructed to obtain the frequency domain branch output;

[0021] Frequency domain and spatial domain fusion: The outputs of the spatial domain branch and the frequency domain branch are concatenated along the channel dimension and then feature fusion is performed to obtain the final output.

[0022] Preferably, in the source image pair acquisition step, acquiring source image pairs of infrared and visible light images of the same target includes the following steps:

[0023] Read the infrared image of the target and convert it into a tensor to obtain an infrared image;

[0024] Read the visible light image of the target, convert the visible light image from the RGB channel to the YcbCr channel, extract the Y channel and convert it into a tensor to obtain the visible light image;

[0025] The shape of each source image pair is 1xH x W, where H and W are the height and width of the image, respectively.

[0026] Preferably, the spatial domain branching process includes the following steps for feature extraction on the source image pairs:

[0027] Feature extraction is performed on the source image pairs using a set of convolutional layers and activation layers to obtain features.

[0028] Preferably, the three-branch structure includes a texture information branch, an attention branch, and a complementary information branch, and its processing steps include:

[0029] Texture information branch: Extract gradient information of the current source image using the Sobel operator;

[0030] Attention branch: The current source image is passed through a convolutional layer and a sigmoid activation layer to obtain an attention map, which is then multiplied with the input current source image to obtain the attention branch result;

[0031] Complementary information branch: Infrared feature map and visible light feature map The elements are added one by one, and then complementary information from different modalities is obtained through two attention blocks.

[0032] Preferably, the frequency domain branching process for calculating the amplitude and phase of the frequency domain image pairs includes the following steps:

[0033] The amplitude and phase diagrams of the infrared and visible light images are calculated separately using the following formulas:

[0034] {AMP ir AMP vis}={|FFT(I ir )|,|FFT(I vis )|}

[0035] {PHA ir PHA vis}={θ(FFT(I ir ),θ(FFT(I vis ))}

[0036] Where |·| represents taking the absolute value; θ calculates the angle element by element; AMP ir AMP vis These represent the amplitudes of the infrared image and the visible light image, respectively; I ir I vis These represent infrared and visible light images, respectively; PHA ir PHA vis These represent the phases of the infrared and visible light images, respectively; FFT stands for Fast Fourier Transform.

[0037] Preferably, the frequency domain branching process includes the following steps to obtain the amplitude fusion result and the phase fusion result:

[0038] The amplitude and phase maps of the two modes are concatenated along the channel dimension and then input into a fusion block consisting of two sets of convolutional layers and LeakyReLU activation layers to complete the fusion of amplitude and phase.

[0039] Preferably, in the frequency domain branching process, the formula for converting the amplitude fusion result and the phase fusion result back to the spatial domain by inverse Fourier transform is as follows:

[0040]

[0041] in, It is a spatial domain output; Indicates concatenation operation; IFFT represents inverse Fourier transform, AMP ir AMP vis These represent the amplitudes of the infrared and visible light images, respectively; PHA ir ,PHA vis These represent the phase of the infrared image and the visible light image, respectively.

[0042] Preferably, the frequency domain branching process includes the following steps for enhancing the spatial domain output:

[0043] The maximum and mean values ​​are extracted from the spatial domain output, and then the maximum and mean values ​​are concatenated along the channel dimension to obtain the enhanced result.

[0044] Preferably, the frequency domain branching process includes the following steps for refining and reconstructing the enhancement results:

[0045] The enhancement results are input into two sets of convolutional layers and ReLU activation layers for refinement and reconstruction, as shown in the following formula:

[0046]

[0047] Among them, F fre It is the output of the frequency domain branch; It is a spatial domain output; This indicates a splicing operation.

[0048] Preferably, the frequency domain and spatial domain fusion step includes:

[0049] The output F of the spatial domain branch ir ,F vis and the output F of the frequency domain branch fre The data is concatenated along the channel dimension and then fed into a fusion block consisting of four sets of convolutional and activation layers. The first three sets of activation layers are ReLU, and the last set is Tanh. The final output is the fused image. The formula is as follows:

[0050]

[0051] Among them, F ir Indicates the spatial domain branch output of the infrared image; F vis C represents the spatial domain branch output of a visible light image; C represents the stitching operation.

[0052] The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information proposed in this invention has the following advantages over existing technologies:

[0053] (1) The method of the present invention relates to a lightweight fusion network for infrared and visible light images that combines frequency domain and spatial domain. This network can not only capture effective information in the spatial domain, but also supplement the spatial domain information through the fusion of frequency domain information, thereby achieving excellent performance in fusion effect and downstream tasks.

[0054] (2) The method of the present invention extracts complementary information in the spatial domain as much as possible through a dual-modal refinement module, filters redundant information and retains texture details through simple attention blocks and gradient information extraction blocks, and performs amplitude and phase fusion of the two modes in the frequency domain through a frequency domain fusion module. The result of frequency domain fusion can make up for the lack of global interaction in the spatial domain.

[0055] (3) The method of the present invention effectively combines frequency domain information, and the quality of the generated fused image is better than other advanced fusion methods. It also performs better than the compared methods in downstream monitoring tasks. In addition, due to the low number of parameters and computation, it has excellent inference efficiency compared with other advanced methods and can meet the real-time requirements in practical application scenarios. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention, and those skilled in the art can obtain other drawings based on the provided drawings without creative effort.

[0057] Figure 1 A flowchart illustrating the lightweight fusion method for infrared and visible light images combining frequency and spatial domains provided in this embodiment of the invention.

[0058] Figure 2 A schematic diagram illustrating the fused image generated by this method and other methods according to embodiments of the present invention;

[0059] Figure 3 This is a schematic diagram illustrating the detection performance of the fused image generated by the method of this invention and other methods in a target monitoring task. Detailed Implementation

[0060] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] This invention provides a lightweight fusion method for infrared and visible light images that integrates spatial and frequency domain information. First, the input image is preprocessed; for RGB images, it needs to be converted to the YcbCr color space, and the Y channel is extracted as input. Second, the inputs are processed in both the spatial and frequency domain branches. Then, the spatial domain branch refines and reconstructs the source image, while the frequency domain branch fuses the amplitude and phase of different modal images in the frequency domain. Finally, the outputs of the two branches are concatenated, and the fusion of the infrared and visible images is achieved through convolution.

[0062] Includes the following steps:

[0063] Source image pair acquisition steps:

[0064] Acquire source image pairs of infrared and visible light images of the same target;

[0065] Both infrared and visible light images are input into the spatial domain branch and the frequency domain branch, and the two branches process the source image pairs in parallel.

[0066] Spatial domain branching processing steps:

[0067] Feature extraction is performed on the source image pairs to obtain infrared feature maps and visible light feature maps;

[0068] The infrared feature map and the visible light feature map are respectively input into two three-branch structures. The three-branch structures are used to extract gradient information of the current source image, suppress redundancy information, and extract complementary information of another source image, respectively, to obtain the three-branch structure processing results of the two modes.

[0069] The results of the three-branch structure processing of the two modes are added element by element to obtain the output of the spatial domain branches of the two modes;

[0070] Frequency domain branching processing steps:

[0071] The source image pairs are transformed from the spatial domain to the frequency domain by using Fast Fourier Transform to obtain frequency domain image pairs;

[0072] Calculate the amplitude and phase of each frequency domain image pair separately;

[0073] The amplitudes of the two modes are concatenated along the channel dimension and then fused to obtain the amplitude fusion result; the phases of the two modes are concatenated along the channel dimension and then fused to obtain the phase fusion result.

[0074] By using inverse Fourier transform, the amplitude fusion result and the phase fusion result are converted back to the spatial domain to obtain the spatial domain output;

[0075] After enhancing the spatial domain output, the enhancement result is refined and reconstructed to obtain the frequency domain branch output;

[0076] Frequency domain and spatial domain fusion: The outputs of the spatial domain branch and the frequency domain branch are concatenated along the channel dimension and then feature fusion is performed to obtain the final output.

[0077] This embodiment extracts complementary information based on bimodal refinement. Useful spatial domain information is extracted from infrared and visible modes, enhancing fine-grained spatial details. To incorporate frequency domain information, frequency domain fusion is performed, converting the spatial domain to the frequency domain using a Fast Fourier Transform (FFT) and then integrating the frequency domain information.

[0078] This embodiment has been extensively tested on public datasets, demonstrating significant advantages in various fusion evaluation metrics and visual effects. Furthermore, it excels in both the efficiency of image fusion and the performance of downstream detection tasks, thus meeting the real-time requirements of advanced vision tasks.

[0079] Figure 1 This is a schematic diagram of the technical process of the present invention, which mainly includes 12 steps. The sequence numbers of the steps do not limit the order of execution.

[0080] Step A: Acquire a pair of infrared and visible light images. For the infrared image, it can be directly read and converted into a tensor. For the visible light image, first convert the image from the RGB channel to the YcbCr channel, then extract the Y channel and convert it into a tensor. After the above processing, the original input shape of the two images is 1xH x W, where H and W are the height and width of the image, respectively.

[0081] Step B involves simultaneously processing the infrared and visible image pairs in the spatial and frequency domain branches.

[0082] Step C, the spatial domain branch, is mainly handled by the bimodal thinning module, which processes the original infrared and visible light images. First, a set of convolutional layers and ReLU activation layers are used to perform simple feature extraction on the original image pairs, with a 3x3 kernel size, a stride of 1, and padding of 1, to obtain the features.

[0083] Step D: For the inputs in the infrared and visible light modes, each is processed through three branches: a texture information branch, an attention branch, and a complementary information branch. The texture information branch processes the input using a gradient extraction block (GEB). The GEB calculates the horizontal and vertical gradients of the image using the Sobel operator, then sums their absolute values ​​element-wise to obtain the GEB output. Its purpose is to enhance the learning ability of image texture details. The attention branch processes the input using an attention block (ATB). The ATB generates an attention map through a convolutional layer and a sigmoid activation layer, which is then multiplied with the ATB input. The purpose of this attention block is to enhance effective information and suppress redundant information. The complementary information branch combines the inputs of the two modalities... The elements are added one by one, and then the effective information is obtained through two ATBs respectively. This branch is designed to capture complementary information from different modalities.

[0084] Step E: Sum the outputs of the three branches to obtain the output of the bimodal refinement module, as shown in the following formula:

[0085]

[0086] Where F ir ,F vis These are the refined infrared and visible light images, and ⊕ indicates element-wise addition.

[0087] Step F, the frequency domain branch involved, is mainly completed by the frequency domain fusion module. First, a Fast Fourier Transform is performed on the input infrared and visible images to transform the original input from the spatial domain to the frequency domain.

[0088] Step G: Calculate the amplitude and phase maps of the infrared and visible light images respectively, using the following formulas:

[0089] {AMP ir AMP vis}={|FFT(I ir )||,|FFT(I vis )||}

[0090] {PHA ir PHA vis}={θ(FFT(I ir ),θ(FFT(I vis ))}

[0091] Where |·| represents taking the absolute value; θ calculates the angle element by element; I ir I vis These represent infrared and visible light images, respectively; AMP ir AMP visThese represent the amplitudes of the infrared and visible light images, respectively; PHA ir PHA vis These represent the phase of the infrared image and the visible light image, respectively.

[0092] In step H, after obtaining the amplitude and phase maps of different modes, the amplitude and phase maps of the two modes are stitched together along the channel dimension and input into a fusion block consisting of two sets of convolutional layers and LeakyReLU activation layers to complete the fusion of amplitude and phase. The convolutional kernel size is 3x3, the stride is 1, and the padding is 1.

[0093] Step I: Perform an inverse Fourier transform on the fused amplitude and phase to convert them from the frequency domain to the spatial domain, obtaining the frequency domain fused result, as shown in the following formula:

[0094]

[0095] in It is the output after frequency domain fusion; 1 represents the splicing operation; IFFT represents the inverse Fourier transform.

[0096] Step J: Extract the maximum and mean values ​​from the frequency domain fusion results and then stitch them together along the channel dimension.

[0097] Step K involves inputting the concatenation result from step J into two sets of convolutional layers and ReLU activation layers for processing, yielding the output of the frequency domain fusion module. The convolutional kernel size is 3x3, the stride is 1, and the padding is 1, as shown in the following formula:

[0098]

[0099] Step L, convert the output F of the two branches ir F vis and F fre The data is concatenated along the channel dimension and then fed into a fusion block consisting of four sets of convolutional and activation layers. The first three activation layers are ReLU, and the last activation layer is Tanh. The kernel size is 3x3, the stride is 1, and the padding is 1. The final output is the fused image, as shown in the following formula:

[0100]

[0101] In one embodiment, a frequency domain fusion loss is designed to guide the fusion process in the frequency domain.

[0102] During network training, the loss function plays a decisive role in the final fusion result. This method uses four loss functions: content loss function, structural similarity loss function, saliency target loss function, and frequency domain fusion loss function. Among them, the frequency domain fusion loss function is a loss function specifically proposed by this method to guide the frequency domain fusion process, and its formula is as follows:

[0103]

[0104] in, Represents the correlation coefficient of an image. This indicates element-wise multiplication. A mask representing a salient target, I ir ,I vis These are the original infrared image and the visible light image, respectively. This represents the image transformed into the spatial domain after frequency domain fusion. The design idea of ​​this loss function is to make the salient targets in the frequency domain fused image consistent with those in the infrared image, while maintaining consistency in background details with those in the visible light image. This loss function guides the frequency domain fusion result to have more salient foreground targets and richer texture details.

[0105] For downstream computer vision tasks such as autonomous driving, pedestrian detection, and image segmentation, where computing and storage resources are limited in hardware deployment, this invention serves as a preprocessing step for visual tasks such as detection, segmentation, and classification. It can compensate for the shortcomings of single-modal infrared and visible light images, generating fused images with good visual quality and downstream task performance in real time and effectively, thus ensuring the implementation effect of subsequent tasks.

[0106] Figure 2 The experimental results shown compare the fusion results of our method with those of seven other advanced image fusion methods. The images used in the experiments are from the publicly available infrared-visible light fusion dataset MSRS, and all experiments were conducted under the same hardware and software environment. The experimental results demonstrate that the fused images generated by our method not only highlight significant thermal targets but also preserve the texture details of both the target and the background very well. Figure 3 The experimental results shown compare the fusion results of our method with those of seven other advanced image fusion methods in the target detection task. The images used in the experiments are from the publicly available infrared-visible light fusion dataset M3FD, and all experiments were conducted under the same hardware and software environment. Figure 3As shown, the original infrared and visible light images, along with fused images generated by other methods, all exhibited missed detections and false detections in the target detection task (as seen in the image of the person bending over behind the stall). In contrast, the fused image generated by our proposed method performed well in the target detection task. Therefore, the experimental results demonstrate that the fused image generated by our method can help improve the performance of downstream vision tasks.

[0107] The present invention provides a detailed description of a lightweight fusion method for infrared and visible light images based on spatial and frequency domain information. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0108] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A lightweight fusion method for infrared and visible light images based on spatial and frequency domain information, characterized in that: Includes the following steps: Source image pair acquisition steps: Acquire source image pairs of infrared and visible light images of the same target; Both the infrared image and the visible light image are input into the spatial domain branch and the frequency domain branch, and the two branches process the source image pairs in parallel. Spatial domain branching processing steps: Feature extraction is performed on the source image pairs to obtain infrared feature maps and visible light feature maps; The infrared feature map and the visible light feature map are respectively input into two three-branch structures. The three-branch structures are used to extract gradient information of the current source image, suppress redundancy information, and extract complementary information of another source image, respectively, to obtain the three-branch structure processing results of the two modes. The three-branch structure processing results of the two modes are added element by element to obtain the output of the spatial domain branches of the two modes; Frequency domain branching processing steps: The source image pair is transformed from the spatial domain to the frequency domain by using Fast Fourier Transform to obtain a frequency domain image pair; Calculate the amplitude and phase for each frequency domain image pair; The amplitudes of the two modes are concatenated along the channel dimension and then fused to obtain the amplitude fusion result; the phases of the two modes are concatenated along the channel dimension and then fused to obtain the phase fusion result. By using inverse Fourier transform, the amplitude fusion result and the phase fusion result are converted back to the spatial domain to obtain the spatial domain output; After enhancing the spatial domain output, the enhancement result is refined and reconstructed to obtain the frequency domain branch output; Frequency domain and spatial domain fusion: The outputs of the spatial domain branch and the frequency domain branch are concatenated along the channel dimension and then feature fusion is performed to obtain the final output.

2. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, The source image pair acquisition step, which involves acquiring source image pairs of infrared and visible light images of the same target, includes the following steps: Read the infrared image of the target and convert it into a tensor to obtain an infrared image; Read the visible light image of the target, convert the visible light image from the RGB channel to the YcbCr channel, extract the Y channel and convert it into a tensor to obtain the visible light image; The shape of each source image pair is 1xH x W, where H and W are the height and width of the image, respectively.

3. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, The spatial domain branching process includes the following steps for feature extraction on the source image pairs: Feature extraction is performed on the source image pairs using a set of convolutional layers and activation layers to obtain features.

4. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, The three-branch structure includes a texture information branch, an attention branch, and a complementary information branch, and its processing steps include: Texture information branch: Extract gradient information of the current source image using the Sobel operator; Attention branch: The current source image is passed through a convolutional layer and a sigmoid activation layer to obtain an attention map, which is then multiplied with the input current source image to obtain the attention branch result; Complementary information branch: Infrared feature map and visible light feature map The elements are added one by one, and then complementary information from different modalities is obtained through two attention blocks.

5. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, The frequency domain branching process involves calculating the amplitude and phase of the frequency domain image pairs, including the following steps: The amplitude and phase diagrams of the infrared and visible light images are calculated separately using the following formulas: {AMP ir ,AMP vis }={|FFT(I ir )|,|FFT(I vis )|} {PHA ir ,PHA vis }=(θ(FFT(I ir )),θ(FFT(I vis ))} Where |·| represents taking the absolute value; θ calculates the angle element by element; I ir I vis These represent infrared and visible light images, respectively; AMP ir AMP vis These represent the amplitudes of the infrared and visible light images, respectively; PHA ir PHA vis These represent the phases of the infrared and visible light images, respectively; FFT stands for Fast Fourier Transform.

6. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, The frequency domain branching process includes the following steps to obtain the amplitude fusion result and the phase fusion result: The amplitude and phase maps of the two modes are concatenated along the channel dimension and then input into a fusion block consisting of two sets of convolutional layers and LeakyReLU activation layers to complete the fusion of amplitude and phase.

7. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, In the frequency domain branching process, the formula for converting the amplitude fusion result and the phase fusion result back to the spatial domain through inverse Fourier transform is as follows: in, It is the spatial domain output; C represents the concatenation operation; IFFT represents the inverse Fourier transform; AMPi r AMP vis These represent the amplitudes of the infrared and visible light images, respectively; PHAi r PHA vis These represent the phase of the infrared image and the visible light image, respectively.

8. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, The frequency domain branching process includes the following steps to enhance the spatial domain output: The maximum and mean values ​​are extracted from the spatial domain output, and then the maximum and mean values ​​are concatenated along the channel dimension to obtain the enhanced result.

9. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, The frequency domain branching process includes the following steps for refining and reconstructing the enhancement results: The enhancement results are input into two sets of convolutional layers and ReLU activation layers for refinement and reconstruction, as shown in the following formula: Among them, F fre It is the output of the frequency domain branch; It is a spatial domain output; This indicates a splicing operation.

10. The lightweight fusion method for infrared and visible light images based on spatial and frequency domain information according to claim 1, characterized in that, The frequency domain and spatial domain fusion steps include: The output F of the spatial domain branch ir F vis and the output F of the frequency domain branch fre The data is concatenated along the channel dimension and then fed into a fusion block consisting of four sets of convolutional and activation layers. The first three sets of activation layers are ReLU, and the last set is Tanh. The final output is the fused image. The formula is as follows: Among them, F ir Indicates the spatial domain branch output of the infrared image; F vis C represents the spatial domain branch output of a visible light image; C represents the stitching operation.

Images