Method and system for converting infrared image into visible light image

Abstract

The invention relates to the technical field of video monitoring, in particular to a method and system for converting an infrared image into a visible light image, and the method comprises the steps: obtaining a non-paired infrared and visible light image data set, and employing a dual-branch feature decoupling network to extract a shared semantic feature and a mode-specific feature, after an initial image is generated through multi-scale fusion and progressive up-sampling, semantic consistency verification is carried out through a CLIP model, and finally a high-quality visible light image is output through adaptive detail enhancement and intelligent fusion based on semantic segmentation. The problems that in the prior art, data are strictly aligned, details of the generated image are fuzzy, and semantics are inconsistent are effectively solved, high-precision visible light conversion of the infrared image in a complex scene is achieved, and the detail authenticity and scene adaptability of the generated image are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of video surveillance technology, and in particular to a method and system for converting infrared images into visible light images. Background Technology

[0002] The conversion technology between infrared and visible light images is an important research direction in the field of computer vision, with wide-ranging value in applications such as security monitoring, autonomous driving, and military reconnaissance. Infrared images are based on the thermal radiation characteristics of targets, while visible light images rely on the reflection characteristics of ambient light; the two have fundamentally different imaging mechanisms. This difference results in weak correlation between the two modalities in terms of grayscale distribution and texture features, posing a significant challenge to cross-modal conversion. Furthermore, due to sensor performance limitations, the resolution of infrared images is usually lower than that of visible light images, making the recovery of high-resolution visible light images from low-resolution infrared images a technical challenge.

[0003] Currently, some studies have attempted to address the problem of converting infrared images to visible light images. Among them, publication number CN101202845B proposes a method and apparatus for converting infrared images to visible light images. This method first constructs a strictly aligned infrared-visible light face image database, extracts the feature projection directions of the two modalities through canonical correlation analysis (CCA), and establishes a block-level feature mapping relationship. In the working stage, the input infrared image is processed into blocks, and a pre-trained feature transformation function is used to map the infrared features to visible light features. Finally, the visible light image is reconstructed through Local Linear Embedding (LLE).

[0004] However, while the aforementioned methods for converting infrared images to visible light images can achieve basic conversion functions, they rely on strictly aligned pairwise training data, which is difficult to acquire on a large scale in practical applications. Furthermore, due to the limited information content of infrared images themselves, visible light images generated based on super-resolution frameworks still fall short in detail representation, particularly in the recovery of high-frequency information such as texture and edges. These issues limit the applicability and practicality of existing technologies in complex scenarios. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for converting infrared images into visible light images, which solves the problem of insufficient applicability and practicality of existing methods for converting infrared images into visible light images in complex scenarios.

[0006] To achieve the above objectives, the present invention provides a method for converting an infrared image into a visible light image, comprising the following steps: Acquire unpaired infrared and visible light image datasets and process them separately; The processed image is input into a dual-branch feature decoupling network to extract shared and specific features. After multi-scale fusion of shared features and special features, the data is input into a progressive upsampling network to first generate a low-resolution visible light image, and then gradually restore high-resolution details through a cascaded RefineNet structure. The generated images are semantically consistent using a pre-trained CLIP model. Adaptive detail enhancement and noise suppression are applied to the verified images. Finally, based on the semantic segmentation results, intelligent fusion images are output by generating image features in important regions and preserving the original infrared image features in background regions.

[0007] This involves acquiring unpaired infrared and visible light image datasets and processing them separately. Infrared images are subjected to radiation normalization, and visible light images are subjected to illumination equalization.

[0008] The processed image is input into a dual-branch feature decoupling network to extract shared and specific features. The dual-branch feature decoupling network includes a shared feature extraction branch for extracting common semantic features from infrared and visible light images; The infrared feature extraction branch includes a differentiable thermodynamic calculation layer to preserve thermal radiation properties; The visible light feature extraction branch includes a high-frequency attention module for enhancing texture detail features.

[0009] The process involves multi-scale fusion of shared and specialized features, followed by input into a progressive upsampling network. This first generates a low-resolution visible light image, which is then progressively restored to high-resolution details through a cascaded RefineNet structure. Specific steps include: The shared features are combined with infrared-specific features and visible light-specific features according to the channel dimension to form a fused feature; The fused features are sequentially input into three cascaded multi-scale fusion modules. Each module first extracts local features through 3×3 convolution, and then performs skip connections to fuse with the high-resolution features output from the previous layer. The features output from the last-stage multi-scale fusion module are input into the initial generator to generate a low-resolution visible light image. The low-resolution image is sequentially input into three cascaded RefineNet modules. Each RefineNet module combines high-frequency attention features of the corresponding scale through residual connections to gradually restore image details and improve resolution. The final output is a high-resolution visible light image.

[0010] The final output is a high-resolution visible light image, in which: The output resolution of each RefineNet module is twice the input resolution.

[0011] The semantic consistency verification of the generated images is performed using a pre-trained CLIP model, and the specific steps include: The generated image and the input infrared image are respectively input into the image encoder of the CLIP model to extract image feature vectors. The cosine similarity between the two feature vectors in the semantic space is calculated as the semantic consistency score. When the semantic consistency score is lower than the preset threshold, the parameters of the generating network are adjusted and the image is regenerated until the semantic consistency score meets the requirements. The CLIP model uses pre-trained weights with fixed parameters, and the image encoding process remains end-to-end differentiable.

[0012] The process involves adaptive detail enhancement and noise suppression on the verified images, and finally, based on the semantic segmentation results, generating image features in important regions and preserving the original infrared image features in background regions to output an intelligent fused image. Specific steps include: The generated image that passes semantic verification is input into the detail enhancement network. The detail enhancement network establishes an adaptive filtering kernel by analyzing the noise distribution of the infrared image, and at the same time uses the texture feature map extracted by the high-frequency attention module to guide the detail enhancement, generating an optimized visible light image. The optimized visible light image and the original infrared image are input into the semantic segmentation network to obtain the mask of important regions; The optimized visible light image and the original infrared image are fused pixel-level based on the important region mask, and the final output is a fused image that retains the visible light details of the important region and the infrared features of the background region.

[0013] A system for converting infrared images into visible light images includes a processing module, a feature extraction module, a generation module, a verification module, and an output module. The feature extraction module is connected to the processing module, the generation module is connected to the feature extraction module, the verification module is connected to the generation module, and the output module is connected to the verification module. The processing module is used to acquire unpaired infrared and visible light image datasets and process them separately. The feature extraction module is used to input the processed image into the dual-branch feature decoupling network to extract shared features and specific features; The generation module is used to fuse shared features and special features at multiple scales and then input them into the progressive upsampling network to first generate a low-resolution visible light image, and then gradually restore high-resolution details through a cascaded RefineNet structure. The verification module is used to perform semantic consistency verification on the generated images using a pre-trained CLIP model. The output module is used to perform adaptive detail enhancement and noise suppression on the verified images. Finally, based on the semantic segmentation results, it outputs an intelligent fused image by generating image features in important regions and preserving the original infrared image features in background regions.

[0014] This invention discloses a method and system for converting infrared images into visible light images. By acquiring unpaired infrared and visible light image datasets, a dual-branch feature decoupling network is used to extract shared semantic features and modality-specific features. After generating an initial image through multi-scale fusion and progressive upsampling, the CLIP model is used for semantic consistency verification. Finally, a high-quality visible light image is output through adaptive detail enhancement and intelligent fusion based on semantic segmentation. This effectively solves the problems of existing technologies, such as reliance on strictly aligned data, blurred details in the generated image, and semantic inconsistencies. It achieves high-precision visible light conversion of infrared images in complex scenes, significantly improving the detail realism and scene adaptability of the generated image. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0016] Figure 1 This is a flowchart of the steps of the method for converting an infrared image into a visible light image according to the first embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the system for converting infrared images into visible light images according to the second embodiment of the present invention.

[0018] In the diagram: 201 - Processing module, 202 - Feature extraction module, 203 - Generation module, 204 - Verification module, 205 - Output module. Detailed Implementation

[0019] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0020] The first embodiment of this application is as follows: Please see Figure 1 ,in, Figure 1 This is a flowchart of the steps of the method for converting an infrared image into a visible light image according to the first embodiment of the present invention.

[0021] This invention provides a method for converting an infrared image into a visible light image, comprising the following steps: S101: Obtain unpaired infrared and visible light image datasets and process them separately; Specifically, firstly, infrared and visible light image datasets containing diverse scenarios are established. These datasets do not need strict pairing relationships, but should cover similar application scenarios, such as various targets in security monitoring and autonomous driving scenarios. For infrared image processing, radiation normalization is used to eliminate the influence of environmental temperature differences. In practice, environmental temperature parameters of the scene are collected, and a temperature-grayscale value mapping relationship is established based on Planck's radiation law. The pixel values ​​of the original infrared images are linearly transformed to adjust all images to the grayscale range corresponding to a standard temperature range (e.g., 20-30℃), ensuring the comparability of infrared images acquired at different times and under different environments. For visible light image processing, an illumination equalization algorithm based on Retinex theory is used. First, the illuminance component of the image is estimated. Then, through methods such as gamma correction and histogram matching, visible light images captured under different lighting conditions are adjusted to a unified brightness standard, with noise suppression and detail enhancement specifically performed on low-illuminance images. After the above processing, for some alignable scenes, such as images containing structured targets like faces and vehicles, the SIFT feature point detection algorithm is used to extract key points, and the optimal affine transformation matrix is ​​calculated using the RANSAC algorithm to achieve coarse alignment between infrared and visible light images. For natural scene images that cannot be aligned, their unpaired characteristics are retained and they directly enter the subsequent processing flow. This series of processes ensures that the subsequent two-branch feature decoupling network can effectively learn cross-modal features, while avoiding the dependence of traditional methods on strictly paired data, providing a high-quality input data foundation for the entire transformation process.

[0022] S102: Input the processed image into a dual-branch feature decoupling network to extract shared features and specific features; Specifically, a dual-branch feature decoupling network is first constructed, featuring a shared feature extraction branch and two dedicated feature extraction branches. The shared feature extraction branch uses a pre-trained ResNet-50 as its base network, removing the final fully connected layer while retaining its convolutional layers and residual structure. This branch extracts high-level semantic features common to both infrared and visible light images, such as object contours and scene layout. The infrared feature extraction branch employs a convolutional neural network with a differentiable thermodynamics computation layer. This branch first extracts low-level thermal radiation features through three 3×3 convolutional layers, then passes them through a differentiable thermodynamics computation layer. This layer, based on the Stefan-Boltzmann law, establishes a thermal radiation-grayscale value mapping relationship, converting the extracted features into a physically consistent representation. Finally, two 1×1 convolutional layers output dedicated features that preserve thermal radiation characteristics. The visible light feature extraction branch employs the encoder portion of a U-Net structure, comprising four downsampling blocks. Each block consists of two 3×3 convolutional layers and a max-pooling layer. A high-frequency attention module is connected after each downsampling block. This module generates attention weight maps by calculating the gradient magnitude of local regions, which are used to enhance the responses in edge and texture regions. The network parameters of the three branches are updated independently but share the same input image size (uniformly adjusted to 256×256 pixels). During training, an adversarial loss function constrains the shared feature space, ensuring semantic alignment of shared features across different modalities. The feature map size output by each branch remains consistent (64×64×256), facilitating subsequent multi-scale fusion processing. This feature extraction process effectively separates modality-shared information from modality-specific information while maintaining feature expressive power, laying a solid foundation for subsequent image generation.

[0023] S103: After multi-scale fusion of shared features and special features, the data is input into a progressive upsampling network to first generate a low-resolution visible light image, and then gradually restore high-resolution details through a cascaded RefineNet structure. Specifically, the shared features extracted in the previous step are first concatenated with infrared-specific features and visible light-specific features along the channel dimension to form a 512-dimensional fused feature map. This fused feature is then input into a processing flow consisting of three cascaded multi-scale fusion modules. Each module adopts a standard U-Net skip connection structure: the first-level module receives the 64×64 resolution fused features, extracts local features through 3×3 convolution, and then performs element-wise addition and fusion with the 128×128 resolution high-level features that have undergone bilinear upsampling; the second-level module repeats the same operation to increase the feature resolution to 256×256; the third-level module outputs the final multi-scale fused features. These fused features are then input into an initial generator, which uses a network structure containing four transposed convolutional layers. Each layer is equipped with a ReLU activation function and batch normalization to progressively upsample the features to the target resolution, initially generating a low-quality visible light image with a 64×64 resolution. The low-resolution image is then refined sequentially through three cascaded RefineNet modules: the first RefineNet receives a 64×64 input, processes it using residual blocks, and combines it with high-frequency attention features (from the visible light feature extraction branch) at the corresponding scale to output a 128×128 resolution image; the second RefineNet uses the same structure to process the 128×128 image, outputting a 256×256 image; the final RefineNet further refines the 256×256 image, ultimately outputting a high-quality 512×512 resolution visible light image. Throughout the upsampling process, each RefineNet module retains the high-frequency attention features from the corresponding scale, injecting texture detail information through feature addition to ensure that the final generated image maintains both the accuracy of the overall structure and rich detail representation. Through progressive feature fusion and image refinement, the problem of detail loss caused by insufficient information in infrared images is effectively solved.

[0024] S104: Use a pre-trained CLIP model to verify the semantic consistency of the generated images; Specifically, the pre-trained CLIP model (ViT-B / 32 version) released by OpenAI is first loaded, keeping all its parameters fixed to ensure the objectivity of semantic judgment. The generated high-resolution visible light image and the original input infrared image are resized to the input size required by the CLIP model (224×224 pixels) and subjected to CLIP's predefined normalization processing (including mean normalization and standard deviation scaling). The processed image is then input into CLIP's image encoder, which uses a Vision Transformer structure and extracts global semantic features of the image through a multi-head self-attention mechanism, ultimately outputting a 512-dimensional feature vector. The cosine similarity between these two feature vectors is then calculated as the semantic consistency score, which ranges from [-1, 1], with a value closer to 1 indicating higher semantic consistency. In practical applications, 0.85 is set as the validation threshold. When the calculated semantic consistency score is lower than this threshold, the parameters of the generator network are automatically adjusted (mainly by fine-tuning the generator weights through backpropagation), the visible light image is regenerated, and validation is performed again until the semantic consistency score meets the requirements. This verification process is fully end-to-end differentiable and can be jointly trained with the preceding generative network. However, to maintain the objectivity of the CLIP model, its parameters are fixed during actual deployment. This verification mechanism ensures that the final output visible light image not only meets visual quality standards but, more importantly, maintains a high degree of semantic consistency with the original infrared image, avoiding the semantic distortion problems common in traditional methods. The entire process fully leverages the CLIP model's powerful cross-modal understanding capabilities, providing reliable semantic guarantees for infrared-to-visible light conversion.

[0025] S105: Adaptive detail enhancement and noise suppression are applied to the verified image. Finally, based on the semantic segmentation results, an intelligent fusion image is output by generating image features in important regions and retaining the original infrared image features in the background regions.

[0026] Specifically, the semantically validated visible light image is first input into a detail enhancement network. This network employs a guided filtering architecture, establishing an adaptive filtering kernel by analyzing the noise distribution characteristics of the original infrared image, specifically targeting and suppressing common thermal imaging noise such as salt-and-pepper noise and stripe noise. Simultaneously, a high-frequency attention feature map generated from the previous visible light feature extraction branch is used as a guide to selectively enhance edge and texture regions in the image. The enhancement process employs a Laplacian pyramid decomposition method, enhancing detail features at different scales. The detail-optimized visible light image, along with the original infrared image, is then input into a pre-trained DeepLabV3+ semantic segmentation network. This network uses weights pre-trained on the Cityscapes and ADE20K datasets, accurately identifying important regions (such as faces, vehicles, and roads) and background regions (such as sky, vegetation, and buildings) in the image. A binary mask is generated based on the probability map output by the segmentation network. In important regions (probability > 0.8), 90% of visible light image features and 10% of infrared image features are fused. In transition regions (0.5 < probability ≤ 0.8), a linear blending approach is used. In background regions (probability ≤ 0.5), 70% of infrared features are retained, and only 30% of visible light features are added for color correction. The fusion process employs multi-resolution stitching technology, first dividing the region at a low resolution, and then gradually increasing the resolution for fine-grained fusion, ensuring a natural and smooth transition between different regions. The final intelligent fused image retains rich visible light details in important regions while maintaining the thermal radiation characteristics of the background region. Adaptive blending ensures the coherence of the overall visual effect, effectively solving the problems of detail distortion and semantic inconsistency commonly found in traditional methods. The entire processing workflow fully utilizes existing deep learning models and image processing techniques, achieving high efficiency while ensuring processing quality.

[0027] It effectively solves the problems of relying on strictly aligned data, blurry details in generated images, and semantic inconsistencies in existing technologies, and achieves high-precision visible light conversion of infrared images in complex scenes, significantly improving the detail realism and scene adaptability of generated images.

[0028] The second embodiment of this application is as follows: Based on the first embodiment, please refer to Figure 2 ,in, Figure 2 This is a schematic diagram of the system for converting infrared images into visible light images according to the second embodiment of the present invention.

[0029] The system for converting infrared images into visible light images in this embodiment includes a processing module 201, a feature extraction module 202, a generation module 203, a verification module 204, and an output module 205.

[0030] In this specific embodiment, the feature extraction module 202 is connected to the processing module 201, the generation module 203 is connected to the feature extraction module 202, the verification module 204 is connected to the generation module 203, and the output module 205 is connected to the verification module 204. Processing module 201 is used to acquire unpaired infrared and visible light image datasets and process them separately. Feature extraction module 202 is used to input the processed image into a dual-branch feature decoupling network to extract shared features and specific features; The generation module 203 is used to fuse shared features and special features at multiple scales and then input them into a progressive upsampling network to first generate a low-resolution visible light image, and then gradually restore high-resolution details through a cascaded RefineNet structure. The verification module 204 is used to perform semantic consistency verification on the generated image using a pre-trained CLIP model; The output module 205 is used to perform adaptive detail enhancement and noise suppression on the verified image, and finally outputs an intelligent fused image by generating image features in important regions and retaining the original infrared image features in background regions based on the semantic segmentation results.

[0031] Using the infrared image to visible light image conversion system of this embodiment, the processing module 201 first receives unpaired infrared and visible light image datasets and unifies the input standard through preprocessing operations such as radiation normalization and illumination equalization. Then, the feature extraction module 202 employs a dual-branch network architecture, where the shared branch extracts cross-modal semantic features using ResNet-50, and the two dedicated branches respectively combine a differentiable thermodynamic computation layer and a high-frequency attention mechanism to process modality-specific features. The generation module 203 adopts a three-level cascaded multi-scale fusion structure, first fusing features through a U-Net architecture to generate a low-resolution image, and then progressively upsampling through a RefineNet module to recover high-resolution details. The verification module 204 fixes the ViT-B / 32 encoder parameters of the CLIP model and ensures the consistency of the generated content by calculating the cosine similarity of the semantic feature vectors. Finally, the output module 205 integrates guided filtering denoising, Laplacian pyramid enhancement, and a semantic fusion strategy based on DeepLabV3+ to achieve intelligent mixing of visible light details in important regions with background infrared features. This system innovatively combines physical models (thermodynamic calculation layers) with deep learning architectures (CLIP, RefineNet, etc.). While maintaining the reliability of existing technologies, it effectively solves key problems in cross-modal conversion such as semantic alignment, detail restoration, and feature fusion through a modular pipeline design. The final output is a fused image that retains both thermal radiation characteristics and natural visual effects. The data flow between modules strictly follows a progressive relationship of "preprocessing-feature extraction-generation-verification-output," ensuring a balance between processing efficiency and conversion quality.

[0032] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. A method for converting an infrared image into a visible light image, characterized in that, Includes the following steps: Acquire unpaired infrared and visible light image datasets and process them separately; The processed image is input into a dual-branch feature decoupling network to extract shared and specific features. After multi-scale fusion of shared features and special features, the data is input into a progressive upsampling network to first generate a low-resolution visible light image, and then gradually restore high-resolution details through a cascaded RefineNet structure. The generated images are semantically consistent using a pre-trained CLIP model. Adaptive detail enhancement and noise suppression are applied to the verified images. Finally, based on the semantic segmentation results, intelligent fusion images are output by generating image features in important regions and preserving the original infrared image features in background regions.

2. The method for converting an infrared image into a visible light image as described in claim 1, characterized in that, Obtain unpaired infrared and visible light image datasets and process them separately. Infrared images are subjected to radiation normalization, and visible light images are subjected to illumination equalization.

3. The method for converting an infrared image into a visible light image as described in claim 2, characterized in that, The processed image is input into a dual-branch feature decoupling network to extract shared and specific features. The dual-branch feature decoupling network includes a shared feature extraction branch for extracting common semantic features from infrared and visible light images; The infrared feature extraction branch includes a differentiable thermodynamic calculation layer to preserve thermal radiation properties; The visible light feature extraction branch includes a high-frequency attention module for enhancing texture detail features.

4. The method for converting an infrared image into a visible light image as described in claim 3, characterized in that, After multi-scale fusion of shared and specialized features, the data is input into a progressive upsampling network to first generate a low-resolution visible light image. Then, a cascaded RefineNet structure is used to gradually recover high-resolution details. The specific steps include: The shared features are combined with infrared-specific features and visible light-specific features according to the channel dimension to form a fused feature; The fused features are sequentially input into three cascaded multi-scale fusion modules. Each module first extracts local features through 3×3 convolution, and then performs skip connections to fuse with the high-resolution features output from the previous layer. The features output from the last-stage multi-scale fusion module are input into the initial generator to generate a low-resolution visible light image. The low-resolution image is sequentially input into three cascaded RefineNet modules. Each RefineNet module combines high-frequency attention features of the corresponding scale through residual connections to gradually restore image details and improve resolution. The final output is a high-resolution visible light image.

5. The method for converting an infrared image into a visible light image as described in claim 4, characterized in that, The final output is a high-resolution visible light image, in the middle: The output resolution of each RefineNet module is twice the input resolution.

6. The method for converting an infrared image into a visible light image as described in claim 4, characterized in that, The generated images are semantically consistent using a pre-trained CLIP model. The specific steps include: The generated image and the input infrared image are respectively input into the image encoder of the CLIP model to extract image feature vectors. The cosine similarity between the two feature vectors in the semantic space is calculated as the semantic consistency score. When the semantic consistency score is lower than the preset threshold, the parameters of the generating network are adjusted and the image is regenerated until the semantic consistency score meets the requirements. The CLIP model uses pre-trained weights with fixed parameters, and the image encoding process remains end-to-end differentiable.

7. The method for converting an infrared image into a visible light image as described in claim 6, characterized in that, Adaptive detail enhancement and noise suppression are performed on the verified images. Finally, based on the semantic segmentation results, intelligent fusion images are output by generating image features in important regions and preserving the original infrared image features in background regions. The specific steps include: The generated image that passes semantic verification is input into the detail enhancement network. The detail enhancement network establishes an adaptive filtering kernel by analyzing the noise distribution of the infrared image, and at the same time uses the texture feature map extracted by the high-frequency attention module to guide the detail enhancement, generating an optimized visible light image. The optimized visible light image and the original infrared image are input into the semantic segmentation network to obtain the mask of important regions; The optimized visible light image and the original infrared image are fused pixel-level based on the important region mask, and the final output is a fused image that retains the visible light details of the important region and the infrared features of the background region.

8. A system for converting an infrared image into a visible light image, applicable to the method for converting an infrared image into a visible light image as described in claim 1, characterized in that, It includes a processing module, a feature extraction module, a generation module, a verification module, and an output module. The feature extraction module is connected to the processing module, the generation module is connected to the feature extraction module, the verification module is connected to the generation module, and the output module is connected to the verification module. The processing module is used to acquire unpaired infrared and visible light image datasets and process them separately. The feature extraction module is used to input the processed image into the dual-branch feature decoupling network to extract shared features and specific features; The generation module is used to fuse shared features and special features at multiple scales and then input them into the progressive upsampling network to first generate a low-resolution visible light image, and then gradually restore high-resolution details through a cascaded RefineNet structure. The verification module is used to perform semantic consistency verification on the generated images using a pre-trained CLIP model. The output module is used to perform adaptive detail enhancement and noise suppression on the verified images. Finally, based on the semantic segmentation results, it outputs an intelligent fused image by generating image features in important regions and preserving the original infrared image features in background regions.

Images