An infrared image edge feature guided visible light image defogging method

Abstract

The application discloses an infrared image edge feature guided visible light image defogging method, comprising the following steps: constructing a fusion defogging network, wherein the fusion defogging network comprises a first encoder, a first decoder, a second encoder, a channel attention fusion module, a second decoder and an edge extraction module; training the fusion defogging network by adopting a phased training strategy; in the first phase, inputting an infrared image into an infrared image input end, inputting an edge map extracted by the edge extraction module based on a clear visible light image as true value label data, and training the first encoder; in the second phase, inputting a fog-containing visible light image into a fog-containing image input end, inputting an infrared image into an infrared image input end, inputting a clear visible light image as true value label data, and training the whole fusion defogging network. The application makes full use of clear edge structure information provided by an infrared image under foggy weather conditions, and can significantly improve the defogging effect of the fog-containing visible light image.

Description

Technical Field

[0001] This invention relates to the fields of image processing and target detection technology, and specifically to a visible light image dehazing method guided by infrared image edge features. Background Technology

[0002] Image dehazing is a crucial task in computer vision, aiming to restore clear scenes from visible light images affected by haze. Due to the scattering of light from the atmosphere, haze reduces image visibility and causes color distortion. Efficient dehazing techniques are of great significance for applications such as autonomous driving, remote sensing, outdoor monitoring, and transportation, and also provide strong support for downstream vision tasks such as target detection, recognition, and tracking. Therefore, image dehazing methods have attracted widespread attention.

[0003] Early image dehazing methods primarily relied on manually designed prior knowledge and explicit physical models. These methods typically relied on domain-specific assumptions, such as dark channel priors, color attenuation models, and fog line models, to estimate the key parameters required for dehazing. However, these methods were generally computationally expensive and had poor generalization ability across different scenes and hazy conditions.

[0004] With the development of deep learning, data-driven image dehazing methods have emerged, demonstrating high efficiency by learning the mapping from hazy images to clear images on large-scale datasets. However, most of these methods are based on a single modality, relying solely on the hazy visible light image as input for direct restoration. This single-modal deep learning process depends entirely on feature extraction from the hazy image itself. Even when combined with attention mechanisms or feature enhancement strategies, it is difficult to fully perceive or restore the severely obscured structural and edge information, especially in dense fog scenes, easily resulting in dehazing artifacts, loss of detail, or even dehazing failure. Therefore, single-modal methods exhibit significant limitations when facing dense fog conditions, urgently requiring the introduction of more supplementary information to enhance the robustness and accuracy of dehazing.

[0005] To overcome the bottlenecks of single-modal methods, multimodal fusion offers a promising direction. Infrared images, due to their longer wavelength and stronger haze penetration, provide more robust structural information than visible light images under hazy conditions. Therefore, introducing infrared modes is expected to significantly improve the dehazing effect of visible light images. However, most existing visible-infrared fusion methods primarily focus on all-day visual perception tasks, with relatively little research specifically for visible light dehazing. Furthermore, many visible-infrared fusion techniques aim to mix the brightness or texture of visible light images with the thermal information of infrared images to generate a synthetic image with complete content. These methods aim to generate cross-modal fused images, which differs fundamentally from infrared-guided visible light dehazing tasks: the latter aims to utilize infrared images as prior data to assist in recovering a clear visible light image from a hazy visible light image, meaning the final result is still in the visible light modality.

[0006] VIFNet is an infrared-guided visible light image dehazing method. While this method has achieved higher dehazing performance than single-modal image dehazing by leveraging the advantages of infrared images in foggy conditions, it suffers from a fundamental drawback: it employs a general dual-modal fusion structure, directly fusing infrared and visible light features, and ultimately using the clear visible light image as a supervisory signal for reconstruction and regression. This approach easily introduces infrared modal differences and redundant information, limiting the quality of the restored visible light image. Therefore, further exploration of more effective infrared-guided visible light dehazing methods is needed.

[0007] Currently, the utilization of features from visible light and infrared images focuses on texture and color information (visible light) and thermal information (infrared), respectively. Although some researchers have proposed using infrared images to extract edge features for applications, in the field of image dehazing, hastily combining edge features extracted from infrared images with other features extracted from visible light images can lead to feature conflicts and artifact enhancement problems due to semantic differences between modalities and inconsistent feature distributions. Summary of the Invention

[0008] To address the fusion distortion problem caused by modal differences in hazy image processing, this invention aims to provide a visible light image dehazing method guided by infrared image edge features. A novel fusion dehazing network, FusionDehazeNet, is proposed. Through segmented training, it fully utilizes important edge feature information in infrared images, and through joint learning, it improves haze recovery and suppresses redundant information, thereby significantly enhancing the dehazing effect of hazy visible light images.

[0009] To achieve the above-mentioned technical objectives, the technical solution adopted by the present invention is as follows:

[0010] A visible light image dehazing method guided by infrared image edge features, the method comprising the following steps:

[0011] S1. Construct the training dataset: ,in This is a visible light image with a hazy background. Infrared image, To obtain clear visible light images, all three images underwent registration preprocessing to align the content;

[0012] S2, construct the FusionDehazeNet network. The FusionDehazeNet network includes an infrared image input terminal, a foggy image input terminal, a first encoder (IREncoder), a first decoder (IRDecoder), a second encoder (VisEncoder), a channel attention fusion module, a second decoder (FusionDecoder), an edge map output terminal, a dehaze map output terminal, and an edge extraction module. The input terminal of the first encoder (IREncoder) is connected to the infrared image input terminal, and its output terminal is connected to the input terminal of the first decoder (IRDecoder) and the first input terminal of the channel attention fusion module, respectively. The output terminal of the first decoder (IRDecoder) is connected to the edge map output terminal. The input terminal of the second encoder (VisEncoder) is connected to the foggy image input terminal, and its output terminal is connected to the fusion module. The second input terminal of the fusion module; the output terminal of the channel attention fusion module is connected to the dehazed image output terminal through the second decoder FusionDecoder; the clear visible light image serves as the supervision signal for the dehazed image output terminal; the clear visible light image serves as the supervision signal for the edge image output terminal through the edge extraction module; the first encoder IREncoder extracts infrared edge features from the input infrared image and imports them into the first decoder IRDecoder to reconstruct the infrared edge image; the second encoder VisEncoder extracts visible light features from the input hazy visible light image; the channel attention fusion module fuses the infrared edge features extracted by the first encoder IREncoder and the visible light features extracted by the second encoder VisEncoder to obtain fused features, which are then reconstructed by the second decoder FusionDecoder to obtain the dehazed image;

[0013] S3 employs a phased training strategy to train the FusionDehazeNet network; in the first phase, infrared images are used... The edge extraction module uses a clear visible light image as the input infrared image input terminal. Extracted edge map The ground truth label data, used as the output of the edge map, is used to train the first encoder, IREncoder; in the second stage, the hazy visible light image is used... Input the foggy image to the input terminal, and input the infrared image. Input infrared image input terminal to get clear visible light image As the ground truth label data at the output of the dehazed map, the edge map output by the edge extraction module is used. The ground truth label data, used as the output of the edge map, is used to train the entire FusionDehazeNet network.

[0014] S4. Transfer the newly acquired hazy visible light image With infrared images Input the trained FusionDehazeNet network to output a dehazed visible light image. .

[0015] Furthermore, the first phase of training adopts the following scheme:

[0016] An edge extraction module is used, based on clear visible light images. Extracting edge maps This is used as the truth label data at the output of the edge map:

[0017] ;

[0018] In the formula, For image edge extraction functions, when the image is clearly visible... When the size is H×W×3, the resulting edge map The dimension is H×W×1;

[0019] infrared images Infrared edge features are obtained by inputting the first encoder IREncoder. :

[0020] ;

[0021] in, The dimension is H×W×1. The network layer number of the first encoder IREncoder is in the range [1,5], and the dimension is... , , and They were respectively in the second The height, width, and number of channels of the feature map extracted from the layer;

[0022] infrared edge features Inputting the first decoder IRDecoder completes edge map reconstruction, resulting in:

[0023] ;

[0024] in The reconstructed infrared edge map, aligned with the visible light edge, has dimensions of H×W×1.

[0025] Based on edge map and the reconstructed infrared edge map The first loss function targeting infrared edge features is adopted. Train the first encoder IREncoder to generate infrared edge features that are closer to the edges of clear visible light.

[0026] Furthermore, the second phase of training adopts the following scheme:

[0027] Clear visible light images As the ground truth label data at the output of the dehazed map, the edge map output by the edge extraction module is used. As the truth label data at the output of the edge map;

[0028] infrared images Input the first encoder IREncoder to convert the hazy visible light image The infrared edge map is reconstructed by inputting the second encoder, VisEncoder, and using the IRDecoder and FusionDecoder in the FusionDehazeNet fusion dehazing network. and the visible light image after defogging. ;

[0029] The FusionDehazeNet network is trained using a joint loss function:

[0030] ;

[0031] in, Denotes the joint loss function. and Let each represent the first loss function for infrared edge features. and a second loss function for hazy visible light characteristics The weight.

[0032] Preferably, the first loss function Represented as:

[0033] ;

[0034] in Represents the pixel number of the image. Decimals were introduced to prevent the denominator from being zero. For summation operations.

[0035] The second loss function Represented as:

[0036] ;

[0037] in, represent The weighting coefficients, The L1 loss function is expressed as:

[0038] ;

[0039] in This indicates the defogging prediction result. Represents a clear visible light image. Represents the pixel number of the image. This represents the total number of pixels in the image. represent The weighting coefficients, The structural similarity loss is expressed as:

[0040] ;

[0041] Where SSIM represents the structural similarity function; represent The weighting coefficients, To represent perceived loss, it is:

[0042] ;

[0043] in, Indicates the number of VisEncoder's... Feature mapping extracted from layers, This represents the feature layer.

[0044] Further, in step S2, the process of fusing the infrared edge features extracted by the first encoder IREncoder and the visible light features extracted by the second encoder VisEncoder to obtain fused features, and then reconstructing the fused features by the second decoder FusionDecoder to obtain the dehazed image, includes the following steps:

[0045] infrared images Infrared edge features are obtained by inputting the first encoder IREncoder. :

[0046] ;

[0047] In the formula, The dimension is H×W×1. The network layer number of the first encoder IREncoder is in the range [1,5], and the dimension is... , , and They were respectively in the second The height, width, and number of channels of the feature map extracted from the layer;

[0048] Hazy visible light images Inputting the second encoder, VisEncoder, extracts the hazy visible light features:

[0049] ;

[0050] In the formula, The network layer number of the second encoder, VisEncoder, ranges from [1,5], when the visible light image contains fog. When the dimensions are H×W×3, In the first Features extracted from hazy visible light images by layer, with dimensions of , , and The first The height, width, and number of channels of the layer feature map;

[0051] The channel attention fusion module is used to integrate the features of hazy visible light. With infrared edge features By fusing, we obtain:

[0052] ;

[0053] In the formula, In the first Layer fusion features, dimension is , For channel attention function, For feature splicing operations;

[0054] The second decoder, FusionDecoder, uses a deep-to-shallow layer-by-layer upsampling decoding method to... The process involves using skip connections to stitch together and upsample the features from the previous decoding layer, the fused features from the corresponding layer of the channel attention fusion module, and the foggy visible light features from the corresponding layer of the second encoder. In the deep decoding stage, the semantic and shape information of the foggy visible light image is fused with the overall thermal distribution features of the infrared image to enhance the global structural understanding of the scene and the semantic integrity of objects. In the shallow decoding stage, the clear thermal radiation edge structure information of the infrared image is used to guide the detail recovery of the visible light image, enhancing its color and texture representation. Finally, a visible light image with clear structure and realistic colors is reconstructed.

[0055] ;

[0056] In the formula, This represents the visible light image after dehazing, with dimensions of H×W×3.

[0057] Furthermore, the second encoder, VisEncoder, uses a VGG16 network as a feature extraction network to extract features from the input hazy visible light image.

[0058] Furthermore, the channel attention fusion module includes an aggregation unit, an adaptive average pooling unit, a multilayer perceptron unit, a sigmoid activation function unit, and a fusion unit;

[0059] The aggregation unit concatenates the five layers of visible light features and infrared edge features in the channel dimension to form an original fused feature map; the adaptive average pooling unit compresses the spatial information of each channel in the original fused feature map to generate a channel descriptor, and sends the channel descriptor to the multilayer perceptron unit.

[0060] The multilayer perceptron unit includes a first fully connected layer, a ReLU activation function unit, and a second fully connected layer connected in sequence. The first fully connected layer reduces the dimensionality of the channels of the channel descriptor. The dimensionality reduction result is processed by the ReLU activation function unit and then restored by the second fully connected layer. The Sigmoid activation function unit processes the output of the second fully connected layer to generate a channel importance weight vector.

[0061] The fusion unit performs element-wise multiplication of the channel importance weight vector with the original fused feature map, integrating the color and weak texture information of visible light features and the structural information of infrared edge features, and outputs a five-layer deep fused feature map that includes the advantages of both modes.

[0062] Further, in step S2, the second decoder, FusionDecoder, adopts a U-shaped architecture and reconstructs a clear visible light image through a layer-by-layer upsampling decoding method from deep to shallow, including a first decoding layer, a second decoding layer, a third decoding layer, a fourth decoding layer, and a decoding output layer; the calculation formula of the second decoder, FusionDecoder, and the formula for decoding the dehazed visible light image are expressed as follows:

[0063] ;

[0064] ;

[0065] in, This refers to the number of network layers in the second decoder, FusionDecoder. The range is [1,5]. The l-th layer features are output by the second decoder. The output of the second decoder Layer features, and Corresponding to the Layer fusion features and hazy visible light images, For feature splicing operations, This is an upsampling operation; This represents the processing of the decoded output convolutional layer. This represents the activation function layer processing.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0067] First, compared with existing single-mode techniques, which rely on the weak features of the visible light image itself to reconstruct the dehazed image, this invention guides the dehazing of the visible light image based on infrared image. It makes full use of the physical property that infrared images can still maintain clear structural contour features in hazy weather, breaks through the performance bottleneck of single-mode dehazing methods under dense fog conditions, and significantly improves the dehazing effect of visible light images.

[0068] Second, compared to the common one-time fusion strategy in existing multimodal technologies, this invention proposes a multimodal fusion dehazing method based on infrared edge guidance and staged training. First, an infrared-to-visible-light edge feature encoder is trained independently to extract edge contour information from the infrared image. Then, it is jointly trained and fine-tuned with a visible-light dehazing network to achieve complementary fusion of infrared edge features and color features of the hazy image. This effectively reduces information redundancy caused by one-time fusion, thereby generating a high-quality dehazed image with clearer contours and fewer artifacts. Attached Figure Description

[0069] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0070] Figure 2 A schematic diagram of the overall structure of the FusionDehazeNet dehazing network;

[0071] Figure 3 This is a schematic diagram of the channel attention fusion module.

[0072] Figure 4 A schematic diagram illustrating the defogging effect of the FusionDehazeNet network under different fog concentrations. Detailed Implementation

[0073] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0074] See Figure 1 This invention discloses a method for dehazing visible light images guided by infrared image edge features, the method comprising the following steps:

[0075] S1. Construct the training dataset: ,in This is a visible light image with a hazy background. Infrared image, To obtain clear visible light images, all three images underwent registration preprocessing to align the content;

[0076] S2, construct the FusionDehazeNet network. The FusionDehazeNet network includes an infrared image input terminal, a foggy image input terminal, a first encoder (IREncoder), a first decoder (IRDecoder), a second encoder (VisEncoder), a channel attention fusion module, a second decoder (FusionDecoder), an edge map output terminal, a dehaze map output terminal, and an edge extraction module. The input terminal of the first encoder (IREncoder) is connected to the infrared image input terminal, and its output terminal is connected to the input terminal of the first decoder (IRDecoder) and the first input terminal of the channel attention fusion module, respectively. The output terminal of the first decoder (IRDecoder) is connected to the edge map output terminal. The input terminal of the second encoder (VisEncoder) is connected to the foggy image input terminal, and its output terminal is connected to the fusion module. The second input terminal of the fusion module; the output terminal of the channel attention fusion module is connected to the dehazed image output terminal through the second decoder FusionDecoder; the clear visible light image serves as the supervision signal for the dehazed image output terminal; the clear visible light image serves as the supervision signal for the edge image output terminal through the edge extraction module; the first encoder IREncoder extracts infrared edge features from the input infrared image and imports them into the first decoder IRDecoder to reconstruct the infrared edge image; the second encoder VisEncoder extracts visible light features from the input hazy visible light image; the channel attention fusion module fuses the infrared edge features extracted by the first encoder IREncoder and the visible light features extracted by the second encoder VisEncoder to obtain fused features, which are then reconstructed by the second decoder FusionDecoder to obtain the dehazed image;

[0077] S3 employs a phased training strategy to train the FusionDehazeNet network; in the first phase, infrared images are used... The edge extraction module uses a clear visible light image as the input infrared image input terminal. Extracted edge map The ground truth label data, used as the output of the edge map, is used to train the first encoder, IREncoder; in the second stage, the hazy visible light image is used... Input the foggy image to the input terminal, and input the infrared image. Input infrared image input terminal to get clear visible light image As the ground truth label data at the output of the dehazed map, the edge map output by the edge extraction module is used. The ground truth label data, used as the output of the edge map, is used to train the entire FusionDehazeNet network.

[0078] S4. Transfer the newly acquired hazy visible light image With infrared images Input the trained FusionDehazeNet network to output a dehazed visible light image. .

[0079] Example

[0080] Figure 2 This is a schematic diagram of the overall structure of the FusionDehazeNet dehazing network. The principles of each step of this invention are explained below with specific examples. The numerical settings such as dataset partitioning ratio and image resolution are only for illustrative purposes and are not technical limitations.

[0081] (a) Training dataset

[0082] Building the training dataset: ,in This is a visible light image with fog (3 channels). For infrared images (1 channel), and The images are clear visible light images (3 channels); the three images undergo registration preprocessing to ensure content alignment, and then... As a bimodal input to the dehazing model, The dataset was randomly divided into training and validation sets in an 8:2 ratio, serving as ground truth labels for the dehazing model. All images in the dataset had a resolution of 256×256.

[0083] (II) Constructing the FusionDehazeNet network

[0084] A FusionDehazeNet network is constructed, which includes an infrared image input terminal, a foggy image input terminal, a first encoder IREncoder, a first decoder IRDecoder, a second encoder VisEncoder, a channel attention fusion module, a second decoder FusionDecoder, an edge map output terminal, a dehazed map output terminal, and an edge extraction module.

[0085] The first encoder IREncoder and the first decoder IRDecoder are interconnected and adopt a U-shaped structure to use infrared images. As input, a clear visible light image The edge map is extracted using the Canny edge extraction algorithm. The data, with a size of 256×256×1, serves as the ground truth label data at the output of the first decoder IRDecoder to achieve the first stage of the training process.

[0086] Specifically, the first encoder, IREncoder, contains five convolutional blocks. The first convolutional block contains a two-dimensional convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1. After the input data of 256×256×1 passes through this module, the number of channels changes from 1 to 64, and the size of the output feature map is 256×256×64. The second convolutional block contains a two-dimensional convolutional layer with a kernel size of 3×3, a stride of 2, and padding of 1, and the size of the output feature map is 128×128×128. The third convolutional block contains a convolutional layer with a kernel size of 3×3, a stride of 2, and padding of 1, and the size of the output feature map is 64×64×256. The fourth convolutional block contains a convolutional layer with a kernel size of 3×3, a stride of 2, and padding of 1, and the size of the output feature map is 32×32×512. The fifth convolutional block contains a convolutional layer with a kernel size of 3×3, a stride of 2, and padding of 1, and the output feature map has a size of 16×16×512.

[0087] The first decoder, IRDecoder, consists of 5 layers, including 4 deconvolutional blocks and 1 output convolutional layer. The first deconvolutional block contains only a single 2D transposed convolutional layer with a 2×2 kernel, a stride of 2, and 0 padding. The input feature map size is 16×16×512, and the output size is 32×32×512. In the second deconvolutional block, the 32×32×512 feature output from the previous step is first concatenated with the 32×32×512 feature skipped by the encoder along the channel dimension, forming a fused feature of size 32×32×1024. This fused feature is then processed by a convolutional block with a 3×3 kernel, a stride of 1, and 1 padding, resulting in an output feature size of 32×32×256. Finally, the 32×32×256 feature is passed through a 2D transposed convolutional layer with a 2×2 kernel, a stride of 2, and 0 padding, resulting in a final output feature size of 64×64×256. The third deconvolutional block includes a channel concatenation operation, a convolutional block, and a 2D transposed convolutional layer. First, the 64×64×256 feature output from the previous step is concatenated with the 64×64×256 feature skipped by the encoder, forming a fused feature of size 64×64×512. This feature is processed by a convolutional block (3×3 kernel, stride of 1, padding of 1), resulting in an output feature size of 64×64×128. Then, this feature is passed through a 2D transposed convolutional layer (2×2 kernel, stride of 2, padding of 0), resulting in a final output feature size of 128×128×128. In the fourth deconvolution block, the 128×128×128 feature output from the previous step is first concatenated with the 128×128×128 feature skipped by the encoder, forming a fused feature of size 128×128×256. This feature is then processed by a convolution block (3×3 kernel, stride 1, padding 1), outputting a feature size of 128×128×64. Subsequently, this feature is passed through a 2D transposed convolutional layer (2×2 kernel, stride 2, padding 0), resulting in a final output feature size of 256×256×64. In the final output convolutional layer, the 256×256×64 feature output from the previous step is first concatenated with the 256×256×64 feature from the shallowest layer of the encoder, forming a final fused feature of size 256×256×128. The feature is finally processed by a two-dimensional convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1, outputting an image with a final size of 256×256×1. The image is then processed by a Sigmoid activation function, which maps the pixels of the final 256×256×1 image to the [0,1] interval, outputting the edge image.

[0088] Hazy visible light images Infrared image as input to the second encoder VisEncoder end As input to the first encoder IREncoder, a clear visible light image is generated. As the ground truth label data at the output of the second decoder, FusionDecoder, the FusionDehazeNet network is trained to generate a dehazed visible light image from a hazy visible light image, and effectively integrates the infrared edge features provided by the first encoder, IREncoder, to improve the dehazing quality.

[0089] The second encoder, VisEncoder, serves as the visible light feature extraction branch. It uses several layers from the VGG16 backbone network as feature extraction layers. After a series of convolutions and pooling operations by VGG16, it outputs five layers of visible light features with dimensions of 256×256×64, 128×128×128, 64×64×256, 32×32×512, and 16×16×512. These features contain color and subtle texture information from the visible light image, providing a foundation for subsequent cross-modal fusion and dehazing.

[0090] The structure of the channel attention fusion module is as follows: Figure 3 As shown, the channel attention fusion module's processing flow begins by concatenating the two input feature maps (assuming both are 16×16×512 visible light features and edge features) along the channel dimension, forming a fused feature of size 16×16×1024. This fused feature then enters the processing flow within the channel attention fusion module: First, the adaptive average pooling unit compresses the spatial information of each channel into a single value, transforming the feature map size from 16×16×1024 into a channel descriptor. Next, this descriptor is fed into a multilayer perceptron (MLP) unit, which contains two fully connected layers: the first fully connected layer reduces the channel dimension from 1024 to 1024 / r (where r is the dimensionality reduction coefficient) and is processed by the ReLU activation function unit; the second fully connected layer restores the dimension to 1024 and is finally processed by the Sigmoid activation function unit to generate a channel importance weight vector. Finally, this weight vector is reshaped to 1024×1×1 and element-wise multiplied with the stitched original fused feature map. Through this process, the channel attention fusion module utilizes the weights learned by the attention mechanism to more effectively integrate the texture information of visible light features and the structural information of infrared edge features, ultimately outputting a deep fusion feature with a size of 16×16×1024 that incorporates the advantages of both modalities. This deep fusion feature also contains five layers of features, from shallowest to deepest: the first layer of deep fusion features, the second layer of deep fusion features, the third layer of deep fusion features, the fourth layer of deep fusion features, and the fifth layer of deep fusion features.

[0091] The second decoder, FusionDecoder, employs a U-shaped architecture, comprising four decoding layers and one decoding output layer. In the first decoding layer, the deepest fused feature layer, with a size of 16×16×1024, is first fed into an upsampling block. Within this block, the feature map's spatial resolution is first magnified by a factor of two to 32×32×1024 using bilinear interpolation. Then, it passes through a convolutional block with a kernel size of ×3, a stride of 1, and padding of 1, reducing the number of channels from 1024 to 512, ultimately outputting a feature map with a size of 32×32×512. In the second decoding layer, the 32×32×512 feature output from the previous step is concatenated along the channel dimension with the fused feature of 32×32×1024 from the corresponding layer and the visible light feature of 32×32×512, which serve as skip connections, forming a fused feature with a size of 32×32×2048. The feature is processed by a convolutional block (3×3 kernel, stride 1, padding 1), compressing its channel count to 256 and outputting a size of 32×32×256. Subsequently, this feature is passed through a separate upsampling block, first amplified to 64×64×256 by bilinear interpolation, and then processed by the convolutional block, finally outputting a feature map of size 64×64×256. Next, in the third decoding layer, the process is similar to the previous stage, concatenating the 64×64×256 feature with the corresponding layer's fused feature (64×64×512) and visible light feature (64×64×256) as skip connections, forming a fused feature of size 64×64×1024. This feature is first processed by a convolutional block, reducing the channel count to 128 and changing its size to 64×64×128; then processed by an upsampling block, finally outputting a feature map of size 128×128×128. Subsequently, in the fourth decoding layer, the 128×128×128 feature is concatenated with the corresponding layer's fused feature of 128×128×256 and the visible light feature of 128×128×128, which serve as skip connections, to form a fused feature of 128×128×512. This feature is first processed by a convolutional block, reducing the number of channels to 64 and changing its size to 128×128×64; then it is processed by an upsampling block, finally outputting a feature map with a size of 256×256×64, consistent with the resolution of the original image. Finally, in the decoding output layer, the 256×256×64 features from the previous step are concatenated with the 256×256×128 features (serving as skip connections) and the 256×256×64 visible light features to form a 256×256×256 feature map. This feature map is then fed into an output convolutional layer, which uses a 3×3 kernel, a stride of 1, and padding of 1 to map the 256 channels into three output channels (corresponding to the RGB image), generating a 256×256×3 image. This image is finally activated by a sigmoid activation function, completing the entire decoding and reconstruction process.

[0092] (III) Phased Training

[0093] The first phase of training will adopt the following plan:

[0094] An edge extraction module is used, based on clear visible light images. Extracting edge maps This is used as the truth label data at the output of the edge map:

[0095] ;

[0096] infrared images Infrared edge features are obtained by inputting the first encoder IREncoder. :

[0097] ;

[0098] infrared edge features The edge map is reconstructed by inputting the first decoder IRDecoder, resulting in:

[0099] ;

[0100] in The reconstructed infrared edge map, aligned with the visible light edge, has dimensions of H×W×1.

[0101] Based on edge map and the reconstructed infrared edge map The first loss function targeting infrared edge features is adopted. Train the first encoder IREncoder to generate infrared edge features that are closer to the edges of clear visible light.

[0102] The second phase of training will adopt the following plan:

[0103] Clear visible light images As the ground truth label data at the output of the dehazed map, the edge map output by the edge extraction module is used. As the truth label data at the output of the edge map;

[0104] infrared images Input the first encoder IREncoder to convert the hazy visible light image The infrared edge map is reconstructed by inputting the second encoder, VisEncoder, and using the IRDecoder and FusionDecoder in the FusionDehazeNet fusion dehazing network. and the visible light image after defogging. ;

[0105] The FusionDehazeNet network is trained using a joint loss function:

[0106] ;

[0107] in, Denotes the joint loss function. and Let each represent the first loss function for infrared edge features. and a second loss function for hazy visible light characteristics The weight.

[0108] The training parameters for the first stage are: batch size of 2, training epochs of 100, initial learning rate of 0.0001, and the optimizer is Adam. The model is optimized using the first loss function (Dice loss function), and the specific formula is as follows:

[0109]

[0110] in Represents the pixel number of the image. Decimals were introduced to prevent the denominator from being zero. For the summation operation, this stage of training enables the first encoder, IREncoder, to generate features that are closer to the edges of clear visible light.

[0111] The second stage of training uses joint training. The joint training parameter setting process includes: loading the weights of the first encoder IREncoder, loading the ImageNet pre-trained weights into the VGG16 branch of the second encoder VisEncoder; the batch size is 2, the training epochs are 100, the initial learning rate is 0.0001, and the optimizer is Adam.

[0112] The second stage of training uses a joint loss function for optimization: the joint loss function includes L1 loss, SSIM loss, perceptual loss, and Dice loss, and the formula is as follows: ;in (N is the total number of pixels, weight) ), (weight) ), ( For VGG16 Layer feature mapping, weights ), The first loss function (weights are) Every 10 rounds, PSNR and SSIM (to evaluate dehazing performance) are calculated on the validation set, and the model weight with the highest PSNR is saved as the final fusion dehazing network, FusionDehazeNet. The process of performance validation of the dehazing effect and evaluating the model's robustness under different fog concentrations specifically includes:

[0113] (1) Selection of evaluation indicators: Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are used as quantitative indicators. PSNR measures pixel-level distortion in dB, and the higher the value, the better. SSIM measures structural consistency in the range of [-1,1], and the closer the value is to 1, the better.

[0114] (2) Set up comparative experiments: Select several mainstream dehazing methods for comparison, including single-modal non-deep learning methods, such as dark channel dehazing algorithm (DCP), single-modal deep learning methods, such as AOD-Net, FFA-Net, UCL-Dehaze, etc., and multi-modal deep learning methods, such as VIFNet. All methods are trained and tested on the visible light subset (single-modal method) or the complete subset (multi-modal method) of the AirSim-VID dataset.

[0115] The visible light image and infrared image of the foggy day to be defogged are input into the FusionDehazeNet network, which has been trained, to generate the defogged visible light image. Figure 4 A schematic diagram illustrating the defogging effect of the FusionDehazeNet network under different fog concentrations.

[0116] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0117] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for dehazing visible light images guided by infrared image edge features, characterized in that, The method includes the following steps: S1. Construct the training dataset: ,in This is a visible light image with a hazy background. Infrared image, To obtain clear visible light images, all three images underwent registration preprocessing to align the content; S2. Construct a FusionDehazeNet network. The FusionDehazeNet network includes an infrared image input terminal, a foggy image input terminal, a first encoder (IREncoder), a first decoder (IRDecoder), a second encoder (VisEncoder), a channel attention fusion module, a second decoder (FusionDecoder), an edge map output terminal, a dehaze map output terminal, and an edge extraction module. The input terminal of the first encoder (IREncoder) is connected to the infrared image input terminal, and its output terminal is connected to the input terminal of the first decoder (IRDecoder) and the first input terminal of the channel attention fusion module, respectively. The output terminal of the first decoder (IRDecoder) is connected to the edge map output terminal. The input terminal of the second encoder (VisEncoder) is connected to the foggy image input terminal, and its output terminal is connected to the fusion module. The second input terminal of the fusion module; the output terminal of the channel attention fusion module is connected to the dehazed image output terminal through the second decoder FusionDecoder; the clear visible light image serves as the supervision signal for the dehazed image output terminal; the clear visible light image serves as the supervision signal for the edge image output terminal through the edge extraction module; the first encoder IREncoder extracts infrared edge features from the input infrared image and imports them into the first decoder IRDecoder to reconstruct the infrared edge image; the second encoder VisEncoder extracts visible light features from the input hazy visible light image; the channel attention fusion module fuses the infrared edge features extracted by the first encoder IREncoder and the visible light features extracted by the second encoder VisEncoder to obtain fused features, which are then reconstructed by the second decoder FusionDecoder to obtain the dehazed image; S3. A phased training strategy is adopted to train the FusionDehazeNet network; in the first phase, infrared images are used... The edge extraction module uses a clear visible light image as the input infrared image input terminal. Extracted edge map The ground truth label data, used as the output of the edge map, is used to train the first encoder, IREncoder; in the second stage, the infrared image... Input infrared image input terminal, visible light image with fog Input a hazy image to the input terminal, then input a clear visible light image. As the ground truth label data at the output of the dehazed map, the edge map output by the edge extraction module is used. The ground truth label data, used as the output of the edge map, is used to train the entire FusionDehazeNet network. S4. Transfer the newly acquired hazy visible light image With infrared images Input the trained FusionDehazeNet network to output a dehazed visible light image. ; In step S2, the infrared edge features extracted by the first encoder IREncoder and the visible light features extracted by the second encoder VisEncoder are fused to obtain fused features. The process of reconstructing the fused features by the second decoder FusionDecoder to obtain the dehazed image includes the following steps: infrared images Infrared edge features are obtained by inputting the first encoder IREncoder. : ； In the formula, The dimension is H×W×1. The network layer number of the first encoder IREncoder is in the range [1,5], and the dimension is... , , and They were respectively in the second The height, width, and number of channels of the feature map extracted from the layer; Hazy visible light images Inputting the second encoder, VisEncoder, extracts the hazy visible light features: ； In the formula, The network layer number of the second encoder, VisEncoder, ranges from [1,5], when the visible light image contains fog. When the dimensions are H×W×3, In the first Features extracted from hazy visible light images by layer, with dimensions of , , and The first The height, width, and number of channels of the layer feature map; The channel attention fusion module is used to integrate the features of hazy visible light. With infrared edge features By fusing, we obtain: ； In the formula, In the first Layer fusion characteristics, The range is [1,5], and the dimension is , For channel attention function, For feature splicing operations; The second decoder, FusionDecoder, uses a deep-to-shallow layer-by-layer upsampling decoding method to... The process involves using skip connections to stitch together and upsample the features from the previous decoding layer, the fused features from the corresponding layer of the channel attention fusion module, and the foggy visible light features from the corresponding layer of the second encoder. In the deep decoding stage, the semantic and shape information of the foggy visible light image is fused with the overall thermal distribution features of the infrared image to enhance the global structural understanding of the scene and the semantic integrity of objects. In the shallow decoding stage, the clear thermal radiation edge structure information of the infrared image is used to guide the detail recovery of the visible light image, enhancing its color and texture representation. Finally, a visible light image with clear structure and realistic colors is reconstructed. ； In the formula, This represents the visible light image after dehazing, with dimensions of H×W×3.

2. The infrared image edge feature-guided visible light image dehazing method according to claim 1, characterized in that, The first phase of training adopts the following scheme: An edge extraction module is used, based on clear visible light images. Extracting edge maps This is used as the truth label data at the output of the edge map: ； In the formula, For image edge extraction functions, when the image is clearly visible... When the size is H×W×3, the resulting edge map The dimension is H×W×1; infrared images Infrared edge features are obtained by inputting the first encoder IREncoder. : ； in, The dimension is H×W×1. The network layer number of the first encoder IREncoder is in the range [1,5], and the dimension is... , , and They were respectively in the second The height, width, and number of channels of the feature map extracted from the layer; infrared edge features Inputting the first decoder IRDecoder completes edge map reconstruction, resulting in: ； in The reconstructed infrared edge map, aligned with the visible light edge, has dimensions of H×W×1. Based on edge map and the reconstructed infrared edge map The first loss function targeting infrared edge features is adopted. The first encoder, IREncoder, is trained to generate infrared edge features that more closely approximate sharp visible light edges; the first loss function... Represented as: ； in Represents the pixel number of the image. Decimals were introduced to prevent the denominator from being zero. For summation operations.

3. The infrared image edge feature-guided visible light image dehazing method according to claim 1, characterized in that, The second phase of training adopts the following scheme: Clear visible light images As the ground truth label data at the output of the dehazed map, the edge map output by the edge extraction module is used. As the truth label data at the output of the edge map; infrared images Input the first encoder IREncoder to convert the hazy visible light image The infrared edge map is reconstructed by inputting the second encoder, VisEncoder, and using the IRDecoder and FusionDecoder in the FusionDehazeNet fusion dehazing network. and the visible light image after defogging. ; The FusionDehazeNet network is trained using a joint loss function: ； in, Denotes the joint loss function. and Let each represent the first loss function for infrared edge features. and a second loss function for hazy visible light characteristics The weights; The second loss function Represented as: ； in, represent The weighting coefficients, The L1 loss function is expressed as: ； in This indicates the defogging prediction result. Represents a clear visible light image. Represents the pixel number of the image. This represents the total number of pixels in the image. represent The weighting coefficients, The structural similarity loss is expressed as: ； Where SSIM represents the structural similarity function; represent The weighting coefficients, To represent perceived loss, it is: ； in, Indicates the number of VisEncoder's... Feature mapping extracted from layers, This represents the feature layer.

4. The visible light image dehazing method guided by infrared image edge features according to claim 1, characterized in that, The second encoder, VisEncoder, uses a VGG16 network as a feature extraction network to extract features from the input hazy visible light image.

5. The infrared image edge feature-guided visible light image dehazing method according to claim 1, characterized in that, The channel attention fusion module includes an aggregation unit, an adaptive average pooling unit, a multilayer perceptron unit, a sigmoid activation function unit, and a fusion unit. The aggregation unit concatenates the five layers of visible light features and infrared edge features in the channel dimension to form an original fused feature map; the adaptive average pooling unit compresses the spatial information of each channel in the original fused feature map to generate a channel descriptor, and sends the channel descriptor to the multilayer perceptron unit. The multilayer perceptron unit includes a first fully connected layer, a ReLU activation function unit, and a second fully connected layer connected in sequence. The first fully connected layer reduces the dimensionality of the channels of the channel descriptor. The dimensionality reduction result is processed by the ReLU activation function unit and then restored by the second fully connected layer. The Sigmoid activation function unit processes the output of the second fully connected layer to generate a channel importance weight vector. The fusion unit performs element-wise multiplication of the channel importance weight vector with the original fused feature map, integrating the color and weak texture information of visible light features and the structural information of infrared edge features, and outputs a five-layer deep fused feature map that includes the advantages of both modes.

6. The visible light image dehazing method guided by infrared image edge features according to claim 1, characterized in that, In step S2, the second decoder, FusionDecoder, adopts a U-shaped architecture and reconstructs a clear visible light image through a layer-by-layer upsampling decoding method from deep to shallow. This includes a first decoding layer, a second decoding layer, a third decoding layer, a fourth decoding layer, and a decoding output layer. The calculation formula for the second decoder, FusionDecoder, and the formula for decoding the dehazed visible light image are expressed as follows: ； ； in, This refers to the number of network layers in the second decoder, FusionDecoder. The range is [1,5]. The l-th layer features are output by the second decoder. The output of the second decoder Layer features, and Corresponding to the Layer fusion features and hazy visible light images, For feature splicing operations, This is an upsampling operation; This represents the processing of the decoded output convolutional layer. This represents the activation function layer processing.

Images