Dehazing method based on frequency information differential fusion
By employing a frequency-information-based differential fusion dehazing method, utilizing a U-shaped encoder-decoder architecture and wavelet transform to decompose the image, combined with high- and low-frequency information differential processing and a staged training strategy, the problem of low efficiency in utilizing frequency information in existing methods is solved, achieving efficient image dehazing and detail restoration.
Patent Information
- Application Number
- CN202511163952.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-20
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2045-08-20
AI Technical Summary
Existing dehazing methods fail to fully utilize the differences in the frequency characteristics of images, resulting in limited dehazing effects. Furthermore, they neglect the processing requirements of high-frequency and low-frequency information at different stages, affecting the visual quality and detail integrity of the images.
A frequency-information-based differential fusion dehazing method is adopted. The image is decomposed into low-frequency and high-frequency components through a U-shaped encoder-decoder architecture and wavelet transform. The high-frequency and low-frequency information differential fusion module and a staged training strategy are combined to process high-frequency and low-frequency information respectively. Feature extraction is enhanced through channel and spatial attention mechanisms. Finally, the network performance is optimized through deep supervision and curriculum learning.
It significantly improves the clarity and detail fidelity of dehazed images, and can achieve higher dehazing quality and stable generalization performance under various levels of fogging and complex scenes.
Smart Images

Figure CN120725904B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a dehazing method based on frequency information differential fusion. Background Technology
[0002] In severe weather conditions such as smog and sandstorms, the air contains a large number of tiny particulate matter. These particles scatter and absorb light, leading to reduced clarity, color shifts, insufficient contrast, and blurred details in images captured by imaging devices. This not only affects the visual experience for the human eye but also adversely impacts intelligent perception systems that rely on image information. Therefore, image dehazing techniques for image restoration and enhancement in such degraded environments have become an important research direction in the field of computer vision.
[0003] Image dehazing, a typical low-level visual processing task, aims to recover a clearer image closer to the real scene from images affected by haze. In hazy environments, light is scattered multiple times by particles during transmission, resulting in reduced image contrast, unnatural colors, and loss of detail. Directly applying these degraded images to high-level visual tasks (such as intelligent surveillance, autonomous driving, and drone navigation) often leads to a significant decrease in recognition accuracy. Image dehazing technology can effectively improve the visualization quality of images and enhance the robustness and accuracy of subsequent computer vision algorithms.
[0004] Image dehazing techniques can be broadly categorized into two types: traditional methods based on manual priors and deep learning-based methods. The former primarily relies on atmospheric scattering models (ASMs) and manually designed prior knowledge. Traditional methods based on manual priors typically recover haze-free images by estimating transmission maps and atmospheric light intensity, combined with an atmospheric scattering model. Their advantage is strong interpretability, but their disadvantage is heavy reliance on prior assumptions, making them prone to failure in complex scenes. In recent years, with the improvement of computing power, the development of deep learning technologies such as convolutional neural networks (CNNs), and the construction of large-scale image dehazing datasets, deep learning-based dehazing methods have gradually become a research hotspot and are replacing traditional methods. Early deep learning dehazing models typically combined atmospheric scattering models with deep learning networks, using the network to predict transmission maps or atmospheric light values, and then reconstructing the image using the atmospheric scattering model. More recent end-to-end methods, however, abandon explicit physical model estimation and directly learn the mapping relationship from hazy to haze-free images. These methods not only maintain high dehazing performance under various weather and lighting conditions but also demonstrate excellent computational efficiency and visual quality.
[0005] Currently, most dehazing methods focus primarily on spatial domain image restoration, failing to fully utilize the frequency characteristics of hazy and hazy images, thus limiting further improvements in dehazing performance. To address this issue, some studies have introduced tools such as Fourier transform and wavelet transform to decompose image features into high-frequency and low-frequency components, and then process these frequency information to reconstruct hazy images. However, existing dehazing networks that incorporate frequency information still have significant limitations. First, existing methods fail to design differentiated processing strategies for high-frequency and low-frequency information, resulting in inefficient utilization of frequency information. For example, high-frequency information typically corresponds to edges and details, while low-frequency information focuses more on overall structure and illumination, but current methods often process both types of frequency information in the same way, failing to fully leverage their respective advantages. Second, the network's dependence on high-frequency and low-frequency information varies at different stages. For example, in the shallow stages of feature extraction, due to the high image resolution, the network can capture more detailed information and therefore relies more on high-frequency information to enhance edge and texture extraction. However, in the deeper stages of feature extraction, as the input image resolution gradually decreases, the network tends to extract global information (such as illumination and overall structure), at which point low-frequency information plays a more prominent role. However, existing methods generally ignore this stage-specific requirement, resulting in a lack of adaptability in frequency information processing. Furthermore, after introducing frequency domain processing, some methods rely too heavily on frequency domain optimization and fail to adequately balance the recovery requirements in the spatial domain, thus affecting the visual quality and detail integrity of the dehazed image. While frequency domain processing can effectively capture global features and local details, neglecting spatial domain recovery may lead to texture distortion or color deviation. Summary of the Invention
[0006] To address the aforementioned technical problems, this invention provides a defogging method based on frequency information differential fusion that offers good defogging performance.
[0007] The technical solution of this invention to solve the above-mentioned technical problems is: a dehazing method based on frequency information differential fusion, comprising the following steps:
[0008] S1: Create a training dataset containing the original fog-free images and the corresponding foggy images;
[0009] S2: Perform data augmentation on the training dataset, including cropping and flipping the input images;
[0010] S3: Construct a defogging network based on frequency information differential fusion;
[0011] The constructed dehazing network employs a U-shaped encoder-decoder architecture for image dehazing; given any hazy image I, I∈R 3×H×WH and W represent the height and width of the image, respectively, and R is the real number domain. The dehazing network first applies 3×3 convolution to generate shallow features of size C×H×W, where C represents the number of channels. Then, the shallow features are processed by three U-shaped feature extraction modules to obtain multi-scale features.
[0012] During the encoding stage, the features are transformed by Discrete Wavelet Transform (DWT) and decomposed into low-frequency components and high-frequency components. The low-frequency and high-frequency components are then fused with the main path features through a high-low frequency information differentiation fusion module.
[0013] During the decoding stage, the features are gradually recovered to high-resolution features through three additional U-shaped feature extraction modules and upsampling to obtain the final output;
[0014] S4: Train the dehazing network using the training dataset until the pre-set loss function converges; combine the loss function and the frequency domain loss as the objective function for network training to obtain the trained dehazing network;
[0015] S5: Input the image to be dehazed into the trained dehazing network for testing and verification to obtain the dehazing result.
[0016] In the above-mentioned dehazing method based on frequency information differential fusion, the operation steps of the U-shaped feature extraction module in step S3 are as follows:
[0017] S311: Perform a 3×3 convolution operation on the input feature Fin to generate an initial feature map Fe1, and enhance the linear representation of the feature using the activation function DyT. The expression for DyT is:
[0018] Fe1 = γ·tanh(α·(Fin)) + β;
[0019] Where γ, α, and β are all learnable parameters; tanh is the hyperbolic tangent function;
[0020] S312: Perform two more 3×3 convolution operations and the activation function DyT on feature Fe1 in sequence. The specific operations are as follows:
[0021] Fe2=DyT·(Conv(Fe1));
[0022] Fe3=DyT·(Conv(Fe2));
[0023] Where Fe2 represents the intermediate output feature, Fe3 represents the encoder output feature, and Conv represents the 3×3 convolution operation;
[0024] S313: In the bottleneck stage, channel attention (CA) and spatial attention (SA) mechanisms are applied sequentially to Fe3 to enhance the selective extraction of key features.
[0025] In the above-mentioned dehazing method based on frequency information differential fusion, the channel attention CA mechanism in step S313 is as follows:
[0026] Global average pooling (GAP) is performed on Fe3 to compress its spatial dimension. Then, it is sequentially processed through 1×1 convolution, GELU activation, 1×1 convolution, and Sigmoid activation to generate a channel attention weight vector (WCA). This WCA is then element-wise multiplied with Fe3 to obtain the enhanced feature Fe4, expressed as:
[0027] Fe4= Fe3·Sigmoid(PWConv(GELU(PWConv(GAP(Fe3)))));
[0028] Where GELU represents the GELU activation function, PWConv represents a 1×1 convolution, and Sigmoid represents the Sigmoid activation function.
[0029] The above-mentioned dehazing method based on frequency information differential fusion, in step S313, the spatial attention (SA) mechanism is as follows: Fe4 is sequentially subjected to 3×3 convolution, GELU activation, 3×3 convolution, and Sigmoid activation to generate a spatial attention weight map (WSA). The spatial attention weight map is then multiplied element-wise with Fe4 to obtain the enhanced feature Fe5, expressed as:
[0030] Fe5= Fe4·Sigmoid (Conv(GELU(Conv(Fe4)))).
[0031] The aforementioned dehazing method based on frequency information differential fusion performs three 3×3 convolution operations on Fe5 in the decoder stage, followed by an upsampling operation after each convolution layer to restore the spatial resolution of the feature map. The specific operations are as follows:
[0032] Fd1 = ↑(DyT(conv(Fe5)));
[0033] Fd2 = ↑(DyT(conv(Fd1)));
[0034] Fd3 = ↑(DyT(conv(Fd2)));
[0035] In this context, ↑ indicates an upsampling operation, Fd1 and Fd2 are intermediate output features, and Fd3 is the decoder output feature.
[0036] The above-mentioned dehazing method based on frequency information differential fusion, in step S3, the high- and low-frequency information differential fusion module contains three inputs and one output, which fuses the high-frequency information HF obtained by wavelet transform, the low-frequency information LL, and the original features RF extracted by the U-shaped feature extraction module; the operation process of the high- and low-frequency information differential fusion module is as follows:
[0037] The high- and low-frequency information differentiation fusion module first performs differential processing on the three received inputs. RF is input to the Convs module for further feature extraction to obtain the enhanced feature RF'. LL is processed through the low-frequency enhancement channel attention block to enhance the global structure and illumination information in the low-frequency features, generating the enhanced feature LL'. HF is fed into the high-frequency information amplification block to enhance the representation ability of the local structure of the image by amplifying the edge and detail information in the high-frequency features, resulting in the final feature HF'.
[0038] LL' is multiplied by the learnable parameter a, HF' is multiplied by 1-a, and the result of the two multiplications is finally added to RF' to generate the final output O; the specific mathematical expression is as follows:
[0039] RF' = Convs(RF);
[0040] LL' = LFECA (LL);
[0041] HF' = HFEB (HF);
[0042] O = RF' + a·LL' + (1-a)·HF';
[0043] Wherein, Convs represents the Convs module, LFECA represents the low-frequency enhancement channel attention block, and HFEB represents the high-frequency amplification block.
[0044] The above-mentioned dehazing method based on frequency information differential fusion has the following operation steps for the low-frequency enhanced channel attention block: First, the low-frequency information LL is extracted independently through a 3×3 channel-wise convolutional layer to focus on the global structure of the low-frequency features; then, the information between channels is integrated through a 1×1 pointwise convolutional layer to further enhance the semantic expression of the features; finally, the low-frequency enhanced channel attention block introduces a channel attention mechanism, extracts the channel-level global information through global average pooling, and generates channel attention weights using two 1×1 convolutions with ReLU activation in between, and normalizes them to [0,1] using the Sigmoid function.
[0045] The above-mentioned dehazing method based on frequency information differential fusion has the following operation steps for the high-frequency information amplification block: First, the high-frequency information amplification block performs a feature shift operation on the input HF, shifting the HF to the lower right by one pixel. The empty edge part after the shift is filled with zeros to generate a shifted feature map. Then, the shifted feature map is subtracted from the HF element by element, and the absolute value of the result is taken to obtain a difference map. Next, the difference map is input into a sequence containing convolution, batch normalization, GELU activation function, and convolution for further processing. After that, the feature is normalized by Sigmoid activation and added to a one matrix, and then multiplied element by element with the HF to obtain the enhanced feature. Finally, the enhanced feature is residually concatenated with the HF after convolution to generate the final feature HF'.
[0046] The aforementioned dehazing method based on frequency information differential fusion, in step S4, introduces a staged training strategy. This strategy combines deep supervision and curriculum learning to optimize the multi-scale feature learning and overall performance of the dehazing network. The staged training strategy divides the training process into four stages, with n rounds as the boundary. The specific steps are as follows:
[0047] In the first stage, for the low-resolution output, the loss is calculated with the corresponding low-resolution real image to guide the network to learn the basic structure and global features of the image first.
[0048] In the second stage, the supervision shifts to the medium-resolution output, and the loss is calculated with the corresponding medium-resolution real image to gradually enhance the ability to recover local textures and details.
[0049] In the third stage, supervision focuses on high-resolution output and calculates loss with the corresponding high-resolution real image to ensure the overall consistency and fine-grained details of the dehazing results.
[0050] In the fourth stage, the loss of all resolution outputs is optimized to enable the network to fully capture multi-scale features.
[0051] The beneficial effects of this invention are as follows: This invention introduces a feature processing mechanism based on frequency-information differential fusion, which can extract global information and detailed texture information from low-frequency and high-frequency components respectively, thereby significantly improving the clarity and detail fidelity of dehazed images. Furthermore, this invention combines a staged training strategy, organically integrating deep supervision and curriculum learning. Following a progressive training approach from low resolution to high resolution and from structure to detail, the network prioritizes mastering the global contour and main structure in the initial stage, gradually strengthening its ability to restore local details and textures in the later stage, and achieving comprehensive optimization of multi-scale features in the final stage. Therefore, this invention effectively avoids the problem of the network falling into overfitting of details or ignoring global information in the early stages of training, thus achieving higher dehazing quality and more stable generalization performance under various levels of fogging and complex scenes. Attached Figure Description
[0052] Figure 1 This is the overall flowchart of the present invention.
[0053] Figure 2 This is a schematic diagram of the defogging network of the present invention.
[0054] Figure 3 This is a schematic diagram of the U-shaped feature extraction module.
[0055] Figure 4 A schematic diagram of the bottleneck design structure for the U-shaped feature extraction module.
[0056] Figure 5 This is a structural diagram of the Convs module.
[0057] Figure 6 This is a schematic diagram of the high- and low-frequency information differentiation fusion module.
[0058] Figure 7 This is a schematic diagram of the attention block structure for the low-frequency enhancement channel.
[0059] Figure 8 This is a schematic diagram of the high-frequency information amplification block.
[0060] Figure 9 This is a schematic diagram of a phased training strategy.
[0061] Figure 10 Three examples of dehazing results are provided for embodiments of the present invention; wherein, (a) is a foggy image, (b), (c), and (d) represent images after dehazing by DCP, FFANet, and GridDehaze methods, respectively, (e) is an image after dehazing by the present invention, and (f) is the original fog-free image. Detailed Implementation
[0062] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0063] like Figure 1 As shown, a dehazing method based on frequency information differential fusion includes the following steps:
[0064] S1: Create a training dataset containing the original fog-free images and the corresponding foggy images.
[0065] S2: Perform data augmentation on the training dataset, including cropping and flipping the input images. Specifically, for the input image, randomly crop it into 256×256 image blocks and perform data augmentation by randomly rotating it by 90°, 180°, or 270°.
[0066] S3: Construct a defogging network based on frequency information differentiation fusion.
[0067] The constructed dehazing network employs a U-shaped encoder-decoder architecture for image dehazing, such as... Figure 2 As shown; given any foggy image I, I∈R 3×H×W H and W represent the height and width of the image, respectively, and R is the real number domain. The dehazing network first applies 3×3 convolution to generate shallow features of size C×H×W, where C represents the number of channels. Then, the shallow features are processed by three U-shaped feature extraction modules to obtain multi-scale features.
[0068] During the encoding stage, features are transformed using Discrete Wavelet Transform (DWT) to decompose into low-frequency and high-frequency components (LH, HL, HH). These components are then fused with the main path features through a high-low frequency information differentiation fusion module. In the decoding stage, features are progressively restored to high-resolution features through three additional U-shaped feature extraction modules and upsampling to obtain the final output. Assuming the input image size is 256×256 pixels, downsampling to 128×128, 64×64, etc., results in low-resolution feature maps, while upsampling to restore the output feature map to 256×256 results in a high-resolution output. During this process, skip connections are used between the encoder and decoder features to assist in image dehazing, and 1×1 convolutions are applied to reduce the number of channels. Furthermore, this invention employs a staged training strategy (PTS), which supervises only lower-resolution outputs in the early training rounds to ensure the network's robustness in learning and training multi-scale features.
[0069] like Figure 3 As shown, the operation steps of the U-shaped feature extraction module are as follows:
[0070] S311: Perform a 3×3 convolution operation on the input feature Fin to generate an initial feature map Fe1, and enhance the linear representation of the feature using the activation function DyT. The expression for DyT is:
[0071] Fe1 = γ·tanh(α·(Fin)) + β;
[0072] Where γ, α, and β are all learnable parameters, and their initial values are all set to 1; tanh is the hyperbolic tangent function;
[0073] S312: Perform two more 3×3 convolution operations and the activation function DyT on feature Fe1 in sequence to gradually reduce the spatial resolution of the feature map, while increasing the channel depth to capture high-level semantic information. The specific operation is as follows:
[0074] Fe2=DyT·(Conv(Fe1));
[0075] Fe3=DyT·(Conv(Fe2));
[0076] Among them, Fe2 is the intermediate output feature, Fe3 is the encoder output feature, which contains high-level semantic information; Conv represents a 3×3 convolution operation;
[0077] S313: In the bottleneck stage, channel attention (CA) and spatial attention (SA) mechanisms are applied sequentially to Fe3 to enhance the selective extraction of key features, such as... Figure 4 As shown, Figure 4 The sigmoid function in the equation is the sigmoid activation function.
[0078] The process of the channel attention CA mechanism is as follows:
[0079] Global average pooling (GAP) is performed on Fe3 to compress its spatial dimension. Then, it is sequentially processed through 1×1 convolution, GELU activation, 1×1 convolution, and Sigmoid activation to generate a channel attention weight vector (WCA). This WCA is then element-wise multiplied with Fe3 to obtain the enhanced feature Fe4, expressed as:
[0080] Fe4= Fe3·Sigmoid(PWConv(GELU(PWConv(GAP(Fe3)))));
[0081] Where GELU represents the GELU activation function, PWConv represents a 1×1 convolution, and Sigmoid represents the Sigmoid activation function.
[0082] The spatial attention (SA) mechanism works as follows: Fe4 is sequentially subjected to 3×3 convolution, GELU activation, 3×3 convolution, and Sigmoid activation to generate a spatial attention weight map (WSA). The spatial attention weight map is then element-wise multiplied with Fe4 to obtain the enhanced feature Fe5, expressed as:
[0083] Fe5= Fe4·Sigmoid (Conv(GELU(Conv(Fe4)))).
[0084] In the decoder stage, Fe5 is subjected to three 3×3 convolution operations, followed by an upsampling operation after each convolution layer to restore the spatial resolution of the feature map. The specific operations are as follows:
[0085] Fd1 = ↑(DyT(conv(Fe5)));
[0086] Fd2 = ↑(DyT(conv(Fd1)));
[0087] Fd3 = ↑(DyT(conv(Fd2)));
[0088] In this context, ↑ indicates an upsampling operation, Fd1 and Fd2 are intermediate output features, and Fd3 is the decoder output feature.
[0089] like Figure 6 As shown, the high- and low-frequency information differentiation fusion module contains three inputs and one output, which fuses the high-frequency information HF obtained by wavelet transform, the low-frequency information LL, and the original features RF extracted by the U-shaped feature extraction module; the operation process of the high- and low-frequency information differentiation fusion module is as follows:
[0090] The high- and low-frequency information differentiation fusion module first performs differential processing on the three received inputs. RF is input to the Convs module for further feature extraction to obtain the enhanced feature RF'. LL is processed through the low-frequency enhancement channel attention block to enhance the global structure and illumination information in the low-frequency features, generating the enhanced feature LL'. HF is fed into the high-frequency information amplification block to enhance the representation ability of the local structure of the image by amplifying the edge and detail information in the high-frequency features, resulting in the final feature HF'.
[0091] LL' is multiplied by the learnable parameter a, HF' is multiplied by 1-a, and the result of the two multiplications is finally added to RF' to generate the final output O; the specific mathematical expression is as follows:
[0092] RF' = Convs(RF);
[0093] LL' = LFECA (LL);
[0094] HF' = HFEB (HF);
[0095] O = RF' + a·LL' + (1-a)·HF';
[0096] Where Convs represents the Convs module, and the result of the Convs module is as follows: Figure 5 As shown; LFECA represents the low-frequency enhancement channel attention block, and HFEB represents the high-frequency amplification block.
[0097] like Figure 7As shown, the low-frequency enhancement channel attention block is a core component in the high-low frequency information differentiation fusion module, specifically designed for low-frequency information enhancement. It aims to strengthen the global structure and illumination information in low-frequency features through lightweight operations, while simultaneously improving computational efficiency to meet the demands of real-time dehazing tasks. The operation steps of the low-frequency enhancement channel attention block are as follows: First, the low-frequency information LL is extracted independently from the spatial features of each channel through a 3×3 channel-wise convolutional layer, focusing on the global structure of the low-frequency features; then, the information between channels is integrated through a 1×1 pointwise convolutional layer to further enhance the semantic expression of the features; finally, the low-frequency enhancement channel attention block introduces a channel attention mechanism, extracting channel-level global information through global average pooling, and generating channel attention weights using two 1×1 convolutions with ReLU activation in between, which are then normalized to [0,1] by the Sigmoid function, thereby adaptively highlighting channels that contribute significantly to the dehazing task, such as features related to global contrast and illumination.
[0098] like Figure 8 As shown, the high-frequency information amplification block is a core component in the high-low frequency information differentiation fusion module, specifically designed to enhance high-frequency information. Its goal is to improve the representation of local image structures by amplifying edge and detail information in high-frequency features, thereby effectively restoring details blurred by haze in image dehazing tasks. The operation steps of the high-frequency information amplification block are as follows: First, the high-frequency information amplification block performs a feature shift operation on the input HF, shifting the HF one pixel to the lower right. The empty edge portion after the shift is filled with zeros, generating a shifted feature map A1. Then, the shifted feature map A1 is subtracted element-wise from the HF, and the absolute value of the result is taken to obtain the difference map A2. Figure 8 In This represents the subtraction of absolute values; next, the difference map is input into a sequence containing convolution, batch normalization, GELU activation function and convolution for further processing; then, the features are normalized by Sigmoid activation and added to the all-one matrix A3, and then multiplied element-wise with HF to obtain the enhanced features; finally, the enhanced features are residually concatenated with the HF that has been convolved to generate the final feature HF'.
[0099] S4: Train the dehazing network using the training dataset until the pre-set loss function converges; combine the loss function and the frequency domain loss as the objective function for network training to obtain the trained dehazing network.
[0100] In step S4, a staged training strategy is introduced. This strategy combines deep supervision and curriculum learning to optimize the multi-scale feature learning and overall performance of the dehazing network. The staged training strategy is divided into n rounds, such as... Figure 9 As shown, the training process is divided into four stages, and the specific steps are as follows:
[0101] In the first stage, for the low-resolution output, the loss is calculated with the corresponding low-resolution real image to guide the network to learn the basic structure and global features of the image first.
[0102] In the second stage, the supervision shifts to the medium-resolution output, and the loss is calculated with the corresponding medium-resolution real image to gradually enhance the ability to recover local textures and details.
[0103] In the third stage, supervision focuses on high-resolution output and calculates loss with the corresponding high-resolution real image to ensure the overall consistency and fine-grained details of the dehazing results.
[0104] In the fourth stage, the loss of all resolution outputs is optimized to enable the network to fully capture multi-scale features.
[0105] S5: Input the image to be dehazed into the trained dehazing network for testing and verification to obtain the dehazing result.
[0106] This invention uses the non-uniform public dataset NH-HAZE for experiments. The images in NH-HAZE were generated using a professional smog camera simulating real smog conditions, with an image resolution of 1600×1200, and contain 55 pairs of real foggy and fog-free images. For the NH-HAZE dataset, this invention uses the last 5 pairs of images as the test set and the remaining images in the dataset as the training set.
[0107] Figure 10 The results of dehazing some images on the test dataset are shown. (a) is a hazy image; (b), (c), and (d) represent the images after dehazing using the Dark Channel Prior (DCP), Feature Fusion Attention Network (FFANet), and Grid Dehaze Network (GridDehaze) methods, respectively; (e) is the image after dehazing using the present invention; and (f) is the original hazy image. It can be seen from the figure that the dehazed images generated by other methods contain artifacts left by incomplete dehazing, while the method proposed in this invention can effectively restore the detailed information of the image and generate a dehazed image that is close to the real image.
Claims
1. A dehazing method based on frequency information differential fusion, characterized in that, Includes the following steps: S1: Create a training dataset containing the original fog-free images and the corresponding foggy images; S2: Perform data augmentation on the training dataset, including cropping and flipping the input images; S3: Construct a defogging network based on frequency information differential fusion; The constructed dehazing network employs a U-shaped encoder-decoder architecture for image dehazing; given any hazy image I, I∈R 3×H×W H and W represent the height and width of the image, respectively, and R is the real number domain. The dehazing network first applies 3×3 convolution to generate shallow features of size C×H×W, where C represents the number of channels. Then, the shallow features are processed by three U-shaped feature extraction modules UFEM to obtain multi-scale features. During the encoding stage, the features are transformed by Discrete Wavelet Transform (DWT) and decomposed into low-frequency and high-frequency components. The low-frequency and high-frequency components are then fused with the main path features by the High-Low Frequency Information Differentiation Fusion Module (HLFDFM). During the decoding stage, the features are gradually recovered to high-resolution features through three additional U-shaped feature extraction modules and upsampling to obtain the final output; The high- and low-frequency information differentiation fusion module contains three inputs and one output, which fuses the high-frequency information HF obtained by wavelet transform, the low-frequency information LL, and the original features RF extracted by the U-shaped feature extraction module. The operation process of the high- and low-frequency information differentiation fusion module is as follows: The high- and low-frequency information differentiation fusion module first performs differential processing on the three received inputs. RF is input to the Convs module for further feature extraction to obtain the enhanced feature RF'. LL is processed through the low-frequency enhancement channel attention block to enhance the global structure and illumination information in the low-frequency features, generating the enhanced feature LL'. HF is fed into the high-frequency information amplification block to enhance the representation ability of the local structure of the image by amplifying the edge and detail information in the high-frequency features, resulting in the final feature HF'. LL' is multiplied by the learnable parameter a, HF' is multiplied by 1-a, and the result of the two multiplications is finally added to RF' to generate the final output O; the specific mathematical expression is as follows: RF' = Convs(RF); LL' = LFECA (LL); HF' = HFEB (HF); O = RF' + a·LL' + (1-a)·HF'; Wherein, Convs represents the Convs module, LFECA represents the low-frequency enhancement channel attention block, and HFEB represents the high-frequency amplification block; S4: Train the dehazing network using the training dataset until the pre-set loss function converges; combine the loss function and the frequency domain loss as the objective function for network training to obtain the trained dehazing network; S5: Input the image to be dehazed into the trained dehazing network for testing and verification to obtain the dehazing result.
2. The dehazing method based on frequency information differential fusion according to claim 1, characterized in that, In step S3, the operation steps of the U-shaped feature extraction module are as follows: S311: Perform a 3×3 convolution operation on the input feature Fin to generate an initial feature map Fe1, and enhance the linear representation of the feature using the activation function DyT. The expression for DyT is: Fe1 = γ·tanh(α·(Fin)) + β; Where γ, α, and β are all learnable parameters; tanh is the hyperbolic tangent function; S312: Perform two more 3×3 convolution operations and the activation function DyT on feature Fe1 in sequence. The specific operations are as follows: Fe2=DyT·(Conv(Fe1)); Fe3=DyT·(Conv(Fe2)); Where Fe2 represents the intermediate output feature, Fe3 represents the encoder output feature, and Conv represents the 3×3 convolution operation; S313: In the bottleneck stage, channel attention (CA) and spatial attention (SA) mechanisms are applied sequentially to Fe3 to enhance the selective extraction of key features.
3. The dehazing method based on frequency information differential fusion according to claim 2, characterized in that, In step S313, the channel attention CA mechanism process is as follows: Global average pooling (GAP) is performed on Fe3 to compress its spatial dimension. Then, it is sequentially processed through 1×1 convolution, GELU activation, 1×1 convolution, and Sigmoid activation to generate a channel attention weight vector (WCA). This WCA is then element-wise multiplied with Fe3 to obtain the enhanced feature Fe4, expressed as: Fe4= Fe3·Sigmoid(PWConv(GELU(PWConv(GAP(Fe3))))); Where GELU represents the GELU activation function, PWConv represents a 1×1 convolution, and Sigmoid represents the Sigmoid activation function.
4. The dehazing method based on frequency information differential fusion according to claim 3, characterized in that, In step S313, the spatial attention (SA) mechanism works as follows: Fe4 is sequentially subjected to 3×3 convolution, GELU activation, 3×3 convolution, and Sigmoid activation to generate a spatial attention weight map (WSA). The spatial attention weight map is then multiplied element-wise with Fe4 to obtain the enhanced feature Fe5, expressed as: Fe5= Fe4·Sigmoid (Conv(GELU(Conv(Fe4)))).
5. The dehazing method based on frequency information differential fusion according to claim 4, characterized in that, In the decoder stage, Fe5 is subjected to three 3×3 convolution operations, followed by an upsampling operation after each convolution layer to restore the spatial resolution of the feature map. The specific operations are as follows: Fd1 = ↑(DyT(conv(Fe5))); Fd2 = ↑(DyT(conv(Fd1))); Fd3 = ↑(DyT(conv(Fd2))); In this context, ↑ indicates an upsampling operation, Fd1 and Fd2 are intermediate output features, and Fd3 is the decoder output feature.
6. The dehazing method based on frequency information differential fusion according to claim 5, characterized in that, The operation steps of the low-frequency enhanced channel attention block are as follows: First, the low-frequency information LL is extracted independently through a 3×3 channel-wise convolutional layer to focus on the global structure of the low-frequency features; then, the information between channels is integrated through a 1×1 pointwise convolutional layer to further enhance the semantic expression of the features; finally, the low-frequency enhanced channel attention block introduces a channel attention mechanism, extracts the global information at the channel level through global average pooling, and generates channel attention weights by using two 1×1 convolutions with ReLU activation in between, and normalizes them to [0,1] by the Sigmoid function.
7. The dehazing method based on frequency information differential fusion according to claim 6, characterized in that, The operation steps of the high-frequency information amplification block are as follows: First, the high-frequency information amplification block performs a feature shift operation on the input HF, shifting the HF to the lower right by one pixel. The empty edge part after the shift is filled with zeros to generate a shifted feature map. Then, the shifted feature map is subtracted from the HF element by element, and the absolute value of the result is taken to obtain a difference map. Next, the difference map is input into a sequence containing convolution, batch normalization, GELU activation function, and convolution for further processing. After that, the feature is normalized by Sigmoid activation and added to the all-one matrix, and then multiplied element by element with the HF to obtain the enhanced feature. Finally, the enhanced feature is residually concatenated with the HF after convolution to generate the final feature HF'.
8. The dehazing method based on frequency information differential fusion according to claim 1, characterized in that, In step S4, a staged training strategy is introduced. This strategy combines deep supervision and curriculum learning to optimize the multi-scale feature learning and overall performance of the dehazing network. The staged training strategy divides the training process into four stages, with n rounds as the boundary. The specific steps are as follows: In the first stage, for the low-resolution output, the loss is calculated with the corresponding low-resolution real image to guide the network to learn the basic structure and global features of the image first. In the second stage, the supervision shifts to the medium-resolution output, and the loss is calculated with the corresponding medium-resolution real image to gradually enhance the ability to recover local textures and details. In the third stage, supervision focuses on high-resolution output and calculates loss with the corresponding high-resolution real image to ensure the overall consistency and fine-grained details of the dehazing results. In the fourth stage, the loss of all resolution outputs is optimized to enable the network to fully capture multi-scale features.
Citation Information
Patent Citations
HFRC-Diff-based low-illumination superposition fog image enhancement method
CN119048381A
Double-branch defogging method based on Laplacian pyramid
CN119887582A