An efficient image defogging method based on adaptive frequency enhancement and global-local feature aggregation
By employing global-local feature aggregation and adaptive frequency enhancement, the problem of insufficient integration of global and local information in existing technologies is solved, thereby improving image dehazing effect and visual quality, especially in complex hazy scenes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNIV OF PETROLEUM (EAST CHINA)
- Filing Date
- 2025-07-07
- Publication Date
- 2026-05-19
AI Technical Summary
Existing image dehazing methods based on convolutional neural networks struggle to effectively integrate global and local information in complex hazy scenes, leading to inaccurate haze estimation and excessive smoothing of image texture details, which affects visual quality.
We employ a global-local feature aggregation block and an adaptive frequency enhancement module, combining multi-scale local feature extraction, global feature extraction, adaptive frequency enhancement, and contextual information fusion. Through an improved multi-head self-attention mechanism and depthwise separable convolution, we enhance feature extraction and fusion capabilities.
It improves image dehazing performance, enhances the ability to preserve local details, and improves image robustness and visual quality in complex hazy scenes.
Smart Images
Figure CN120852232B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image dehazing, and more specifically, relates to an image dehazing method for complex scenes with uneven fog distribution. Background Technology
[0002] Images captured under adverse weather conditions such as fog and haze often suffer from reduced contrast, color distortion, and loss of detail, severely impacting visual quality and subsequent analysis. To address this issue, image dehazing technology has emerged, aiming to restore image clarity and detail. Furthermore, in computer vision tasks such as object detection, recognition, and scene understanding, dehazing technology can significantly improve the accuracy and reliability of algorithms, particularly valuable in applications like autonomous driving, surveillance, and robot navigation. Simultaneously, in remote sensing and geographic information systems, dehazing technology helps improve the quality of remote sensing images, providing more accurate data support for environmental monitoring, resource management, and disaster assessment. Therefore, research on image dehazing technology not only has significant theoretical importance but also demonstrates broad value in practical applications.
[0003] In recent years, image dehazing methods based on convolutional neural networks (CNNs) have made significant progress, demonstrating strong performance in end-to-end dehazing modeling. However, existing methods still have two prominent problems in terms of global and local feature extraction: on the one hand, traditional CNNs, due to their limited receptive field, have difficulty effectively modeling the dependencies between distant pixels in an image, especially in scenes where haze is widely and unevenly distributed, which can easily lead to inaccurate global haze estimation; on the other hand, CNNs rely on local convolution operations, which have certain shortcomings in detail recovery, and excessive smoothing of image texture and edge details can easily occur during the dehazing process, affecting the realism and clarity of the image.
[0004] The aforementioned issues prevent existing CNN-based dehazing methods from fully integrating global and local information in complex hazy environments, thus limiting further improvements in dehazing performance. Therefore, there is an urgent need for an image dehazing method that can effectively capture global dependencies and enhance the preservation of local details, thereby improving robustness and visual quality in complex scenes. Summary of the Invention
[0005] This invention addresses the aforementioned problems by providing a method based on adaptive frequency enhancement and global-local feature clustering.
[0006] The proposed efficient image dehazing method addresses the technical limitations of existing technologies, such as insufficient global dependency modeling and limited ability to preserve local details. This improves the dehazing effect and visual quality of images in complex hazy scenes. The specific steps are as follows:
[0007] Step S1: Obtain pairs of foggy and clear images, construct a dataset, and divide it into training and test sets;
[0008] Step S2: Construct a global-local feature aggregation block to capture local features at different scales as well as global information;
[0009] Step S3: Construct an adaptive frequency enhancement module to improve the network's dynamic perception of multiple frequency components and promote the restoration of edge and texture details;
[0010] Step S4: Construct a context information fusion module to achieve efficient and lightweight feature fusion by utilizing horizontal and vertical context information;
[0011] Step S5: Input the paired foggy images and clear images into the dehazing network composed of the global-local feature aggregation block, the adaptive frequency enhancement module and the context information fusion module for training to obtain the trained deep learning model;
[0012] Step S6: Input the image to be dehazed into the trained dehazing network, perform global-local feature extraction, frequency feature enhancement and context-guided fusion to estimate the fog distribution in the image, and obtain the final clear dehazed image.
[0013] The global-local feature aggregation block contains two parallel branches: multi-scale local feature extraction and global feature extraction. By co-modeling local and global information, it leverages the synergistic effect brought about by the complementarity of the two in terms of information scale and expressive power to achieve high-performance dehazing.
[0014] Among them, the adaptive frequency enhancement module divides the features into different frequency bands through wavelet transform and models them independently in the frequency domain. In addition, it uses dynamic pooling to coordinate channel attention and spatial attention to improve the dynamic perception capability of multi-frequency components.
[0015] The context information fusion module consists of multiple parallel horizontal and vertical 1D depth separable convolutional groups, which can explicitly capture the correlation of images over equal distances in the horizontal and vertical directions.
[0016] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0017] (1) This invention provides an efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation, which aims to address the problem of local detail loss and unnatural visual effect in dehazing images in non-uniform haze scenes;
[0018] (2) The proposed global-local feature aggregation block can effectively improve the fusion ability of local and global information. By combining multiple dilated convolutions to capture local information of different sizes and expand the receptive field, and by using an improved multi-head self-attention mechanism to extract information that includes cross-regions and cross-scales, the network can break through the limitations of the receptive field of traditional CNNs and achieve the construction of long-distance dependencies.
[0019] Models improve the ability to recover features.
[0020] (3) The adaptive frequency enhancement module proposed in this method is based on wavelet transform and dynamic pooling collaborative attention to extract and process multi-scale frequency features of images. The wavelet transform is responsible for dividing the features into different frequency bands and performing independent modeling in the frequency domain; the dynamic pooling collaborative attention can weight and enhance the frequency features at different channels and spatial locations, thereby improving the expressive power of the input feature map.
[0021] (4) The context information fusion module proposed in this method enables the model to dynamically focus on key regions through multiple sets of horizontal and vertical depth separable convolutions, avoiding uniform processing of the entire image, and helping the model to dynamically adjust the fusion strategy when facing complex scenes, thereby improving the robustness of the model. Attached Figure Description
[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is the overall flowchart of this application;
[0024] Figure 2 This is a schematic diagram of the overall model provided by the present invention;
[0025] Figure 3 This is a structural diagram of the spatial feature processing module (SFP) of the dehazing network in an embodiment of the present invention.
[0026] Figure 4 This is a structural diagram of the Global-Local Feature Aggregation Block (GLFA) in an embodiment of the present invention.
[0027] Figure 5 This is a structural diagram of the adaptive frequency enhancement module AFEM of the defogging network in an embodiment of the present invention;
[0028] Figure 6 This is a structural diagram of the CAFM (Context Information Fusion) module of the defogging network in an embodiment of the present invention;
[0029] Figure 7 This is a comparison chart showing the defogging effect of the present invention. Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention.
[0031] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0032] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings and examples.
[0033] This invention provides an efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation, such as... Figure 1 As shown, the method includes the following steps:
[0034] Step 1: Select the publicly available image dehazing datasets Haze4k and NH-Haze to construct the training and test sets;
[0035] Step 1.1: Randomly crop the images in the training set to 256×256 and randomly flip them by 90°, 180° or 270°;
[0036] Step 1.2: The training set and the test set are divided in an 8:2 ratio;
[0037] Step 2: Construct an image dehazing network and train it using a training set; for example... Figure 2 As shown, the defogging network adopts an encoder-decoder architecture, including: a spatial feature processing module, an adaptive frequency enhancement module, and a context information fusion module;
[0038] Step 2.1: As Figure 3 As shown, the Spatial Feature Processing (SFP) module is used as the main module for feature extraction in both the encoder and decoder. The input features of this module... First, batch normalization is used to stabilize the input distribution and accelerate the convergence speed;
[0039] Furthermore, the features are preliminarily processed through two convolutional layers:
[0040]
[0041]
[0042] Specifically, PWConv represents a 1×1 convolutional layer, and Conv represents a 5×5 convolutional layer.
[0043] Furthermore, Feeding in a global-local feature aggregation block yields intermediate results:
[0044]
[0045] Specifically, GLFA stands for Global-Local Feature Aggregation Block, and y represents the intermediate result.
[0046] Furthermore, after normalizing the y-batch, channel attention (CA) weighting and pixel attention (PA) weighting are applied separately. Channel attention adaptively weights each channel of the feature to improve information utilization. Specifically, this can be expressed as:
[0047]
[0048]
[0049] Specifically, GAP is global average pooling, and Conv is a 1×1 convolutional layer. This is for element-wise multiplication.
[0050] Furthermore, pixel attention focuses on spatial feature enhancement rather than information selection between channels, which can be specifically represented as:
[0051]
[0052]
[0053] Furthermore, the channel attention-weighted result Fc and the pixel attention-weighted result Fp are concatenated along the channel dimension, and the concatenated result is input into two convolutional layers for processing. Finally, a residual connection is made with the intermediate result y.
[0054]
[0055]
[0056] Specifically, Conv is a convolutional layer with a kernel size of 1, and z is the output of the spatial feature processing module.
[0057] Step 2.2: The global-local feature aggregation block, as the core component of the spatial feature processing module, has the following structure: Figure 4 As shown, the input features are fed into two parallel branches for processing.
[0058] Furthermore, the input features The input is fed into the first branch and passes through four parallel residual blocks (composed of dilated convolution, batch normalization, and GELU activation functions concatenated) to extract multi-scale local features:
[0059]
[0060] Specifically, , , , These are dilated convolutions with dilation rates of 1, 3, 5, and 7, respectively.
[0061] Furthermore, the four output features are concatenated by channels, and then the concatenated features are processed through two convolutional layers:
[0062]
[0063] Specifically, Conv is a convolutional layer with a kernel of 1. This is the output of the first branch.
[0064] Meanwhile, input features The second branch incorporates an improved self-attention mechanism to capture global information:
[0065]
[0066]
[0067]
[0068]
[0069]
[0070] Specifically, b, c, h, and w are Size information, For 2×2 max pooling, For 3×3 max pooling, For 5×5 max pooling, It is a 6×6 max pooling, where Q is the query feature, K is the key feature, and V is the value feature.
[0071] Furthermore, the attention weights are obtained by calculating the similarity matrix using query feature Q and key feature K and then performing softmax normalization.
[0072]
[0073] Specifically, This is a dot product operation, where alpha is a learnable scaling factor and sim_map is the self-attention weight matrix.
[0074] Furthermore, the value features V are weighted to obtain global features, which are then processed by convolutional layers and combined with the input features. Perform a residual connection to obtain the final output of the second branch:
[0075]
[0076] Specifically, Conv is a 1×1 convolutional layer. This is the output of the second branch.
[0077] Furthermore, the output of the first branch is processed through two convolutional layers and then added element-wise to the output of the second branch to obtain the final output of the global-local feature aggregation module.
[0078]
[0079] Specifically, Conv is a 1×1 convolutional layer.
[0080] Step 2.3: The adaptive frequency enhancement module uses a wavelet transform and dynamic pooling collaborative attention mechanism to dynamically enhance frequency features;
[0081] The wavelet transform is defined as follows:
[0082]
[0083]
[0084]
[0085]
[0086]
[0087]
[0088] Specifically, Conv implements the Haar wavelet transform using four 2×2 depthwise separable convolutions with a stride of 2. Corresponding to low-frequency components, , , It is horizontal and vertical.
[0089] The high-frequency information in the directional and diagonal directions is represented by DWConv, which is a depthwise separable convolution with a size of 5×5 and a stride of 1, IWT, which is an inverse wavelet transform, and Y, which is an intermediate result of the dynamic multi-scale frequency enhancement module.
[0090] Furthermore, under the premise of controllable parameters and computational cost, a small convolutional kernel is used to obtain a large receptive field, and multi-band feature extraction is achieved. Next, dynamic pooling combined with channel attention and spatial attention is used to adaptively enhance Y.
[0091]
[0092]
[0093]
[0094]
[0095]
[0096]
[0097]
[0098] Specifically, These are learnable parameters used to adaptively select weights for average pooling and max pooling.
[0099] This yields richer feature representations, which are then weighted by concatenated channel attention and spatial attention to obtain the frequency feature-enhanced result. .
[0100] Step 2.4: The context information fusion module captures context information by using four sets of horizontal and vertical depth-separable convolutions at different scales, thereby taking into account both local details and global smoothness during feature fusion and improving the overall naturalness of the dehazed image.
[0101] The definition of fusion features is as follows:
[0102]
[0103]
[0104]
[0105]
[0106]
[0107]
[0108]
[0109] Specifically, and For the two features to be fused, h_conv is the horizontal convolution and v_conv is the vertical convolution. By using pointwise convolution, horizontal and vertical convolution groups, and the Sigmoid function, weight coefficients for each position and channel of the input features can be generated.
[0110] Step 3: Input the foggy image into the image dehazing network and train the image dehazing network under the guidance of the loss function;
[0111] The loss functions in step 3 include the pixel loss function, the contrast loss function, and the perceptual loss function, each defined as follows:
[0112]
[0113]
[0114]
[0115]
[0116] Specifically, output is the dehazed image output by the network, input is the source image, and target is the ground truth image.
[0117] Step 4: Input the image to be dehazed into the trained dehazing network, perform global-local feature extraction, frequency feature enhancement, and context-guided fusion to estimate the fog distribution in the image and obtain the final clear dehazed image.
[0118] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the claims of the present invention are similarly included within the protection scope of the claims of the present invention.
Claims
1. An efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation, characterized in that, include: Step S1: Obtain pairs of foggy and clear images, construct a dataset, and divide it into training and test sets; Step S2: Construct a global-local feature aggregation block, which includes two parallel branches: multi-scale local feature extraction and global feature extraction. By co-modeling local and global information, and utilizing the synergistic effect brought about by the complementarity of the two in terms of information scale and expressive power, high-performance dehazing is achieved. Step S3: Construct an adaptive frequency enhancement module, which divides features into different frequency bands through wavelet transform and models them independently in the frequency domain. In addition, dynamic pooling is used to coordinate channel attention and spatial attention to improve the dynamic perception of multi-frequency components and promote the recovery of edge and texture details. Step S4: Construct a context information fusion module, which consists of multiple parallel horizontal and vertical 1D depth separable convolutional groups. This module can explicitly capture the correlation of images over equal distances in the horizontal and vertical directions, achieving efficient and lightweight feature fusion. Step S5: Input the paired foggy images and clear images into the dehazing network composed of the global-local feature aggregation block, the adaptive frequency enhancement module and the context information fusion module for training to obtain the trained deep learning model; Step S6: Input the image to be dehazed into the trained dehazing network, perform global-local feature extraction, frequency feature enhancement and context-guided fusion to estimate the fog distribution in the image, and obtain the final clear dehazed image.
2. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 1, characterized in that, The process of acquiring paired foggy and clear images and constructing a dataset specifically includes: collecting foggy and clear images under various environments, mainly divided into three categories: indoor environment, outdoor environment, and dense fog environment; then pairing the clear images with their corresponding foggy images one-to-one and storing them in different folders, with paired images using the same image naming format; after completing the pairing, dividing the total dataset into training and test sets.
3. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 1, characterized in that, The multi-scale local feature extraction branch consists of four parallel, progressively residual connected, depthwise separable dilated convolutions with different dilation rates. This allows the network to expand its receptive field while maintaining a lightweight architecture, thereby capturing local information of different sizes.
4. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 1, characterized in that, The global feature extraction branch is constructed based on a self-attention mechanism and replaces convolution with multi-scale pyramid pooling to extract information that spans regions and scales.
5. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 1, characterized in that, The dynamic pooling includes average pooling, max pooling, and weighted balanced pooling based on average pooling and max pooling, making pooling operations more flexible and adaptable to different features.