Efficient image defogging method based on adaptive frequency enhancement and global-local feature aggregation

Abstract

The invention provides a high-efficiency image defogging method based on adaptive frequency enhancement and global-local feature aggregation, belongs to the technical field of image defogging, and solves the problem that the traditional defogging method is poor in effect when processing non-uniform haze distribution. The method comprises the following steps: S1, obtaining a foggy image and a clear image in pairs, and constructing a training set and a test set; s2, constructing a global-local feature aggregation block, and extracting cross-regional and cross-scale information by adopting an improved self-attention mechanism; s3, constructing an adaptive frequency enhancement module, designing dynamic pooling cooperative channel attention and space attention, and improving the dynamic sensing ability of the network to multi-frequency components; s4, a context information fusion module is constructed, the problem of feature mismatching is relieved by using context information in the horizontal and vertical directions, and efficient lightweight fusion is achieved in the decoding stage; and S5, using the foggy image and the clear image in pairs in the training set to train the image defogging model to obtain a trained image defogging model. According to the method, global and local features are comprehensively utilized, the model can better adapt to haze conditions of different areas, in addition, by fusing frequency and space information, details shielded by fog can be effectively recovered, the color and brightness of a picture can be effectively coordinated, and therefore the stability and expressive force of the model in a complex scene are improved.

Description

Technical Field

[0001] This invention belongs to the field of image dehazing, and more specifically, relates to an image dehazing method for complex scenes with uneven fog distribution. Background Technology

[0002] Images captured under adverse weather conditions such as fog and haze often suffer from reduced contrast, color distortion, and loss of detail, severely impacting visual quality and subsequent analysis. To address this issue, image dehazing technology has emerged, aiming to restore image clarity and detail. Furthermore, in computer vision tasks such as object detection, recognition, and scene understanding, dehazing technology can significantly improve the accuracy and reliability of algorithms, particularly valuable in applications like autonomous driving, surveillance, and robot navigation. Simultaneously, in remote sensing and geographic information systems, dehazing technology helps improve the quality of remote sensing images, providing more accurate data support for environmental monitoring, resource management, and disaster assessment. Therefore, research on image dehazing technology not only has significant theoretical importance but also demonstrates broad value in practical applications.

[0003] In recent years, image dehazing methods based on convolutional neural networks (CNNs) have made significant progress, demonstrating strong performance in end-to-end dehazing modeling. However, existing methods still have two prominent problems in global and local feature extraction: First, traditional CNNs, due to their limited receptive field, struggle to effectively model the dependencies between distant pixels in an image, especially in scenes with widespread and uneven haze distribution, which can easily lead to inaccurate global haze estimation. Second, CNNs rely on local convolution operations, which have certain shortcomings in detail recovery. During the dehazing process, excessive smoothing of image textures and edge details can easily occur, affecting the realism and clarity of the image.

[0004] The aforementioned issues prevent existing CNN-based dehazing methods from fully integrating global and local information in complex hazy environments, thus limiting further improvements in dehazing performance. Therefore, there is an urgent need for an image dehazing method that can effectively capture global dependencies and enhance the preservation of local details, thereby improving robustness and visual quality in complex scenes. Summary of the Invention

[0005] This invention provides an efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation, addressing the aforementioned problems. This method solves the technical issues of insufficient global dependency modeling and limited local detail preservation in existing technologies, thereby improving image dehazing effects and visual quality in complex hazy scenes. The specific steps are as follows:

[0006] Step S1: Obtain pairs of foggy and clear images, construct a dataset, and divide it into training and test sets;

[0007] Step S2: Construct a global-local feature aggregation block to capture local features at different scales as well as global information;

[0008] Step S3: Construct an adaptive frequency enhancement module to improve the network's dynamic perception of multiple frequency components and promote the restoration of edge and texture details;

[0009] Step S4: Construct a context information fusion module to achieve efficient and lightweight feature fusion by utilizing horizontal and vertical context information;

[0010] Step S5: Input the pairs of foggy images and clear images into the above dehazing network for training to obtain the trained deep learning model;

[0011] Step S6: Input the image to be dehazed into the trained dehazing network, perform global-local feature extraction, frequency feature enhancement and context-guided fusion to estimate the fog distribution in the image, and obtain the final clear dehazed image.

[0012] The global-local feature aggregation block contains two parallel branches: multi-scale local feature extraction and global feature extraction. By co-modeling local and global information, it leverages the synergistic effect brought about by the complementarity of the two in terms of information scale and expressive power to achieve high-performance dehazing.

[0013] Among them, the adaptive frequency enhancement module divides the features into different frequency bands through wavelet transform and models them independently in the frequency domain. In addition, it uses dynamic pooling to coordinate channel attention and spatial attention to improve the dynamic perception capability of multi-frequency components.

[0014] The context information fusion module consists of multiple parallel horizontal and vertical 1D depth separable convolutional groups, which can explicitly capture the correlation of images over equal distances in the horizontal and vertical directions.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0016] (1) This invention provides an efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation, which aims to address the problem of local detail loss and unnatural visual effect in dehazing images in non-uniform haze scenes;

[0017] (2) The proposed global-local feature aggregation block can effectively improve the fusion ability of local and global information. By combining multiple dilated convolutions to capture local information of different sizes and expand the receptive field, and by using an improved multi-head self-attention mechanism to extract information that includes cross-regions and cross-scales, the network can break through the limitations of the receptive field of traditional CNNs, achieve long-distance dependency modeling, and improve the feature recovery ability.

[0018] (3) The adaptive frequency enhancement module proposed in this method is based on wavelet transform and dynamic pooling collaborative attention to extract and process multi-scale frequency features of images. Wavelet transform is responsible for dividing the features into different frequency bands and performing independent modeling in the frequency domain; dynamic pooling collaborative attention can weight and enhance the frequency features at different channels and spatial locations, thereby improving the expressive power of the input feature map.

[0019] (4) The context information fusion module proposed in this method enables the model to dynamically focus on key regions through multiple sets of horizontal and vertical depth separable convolutions, avoiding uniform processing of the entire image, and helping the model to dynamically adjust the fusion strategy when facing complex scenes, thereby improving the robustness of the model. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is the overall flowchart of this application;

[0022] Figure 2 This is a schematic diagram of the overall model provided by the present invention;

[0023] Figure 3 This is a structural diagram of the spatial feature processing module (SFP) of the dehazing network in an embodiment of the present invention.

[0024] Figure 4 This is a structural diagram of the Global-Local Feature Aggregation Block (GLFA) in an embodiment of the present invention.

[0025] Figure 5 This is a structural diagram of the adaptive frequency enhancement module AFEM of the defogging network in an embodiment of the present invention;

[0026] Figure 6 This is a structural diagram of the CAFM (Context Information Fusion) module of the defogging network in an embodiment of the present invention;

[0027] Figure 7 This is a comparison chart showing the defogging effect of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention.

[0029] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0030] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings and examples.

[0031] This invention provides an efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation, such as... Figure 1 As shown, the method includes the following steps:

[0032] Step 1: Select the publicly available image dehazing datasets Haze4k and NH-Haze to construct the training and test sets;

[0033] Step 1.1: Randomly crop the images in the training set to 256×256 and randomly flip them by 90°, 180° or 270°;

[0034] Step 1.2: The training set and the test set are divided in an 8:2 ratio;

[0035] Step 2: Construct an image dehazing network and train it using a training set; for example... Figure 2 As shown, the defogging network adopts an encoder-decoder architecture, including: a spatial feature processing module, an adaptive frequency enhancement module, and a context information fusion module;

[0036] Step 2.1: As Figure 3 As shown, the Spatial Feature Processing (SFP) module is used as the main module for partial feature extraction in the encoder and decoder. The input feature x of this module is first batch normalized to stabilize the input distribution and speed up the convergence.

[0037] Furthermore, the features are preliminarily processed through two convolutional layers:

[0038] x1 = PWConv(x)

[0039] x2 = Conv(x1)

[0040] Specifically, PWConv represents a 1×1 convolutional layer, and Conv represents a 5×5 convolutional layer.

[0041] Furthermore, x2 is fed into the global-local feature aggregation block to obtain intermediate results:

[0042] y = GLFA(x2)

[0043] Specifically, GLFA stands for Global-Local Feature Aggregation Block, and y represents the intermediate result.

[0044] Furthermore, after normalizing the y-batch, channel attention (CA) weighting and pixel attention (PA) weighting are applied separately. Channel attention adaptively weights each channel of the feature to improve information utilization. Specifically, this can be expressed as:

[0045] CA=Sigmoid(Conv(GELU(Conv(GAP(y)))))

[0046]

[0047] Specifically, GAP is global average pooling, and Conv is a 1×1 convolutional layer. This is for element-wise multiplication.

[0048] Furthermore, pixel attention focuses on spatial feature enhancement rather than information selection between channels, which can be specifically represented as:

[0049] PA=Sigmoid(Conv(GELU(Conv(y))))

[0050]

[0051] Furthermore, the channel attention-weighted result Fc and the pixel attention-weighted result Fp are concatenated along the channel dimension, and the concatenated result is input into two convolutional layers for processing. Finally, a residual connection is made with the intermediate result y.

[0052] F = Concat(Fc, Fp)

[0053] z = y + Conv(GELU(Conv(F)))

[0054] Specifically, Conv is a convolutional layer with a kernel size of 1, and z is the output of the spatial feature processing module.

[0055] Step 2.2: The global-local feature aggregation block, as the core component of the spatial feature processing module, has the following structure: Figure 4As shown, the input features are fed into two parallel branches for processing.

[0056] Furthermore, the input features F in The input is fed into the first branch and passes through four parallel residual blocks (composed of dilated convolution, batch normalization, and GELU activation functions concatenated) to extract multi-scale local features:

[0057]

[0058] Specifically, DilatedConv1, DilatedConv2, DilatedConv3, and DilatedConv4 are dilated convolutions with dilation rates of 1, 3, 5, and 7, respectively.

[0059] Furthermore, the four output features are concatenated by channels, and then the concatenated features are processed through two convolutional layers:

[0060] F loc =Conv(GELU(Conv(Concat(Floc1+Floc2+Floc3+Floc4))))

[0061] Specifically, Conv is a convolutional layer with a kernel of 1, and F... loc This is the output of the first branch.

[0062] Meanwhile, input feature F in The second branch incorporates an improved self-attention mechanism to capture global information:

[0063] Q = (F in .view(b,c,h*w)).permute(0,2,1)

[0064] K i =MaxPool i (F in (i = 1, 2, 3, 4)

[0065] K=Concat(K1,K2,K3,K4).permute(0,2,1)

[0066] V i =MaxPool i (F in (i = 1, 2, 3, 4)

[0067] V = Concat(V1, V2, V3, V4)

[0068] Specifically, b, c, h, and w are F. inThe size information is as follows: MaxPool1 is 2×2 max pooling, MaxPool2 is 3×3 max pooling, MaxPool3 is 5×5 max pooling, MaxPool4 is 6×6 max pooling, Q is the query feature, K is the key feature, and V is the value feature.

[0069] Furthermore, the attention weights are obtained by calculating the similarity matrix using query feature Q and key feature K and then performing softmax normalization.

[0070] sim_map=softmax(((Q⊙K)*alpha))

[0071] Specifically, ⊙ represents the dot product operation, alpha is the learnable scaling factor, and sim_map is the self-attention weight matrix.

[0072] Furthermore, the value features V are weighted to obtain global features, which are then processed by convolutional layers and combined with the input features F. in Perform a residual connection to obtain the final output of the second branch:

[0073] F gol =Conv((sim_map⊙V).permute(0,2,1))+F in

[0074] Specifically, Conv is a 1×1 convolutional layer, F gol This is the output of the second branch.

[0075] Furthermore, the output of the first branch is processed through two convolutional layers and then added element-wise to the output of the second branch to obtain the final output of the global-local feature aggregation module.

[0076] F GLFA =Conv(GELU(Conv(F) loc )))+F gol

[0077] Specifically, Conv is a 1×1 convolutional layer.

[0078] Step 2.3: The adaptive frequency enhancement module uses a wavelet transform and dynamic pooling collaborative attention mechanism to dynamically enhance frequency features;

[0079] The wavelet transform is defined as follows:

[0080] [X LL ,X LH ,X HL ,X HH ] = Conv([f LL ,f LH ,fHL ,f HH ],X,stride=2)

[0081] X LL =DWConv(X) LL )+X LL

[0082] X LH =DWConv(X) LH )+X LH

[0083] X HL =DWConv(X) HL )+X HL

[0084] X HH =DWConv(X) HH )+X HH

[0085] Y = IWT(Concat(X) LL ′,X LH ′,X HL ′,X HH ′))

[0086] Specifically, Conv implements the Haar wavelet transform using four 2×2 depthwise separable convolutions with a stride of 2, X LL Corresponding to the low-frequency component, X LH X HL X HH It contains high-frequency information in the horizontal, vertical, and diagonal directions. DWConv is a depthwise separable convolution with a size of 5×5 and a stride of 1. IWT is the inverse wavelet transform. Y is the intermediate result of the dynamic multi-scale frequency enhancement module.

[0087] Furthermore, under the premise of controllable parameters and computational cost, a small convolutional kernel is used to obtain a large receptive field, and multi-band feature extraction is achieved. Next, dynamic pooling combined with channel attention and spatial attention is used to adaptively enhance Y.

[0088] avg_out=Conv(ReLU(Conv(AdaptiveAvgPool2d(Y))))

[0089] max_out=Conv(ReLU(Conv(AdaptiveMaxPool2d(Y))))

[0090] mix_pool=α*AvgPool2d(Y)+(1-α)*MaxPool2d(Y)

[0091] mix_out=Conv(ReLU(Conv(mix_pool)))

[0092] ca_out=Y*σ(avg_out+max_out+mix_out)

[0093] sa=σ(Conv(Concat(Mean(ca_out,dim=1),MaxPool(ca_out))))

[0094] F DMFE =ca_out*sa

[0095] Specifically, α is a learnable parameter used to adaptively select the weights for average pooling and max pooling, thereby obtaining richer feature representations. After being weighted by concatenated channel attention and spatial attention, the result F with enhanced frequency features is obtained. DMFE .

[0096] Step 2.4: The context information fusion module captures context information by using four sets of horizontal and vertical depth-separable convolutions at different scales, thereby taking into account both local details and global smoothness during feature fusion and improving the overall naturalness of the dehazed image.

[0097] The definition of fusion features is as follows:

[0098] in_feats=Concat(feat1,feat2).view(b,2,c,h,w)

[0099] feat_sum=sum(in_feats,dim=1)

[0100] x=PWConv(AvgPool2d(feat_sum))

[0101] hv_conv i =v_conv i (h_conv i (x))i=1,2,3,4

[0102] hv_outputs=Concat(hv_conv1, hv_conv2, hv_conv3, hv_conv4)

[0103] attn=Sigmoid(PWConv(hv_outputs).view(b, 2, c, h, w))

[0104] F CGF=sum(in_feats*attn,dim=1)

[0105] Specifically, feat1 and feat2 are two features to be fused, h_conv is a horizontal convolution, and v_conv is a vertical convolution. Through pointwise convolution, horizontal and vertical convolution groups, and the Sigmoid function, weight coefficients for each position and channel of the input features can be generated.

[0106] Step 3: Input the foggy image into the image dehazing network and train the image dehazing network under the guidance of the loss function;

[0107] The loss functions in step 3 include the pixel loss function, the contrast loss function, and the perceptual loss function, each defined as follows:

[0108]

[0109] Specifically, output is the dehazed image output by the network, input is the source image, and target is the ground truth image.

[0110] Step 4: Input the image to be dehazed into the trained dehazing network, perform global-local feature extraction, frequency feature enhancement, and context-guided fusion to estimate the fog distribution in the image and obtain the final clear dehazed image.

[0111] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the claims of the present invention are similarly included within the protection scope of the claims of the present invention.

Claims

1. An efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation, characterized in that, include: Step S1: Obtain pairs of foggy and clear images, construct a dataset, and divide it into training and test sets; Step S2: Construct a global-local feature aggregation block to capture local features at different scales as well as global information; Step S3: Construct an adaptive frequency enhancement module to improve the network's dynamic perception of multiple frequency components and promote the restoration of edge and texture details; Step S4: Construct a context information fusion module to achieve efficient and lightweight feature fusion by utilizing horizontal and vertical context information; Step S5: Input the pairs of foggy images and clear images into the above dehazing network for training to obtain the trained deep learning model; Step S6: Input the image to be dehazed into the trained dehazing network, perform global-local feature extraction, frequency feature enhancement and context-guided fusion to estimate the fog distribution in the image, and obtain the final clear dehazed image.

2. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 1, characterized in that, The process of acquiring paired foggy and clear images and constructing a dataset specifically includes: collecting foggy and clear images under various environments, mainly divided into three categories: indoor environment, outdoor environment, and dense fog environment; then pairing the clear images with their corresponding foggy images one-to-one and storing them in different folders, with paired images using the same image naming format; after completing the pairing, dividing the total dataset into training and test sets.

3. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 1, characterized in that, The global-local feature aggregation block contains two parallel branches: multi-scale local feature extraction and global feature extraction. By co-modeling local and global information, and utilizing the synergistic effect brought about by the complementarity of the two in terms of information scale and expressive ability, high-performance dehazing is achieved.

4. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 1, characterized in that, The adaptive frequency enhancement module specifically divides features into different frequency bands through wavelet transform and models them independently in the frequency domain. In addition, it utilizes dynamic pooling to coordinate channel attention and spatial attention to improve the dynamic perception capability of multiple frequency components.

5. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 1, characterized in that, The context information fusion module consists of multiple parallel horizontal and vertical 1D depth separable convolutional groups, which can explicitly capture the correlation of images over equal distances in the horizontal and vertical directions.

6. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 3, characterized in that, The multi-scale local feature extraction branch consists of four parallel, progressively residual connected, depthwise separable dilated convolutions with different dilation rates. This allows the network to expand its receptive field while maintaining a lightweight architecture, thereby capturing local information of different sizes.

7. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 3, characterized in that, The global feature extraction branch is constructed based on a self-attention mechanism and replaces convolution with multi-scale pyramid pooling to extract information that spans regions and scales.

8. The efficient image dehazing method based on adaptive frequency enhancement and global-local feature aggregation according to claim 4, characterized in that, The dynamic pooling includes average pooling, max pooling, and weighted balanced pooling based on average pooling and max pooling, making pooling operations more flexible and adaptable to different features.

Images