Frequency domain-space multi-stage image defogging method based on fog concentration guidance

Abstract

The invention provides a frequency domain-space parallel image defogging method based on fog concentration guidance. The frequency domain-space parallel image defogging method comprises the step of constructing a training set, a verification set and a test set of a defogging data set. The FSDG-Net defogging network is trained by using foggy images and corresponding fogless images in the training set, performance evaluation is periodically performed on the verification set in the training process, and the learning rate is adjusted when the evaluation index is not improved; judging whether a preset training round number is reached, and storing the training model after the preset training round number is reached; and carrying out defogging processing on a foggy image which does not participate in training by using the stored model, and outputting a defogging result image. According to the FSDG-Net network, a spatial domain and frequency domain parallel modeling structure is adopted, a fog concentration guiding module is introduced to perform adaptive modeling on fog distribution in different areas, and different depth features are integrated through a multi-stage feature fusion strategy, so that the defogging effect and the result stability of a non-uniform fog field are improved.

Description

A frequency-domain-spatial multi-stage image dehazing method guided by fog concentration Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a frequency-domain-spatial multi-stage image dehazing method guided by fog concentration, which is suitable for image enhancement and visual perception tasks in complex foggy environments. Background Technology

[0002] In the natural environment, due to the scattering and absorption of light by suspended particulate matter in the atmosphere, images are often affected by haze during the acquisition process, resulting in reduced contrast, color distortion and severe degradation of detail information. Image dehazing, as a key research direction in the field of image enhancement and restoration [1], is of great significance in application scenarios such as autonomous driving, video surveillance, remote sensing imaging and intelligent robots. High-quality dehazing processing can effectively improve the accuracy and stability of subsequent high-level visual tasks such as target detection, semantic understanding and scene analysis.

[0003] Early image dehazing methods mainly relied on physical models and artificial priors, such as dark channel priors based on atmospheric scattering models [2]. These methods recover clear images by estimating transmittance or atmospheric light parameters, and have achieved good results under certain conditions. However, since fog distribution in real scenes often has obvious non-uniformity, the above methods are prone to fog residue, over-enhancement or color distortion in complex fog environments, and have a strong dependence on prior assumptions and limited generalization ability.

[0004] With the development of deep learning technology, end-to-end image dehazing methods based on convolutional neural networks [3] and Transformer [7] have gradually become a research hotspot. These methods significantly improve the dehazing effect by learning the mapping relationship between foggy and fog-free images. However, traditional convolutional neural networks mainly rely on local receptive fields for feature extraction, and have limited ability to model the global structural information of images, which can easily cause structural shifts or a decrease in overall consistency during the dehazing process. Especially in heavy fog or distant areas, local modeling methods are difficult to effectively restore global contrast and structural information.

[0005] To compensate for the shortcomings of spatial domain modeling, some studies [4][5] have attempted to introduce frequency domain information to process images. Frequency domain representation can decompose an image into different frequency components, where the low-frequency part mainly describes the global structure of the image, while the high-frequency part contains detailed information such as texture and edges [6]. By performing targeted processing on different frequency components in the frequency domain, it is helpful to improve the detail restoration effect. However, most existing frequency domain dehazing methods adopt a single scale or simple frequency domain enhancement strategy, which makes it difficult to achieve a balance between high-frequency detail restoration and low-frequency structure preservation, and lacks an effective collaborative mechanism with spatial domain features.

[0006] In addition, the distribution of fog in real scenes usually has significant spatial non-uniformity, and the fog concentration varies greatly in different areas. Most existing deep learning dehazing methods adopt a uniform feature enhancement strategy for the entire image [8], which fails to fully consider the differences in the degree of degradation in different areas, and is prone to insufficient dehazing in dense fog areas and over-enhanced artifacts in thin fog areas. At the same time, as the network depth increases, there is a lack of effective fusion mechanism between the multi-stage feature extraction results, which may also cause problems such as information redundancy or gradient decay.

[0007] Therefore, there is a need for an image dehazing method that can combine the advantages of frequency domain and spatial domain features, introduce adaptive guidance based on fog concentration, and improve the stability and robustness of dehazing through multi-stage feature fusion, so as to better meet the application requirements of high-quality visual perception in complex foggy environments.

[0008] References:

[0009] [1]Fattal R. Single image dehazing[J]. ACM transactions on graphics(TOG), 2008, 27(3): 1-9.

[0010] [2]He K, Sun J, Tang X. Single image haze removal using dark channel prior[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 33(12): 2341-2353.

[0011] [3]Li Z, Liu F, Yang W, et al. A survey of convolutional neural networks: analysis, applications, and prospects[J]. IEEE transactions onneural networks and learning systems, 2021, 33(12): 6999-7019.

[0012] [4] Chi L, Jiang B, Mu Y. Fast fourier convolution[J]. Advances in Neural Information Processing Systems, 2020, 33: 4479-4488.

[0013] [5] Wang M, Liao L, Huang D, et al. Frequency and content dual stream network for image dehazing[J]. Image and Vision Computing, 2023, 139: 104820.

[0014] [6] Li S, Wang H, Zhu A. FFTDiff: Tuning-free image texture transfer based on diffusion model[J]. Journal of Visual Communication and Image Representation, 2025: 104681.

[0015] [7] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.

[0016] [8] Zhang H, Patel V M. Densely connected pyramid dehazing network[C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 3194-3203. Summary of the Invention

[0017] To address the shortcomings of existing image dehazing methods in complex foggy environments, such as difficulty in balancing detail restoration and structural stability, insufficient adaptability to non-uniform fog, and low efficiency in utilizing multi-stage features, this invention proposes a frequency-domain-spatial multi-stage image dehazing method guided by fog concentration. This invention introduces a parallel spatial and frequency domain modeling mechanism, a multi-stage feature fusion strategy, and a fog concentration adaptive guidance mechanism. This enhances high-frequency detail restoration while maintaining the overall structural consistency of the image, thereby achieving high-quality dehazing processing for images under complex fog conditions.

[0018] To achieve the above objectives, the present invention adopts the following technical solution:

[0019] This invention proposes a frequency-domain-spatial multi-stage image dehazing method guided by fog concentration, comprising the following steps:

[0020] 1) Acquire the input foggy image and preprocess it to obtain network input features; 2) Estimate the fog concentration distribution map from the foggy image using the fog concentration guidance module, and dynamically weight modulate the intermediate features using the fog concentration distribution map; 3) Feed the input features into at least one frequency domain-spatial parallel processing module, extract local structural features in the spatial domain branch, model frequency features in the frequency domain branch using fast Fourier transform, and restore them to the spatial domain using inverse fast Fourier transform; 4) Adaptively fuse the spatial domain branch features and the frequency domain branch features to obtain enhanced intermediate features; 5) Perform multi-stage weighted fusion of the features output by multiple frequency domain-spatial parallel processing modules to obtain the final defogging features; 6) Reconstruct the output defogging image based on the final defogging features.

[0021] Preferably, the image dehazing method proposed in this invention is implemented by constructing an end-to-end trainable image dehazing network model, wherein the network model is preferably FSDG-Net (a Frequency-Spatial Parallel Network with Density Guidance for Image Dehazing). This network model includes a fog density guidance module, a frequency-spatial parallel processing module, and a multi-stage feature fusion module, used to perform stage-by-stage feature enhancement and dehazing reconstruction on the input foggy image.

[0022] In any of the above schemes, preferably, the fog concentration guidance module is used to estimate the spatial distribution features of fog concentration from the input foggy image. The fog concentration guidance module employs a lightweight convolutional network structure to extract features from the input image and output a fog concentration distribution map. This fog concentration distribution map is used to dynamically modulate the feature responses of different spatial regions during subsequent feature extraction, giving higher feature enhancement weights to regions with higher fog concentrations, thereby improving the network's adaptability to non-uniform fog fields.

[0023] In any of the above schemes, it is preferred that the frequency domain-spatial parallel processing module includes a spatial domain branch and a frequency domain branch set in parallel. The spatial domain branch is used to extract local structural information and spatial texture features of the image, while the frequency domain branch is used to model the features in the frequency domain. The two branches run in parallel, and feature fusion is performed at the end of the module to achieve collaborative modeling of spatial and frequency information.

[0024] In any of the above schemes, a preferred embodiment is that the frequency domain branch maps the features from the spatial domain to the frequency domain by performing a Fast Fourier Transform on the input features. In the frequency domain, convolution operations of different scales are used to process the frequency domain features in parallel to model low-frequency structural information and high-frequency texture detail information respectively. After completing the frequency domain feature enhancement, the processed frequency domain features are mapped back to the spatial domain through an Inverse Fast Fourier Transform, thereby obtaining a spatial feature representation that fuses frequency information.

[0025] In any of the above schemes, it is preferred that the spatial domain branch employs a multi-scale parallel convolutional structure to extract spatial context information under different receptive fields. The spatial domain branch further introduces channel attention and pixel attention mechanisms to adaptively weight the feature responses of the channel dimension and spatial location, respectively, thereby enhancing the spatial feature representation capability of key regions.

[0026] In any of the above schemes, it is preferred that the frequency domain-spatial parallel processing module is configured in multiple cascaded configurations to form a multi-stage feature extraction structure. Each module enhances the features stage by stage to achieve progressive feature refinement from shallow to deep layers.

[0027] In any of the above schemes, it is preferred that the multi-stage feature fusion module is used to fuse the features output by multiple frequency domain-spatial parallel processing modules. The multi-stage feature fusion module uses a weighted fusion mechanism to adaptively weight and combine features from different stages, thereby achieving information complementarity between features of different depths and improving the stability and robustness of the dehazing results.

[0028] In any of the above schemes, a preferred approach is to generate an output dehazed image based on the final dehazed features obtained through multi-stage fusion using an image reconstruction module. The image reconstruction module employs convolutional mapping to map the final features back into the image space, resulting in a clear, dehazed image.

[0029] FSDG-Net stands for Frequency-Spatial Parallel Network with DensityGuidance for Image Dehazing, which is a frequency-spatial multi-stage image dehazing network guided by fog density. Attached Figure Description

[0030] Figure 1 is a flowchart of a preferred embodiment of the frequency-spatial multi-stage image dehazing method based on fog concentration guidance according to the present invention.

[0031] Figure 2 is a schematic diagram of the DEB module of the FSDG-Net network according to the fog concentration-guided frequency-space multi-stage image dehazing method of the present invention.

[0032] Figure 3 is a schematic diagram of the FSPP module of the FSDG-Net network according to the fog concentration-guided frequency-space multi-stage image dehazing method of the present invention.

[0033] Figure 4 is a schematic diagram of the MS module of the FSDG-Net network according to the fog concentration-guided frequency-space multi-stage image dehazing method of the present invention.

[0034] Figure 5 is a schematic diagram of the overall network structure of an embodiment of the FSDG-Net network based on the frequency domain-space multi-stage image dehazing method guided by fog concentration according to the present invention. Detailed Implementation

[0035] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0036] Example 1

[0037] As shown in Figure 1, this embodiment provides a training and testing process for a frequency-spatial parallel image dehazing method guided by fog concentration, including the following steps.

[0038] Perform step 100 to prepare the training and test sets for the dehazing dataset, where the training set is used for model parameter learning and the test set is used to evaluate the dehazing effect.

[0039] Execute step 110, using the foggy images and corresponding fog-free images in the training set to train the FSDG-Net network and generate a training model.

[0040] The FSDG-Net network includes a fog concentration guidance module (DEB), a frequency domain-spatial parallel processing module (FSPP), and a multi-stage feature fusion module (MF).

[0041] The fog concentration guidance module (DEB) is used to estimate the spatial distribution of fog from the input foggy image, generate a fog concentration weight map, and perform adaptive weighted modulation on the features based on the fog concentration weight map to enhance the feature response in areas with denser fog, while suppressing excessive enhancement in areas with lighter fog.

[0042] The Frequency-Spatial Parallel Processing (FSPP) module consists of parallel spatial and frequency domain branches. The spatial branch processes the input features using multi-scale convolutional structures to extract spatial structural information and local texture features from the image, and adaptively enhances key features using channel attention and pixel attention mechanisms. The frequency branch performs a frequency domain transformation on the input features, mapping them to the frequency space. It then models different frequency components using multi-scale frequency domain convolutions to enhance high-frequency detail information while maintaining low-frequency structural stability. Finally, an inverse frequency domain transformation maps the frequency domain features back to the spatial domain. The output features from the spatial and frequency domain branches are fused to obtain a joint frequency-spatial feature representation.

[0043] The multi-stage feature fusion module (MF) is used to fuse features output from multiple frequency-space parallel processing modules at different stages. It integrates feature information at different depths through an adaptive weight allocation mechanism to enhance the overall consistency and detail recovery capability of the dehazing results.

[0044] Execute step 120. During the training process, the current training model is validated using the validation set. If the evaluation metrics in the validation results do not show improvement, the current learning rate is reduced by 10% to improve the stability of the training process.

[0045] Execute step 130, during the training process, save the training model every preset number of rounds, and verify the model after saving; in this embodiment, the total number of training rounds is 150 rounds, and when the model saving period is not reached, return to step 110 to continue training.

[0046] Execute step 140. When the model saving period is reached, save the current training model to the local folder and perform performance verification on the saved model based on the validation set.

[0047] Execute step 150, use the saved training model to dehaze the foggy images that were not trained, and output the corresponding dehazed image results.

[0048] Example 2

[0049] This invention is a frequency-spatial parallel image dehazing network guided by fog density. This network is a trainable end-to-end image dehazing network, named FSDG-Net (A Frequency-Spatial Parallel Network with Density Guidance for Image Dehazing).

[0050] The network structure of FSDG-Net includes a Density Estimation Block (DEB), a Frequency-Spatial Parallel Processing (FSPP) module, and a Multi-stage Fusion (MF) module.

[0051] FSDG-Net trains a model on a dehazing dataset. The network structure parameters and weight parameters in the training model are loaded into the network model, thereby enabling the dehazing of input foggy images and outputting corresponding fog-free images. This network can effectively recover image structure information and texture details under complex fog conditions, improving the visual quality and stability of the dehazing results.

[0052] The steps for using FSDG-Net (implemented using the Python programming language) include:

[0053] 1. Prepare the training and testing sets for the image dehazing dataset. The training set contains foggy images and their corresponding fog-free reference images, while the testing set is used to evaluate the dehazing effect of the trained model.

[0054] 2. Start the network training process using the training file. Set the network training parameters according to actual needs, including batch size, initial learning rate, number of training epochs, and validation period, and train the FSDG-Net network.

[0055] 3. During training, save the current training model to a local folder and use the validation set to verify the performance of the training model. When the evaluation metrics in the validation results do not show improvement, reduce the current learning rate by a preset percentage to improve the stability of model training. Training can be continued through breakpoint resume training until a dehazing model that meets the performance requirements is obtained.

[0056] 4. Save the trained model weights to a local folder, and use the trained model to dehaze the foggy images that were not trained, and output the corresponding dehazed result images.

[0057] Example 3

[0058] Because fog in real-world images often exhibits uneven spatial distribution and complex degradation patterns, achieving a balance between detail restoration, structural preservation, and overall visual consistency is challenging in image dehazing. Especially under complex fog conditions, traditional methods relying solely on spatial domain modeling easily lead to the loss of high-frequency texture details, while methods based solely on frequency domain modeling may cause structural distortion and artifacts, thus limiting further improvements in dehazing performance. To address these issues, this embodiment proposes a frequency-spatial parallel image dehazing method (FSDG-Net) guided by fog concentration.

[0059] The overall framework of FSDG-Net is shown in the figure. The network first explicitly models the spatial distribution of fog in the input foggy image through a fog concentration guidance module, and then adaptively modulates the features accordingly. Subsequently, it uses multiple frequency domain-spatial parallel processing modules to enhance the image features layer by layer. Finally, it integrates features at different depths through a multi-stage feature fusion module to generate the final dehazed image.

[0060] The functions of each module are as follows:

[0061] 1. Fog Concentration Guidance Module (DEB)

[0062] The fog concentration guidance module estimates the spatial distribution of fog in the input foggy image and provides explicit degradation guidance information for subsequent feature extraction. This module processes the input image or its shallow features using a lightweight convolutional structure to learn a weight map that reflects changes in fog density.

[0063] The weight map spatially aligns with the fog distribution trend: it exhibits higher response values ​​in areas of denser fog and lower response values ​​in areas of lighter fog or near-fog-free fog. By element-wise weighting the fog concentration with the feature map, the network can enhance feature responses in heavily foggy areas, guiding subsequent modules to focus on restoring severely degraded regions; simultaneously, it suppresses unnecessary enhancements in lightly foggy areas, avoiding over-sharpening or color distortion. Through this fog concentration-guided mechanism, the network's adaptability to non-uniform fog fields is significantly improved.

[0064] 2. Frequency Domain-Spatial Parallel Processing Module (FSPP)

[0065] The frequency domain-spatial parallel processing module is the core feature modeling unit in this embodiment. Its purpose is to simultaneously model the spatial structure information and frequency domain spectrum information of the image in the same stage, thereby achieving the coordinated restoration of details and structure.

[0066] This module consists of parallel spatial domain branches and frequency domain branches.

[0067] The spatial domain branch employs a multi-scale convolutional structure to process input features, extracting local structure and texture information from the image. Convolutional operations at different scales can capture spatial context within different receptive fields, helping to characterize edges, contours, and local details. Building upon this, channel attention and pixel attention mechanisms are introduced, adaptively weighting features from the channel dimension and spatial location dimension, respectively. This allows the network to highlight feature channels and spatial regions that are more critical to the dehazing task, thereby enhancing structural consistency and detail representation capabilities.

[0068] The frequency domain branch first performs a frequency domain transformation on the input features, mapping them from the spatial domain to a frequency domain representation, thus endowing the features with global receptive field characteristics. Then, in the frequency domain, multi-scale convolutions are used in parallel to model different frequency components. Smaller-scale convolutions are used to enhance high-frequency details, such as texture and edges, while larger-scale convolutions are used to characterize low-frequency structural information, such as overall brightness and contour structure. After frequency domain feature enhancement, an inverse frequency domain transformation is used to map the processing result back to the spatial domain for fusion with the output of the spatial domain branch.

[0069] The outputs of the spatial domain branch and the frequency domain branch are integrated through an adaptive fusion method to form a joint frequency-spatial feature representation. Through this parallel structure, the network can simultaneously utilize the local modeling capabilities of the spatial domain and the global modeling advantages of the frequency domain, effectively mitigating the limitations of single-domain processing.

[0070] 3. Multi-stage Feature Fusion Module (MF)

[0071] Because different levels of frequency domain-spatial parallel processing modules have different focuses in feature representation, shallow features are more inclined to local texture and edge information, while deep features are more focused on global structure and semantic consistency. Relying solely on features from a single stage for reconstruction can easily lead to missing details or structural instability.

[0072] To address this, this embodiment introduces a multi-stage feature fusion module, which progressively fuses features output from multiple frequency-space parallel processing modules at different stages. This module employs an adaptive weight allocation mechanism to dynamically adjust the contribution ratio of features at each stage to the final feature representation based on their importance, thereby achieving complementarity and synergy of multi-level information.

[0073] By fusing features in multiple stages, the network can further enhance the expression of detailed information while maintaining the overall structural stability, improve the robustness and consistency of the dehazing results, and effectively alleviate the gradient decay and overfitting problems caused by deepening the network.

[0074] 4. Defogging result reconstruction

[0075] The final dehazing features obtained after multi-stage feature fusion are mapped back to the image space through the reconstruction module to generate the final haze-free image. Due to the synergistic effect of the aforementioned fog concentration guidance mechanism, frequency domain-spatial parallel modeling, and multi-stage fusion strategy, the output image shows good results in terms of fog removal degree, structural consistency, and texture detail restoration.

[0076] To better understand the present invention, the proposed defogging method has been described in detail above with reference to specific embodiments of the present invention. However, the present invention is not limited to the above embodiments. Any equivalent substitutions or simple modifications made to the above embodiments without departing from the technical essence of the present invention should fall within the protection scope of the present invention. Identical or similar parts in the various embodiments of this specification can be referred to mutually and will not be repeated.

Claims

1. A single-image dehazing method based on parallel processing of the frequency and spatial domains and fog density guidance, characterized in that, The process includes the following steps: 1) Acquiring an input foggy image and preprocessing it to obtain network input features; 2) Estimating a fog concentration distribution map from the foggy image using a fog concentration guidance module, and dynamically weighting and modulating the intermediate features using the fog concentration distribution map; 3) Feeding the input features into at least one frequency-space parallel processing module, extracting local structural features in the spatial domain branch, modeling frequency features in the frequency domain branch using fast Fourier transform, and restoring them to the spatial domain using inverse fast Fourier transform; 4) Adaptively fusing the spatial domain branch features and the frequency domain branch features to obtain enhanced intermediate features. 5) Perform multi-stage weighted fusion on the features output by multiple frequency domain-spatial parallel processing modules to obtain the final dehazing features; 6) Reconstruct and output the dehazed image based on the final dehazing features.

2. The single-image dehazing method as described in claim 1, characterized in that, The frequency-space parallel processing module includes a frequency domain branch and a spatial domain branch set in parallel, and the output features of the two branches are fused at the end of the module.

3. The single-image dehazing method as described in claim 2, characterized in that, The frequency domain branch includes: performing a fast Fourier transform on the input features, performing parallel convolution operations in the frequency domain using convolution kernels of different sizes to model low-frequency structural information and high-frequency texture detail information respectively, and mapping the frequency domain features back to the spatial domain through an inverse fast Fourier transform.

4. The single-image dehazing method as described in claim 2, characterized in that, The spatial domain branch employs a multi-scale parallel convolutional structure to extract spatial context information at different scales, and achieves spatial feature fusion through feature reshaping.

5. The single-image dehazing method as described in claim 4, characterized in that, The spatial domain branch further introduces channel attention mechanism and pixel attention mechanism to adaptively weight the feature responses of spatial location and channel dimension.

6. The single-image dehazing method as described in claim 1, characterized in that, The fog concentration guidance module includes a lightweight convolutional network for estimating the fog concentration distribution map and performing region-adaptive modulation on subsequent features based on the fog concentration distribution map to enhance the defogging effect in non-uniform fog regions.

7. The single-image dehazing method as described in claim 1, characterized in that, The multi-stage weighted fusion adopts a stage-by-stage feature weighted fusion strategy to achieve synergistic enhancement of high-level semantic information and low-level detailed information.

8. The single-image dehazing method as described in claim 1, characterized in that, The frequency domain-spatial parallel processing module is configured in multiple cascaded configurations to form a multi-stage feature extraction structure.

9. An image dehazing system constructed based on the method of any one of claims 1 to 8, characterized in that, include: The system includes an image input module, a fog density guidance module, a frequency domain-spatial parallel feature extraction module, a multi-stage weight fusion module, and an image reconstruction module.

Images