Image defogging method based on frequency domain information fusion and domain self-adaption

Abstract

The invention discloses an image defogging method based on frequency domain information fusion and domain self-adaption, which comprises the following steps of: when defogging an image based on a contrast regularization network AECRNet of an auto-encoder, correcting low-frequency global information and high-frequency local information after image decomposition through a frequency domain information enhancement and fusion strategy; statistical adjustment is carried out on deep features extracted by an AECRNet pre-training model encoder through a visual prompt guided domain adaptive strategy, so that the mean value and the standard deviation of the deep features are close to statistical distribution of synthetic domain data. The method has the advantages that the problem of image definition in rainy and foggy weather is solved; according to the method, Fourier transform is utilized to separate the amplitude spectrum and the phase spectrum of the foggy day image, and the amplitude spectrum and the phase spectrum are respectively enhanced and corrected, so that the overall brightness and detail performance of the image are remarkably improved. In addition, the domain difference between the synthetic data and the real foggy day data is effectively reduced through a visual prompt adaptive strategy, the generalization ability of the model is enhanced, and the definition and effect of the image after defogging are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image dehazing method based on frequency domain information fusion and domain adaptation. Background Technology

[0002] With the rapid development of intelligent transportation and autonomous driving technologies, the reduced image clarity in adverse weather conditions (especially foggy weather) has seriously affected the accuracy and safety of vehicle and pedestrian target detection.

[0003] With the rapid development of deep learning technology, image dehazing, as a low-level visual task, plays a crucial role in practical applications such as autonomous driving, surveillance, and smart cities. While traditional dehazing methods based on spatial domain features have achieved good results in some scenarios, they often struggle to balance global and local information when dealing with multiple degradations caused by fog, such as global brightness decay and local detail blurring. This leads to problems like color imbalance and missing details in the reconstructed results. Furthermore, limited by the fact that training data mainly relies on synthetic images, pre-trained models often experience performance degradation on real foggy images due to significant differences in statistical distribution. Domain difference becomes a key factor affecting the generalization ability of dehazing algorithms. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an image dehazing method based on frequency domain information fusion and domain adaptation to solve the problem of image clarity in rainy and foggy weather.

[0005] To achieve the above objectives, the technical solution adopted in this invention is as follows: an image dehazing method based on frequency domain information fusion and domain adaptation, which includes correcting the low-frequency global information and high-frequency local information after image decomposition by means of frequency domain information enhancement and fusion strategy when performing image dehazing processing on the contrast regularization network AECRNet based on autoencoder; and statistically adjusting the deep features extracted by the encoder of the AECRNet pre-trained model by means of a domain adaptation strategy guided by visual cues, so that its mean and standard deviation are close to the statistical distribution of synthetic domain data.

[0006] At the encoder end of the AECRNet network, the amplitude enhancement module AEB repairs low-frequency global information to restore the brightness, contrast and color balance of the image.

[0007] In the decoder front end of the AECRNet network, high-frequency local information is enhanced by the Phase Reconstruction Block (PRB) to restore edges and details in the image.

[0008] A PDA module is set after the output of each encoder layer in the AECRNet network, and a visual cue guidance strategy is adopted for domain adaptation. During the inference phase, the PDA module adjusts the statistics of the deep features extracted by the encoder by analyzing the feature distribution of the cue image, so as to ensure that its mean and standard deviation are consistent with the statistical distribution of the synthetic data, thereby reducing the domain difference between training and testing data and improving the generalization ability of the network.

[0009] The frequency domain information enhancement and fusion strategy decomposes the image from the spatial domain into amplitude spectrum and phase spectrum through Fourier transform, corresponding to low-frequency and high-frequency information, respectively. The low-frequency part reflects the overall brightness, contrast and color distribution of the image, while the high-frequency part carries the edge, texture and structural details of the image. The interference of rain and fog is compensated by correcting the amplitude and phase separately.

[0010] Both the AEB and PRB modules adopt a basic architecture of "spatial branch - frequency domain branch - feature fusion". The AEB or PRB module receives spatial domain features from the previous layer or skip connections and extracts basic information through regular convolution. Then, the input features are transformed into the frequency domain through fast Fourier transform to obtain the amplitude spectrum and phase spectrum. The frequency domain branch of AEB focuses on processing the amplitude spectrum, using a series of lightweight 1×1 convolutional layers to enhance the amplitude spectrum, thereby compensating for the global energy attenuation caused by haze. The frequency domain branch of PRB focuses on correcting the phase spectrum, strengthening edge and texture information through convolution operations.

[0011] After completing the frequency domain processing, the AEB and PRB modules respectively use inverse Fourier transform to reconstruct the processed frequency domain information back into the spatial domain, and then fuse it with the original spatial domain features through channel splicing or addition. After a 1×1 convolution layer, the output is integrated to form the corrected feature representation.

[0012] An AEB module is deployed at the end of the encoder of the AECRNet network to effectively correct the amplitude spectrum of the image; a PRB module is inserted in the first two layers of the decoder to correct the phase spectrum.

[0013] An adaptive mixup strategy is adopted to dynamically fuse the outputs of AEB and PRB, achieving more refined compensation between global equalization and local detail restoration, and providing higher quality visual input for subsequent object detection tasks.

[0014] The domain-adaptive strategy for visual cue guidance during the testing phase includes utilizing the visual cue generation module PGM from real foggy images during the preprocessing stage. r and the composite domain haze-free image y s First, a prompt image p is generated. Second, in the feature transformation stage, the feature adaptation module FAM is used to process the real image features f output by each layer of the encoder. xrFeature-level normalization is performed to make its statistics gradually approach the statistics of the corresponding layer of the prompt image.

[0015] In the visual cue generation module, a haze-free image y is first selected. s For reference, and compared with a real foggy image x r Perform size matching;

[0016] Subsequently, for x r and y s Simultaneously, image-level normalization is employed, the core operation of which is implemented through the following formula:

[0017]

[0018] Where, μ c (·) and σ c (·) represent the mean and standard deviation of the image in spatial dimension H×W and for each channel R, G, B, respectively; to capture local haze distribution more precisely, PGM introduces a partitioning strategy: x r and y s Each image is divided into multiple non-overlapping blocks, and a normalization operation is performed independently on each block, thereby generating a more accurate cue image with local features; to address the color imbalance problem, if x is detected... r If there is a significant monochrome deviation, a color balance strategy is implemented during normalization, which means averaging the statistics of each channel to avoid overall color shift due to a certain channel being too high or too low.

[0019] In the Feature Adaptation Module (FAM), the real image x r Multi-layer features f are extracted by the encoder xr Simultaneously, the prompt image p is processed in the same layer to obtain the corresponding prompt feature f. p The goal of FAM is to make f xr The channel mean and standard deviation are adjusted to be consistent with f. p This makes the features output by the encoder more statistically consistent with the pattern of fog-free images in the synthetic domain.

[0020] In the encoder, the FAM module is inserted after each basic convolution or residual block, utilizing the cumulative adjustment of multi-level features to improve f at each scale. xr They can all gradually approach the statistical characteristics of the layer corresponding to the prompt image p.

[0021] The advantages of this invention are: it solves the image clarity problem in rainy and foggy weather conditions; the method uses Fourier transform to separate and enhance the amplitude and phase spectra of foggy images, thereby significantly improving the overall brightness and detail of the image. Furthermore, the visual cue adaptive strategy effectively reduces the domain difference between synthetic data and real foggy data, enhances the model's generalization ability, and improves the clarity and effect of the image after defogging. Attached Figure Description

[0022] The following is a brief explanation of the contents of each of the accompanying drawings and the markings in the drawings:

[0023] Figure 1 This is a diagram of the FIDA-Net network structure of the present invention;

[0024] Figure 2 This is a structural diagram of the amplitude enhancement block and phase reconstruction block of the present invention;

[0025] Figure 3 This is a domain adaptive structure diagram for visual cues guided by the present invention;

[0026] Figure 4 This is a simplified structural diagram of the domain adaptive design guided by visual cues in this invention.

[0027] Figure 5 This is a diagram of the target recognition network model architecture obtained by the present invention based on the improvement of YOLOv8s. Detailed Implementation

[0028] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and the description of the preferred embodiments.

[0029] With the rapid development of intelligent transportation and autonomous driving technologies, the decreased image clarity in adverse weather conditions (especially foggy weather) severely impacts the accuracy and safety of vehicle and pedestrian target detection. This study focuses on image dehazing and target detection techniques in foggy environments. To address the challenge of traditional spatial domain dehazing algorithms in balancing image brightness and detail, this paper proposes a frequency-domain and information domain adaptive network (FIDA-Net). This method utilizes Fourier transform to separate and enhance the amplitude and phase spectra of foggy images, significantly improving overall image brightness and detail. Furthermore, a visual cue adaptation strategy effectively reduces the domain difference between synthetic and real foggy data, enhancing the model's generalization ability.

[0030] This paper proposes a frequency-domain and information domain adaptive network (FIDA-Net), aiming to recover overall illumination and local details by utilizing the low-frequency and high-frequency characteristics of the image frequency domain, respectively. Simultaneously, during the inference stage, a feature statistical adaptive mechanism guided by visual cues achieves inter-domain distribution alignment. Specifically, the method in this chapter mainly includes two core improvements:

[0031] (1) Frequency Domain Information Enhancement and Fusion (FDE-Fusion) Strategy

[0032] The image is decomposed into amplitude spectrum and phase spectrum by Fourier transform, and special modules are designed to correct low-frequency global information and high-frequency local information respectively.

[0033] (2) Prompt-guided Domain Adaptation (PDA) adopts a prompt-guided domain adaptation strategy during the testing phase. It statistically adjusts the deep features extracted by the pre-trained model encoder so that its mean and standard deviation are closer to the statistical distribution of the synthetic domain data, thereby reducing the difference between the real domain and the synthetic domain and improving the reconstruction effect.

[0034] The FIDA-Net model proposed in this scheme, based on the encoder-decoder architecture of an autoencoder-based contrastive regularization network (AECRNet), achieves global equalization and local detail restoration of image haze degradation by fusing two major strategies: frequency domain information enhancement (FDE-Fusion) and visual cue-guided domain adaptive fading (PDA). Simultaneously, it effectively reduces the statistical difference between the synthetic and real domains. The network structure diagram of FIDA-Net is shown below. Figure 1 As shown. This scheme is based on the AECRNet network and improves it to obtain the FIDA-Net network. The original image is processed using the FIDA-Net network and the dehazed image is output.

[0035] Improvements to AECRNet include: at the end of the encoder of the AECRNet network, the amplitude enhancement module AEB is used to repair low-frequency global information, restoring the brightness, contrast and color balance of the image;

[0036] In the decoder front end of the AECRNet network, high-frequency local information is enhanced by the Phase Reconstruction Block (PRB) to restore edges and details in the image.

[0037] Set a PDA module after the output of each encoder layer in the AECRNet network.

[0038] First, a frequency domain information enhancement strategy is employed to repair frequency domain information in certain layers of the encoder and decoder outputs. At the encoder end, the Amplitude Enhancement Block (AEB) primarily repairs low-frequency global information, restoring the image's brightness, contrast, and color balance. Subsequently, at the decoder end, the Phase Reconstruction Block (PRB) mainly enhances high-frequency local information, repairing edges and details in the image. Through this enhancement and fusion of frequency domain information, the network can simultaneously improve dehazing performance at both the global and local levels.

[0039] The PDA module, located after the output of each encoder layer, employs a visual cue-guided strategy for domain adaptation. During the inference phase, the PDA module analyzes the feature distribution of the cue images and adjusts the statistics of the deep features extracted by the encoder to ensure that their mean and standard deviation are more consistent with the statistical distribution of the synthetic data. This reduces the domain difference between training and testing data and improves the network's generalization ability.

[0040] Furthermore, the network employs an adaptive mixup fusion strategy, which differs from traditional fusion methods by performing feature fusion at the output layers of AEB and PRB. By integrating information at an early stage, the new fusion strategy effectively optimizes the balance between global illumination and local details, thereby improving the dehazing effect.

[0041] Frequency domain information enhancement and fusion are crucial in dehazing images. Due to the scattering effect of haze, images often exhibit both overall brightness and color degradation, as well as blurring of local details and textures. Traditional dehazing methods based on spatial domain features struggle to simultaneously address these two forms of degradation, resulting in deficiencies in both global and local representation. To overcome this issue, this study introduces a frequency domain information enhancement and fusion mechanism based on AECR-Net, proposing an amplitude enhancement block (AEB) and a phase reconstruction block (PRB). This mechanism, based on Fourier transform theory, decomposes the image from the spatial domain into amplitude and phase spectra, corresponding to low-frequency and high-frequency information, respectively. The low-frequency component primarily reflects the overall brightness, contrast, and color distribution of the image, while the high-frequency component carries the image's edges, textures, and structural details. By separately correcting the amplitude and phase, haze interference can be compensated more effectively, resulting in a dehazing result that achieves both a globally balanced visual effect and preserves rich local details.

[0042] like Figure 2 As shown, both the AEB and PRB modules adopt a basic architecture of "spatial branch—frequency domain branch—feature fusion". First, the module receives spatial domain features from the previous layer or skip connections, extracting basic information through conventional convolutions, such as 3×3 convolutions and nonlinear activation functions. Next, a Fast Fourier Transform (FFT) is used to transform the input features to the frequency domain, obtaining the amplitude and phase spectra. The frequency domain branch of AEB focuses on processing the amplitude spectrum, primarily using a series of lightweight 1×1 convolutional layers to enhance the amplitude spectrum, thereby compensating for the global energy attenuation caused by haze. The frequency domain branch of PRB, on the other hand, focuses on correcting the phase spectrum, enhancing edge and texture information through specially designed convolutional operations. After completing the frequency domain processing, both modules use inverse Fourier transforms to reconstruct the processed frequency domain information back to the spatial domain, fusing it with the original spatial domain features through channel concatenation or addition. The resulting output is then integrated through a 1×1 convolution layer, forming the corrected feature representation.

[0043] In the encoder-decoder architecture of AECR-Net, the encoder compresses image features and aggregates global information through downsampling, while the decoder is responsible for upsampling to recover high-resolution details. Considering that the low-resolution features obtained in the downsampling stage mainly reflect the global brightness, contrast, and color distribution of the image, deploying AEB at the end of the encoder can effectively correct the amplitude spectrum at lower resolution, thereby improving the overall energy attenuation problem caused by haze. Frequency domain processing at low resolution not only has lower computational cost but also provides a more balanced basic representation for subsequent deep feature extraction. Correspondingly, in the first two layers of the decoder, since detail information and high-frequency structures are easily lost during upsampling, inserting PRB can correct the phase spectrum early in image restoration, enhancing edge and texture details, thus ensuring that the local structure remains clear after upsampling. By placing these two modules at key positions in the encoder and decoder respectively, and using an adaptive mixup strategy to dynamically fuse their outputs, the network can achieve finer compensation between global equalization and local detail recovery, providing higher-quality visual input for subsequent object detection tasks.

[0044] Domain adaptation guided by visual cues:

[0045] In image dehazing tasks, there is a significant difference in statistical distribution between the synthetic and real domains, known as domain difference. Dehazing models trained solely on synthetic data often struggle to adapt to the complex degradation phenomena of real foggy images. To mitigate this difference, this paper proposes a domain-adaptive strategy guided by visual cues during the testing phase. The core of this strategy lies in using visual cues to guide adaptive adjustments to encoder feature statistics (such as mean and standard deviation), ensuring that the feature distribution of the real foggy image after encoding more closely resembles the statistical characteristics in the synthetic domain. This provides a valid basis for the decoder to generate more accurate dehazing results. This method does not require retraining the model; instead, it dynamically adjusts the deep features extracted from the encoder of the pre-trained model during inference, thereby improving the dehazing effect. Figure 3 The overall framework of the mechanism is presented. The visual cue-guided domain adaptation method consists of two core components: a visual cue generation module (PGM) and a feature adaptation module (FAM). First, in the preprocessing stage, the visual cue generation module (PGM) is used to generate domain adaptation data from real foggy images x r and the composite domain haze-free image y s First, a prompt image p is generated. Second, in the feature transformation stage, the feature adaptation module FAM is used to process the real image features f output by each layer of the encoder. xr Feature-level normalization is performed to make its statistics gradually approach the statistics of the corresponding layer of the prompt image.

[0046] In the visual cue generation module, in order to obtain a cue image that both reflects the actual distribution of haze and conforms to the statistical characteristics of haze-free areas in the synthetic domain, a haze-free image y is first selected. s For reference, and compared with a real foggy image x r Perform size matching. Then, for x... r and y s Simultaneously, Image-Level Normalization (ILN) is employed, the core operation of which is implemented through the following formula:

[0047]

[0048] Where, μ c (·) and σ c (·) represent the mean and standard deviation of the image in spatial dimension H×W and for each channel R, G, B, respectively. To capture local haze distribution more precisely, PGM introduces a partitioning strategy: dividing x... r and y s Each image is divided into multiple non-overlapping blocks, and a normalization operation is performed independently on each block, thereby generating a more accurate cue image with local features. Regarding the color imbalance problem, if x is detected... r If there is a significant monochrome deviation, a color balance strategy (Color Balanced Image-Level Normalization, CBILN) is used during normalization, which means averaging the statistics of each channel to avoid overall color shift due to a certain channel being too high or too low.

[0049] The generated prompt image p has two characteristics: on the one hand, its local statistical properties are consistent with the real image xr, which can reflect the distribution of real haze; on the other hand, its overall statistical properties are close to the distribution learned by the haze-free image ys in the synthetic domain, providing an effective "target domain" reference for subsequent feature alignment.

[0050] In the Feature Adaptation Module (FAM), the real image x r Multi-layer features f are extracted by the encoder xr Simultaneously, the prompt image p is processed in the same layer to obtain the corresponding prompt feature f. p FAM's goal is to make f xr The channel mean and standard deviation are adjusted to be consistent with f. p This makes the features output by the encoder statistically more consistent with the patterns of haze-free images in the synthetic domain. The specific normalization process can be described by the following formula:

[0051]

[0052] Where, μ c (f xr ) and σ c (f xr ) respectively represent f xr The channel mean and standard deviation; while μ c (f p,xr ) and σ c (f p,xr This means based on the prompt feature f p For f xr The target statistical value. It should be noted that, to avoid directly using f... p The statistics led to overly aggressive adjustments, so FAM internally incorporated conditional constraints, such as when f xr with f p When the signs of the means are inconsistent, simply retain f. xr The mean of the standard deviation is taken; when the difference in standard deviation exceeds the preset threshold α, only mild correction is made. The detailed pseudocode of this process can be found in Algorithm 1. Its core idea is to ensure that the feature statistical adjustment makes full use of the prompt information while avoiding feature distortion caused by excessive statistical bias.

[0053]

[0054] In the encoder, the FAM module is inserted after each basic convolution or residual block, utilizing the cumulative adjustment of multi-level features to improve f at each scale. xr All of these features can gradually approach the statistical characteristics of the layer corresponding to the prompt image p. Since the decoder part was not modified, the adjusted features, after being reconstructed by the decoder, can be mapped back to the haze-free image more accurately, thereby improving the dehazing effect.

[0055] The entire visual cue-guided domain adaptation requires no backpropagation or additional parameter updates during the inference phase, relying solely on arithmetic adjustments to feature statistics during the forward computation. Therefore, it has a low computational burden and high real-time adaptability. By leveraging the target statistical information provided by the PGM-generated cue images and the dynamic correction of deep features by FAM, the pre-trained model achieves adaptation to the statistical distribution of real foggy images without altering the original decoder structure, significantly reducing the difference between the synthetic and real domains.

[0056] In image dehazing tasks, the loss function is crucial for model training and performance. To simultaneously achieve global illumination restoration and local detail enhancement in dehazed images, this paper employs a combination of L1 image reconstruction loss and contrastive regularization (CR) to ensure the restored image performs well in both visual quality and detail preservation.

[0057] In dehazing tasks, image reconstruction loss is often used to minimize the difference between the dehazed image and the sharp target image. We use L1 loss to evaluate the dehazed image. The pixel-level difference between the target image J and the target image J. L1 loss can effectively suppress color distortion and preserve image details. Its expression is:

[0058]

[0059] Where J is the sharp image of the target. It is a dehazed image generated by the network, where ||·||1 represents the L1 norm, which is the absolute sum of pixel differences.

[0060] To better utilize the contrast information between positive and negative samples, this paper proposes Contrast Regularization (CR). Unlike traditional training methods that only use positive samples as the target, CR compares the restored image with both sharp images (positive samples) and hazy images (negative samples), encouraging the network to narrow the distance between the restored image and the sharp image, and widen the distance between the restored image and the hazy image in the feature space. Specifically, CR introduces a contrast loss of positive and negative sample pairs in the feature space, as shown in the following formula:

[0061]

[0062] Where D(x,y) is the L1 distance, representing the difference between x and y, and G... i (·) is the i-th layer feature extracted from the pre-trained model, ω i β is the weight coefficient of the features of this layer, and β is the hyperparameter that balances the reconstruction loss and CR loss.

[0063] Combining image reconstruction loss and contrast regularization, the final dehazing loss function is a weighted sum of the two. This allows the network to simultaneously optimize global illumination, contrast, and local details of the image, ensuring a balance between visual quality and detail restoration in the dehazing result. The final loss function is as follows:

[0064] L = L1 + L CR (0.6)

[0065] By introducing contrast regularization, FIDA-Net can better utilize the contrast information between clear and hazy images, which not only enhances the image detail recovery capability but also improves the model's robustness in different scenarios.

[0066] Experimental verification and result analysis:

[0067] This experiment was conducted on two servers equipped with NVIDIA 3090RTX GPUs. The deep learning framework PyTorch 2.3.0 was used for network training and inference. During training, Foggy Cityscapes was used as the synthetic dataset for end-to-end training of AECR-Net. During testing, the generalization performance of the model was evaluated on the real-world foggy datasets RTTS and Foggy Driving. During inference, PDA was introduced to adaptively adjust the feature distribution. The core training process of AECR-Net followed the Adam optimizer strategy, with momentum decay coefficients β1 = 0.9 and β2 = 0.999, an initial learning rate of 0.0002, a batch size of 16, and cosine annealing for learning rate adjustment. The total number of training epochs was set to 100. A penalty term β (with a value of 0.1) was also introduced into the loss function to enhance the network's suppression of undesirable artifacts. Furthermore, to better preserve image content and perceptual features, this experiment added L1 norm constraints to layers 1, 3, 9, and 13 of the pre-trained VGG-19, with the corresponding coefficients set sequentially to... 1. This improves image quality while maintaining a lightweight network.

[0068] During the inference phase, a PDA module is introduced to dehaze the trained AECR-Net during testing. The prompt generation part of this module first selects a haze-free image from Foggy Cityscapes, denoted as y. s The algorithm employs a block-based strategy, ensuring the block size is approximately one-tenth the width of the input image. Simultaneously, Color Balance Normalization (CBILN) is used to statistically align the block with the real foggy image, generating the cue image p. Subsequently, Feature Adaptation (FAM) is inserted after each stage of the AECR-Net encoder output. Based on limiting the adjustment range of the mean and variance, FAM subtly modifies the real image features to bridge the difference between the synthetic and real domains. Specifically, PGM uses a threshold τ = 0.005 to determine whether to perform color balancing; the hyperparameter α = 2 in FAM controls the maximum adjustment range of the feature standard deviation, balancing dehazing intensity and visual naturalness. Notably, PDA only adjusts the feature statistical distribution during the inference stage, requiring no additional training or fine-tuning of model parameters, thus ensuring the simplicity and efficiency of the overall process. Parameter settings are shown in Table 3.1.

[0069] Table 3.1 Parameter Settings

[0070]

[0071] To evaluate the performance of FIDA-Net in the dehazing task, it was compared with several existing dehazing algorithms, including Dark Channel Prior (DCP), End-to-End Single Image Dehazing Network (DehazeNet), All-in-One Dehazing Network (AOD-Net), Feature Fusion Attention Network (FFA-Net), Vision Transformers for Single Image Dehazing (DehazeFormer), and AECR-Net. Experimental results were used to quantitatively compare the dehazing performance of different methods using standard image quality evaluation metrics such as PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index).

[0072] The image quality evaluation indexes for each method after dehazing are shown in the table below:

[0073] Comparison table of experimental results for basic defogging networks

[0074]

[0075] As can be seen, FIDA-Net performs exceptionally well among all methods, achieving the highest PSNR (24.35) and SSIM (0.901) values, far exceeding other benchmark methods such as DCP and DehazeNet. This indicates that FIDA-Net not only recovers more detailed information during the dehazing process but also maintains high structural consistency.

[0076] Compared with traditional methods, FIDA-Net significantly improves the dehazing effect by introducing frequency domain information enhancement and fusion, as well as a domain adaptation strategy guided by visual cues, and shows its powerful advantages, especially when processing complex hazy images.

[0077] After image dehazing, the next step is to perform target recognition on the dehazed image to identify specific targets within it. Dehazing recognition can be performed on both dehazed and undehazed images, effectively reducing the target recognition problems caused by rain and fog. This solution optimizes the YOLOv8s target detection framework by proposing an improved scheme that combines content-focused attention mechanism and multi-scale feature optimization. In foggy environments, traditional target detection algorithms often suffer from limited accuracy due to decreased image contrast and blurred details. To address this, this paper introduces a content-focused attention mechanism, which adaptively enhances the detail information of the target region, thereby improving the accuracy of target detection. Simultaneously, a hybrid fusion strategy based on multi-scale feature optimization is designed to effectively fuse features from different scales, further improving the model's robustness and detection accuracy.

[0078] To address the issue of weakened target features in foggy images, a Content-Focused Multi-Scale Feature Network (CFMNet) is proposed. The improved scheme primarily focuses on the feature fusion part of the detection network. By introducing two key improvement modules, the fusion process of multi-scale features is optimized to cope with the challenges posed by low contrast and blurred details in foggy environments. Specifically, the improvements in this paper include the following two aspects:

[0079] (1) Introduce the Content Focused Attention (CFA) module.

[0080] This module calculates attention weights in both channel and spatial dimensions using global and local statistical information, thereby generating a detailed spatial importance map specific to each channel. This mechanism can adaptively enhance the detail information of target areas in foggy images and effectively suppress background interference caused by fog and haze.

[0081] (2) Design a Content-Focused Attention-based Mixup Fusion Module (CFA-MF)

[0082] This module utilizes a fine-grained attention map generated by CFA to perform pixel-by-pixel weighted fusion of features from different scales, thereby effectively integrating low-level detail information with deep semantic information. Through this adaptive hybrid fusion strategy, the network can more accurately retain key target information during the feature fusion stage, thus improving overall detection performance.

[0083] This chapter will systematically introduce the theoretical basis, design principles and specific implementation details of the improved method, and analyze its actual performance and advantages in vehicle and pedestrian detection in foggy weather based on experimental results.

[0084] An improved YOLOv8s model is used for target recognition, such as... Figure 5 As shown, the basic network structure uses YOLOv8s, mainly consisting of three parts: the backbone, the neck, and the head. The backbone extracts multi-scale features from the input image through a series of convolutional operations and module stacking. The feature fusion part uses upsampling, stitching, and subsequent improved attention fusion strategies to efficiently integrate features of different resolutions. The head finally uses the fused features to complete the target localization and classification tasks.

[0085] The backbone network follows the standard design of YOLOv8s, including basic convolutional layers (Conv), cross-stage partial connection modules (C2f), and spatial pyramid pooling fast modules (SPPF). The Conv module achieves preliminary feature extraction and spatial downsampling by setting appropriate convolutional kernels, strides, and padding methods. The C2f module effectively alleviates the problem of gradient information loss by utilizing cross-connections of partial feature branches, and further refines multi-scale and deep semantic information through multi-layer stacking. Unlike the original YOLOv8s, this paper introduces an additional CGAFusion module, namely the content-focused attention-based hybrid fusion module (CFAMFM), after completing upsampling, concatenation, and partial convolution processing in each scale branch (P3, P4, P5). In this module, Content Focused Attention (CFA) is first used to weight the input features in both channel and spatial dimensions, generating a refined spatial importance map. Then, an adaptive weighted fusion strategy is employed to blend features from different resolutions or sources pixel-by-pixel, effectively enhancing weakened details in foggy images and suppressing noise interference during the fusion process. In this way, the network can more accurately capture key information about vehicles and pedestrians, mitigating the performance degradation caused by decreased contrast and blurred edges.

[0086] Finally, the high-quality multi-scale features obtained after feature fusion are passed to the detection module. In this part, the fused feature maps are used by the detection head to predict the target location and category, achieving accurate detection of vehicle and pedestrian targets in foggy scenes.

[0087] Content-Focused Attention Module: In complex foggy scenes, due to atmospheric scattering and light attenuation, local details in images, such as the edges and textures of vehicles and pedestrians, are often severely weakened, making it difficult for conventional convolutional networks to capture sufficient discriminative information when extracting features. To overcome this problem, this study proposes an attention-guided mechanism based on input content. This mechanism aims to adaptively weight features from both channel and spatial dimensions, thereby generating a Spatial Importance Map (SIM) with unique attention capabilities for each channel.

[0088] Let the input features be:

[0089] X∈R C×H×W (0.7)

[0090] Where C represents the number of channels, and H and W are the height and width of the feature map, respectively. First, to capture global statistical characteristics, global average pooling (GAP) is performed on X in the spatial dimension to obtain the global description vector of each channel. Next, in order to generate attention weights along the channel dimension, the following steps are performed: Two 1×1 convolution layers are performed, and a ReLU activation function is introduced after the first layer to obtain the channel attention vector W. c ∈R C×1×1 The calculation formula is as follows:

[0091]

[0092] C1×1 represents a convolution with a size of 1x1.

[0093] Here, max(0,·) represents the ReLU activation function, which ensures that the attention weights in the output are non-negative, thus better aligning with physical intuition. This process can effectively capture the overall importance of each channel to the target features.

[0094] Meanwhile, to obtain more detailed spatial distribution information, global average pooling and global max pooling are applied simultaneously to the input feature X, yielding the following results: and Global average pooling reflects the overall illumination and color distribution of an image, while global max pooling focuses more on information from the most salient local regions. Subsequently, these two spatial descriptions are concatenated along the channel dimension to form a feature map containing dual spatial statistical information, denoted as:

[0095]

[0096] The stitched result is then fed into a 7×7 convolutional layer (with a kernel size of 7×7, typically using reflection padding to reduce edge effects) to generate a spatial attention map W. s Its formula is expressed as follows:

[0097]

[0098] C7×7 represents a 7x7 convolution.

[0099] The above process utilizes the advantages of both average pooling and max pooling, enabling the generated spatial attention map to fully reflect both local salient regions and overall background information in the image, thereby more accurately representing the weakened target region in foggy images.

[0100] In obtaining channel attention W c After spatial attention Ws, since the former is a single value for each channel and the latter is a global spatial distribution map, the two can be directly used to achieve preliminary cross-dimensional information fusion. Therefore, this paper adopts an element-wise addition method to fuse Ws... c Combined with Ws, a rough fusion attention map W is formed. coa The formula is as follows:

[0101] W coa =W c +W s (0.11)

[0102] It should be noted that, due to W c The original dimensions are C×1×1, and Ws is 1×H×W. Before addition, the dimensions are automatically expanded through a broadcast mechanism to align the channels with the spatial information. Although this step is simple, it has already achieved a preliminary fusion of global channel information and local spatial information, forming a basic attention estimate.

[0103] However, directly using Wcoa is still insufficient to accurately reflect the differences in local regions within each channel. To further refine the attention map, this method introduces an adaptive adjustment strategy based on input features. Specifically, Wcoa is concatenated with the original input features X along the channel dimension, and cross-channel information exchange is achieved through channel shuffle (CS) operation. Then, group convolution (GC) is used to extract local details. Finally, the output is mapped to the [0,1] range using the sigmoid activation function, thus obtaining a fine-grained spatial importance map W specific to each channel, expressed by the following formula:

[0104] W = σ(GC) 7×7 (CS([X,W coa]))) (0.12)

[0105] Among them GC 7×7 This is represented as a 7x7 grouped convolution.

[0106] In this process, channel rearrangement facilitates effective information exchange between different channels, while grouped convolution further mines fine-grained structural information through multi-scale convolutional kernel learning of local regions. The final generated W accurately reflects the importance of each pixel position in the image to each channel, allowing for targeted adjustment of the contribution of each feature during subsequent feature fusion. (See structural diagram) Figure 5 The diagram visually illustrates the entire process from input feature X to final output W, sequentially showing operations such as global pooling, 1×1 convolution, 7×7 convolution, channel concatenation, channel rearrangement, grouped convolution, and sigmoid activation. A hybrid fusion strategy based on content-focused attention is also presented.

[0107] In multi-scale feature fusion, traditional simple fusion methods (such as direct summation or concatenation) often fail to fully leverage the advantages of features at different levels due to significant differences in resolution, semantic information, and local detail representation. This is especially critical in foggy scenarios where the fusion of low-resolution deep features and high-resolution shallow features is even more challenging. To address this, this method, based on generating a fine spatial importance map W, designs an adaptive weighted hybrid fusion strategy that allows features from different sources to complement and synergize while retaining their respective advantages.

[0108] Let x and y be the two features to be fused, where x and y come from different branches or scales of the network. First, x and y are initially fused to obtain preliminary fused features:

[0109] The initial fusion result can integrate the information from the two feature streams to a certain extent. However, due to the different emphases of the features in terms of detail and semantics, simple addition may dilute some key details. Therefore, the fine attention map W generated earlier is used to weight and adjust the two feature streams. Specifically, element-wise multiplication is used to multiply W by x, while complementary weights 1-W are used to modulate y, thereby achieving adaptive fusion. Its mathematical expression is:

[0110] F fuse =x + y + W⊙x + (1 - W)⊙y

[0111] Here, ⊙ represents element-wise multiplication. This formula shows that by adjusting the attention weights W, regions that are more significant in x can be further amplified based on the initial fusion, while compensating for weaker parts in y. Ultimately, the fused feature Ffuse achieves a balance between preserving low-level details and deep semantic information. The entire fusion process is shown in the structure diagram (see Figure Y), which details how fine attention maps are used to perform pixel-wise weighted adjustment of the input features, and how 1×1 convolutions are used to further integrate them to obtain the final fused feature.

[0112] In practical implementation, to further reduce noise interference and optimize feature representation during the fusion process, the hybrid fusion module performs a linear mapping through a 1×1 convolution after the aforementioned weighted fusion. This convolution operation effectively compresses redundant information and integrates multi-scale features, thereby outputting a feature map Fuse with higher discriminative power for use by subsequent detection modules.

[0113] This content-focused attention-based hybrid fusion scheme can theoretically better coordinate feature information at different scales and levels, especially in real foggy scenes, effectively mitigating the performance degradation of object detection caused by low contrast and blurred details. Through precise attention guidance, the network can accurately capture key features of vehicles and pedestrians at each channel and pixel level, thus providing the detector with richer and more discriminative input information.

[0114] Obviously, the specific implementation of this invention is not limited to the above-described methods. Any non-substantial improvements made using the inventive concept and technical solution of this invention are within the protection scope of this invention.

Claims

1. An image dehazing method based on frequency domain information fusion and domain adaptation, characterized in that: This includes correcting low-frequency global information and high-frequency local information after image decomposition by using frequency domain information enhancement and fusion strategies when the autoencoder-based contrastive regularization network AECRNet performs image dehazing; and statistically adjusting the deep features extracted by the encoder of the AECRNet pre-trained model through a domain adaptation strategy guided by visual cues, so that its mean and standard deviation are close to the statistical distribution of synthetic domain data.

2. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 1, characterized in that: At the encoder end of the AECRNet network, the amplitude enhancement module AEB repairs low-frequency global information to restore the brightness, contrast and color balance of the image. In the decoder front end of the AECRNet network, high-frequency local information is enhanced by the Phase Reconstruction Block (PRB) to restore edges and details in the image.

3. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 2, characterized in that: A PDA module is set after the output of each encoder layer in the AECRNet network, and a visual cue guidance strategy is adopted for domain adaptation. During the inference phase, the PDA module adjusts the statistics of the deep features extracted by the encoder by analyzing the feature distribution of the cue image, so as to ensure that its mean and standard deviation are consistent with the statistical distribution of the synthetic data, thereby reducing the domain difference between training and testing data and improving the generalization ability of the network.

4. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 3, characterized in that: The frequency domain information enhancement and fusion strategy decomposes the image from the spatial domain into amplitude spectrum and phase spectrum through Fourier transform, which correspond to low-frequency and high-frequency information, respectively. The low-frequency component reflects the overall brightness, contrast, and color distribution of the image, while the high-frequency component carries the image's edges, textures, and structural details; the interference from rain and fog is compensated by correcting the amplitude and phase separately.

5. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 4, characterized in that: Both the AEB and PRB modules adopt a basic architecture of "spatial branch - frequency domain branch - feature fusion". The AEB or PRB module receives spatial domain features from the previous layer or skip connections and extracts basic information through regular convolution. Then, the input features are transformed into the frequency domain through fast Fourier transform to obtain the amplitude spectrum and phase spectrum. Then, the frequency domain branch of AEB focuses on processing the amplitude spectrum and uses a series of lightweight 1×1 convolutional layers to enhance the amplitude spectrum, thereby compensating for the global energy attenuation caused by haze. The frequency domain branch of PRB focuses on the correction of the phase spectrum, enhancing edge and texture information through convolution operations; After completing the frequency domain processing, the AEB and PRB modules respectively use inverse Fourier transform to reconstruct the processed frequency domain information back into the spatial domain, and fuse it with the original spatial domain features through channel splicing or addition. After a 1×1 convolution layer, the output is integrated to form the corrected feature representation.

6. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 5, characterized in that: An AEB module is deployed at the end of the encoder of the AECRNet network to effectively correct the amplitude spectrum of the image; a PRB module is inserted in the first two layers of the decoder to correct the phase spectrum. An adaptive mixup strategy is adopted to dynamically fuse the outputs of AEB and PRB, achieving more refined compensation between global equalization and local detail restoration, and providing higher quality visual input for subsequent object detection tasks.

7. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 6, characterized in that: The domain-adaptive strategy for visual cue guidance during the testing phase includes utilizing the visual cue generation module PGM from real foggy images during the preprocessing stage. r and the composite domain haze-free image y s Generate a prompt image p in the middle; Secondly, in the feature transformation stage, the feature adaptation module (FAM) is used to process the real image features f output by each layer of the encoder. xr Feature-level normalization is performed to make its statistics gradually approach the statistics of the corresponding layer of the prompt image.

8. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 7, characterized in that: In the visual cue generation module, a haze-free image y is first selected. s For reference, and compared with a real foggy image x r Perform size matching; Subsequently, for x r and y s Simultaneously, image-level normalization is employed, the core operation of which is implemented through the following formula: Where, μ c (·) and σ c (·) represent the mean and standard deviation of the image in spatial dimension H×W and for each channel R, G, B, respectively; to capture local haze distribution more precisely, PGM introduces a partitioning strategy: x r and y s Each image is divided into multiple non-overlapping blocks, and a normalization operation is performed independently on each block, thereby generating a more accurate cue image with local features; to address the color imbalance problem, if x is detected... r If there is a significant monochrome deviation, a color balance strategy is implemented during normalization, which means averaging the statistics of each channel to avoid overall color shift due to a certain channel being too high or too low.

9. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 8, characterized in that: In the Feature Adaptation Module (FAM), the real image x r Multi-layer features f are extracted by the encoder xr Simultaneously, the prompt image p is processed in the same layer to obtain the corresponding prompt feature f. p The goal of FAM is to make f xr The channel mean and standard deviation are adjusted to be consistent with f. p This makes the features output by the encoder more statistically consistent with the pattern of fog-free images in the synthetic domain.

10. The image dehazing method based on frequency domain information fusion and domain adaptation as described in claim 7, characterized in that: In the encoder, the FAM module is inserted after each basic convolution or residual block, utilizing the cumulative adjustment of multi-level features to improve f at each scale. xr They can all gradually approach the statistical characteristics of the layer corresponding to the prompt image p.

Images