Frequency domain image defogging method and system based on SAD-Net network, medium, program product and terminal

Abstract

According to the frequency domain image defogging method and system based on the SAD-Net network, the medium, the program product and the terminal provided by the invention, downsampling processing is carried out on the foggy image, and the floating point calculation amount in the subsequent model operation process can be reduced by 75%. And the SAD-Net network model is used, so that high-frequency details can be added in the defogging process, and the loss of the high-frequency details caused by down-sampling is relieved. And meanwhile, the fog distribution map is directly subtracted from the original map, so that information loss can be reduced. According to the image defogging method, details are more prominent, high-frequency information is richer, and target detection is more obvious after the target image is defogged.

Description

Technical Field

[0001] The present application relates to the field of image defogging technology, and in particular to a frequency domain image defogging method, system, medium, program product, and terminal based on a SAD-Net network. Background Art

[0002] Currently, the visual quality of images captured in hazy scenes significantly degrades, primarily manifesting in reduced contrast and color distortion. These issues not only affect the intuitive visual experience but also have a significant negative impact on the performance of high-level vision tasks. For example, in tasks like object detection and semantic segmentation, the quality of the input image directly determines the performance of the task. Therefore, dehazing hazy images to restore image clarity and detail has become an essential step in these visual tasks.

[0003] In recent years, with the rapid development of deep learning technology, convolutional neural networks (CNNs) have made significant progress in image dehazing. For example, the AOD-Net model, based on dark channel prior theory, estimates the transmission map and atmospheric light, incorporating an atmospheric scattering model to output a dehazed image. While the AOD-Net model can achieve image dehazing effects to a certain extent, its performance relies heavily on accurate estimation of the transmission map and atmospheric light, resulting in certain limitations. Consequently, subsequent research has proposed end-to-end image restoration models, such as AOD-Net and FFA-Net, which can directly generate dehazed images and significantly improve performance.

[0004] However, in practical applications, end-to-end technical solutions still face some challenges. For one thing, most of these methods use standard convolution, which cannot effectively extract high-frequency information obscured by haze. This results in a loss of high-frequency information in the dehazed image, affecting the image's detail recovery. Furthermore, while FFA-Net uses pixel-by-pixel attention to generate spatial importance maps, adaptively processing regions of varying haze density, it is limited by the size of its receptive field and struggles to effectively capture interactions between global features. Consequently, it fails to fully consider the image's overall structural information, impacting the dehazing effect. Summary of the Invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a frequency domain image defogging method, system, medium, program product and terminal based on the SAD-Net network, which is used to solve the problem that the end-to-end defogging network based on deep learning algorithm in the prior art has insufficient image defogging effect.

[0006] To achieve the above-mentioned objectives and other related objectives, the first aspect of the present application provides a frequency domain image defogging method based on a SAD-Net network, comprising: obtaining a foggy image to be defogged, downsampling the foggy image to be defogged to obtain a reduced-size foggy image; inputting the reduced-size foggy image into a pre-trained SAD-Net image defogging model to output an initial defogged image; subtracting the reduced-size foggy image from the initial defogged image to obtain a first fog distribution map; upsampling the first fog distribution map to obtain a second fog distribution map; subtracting the foggy image to be defogged from the second fog distribution map to obtain a final defogged image corresponding to the foggy image to be defogged.

[0007] In some embodiments of the first aspect of the present application, the construction process of the pre-trained SAD-Net image dehazing model includes: establishing a SAD-Net network based on the U-Net architecture; introducing a frequency domain detail enhancement convolution module and a frequency domain detail enhancement attention module into the SAD-Net network to obtain a SAD-Net image dehazing model; training the SAD-Net image dehazing model to obtain a pre-trained SAD-Net image dehazing model.

[0008] In some embodiments of the first aspect of the present application, a frequency domain detail enhancement convolution module and a frequency domain detail enhancement attention module are introduced into the SAD-Net network to obtain a SAD-Net image dehazing model, including: the structure of the SAD-Net network includes an encoder, an adaptive feature transformation module and a decoder; a frequency domain detail enhancement convolution module is introduced into the encoder and decoder of the SAD-Net network respectively, and a frequency domain detail enhancement attention module is introduced into the adaptive feature transformation module of the SAD-Net network to obtain the SAD-Net image dehazing model.

[0009] In some embodiments of the first aspect of the present application, the frequency domain detail enhancement convolution module includes: a first wavelet transform processing layer, a first differential convolution processing layer, a first splicing processing layer, a first inverse wavelet transform processing layer, a first 3×3 convolution layer, a first residual connection layer, a second 3×3 convolution layer and a second residual connection layer; wherein, the first wavelet transform processing layer, the first differential convolution processing layer, the first splicing processing layer, the first inverse wavelet transform processing layer, the first 3×3 convolution layer, the first residual connection layer, the second 3×3 convolution layer and the second residual connection layer are connected in sequence; the first residual connection layer is connected to the input layer of the frequency domain detail enhancement convolution module; the second residual connection layer is connected to the input layer of the frequency domain detail enhancement convolution module.

[0010] In some embodiments of the first aspect of the present application, the frequency domain detail enhancement attention module includes: a second wavelet transform processing layer, a second differential convolution processing layer, a second splicing processing layer, a second inverse wavelet transform processing layer, a third 3×3 convolution layer, a third residual connection layer, a frequency domain guided attention layer and a fourth 3×3 convolution layer; wherein, the second wavelet transform processing layer, the second differential convolution processing layer, the second splicing processing layer, the second inverse wavelet transform processing layer, the third 3×3 convolution layer, the third residual connection layer, the frequency domain guided attention layer, the fourth residual connection layer and the fourth 3×3 convolution layer are connected in sequence; the third residual connection layer is connected to the input layer of the frequency domain detail enhancement attention module; the fourth residual connection layer is connected to the input layer of the frequency domain detail enhancement attention module.

[0011] In some embodiments of the first aspect of the present application, the frequency domain guided attention layer includes: a third wavelet transform processing layer, a third splicing processing layer, a first average pooling layer, a first one-dimensional convolution layer, a first Sigmoid function layer, a first 1×1 convolution layer, a second average pooling layer, a second one-dimensional convolution layer, a second Sigmoid function layer, a first point multiplication operation layer, a third average pooling layer, a normalization layer, a second 1×1 convolution layer, a third 1×1 convolution layer, a fourth 1×1 convolution layer, a CA-MHSA layer, a fourth average pooling layer, a third Sigmoid function layer, a second point multiplication operation layer, and a third inverse wavelet transform processing layer; wherein the third wavelet transform processing layer is connected to the third splicing processing layer; the third splicing processing layer is respectively connected to the first average pooling layer, the first 1×1 convolution layer and the second average pooling layer; the first average pooling layer, the first one-dimensional convolution layer, the first The first Sigmoid function layer is connected in sequence; the second average pooling layer, the second one-dimensional convolution layer, and the second Sigmoid function layer are connected in sequence; the first Sigmoid function layer, the first 1×1 convolution layer, and the second Sigmoid function layer are connected in sequence; the first Sigmoid function layer, the first 1×1 convolution layer, and the second Sigmoid function layer are respectively connected to the first dot product operation layer; the first dot product operation layer, the third average pooling layer, and the normalization layer are connected in sequence; the normalization layer is respectively connected to the second 1×1 convolution layer, the third 1×1 convolution layer, and the fourth 1×1 convolution layer; the second 1×1 convolution layer, the third 1×1 convolution layer, and the fourth 1×1 convolution layer are respectively connected to the CA-MHSA layer; the CA-MHSA layer, the fourth average pooling layer, the third Sigmoid function layer, the second dot product operation layer, and the third inverse wavelet transform processing layer are connected in sequence; the first dot product operation layer is connected to the second dot product operation layer.

[0012] To achieve the above-mentioned objectives and other related objectives, the second aspect of the present application provides a frequency domain image defogging system based on the SAD-Net network, including: a downsampling processing module, used to obtain a foggy image to be defogged, and downsample the foggy image to be defogged to obtain a reduced-size foggy image; a model processing module, inputting the reduced-size foggy image into a pre-trained SAD-Net image defogging model, and outputting an initial defogged image; a subtraction processing module, used to subtract the reduced-size foggy image from the initial defogged image to obtain a first fog distribution map; an upsampling processing module, used to upsample the first fog distribution map to obtain a second fog distribution map; a defogged image acquisition module, used to subtract the foggy image to be defogged from the second fog distribution map to obtain a final defogged image corresponding to the foggy image to be defogged.

[0013] To achieve the above-mentioned objectives and other related objectives, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the frequency domain image dehazing method based on the SAD-Net network.

[0014] To achieve the above-mentioned objectives and other related objectives, the fourth aspect of the present application provides a computer program product, which includes a computer program code. When the computer program code is run on a computer, the computer implements the frequency domain image dehazing method based on the SAD-Net network.

[0015] To achieve the above-mentioned objectives and other related objectives, the fifth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the frequency domain image dehazing method based on the SAD-Net network.

[0016] As described above, the frequency domain image defogging method, system, medium, program product, and terminal provided by this application based on the SAD-Net network have the following beneficial effects:

[0017] This application uses a downsampling method to reduce the amount of floating-point operations required during subsequent model execution by 75 percent. Furthermore, the SAD-Net network model can increase high-frequency details during the defogging process, alleviating the loss of high-frequency details caused by downsampling. Directly subtracting the fog distribution map from the original image can also minimize information loss. This application's image defogging method can enhance the details of the defogging target image, enrich the high-frequency information, and make the detected target more distinct. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1Shown is a flow chart of a frequency domain image dehazing method based on a SAD-Net network in one embodiment of the present application.

[0019] Figure 2 Shown is a structural diagram of a SAD-Net image dehazing model in one embodiment of the present application.

[0020] Figure 3 Shown is a structural diagram of a frequency domain detail enhancement convolution module in one embodiment of the present application.

[0021] Figure 4 Shown is a calculation diagram of four differential convolution methods in an embodiment of the present application.

[0022] Figure 5 Shown is a structural diagram of a frequency domain detail enhancement attention module in one embodiment of the present application.

[0023] Figure 6 Shown is a structural diagram of a frequency domain guided attention layer in one embodiment of the present application.

[0024] Figure 7 Shown is a schematic diagram of visual and frequency domain comparison results of different single image defogging methods in one embodiment of the present application.

[0025] Figure 8 Shown is a structural diagram of a frequency domain image defogging system based on a SAD-Net network in one embodiment of the present application.

[0026] Figure 9 Shown is a structural schematic diagram of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0028] Before further explaining the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations:

[0029] <1> Deep learning: Deep learning is a machine learning method in which algorithms automatically learn useful features from data and use these features to perform tasks such as classification and recognition. Deep learning is widely used in fields such as natural language processing, speech recognition, and computer vision. In natural language processing, deep learning can be applied to tasks such as text classification, sentiment analysis, and machine translation. In speech recognition, deep learning can automatically learn and identify speech features, improving speech recognition accuracy. In computer vision, deep learning can build complex neural networks to recognize and classify images, and even achieve advanced functions such as image generation.

[0030] <2> U-net network: The U-net network structure is a type of fully convolutional neural network (FCN), a deep learning network consisting of two parts: an encoding layer and a decoding layer. The encoding layer primarily extracts contextual information from the image, while the decoding layer locates the target area within the image. The U-net network employs data augmentation strategies to achieve accurate learning even with relatively few samples. The lack of fully connected layers in the U-net network structure significantly reduces the number of parameters required for learning, greatly improving the learning efficiency of the network structure.

[0031] <3> Peak signal-to-noise ratio (PSNR): A commonly used metric for measuring image or video quality, PSNR assesses distortion by calculating the ratio of the maximum signal power to the noise power. Higher PSNR values indicate better image quality and are widely used in compression, denoising, and transmission.

[0032] <4> SSIM (Structural Similarity Index): A quantitative metric used to assess the similarity between two images. It not only compares image brightness and contrast but also deeply considers the image's structural information. It has a wide range of applications in image quality assessment, often used to evaluate the effectiveness of algorithms such as image compression, image denoising, and image enhancement.

[0033] To facilitate understanding of the embodiments of this application, first Figure 1 Detailed description. Figure 1 The following is a flow chart showing a frequency domain image dehazing method based on a SAD-Net network in an embodiment of the present invention. The method in this embodiment includes:

[0034] Step S11: obtaining a foggy image to be defogged, and performing downsampling processing on the foggy image to be defogged to obtain a foggy image with a reduced size.

[0035] It should be understood that foggy images refer to images captured during the shooting process that exhibit blurred object details, reduced contrast, and distorted colors due to fog, haze, or gloomy weather. Downsampling an image reduces both its height and width. For example, an original image with a resolution of 1920×1080 may be reduced to 960×540 or smaller after downsampling.

[0036] In image dehazing, high-resolution images require more computing resources and time. Downsampling can reduce the image to an appropriate size, reducing the computational effort and speeding up the subsequent dehazing process while preserving the image's key features to a certain extent. Downsampling the foggy image to be dehazed produces a reduced-size foggy image that retains the key features and structural information of the original foggy image, making it sufficient for subsequent dehazing model processing.

[0037] The downsampling methods include bilinear interpolation sampling method, nearest neighbor interpolation sampling method, bicubic interpolation sampling method, spline interpolation sampling method, non-local average interpolation sampling method and bilateral filter interpolation sampling method, etc. The selection of the downsampling method in this embodiment is not limited.

[0038] Step S12: Input the reduced-size foggy image into the pre-trained SAD-Net image defogging model and output the initial defogging image.

[0039] It should be noted that feeding a downsized foggy image into the pre-trained SAD-Net image dehazing model for dehazing reduces the amount of data processed by the model, lowering computational complexity and speeding up the model's processing speed, compared to feeding the original foggy image into the pre-trained SAD-Net image dehazing model for dehazing. Therefore, downsampling significantly reduces the amount of computation required in image dehazing, improving computational speed and efficiency.

[0040] In one embodiment, the construction process of the pre-trained SAD-Net image dehazing model includes: establishing a SAD-Net network based on the U-Net architecture; introducing a frequency domain detail enhancement convolution module and a frequency domain detail enhancement attention module into the SAD-Net network to obtain a SAD-Net image dehazing model; training the SAD-Net image dehazing model to obtain a pre-trained SAD-Net image dehazing model.

[0041] It should be understood that the structure of the SAD-Net network in this embodiment is designed based on the U-Net architecture and is used for single-image dehazing tasks. The full name of SAD-Net is A Full Spectral Self-Attention Detail Enhancement Network. U-Net is a convolutional neural network architecture mainly used for image segmentation. The structure of U-Net is similar to a U shape, consisting of an encoder (downsampling path) and a decoder (upsampling path). Due to the efficient performance and flexible structure of U-Net, U-Net is widely used in medical image processing, satellite image analysis and other fields.

[0042] The U-Net architecture used in this embodiment is a three-layer U-Net style encoder-decoder architecture. A SAD-Net network is established based on the U-Net architecture, and a frequency domain detail enhancement convolution module (SDEB) and a frequency domain detail enhancement attention module (SDEAB) are introduced into the SAD-Net network to obtain a SAD-Net image defogging model. The frequency domain detail enhancement convolution module is used to extract and enhance high-frequency details in the image. The frequency domain detail enhancement attention module is used to enhance the fine-grained processing of uneven haze areas in the image using the frequency domain guided attention mechanism, thereby improving feature recognition capabilities. The defogging effect can be improved by using the frequency domain detail enhancement convolution module and the frequency domain detail enhancement attention module.

[0043] In one embodiment, a frequency domain detail enhancement convolution module and a frequency domain detail enhancement attention module are introduced into the SAD-Net network to obtain a SAD-Net image dehazing model, including: the structure of the SAD-Net network includes an encoder, an adaptive feature transformation module and a decoder; a frequency domain detail enhancement convolution module is introduced into the encoder and decoder of the SAD-Net network, respectively, and a frequency domain detail enhancement attention module is introduced into the adaptive feature transformation module of the SAD-Net network to obtain the SAD-Net image dehazing model.

[0044] It should be noted that if Figure 2 As shown in the figure, the structure of the SAD-Net network based on the U-Net architecture is divided into three parts: encoder part, adaptive feature transform module, and decoder part. The encoder, adaptive feature transform module, and decoder are connected in sequence. Multiple frequency domain detail enhancement convolution modules are introduced into the encoder of the SAD-Net network, multiple frequency domain detail enhancement attention modules are introduced into the adaptive feature transform module, and multiple frequency domain detail enhancement convolution modules are introduced into the decoder. The SAD-Net network finally generated is the SAD-Net image dehazing model.

[0045] Combine Figure 2 The structure of the encoder includes a first frequency domain detail enhancement convolution layer (SDEB×N1), a first downsampling layer (Down-sampling), a second frequency domain detail enhancement convolution layer (SDEB×N2), and a second downsampling layer (Down-sampling) arranged in sequence; the structure of the adaptive feature transformation module includes a frequency domain detail enhancement attention layer (SDEAB×N3) and a first fusion splicing layer (Fusion) arranged in sequence; the structure of the decoder includes a first upsampling layer (Up-sampling), a third frequency domain detail enhancement convolution layer (SDEB×N4), a second fusion splicing layer (Fusion), a second upsampling layer (Up-sampling), and a fourth frequency domain detail enhancement convolution layer (SDEB×N5) arranged in sequence. The first downsampling layer is connected to the second fusion splicing layer; and the second downsampling layer is connected to the first fusion splicing layer.

[0046] In this embodiment, N1 frequency domain detail enhancement convolution modules (SDEBs) are introduced in the first frequency domain detail enhancement convolution layer, N2 frequency domain detail enhancement convolution modules (SDEBs) are introduced in the second frequency domain detail enhancement convolution layer, N3 frequency domain detail enhancement attention modules (SDEABs) are introduced in the frequency domain detail enhancement attention layer, N4 frequency domain detail enhancement convolution modules (SDEBs) are introduced in the third frequency domain detail enhancement convolution layer, and N5 frequency domain detail enhancement convolution modules (SDEBs) are introduced in the fourth frequency domain detail enhancement convolution layer. Preferably, the corresponding numbers of N1, N2, N3, N4, and N5 are 4, 4, 8, 4, and 4, respectively. Among them, SDEAB stands for a spectral detail enhancement attention block. SDEB stands for SpectralDetail Enhancement Block.

[0047] It should be explained that the encoder structure in the SAD-Net image dehazing model is responsible for compressing the spatial dimensions of the input image while extracting multi-scale feature information. That is, in the encoder, the spatial resolution of the image is gradually reduced through a series of convolutional layers and pooling layers (or downsampling layers), while the number of feature channels is increased to capture richer features. In this embodiment, the encoder compresses the spatial dimensions of the feature map through two-step downsampling processing, retaining more global semantic information, and uses the frequency domain detail enhancement convolution module to effectively extract high-frequency details. The adaptive feature transformation module consists of multiple frequency domain detail enhancement attention modules, which use the frequency domain to guide the attention mechanism to enhance the fine-grained processing of uneven haze areas and improve feature recognition capabilities. In the decoder structure, the spatial resolution of the image is restored through a two-step upsampling process, and combined with the detail features retained by the encoder to finally obtain the dehazed image. This ensures that the dehazed image maintains a good global structure while retaining rich local details.

[0048] The specific process of defogging the input image based on the SAD-Net image defogging model is as follows: Figure 2 As shown:

[0049] The input image (Hazy Input) enters the encoder. First, the input image is processed by a 3×3 standard convolution layer (not shown) to expand the channel dimension of the image from 3 to 32 while keeping the size unchanged, and obtain the feature map after convolution. The feature map after convolution enters the first frequency domain detail enhancement convolution layer. In the first frequency domain detail enhancement convolution layer, four frequency domain detail enhancement convolution modules are used continuously to extract features and obtain the first enhanced feature map. The first enhanced feature map enters the first downsampling layer. The first downsampling layer uses a 3×3 standard convolution layer to expand the channel dimension of the feature map from 32 to 64 and reduce the size of the feature map to half the original size of the input image to obtain the first downsampled feature map. The first downsampled feature map enters the second frequency domain detail enhancement convolution layer. In the second frequency domain detail enhancement convolution layer, four frequency domain detail enhancement convolution modules are used continuously to continue feature extraction and obtain the second enhanced feature map. The second enhanced feature map enters the second downsampling layer. The second downsampling layer uses a 3×3 standard convolution layer to expand the channel dimension of the feature map from 64 to 128, and further reduces the size of the feature map to half the size of the first downsampling feature map to obtain the second downsampling feature map.

[0050] The second downsampled feature map then enters the adaptive feature transformation module. It then passes through the eight frequency domain detail enhancement attention modules in the frequency domain detail enhancement attention layer to enhance the feature map's details, resulting in a third enhanced feature map. The third enhanced feature map then enters the first fusion and splicing layer, where it is fused and spliced with the second downsampled feature map. This leaves the feature map unchanged in size and number of channels, resulting in the first fused and spliced feature map.

[0051] Finally, the first fused and spliced feature map enters the decoder. It first enters the first upsampling layer, which uses a 3×3 standard deconvolution layer. The number of channels in the feature map is reduced from 128 to 64, and its size is doubled, resulting in the first upsampled feature map. The first upsampled feature map enters the third frequency-domain detail enhancement convolution layer, where it undergoes four frequency-domain detail enhancement convolution modules for feature extraction, resulting in the fourth enhanced feature map. The fourth enhanced feature map enters the second fused and spliced layer, where it is fused and spliced with the first downsampled feature map, resulting in the second fused and spliced feature map. The second fused and spliced feature map enters the second upsampling layer, where it uses a 3×3 standard deconvolution layer, where the number of channels in the feature map is reduced from 64 to 32, and its size is doubled, resulting in the second upsampled feature map. The second upsampled feature map enters the fourth frequency-domain detail enhancement convolution layer, where it undergoes four frequency-domain detail enhancement convolution modules for feature extraction, resulting in the fifth enhanced feature map. The fifth enhanced feature map is processed by a 3×3 standard convolution layer (not shown), and the number of channels is reduced from 32 to 3, and the dehazed image (Clear output) is finally output. At this time, the dehazed image is the same size as the input image.

[0052] It's important to note that after building the SAD-Net image dehazing model, it needs to be trained. During training, the model learns the characteristic differences between foggy and clean images, such as the impact of fog on image contrast, brightness, and color distribution. By learning these characteristics, the model can better identify and remove fog, ultimately obtaining a pre-trained SAD-Net image dehazing model. Compared to the untrained model, the trained SAD-Net image dehazing model shows significant improvements in dehazing performance, accuracy, and robustness.

[0053] In one embodiment, if Figure 3 As shown, the frequency domain detail enhancement convolution module (SDEB) includes: a first wavelet transform processing layer (WT), a first differential convolution processing layer, a first splicing processing layer, a first inverse wavelet transform processing layer (IWT), a first 3×3 convolution layer, a first residual connection layer, a second 3×3 convolution layer, and a second residual connection layer;

[0054] Among them, the first wavelet transform processing layer, the first differential convolution processing layer, the first splicing processing layer, the first inverse wavelet transform processing layer, the first 3×3 convolution layer, the first residual connection layer, the second 3×3 convolution layer, and the second residual connection layer are connected in sequence; the first residual connection layer is connected to the input layer of the frequency domain detail enhancement convolution module; the second residual connection layer is connected to the input layer of the frequency domain detail enhancement convolution module.

[0055] The specific processing flow of the input feature map in the frequency domain detail enhancement convolution module (SDEB) is as follows:

[0056] First, the input layer of the frequency domain detail enhancement convolution module inputs the original feature map. The input original feature map enters the first wavelet transform processing layer and is decomposed into four sub-bands after wavelet transform processing. The four sub-bands are low-frequency sub-band (LL), horizontal high-frequency sub-band (HL), vertical high-frequency sub-band (LH), and diagonal high-frequency sub-band (HH). Among them, the LL sub-band contains the low-frequency information of the image and mainly retains the smooth areas and overall contour information of the image. The HL sub-band, LH sub-band, and HH sub-band contain the high-frequency information of the image and focus on extracting edge details. The HL sub-band captures edge features in the horizontal direction, the LH sub-band captures edge features in the vertical direction, and the HH sub-band captures edge features in the diagonal direction.

[0057] Then, in the first differential convolution processing layer, the four sub-bands are subjected to corresponding differential convolution processing, that is, different differential convolution methods are applied to each sub-band to enhance the detail features of image feature extraction. Four differential convolution methods are used in the first differential convolution processing layer, such as Figure 4 As shown in the figure, the four differential convolution methods are the central difference convolution (CDC) method, the horizontal difference convolution (HDC) method, the vertical difference convolution (VDC) method, and the diagonal difference convolution (ADC) method.

[0058] The central difference convolution method is applied to the LL subband to enhance image details by calculating the difference between the central pixel of the image and its surrounding pixels. The horizontal difference convolution method is applied to the HL subband to enhance high-frequency details in the horizontal direction. The vertical difference convolution method is applied to the LH subband to enhance high-frequency details in the vertical direction. The diagonal difference convolution method is applied to the HH subband to enhance high-frequency details in the diagonal direction.

[0059] Finally, after differential convolution processing to obtain the feature maps of each subband, they enter the first splicing processing layer for splicing. The feature maps of each subband are connected to form a feature representation containing multi-frequency information, that is, the spliced feature image. The spliced feature image enters the first inverse wavelet transform processing layer, and is reconstructed into a first feature image of the same size as the original input feature image through inverse wavelet transform. The first feature image is further processed by the first 3×3 convolution layer to obtain the second feature image. The second feature image enters the first residual connection layer. The original feature map input layer of the frequency domain detail enhancement convolution module is also input into the first residual connection layer after the ReLU activation function operation. The splicing process is performed in the first residual connection layer to obtain the third feature image. The third feature image enters the second 3×3 convolution layer for processing to obtain the fourth feature image. The fourth feature image enters the second residual connection layer. In the second residual connection layer, the fourth feature image is spliced with the original feature map input layer of the frequency domain detail enhancement convolution module, and the enhanced detail feature image is finally obtained. The obtained detail-enhanced feature image is enhanced in details while retaining the information of the original feature map.

[0060] It should be noted that in foggy conditions, impurities in the air will affect the propagation of light in the air, resulting in decreased contrast and blurred edges in the image, which will have an adverse effect on the target detection task. In this embodiment, a frequency domain detail enhancement convolution module is constructed, and the wavelet transform method is used to separate and process the high-frequency and low-frequency information of the image, which can accurately extract and enhance the edge and detail features of the image, thereby significantly improving the clarity and quality of the image. And through this structure, the SAD-Net network can process features in different frequency domains to enhance the image details. Detail enhancement is especially important for defogging tasks because fog usually leads to the loss of image details, especially the loss of high-frequency information.

[0061] In one embodiment, if Figure 5 As shown, the frequency domain detail enhancement attention module (SDEAB) includes: a second wavelet transform processing layer (WT), a second differential convolution processing layer, a second splicing processing layer, a second inverse wavelet transform processing layer (IWT), a third 3×3 convolution layer, a third residual connection layer, a frequency domain guided attention layer (FGA), a fourth residual connection layer, and a fourth 3×3 convolution layer;

[0062] Among them, the second wavelet transform processing layer, the second differential convolution processing layer, the second splicing processing layer, the second inverse wavelet transform processing layer, the third 3×3 convolution layer, the third residual connection layer, the frequency domain guided attention layer, the fourth residual connection layer and the fourth 3×3 convolution layer are connected in sequence; the third residual connection layer is connected to the input layer of the frequency domain detail enhancement attention module; the fourth residual connection layer is connected to the input layer of the frequency domain detail enhancement attention module.

[0063] Specifically, the processing process of the frequency domain detail enhancement attention module is:

[0064] The input feature map enters the second wavelet transform (WT) layer, where it is decomposed into four subbands. These subbands then enter the second differential convolution layer, where each subband is processed using a corresponding differential convolution method. Each subband then enters the second splicing layer for splicing. The spliced feature map then enters the second inverse wavelet transform (IWT) layer, where it is processed using IWT and reconstructed into a feature map of the same size as the original input feature map. The feature map then enters the third 3×3 convolution layer, where it enters the third residual connection layer. The input feature map also enters the third residual connection layer after being activated by the ReLU function. In this layer, the convolutional feature map is concatenated with the ReLU-activated input feature map to obtain a residual spliced feature map. This residual spliced feature map then enters the frequency-guided attention layer (FGA), which uses the frequency-guided attention mechanism to enhance the details of the feature map, resulting in an enhanced feature map. The feature map after detail enhancement enters the fourth residual connection layer, and the feature map after detail enhancement is spliced with the input feature map in the fourth residual connection layer to obtain the residual spliced feature map. The residual spliced feature map enters the fourth 3×3 convolution layer, and after convolution processing, the enhanced feature map is finally output.

[0065] It should be noted that the core structure of the frequency-domain detail enhancement attention module is the frequency-guided attention layer (FGA), which can consider both low-frequency and high-frequency information at the same time, helping to restore clear and fog-free images.

[0066] In a specific embodiment, if Figure 6 As shown, the frequency-domain guided attention layer (FGA) includes:

[0067] The third wavelet transform processing layer, the third splicing processing layer, the first average pooling layer, the first one-dimensional convolution layer, the first Sigmoid function layer, the first 1×1 convolution layer, the second average pooling layer, the second one-dimensional convolution layer, the second Sigmoid function layer, the first point product operation layer, the third average pooling layer, the normalization layer, the second 1×1 convolution layer, the third 1×1 convolution layer, the fourth 1×1 convolution layer, the CA-MHSA layer, the fourth average pooling layer, the third Sigmoid function layer, the second point product operation layer, and the third inverse wavelet transform processing layer;

[0068] Among them, the third wavelet transform processing layer is connected to the third splicing processing layer; the third splicing processing layer is respectively connected to the first average pooling layer, the first 1×1 convolution layer and the second average pooling layer; the first average pooling layer, the first one-dimensional convolution layer, and the first Sigmoid function layer are connected in sequence; the second average pooling layer, the second one-dimensional convolution layer, and the second Sigmoid function layer are connected in sequence; the first Sigmoid function layer, the first 1×1 convolution layer and the second Sigmoid function layer are respectively connected to the first dot product operation layer;

[0069] The first dot product operation layer, the third average pooling layer, and the normalization layer are connected in sequence; the normalization layer is respectively connected to the second 1×1 convolution layer, the third 1×1 convolution layer, and the fourth 1×1 convolution layer; the second 1×1 convolution layer, the third 1×1 convolution layer, and the fourth 1×1 convolution layer are respectively connected to the CA-MHSA layer; the CA-MHSA layer, the fourth average pooling layer, the third sigmoid function layer, the second dot product operation layer, and the third inverse wavelet transform processing layer are connected in sequence; the first dot product operation layer is connected to the second dot product operation layer.

[0070] like Figure 6 As shown, the specific processing in FGA includes:

[0071] First, a feature map of dimension B × C × H × W is input, where B represents the batch size of the input image, C represents the number of channels, H represents the image height, and W represents the image width. The feature map passes through the third wavelet transform layer (WT), where it is decomposed into four subbands: the LL subband, the HL subband, the LH subband, and the HH subband. Feature maps are then obtained for each subband, each with dimensions B × C × H / 2 × W / 2. Each subband feature map then passes through the third concatenation layer (Concat), where it is concatenated to obtain a concatenated feature map of dimension B × (C × 4) × H / 2 × W / 2.

[0072] The concatenated feature map then enters the first average pooling layer (H AvgPool) for average pooling to aggregate spatial information. The first average pooling layer performs average pooling on the height dimension of the image, obtaining a first pooled feature map with dimensions of B×(C×4)×W / 2. The first pooled feature map then enters the first one-dimensional convolutional layer (1Conv1d), where convolution is performed to obtain the first one-dimensional convolutional feature map with dimensions of B×C×W / 2. The first one-dimensional convolutional feature map then enters the first Sigmoid function layer, where it is processed using the Sigmoid activation function to obtain the first attention weight. The concatenated feature map also enters the second average pooling layer (W AvgPool), which performs average pooling along the width dimension, resulting in a second pooled feature map with dimensions B × (C × 4) × H / 2. This second pooled feature map then enters the second one-dimensional convolutional layer (1Conv1d), where it undergoes convolution to obtain a second one-dimensional convolutional feature map with dimensions B × C × H / 2. This second one-dimensional convolutional feature map is then processed with the sigmoid activation function in the second sigmoid function layer to obtain the second attention weight. The concatenated feature map also enters the first 1 × 1 convolutional layer (1 × 1Conv), where it undergoes convolution to obtain a convolved feature map with dimensions B × C × H / 2 × W / 2. The first attention weight, the convolved feature map, and the second attention weight are multiplied in the first dot product layer to obtain a spatial attention feature map with dimensions B × C × H / 2 × W / 2. The spatial attention feature map can highlight the hazy areas in the image and suppress irrelevant areas.

[0073] Furthermore, the spatial attention feature map enters the third average pooling layer (AvgPool) for average pooling to obtain the third pooled feature map. The dimension of the third pooled feature map is B×C×H'×W', where H' and W' correspond to the height and width of the image obtained at different super-resolution multiples. It then enters the normalization layer (GroupNorm) for normalization. The normalized feature map enters the second 1×1 convolutional layer (1×1Conv), the third 1×1 convolutional layer (1×1Conv), and the fourth 1×1 convolutional layer (1×1Conv) for dimensionality reduction, generating the corresponding query (query, Q), key (key, K), and value (value, V) parameter matrices. The dimensions of the three parameter matrices are B×C×N', where N=H×W. The three parameter matrices Q, K, and V are input into the CA-MHSA layer. The attention weight matrix between the Q and K parameter matrices is calculated in the CA-MHSA layer, and the attention weight matrix between channels is normalized using the Softmax function. The attention weight matrix is then multiplied with the V parameter matrix to finally generate a weighted channel feature map.

[0074] Finally, the weighted channel feature map undergoes average pooling in the fourth average pooling layer (AvgPool), reducing its dimensions to B×C×1×1. The average pooled weighted channel feature map then enters the third Sigmoid function layer (Sigmoid) for processing. The result is multiplied with the spatial attention feature map in the second point multiplication layer to obtain a frequency domain feature map. The frequency domain feature map enters the third inverse wavelet transform (IWT) layer, which uses an inverse wavelet transform to restore the frequency domain feature map to the spatial domain, maintaining the same size as the input feature map, resulting in a feature map with enhanced details.

[0075] It should be explained that CA-MHSA stands for Channel-wise Attention Based on Self-attention, which means channel attention based on the self-attention mechanism. The CA-MHSA layer mainly enhances the correlation between channels through the channel self-attention mechanism.

[0076] It should be noted that, through wavelet transforms, FGA can process components of different frequencies separately in the frequency domain, which helps to better restore image details and structure. Through spatial and channel attention mechanisms, FGA can assign different weights to different features, enhancing the model's sensitivity to changes in fog distribution, thereby improving dehazing performance. FGA enhances the spatial characteristics of feature maps through spatial attention and the correlation of feature maps in the channel dimension through channel attention, which helps the model better capture and restore image details. The design of FGA enables it to adapt to different fog densities and non-uniform fog conditions, which is very important for dehazing tasks in practical applications.

[0077] Step S13: performing subtraction processing on the reduced-size foggy image and the initial defogged image to obtain a first fog distribution map.

[0078] Specifically, the downsized foggy image is fed into the pre-trained SAD-Net image defogging model, which outputs an initial defogged image. This initial defogged image is a small-scale, fog-free image. The pixel-by-pixel subtraction between the downsized foggy image and the initial defogged image reveals the distribution of fog in the image. This first fog distribution map highlights areas with denser fog.

[0079] Step S14: performing upsampling processing on the first fog distribution map to obtain a second fog distribution map.

[0080] The first fog distribution map is upsampled to enlarge its size, restoring it to the same size as the original foggy image to be defogged. This means that the size of the obtained second fog distribution map is the same as the size of the foggy image to be defogged. The upsampled image can better match the resolution of the original image. Upsampling methods include bilinear interpolation and nearest neighbor interpolation, and are not limited in this embodiment.

[0081] Step S15: performing a subtraction process on the foggy image to be defogged and the second fog distribution map to obtain a final defogged image corresponding to the foggy image to be defogged.

[0082] The original foggy image to be defogged is subtracted pixel by pixel from the second fog distribution map to remove the effects of fog. The second fog distribution map provides detailed information about the fog distribution, allowing for more precise fog removal. The resulting defogged image is clearer and offers better visual quality.

[0083] The present invention first downsamples the image to reduce the amount of computation; then uses the pre-trained SAD-Net model for preliminary defogging, which can make the edge details of the target image more prominent after defogging and the detected target more obvious; then through subtraction and upsampling operations, the fog distribution information is extracted and utilized, which can effectively remove the fog in the image, improve the image quality, and ultimately obtain a high-quality defogging image.

[0084] In order to facilitate the demonstration of the frequency domain image dehazing method based on the SAD-Net network of the present application, the following specific embodiments are provided for illustration.

[0085] SAD-Net is compared with eight other single image dehazing methods, namely DehazeNet, AOD-Net, GridDehazeNet, MSBDN, FFA-Net, DeHamer, AECR-Net and MB-TaylorFormer, and is comprehensively evaluated on the Dense-Haze, NH-Haze and I-Haze datasets.

[0086] Table 1 shows a quantitative comparison of the dehazing performance of different single-image dehazing models on the Dense-Haze, NH-Haze, and I-Haze datasets. The dehazing performance is evaluated using PSNR and SSIM, with higher PSNR and SSIM values indicating better performance. The symbol "--" indicates that the value is unavailable.

[0087] Table 1 Comparison results of different single image dehazing models

[0088]

[0089]

[0090] Furthermore, the downsampling method of the present invention can reduce the computational complexity of the SAD-Net image dehazing model. Table 2 compares the computational complexity of different single-image dehazing models. The results of the SAD-Net image dehazing model represent the computational complexity of downsampling before entering the model. Param represents parameter quantity, and FLOTs represents floating-point operations.

[0091] Table 2 Comparison of computational complexity of different single image dehazing models

[0092] Model Name Param(M) FLOTs(G) DehazeNet 0.008 0.5409 AOD-Net 0.0018 0.1146 GridDehazeNet 0.96 21.43 MSBDN 31.35 41.54 FFA-Net 4.456 287.5 DeHamer 132.4 48.93 AECR-Net 2.611 52.20 MB-TaylorFormer 7.43 88.1 SAD-Net 3.15 5.05

[0093] It should be noted that the results in Tables 1 and 2 above show that the SAD-Net model achieves optimal dehazing results in the Dense-Haze and I-Haze datasets, and also achieves excellent dehazing results in the NH-Haze dataset. Furthermore, the SAD-Net model of the present invention uses 15% fewer floating-point operations than other image dehazing models, achieving nearly the same results as other image dehazing models.

[0094] If the present invention does not perform downsampling, the SAD-Net model runtime parameter volume is 3.15M and the floating-point operation volume is 20.21G. If downsampling is performed, the SAD-Net model runtime parameter volume is 3.15M and the floating-point operation volume is 5.05G. Therefore, the present invention adopts a downsampling method to reduce the floating-point operation volume during the subsequent model operation by 75%. In addition, the use of the SAD-Net network model in this embodiment can increase high-frequency details in the defogging process, alleviating the loss of high-frequency details caused by downsampling. At the same time, directly using the original image to subtract the fog distribution map can reduce information loss.

[0095] Further, combined with Figure 7 illustrate, Figure 7 Visual and frequency domain comparisons of different single image dehazing methods are presented on the Dense-Haze dataset and the NH-Haze dataset. The different single image dehazing methods include FFA-Net, MSBDN, AECR-Net, DeHamer, and SAD-Net. Figure 7 The image above is the Dense-Haze dataset and its corresponding spectrogram. Figure 7 The following images are the NH-Haze dataset and its corresponding spectrogram. Figure 7 It also includes Hazy (original foggy image) and Clear (clear image). PSNR and SSIM are the values obtained by comparing the dehazed image with the original foggy image. low and Ehigh Respectively represent the low-frequency energy and high-frequency energy of the corresponding spectrum graph. The closer these two values are to the low-frequency energy and high-frequency energy of the clear image (Clear), the better the effect. When the low-frequency energy and high-frequency energy are close to the clear image (Clear), R high / low It should also be as close to a clear image as possible.

[0096] The low-frequency and high-frequency energies of the spectrogram obtained by the SAD-Net model on the Dense-Haze dataset are closer to the clear image than those obtained by other models. The low-frequency and high-frequency energies obtained on the NH-Haze dataset are also closer to the clear image than those obtained by other models. Therefore, the SAD-Net model used in this application has a better dehazing effect.

[0097] It is important to emphasize that the present invention employs a downsampling method, which can reduce the amount of floating-point operations required during subsequent model execution by 75 percent. Furthermore, the use of the SAD-Net network model can enhance high-frequency detail during the dehazing process, mitigating the loss of high-frequency detail caused by downsampling. Furthermore, directly subtracting the fog distribution map from the original image can minimize information loss. The present invention's image dehazing method can enhance detail, enrich high-frequency information, and make detected targets more distinct after dehazing.

[0098] In the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects, and do not limit their order. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity or execution order, and words such as "first" and "second" do not necessarily mean different.

[0099] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" represent examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0100] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0101] Figure 8 This is a frequency domain image defogging system based on the SAD-Net network provided in the embodiment of the present application. Figure 8 As shown, the system 800 includes:

[0102] A downsampling processing module 810 is used to obtain a foggy image to be defogged, and perform downsampling processing on the foggy image to be defogged to obtain a foggy image with a reduced size;

[0103] The model processing module 820 inputs the reduced-size foggy image into a pre-trained SAD-Net image defogging model and outputs an initial defogged image;

[0104] a subtraction processing module 830 for performing subtraction processing on the reduced-size foggy image and the initial defogged image to obtain a first fog distribution map;

[0105] an upsampling processing module 840, configured to perform upsampling processing on the first fog distribution map to obtain a second fog distribution map;

[0106] The defogging image acquisition module 850 is configured to perform a subtraction process on the foggy image to be defogged and the second fog distribution map to obtain a final defogged image corresponding to the foggy image to be defogged.

[0107] It should be understood that the specific processes of the above-mentioned corresponding modules have been described in detail in the above-mentioned method embodiments, and for the sake of brevity, they will not be repeated here.

[0108] It should also be understood that the division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present application may be integrated into a single processor, or may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0109] Figure 9 This is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Figure 9 As shown, the electronic terminal includes: at least one processor 901, a memory 902, at least one network interface 903 and a user interface 905. The various components in the device are coupled together via a bus system 904. It is understood that the bus system 904 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 9 Various buses are labeled as bus systems.

[0110] The user interface 905 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0111] It will be appreciated that the memory 902 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM) or a programmable read-only memory (PROM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0112] The memory 902 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 900. Examples of such data include: any executable program for operating on the electronic terminal 900, such as an operating system 9021 and an application 9022; the operating system 9021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 9022 may include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The frequency domain image dehazing method based on the SAD-Net network provided in the embodiment of the present invention may be included in the application 9022.

[0113] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 901. Processor 901 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 901 or by software instructions. The above processor 901 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 901 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 901 may be a microprocessor or any conventional processor. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium located in a memory. The processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0114] In an exemplary embodiment, the electronic terminal 900 may be configured to execute the aforementioned method using one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs).

[0115] According to the method provided in the embodiments of the present application, the present application also provides a computer program product, which includes: computer program code, which, when running on a computer, enables the computer to execute the frequency domain image dehazing method based on the SAD-Net network of any embodiment of the illustrated embodiments.

[0116] According to the method provided in the embodiments of the present application, the present application also provides a computer-readable storage medium, which stores a program code. When the program code is run on a computer, the computer executes the frequency domain image dehazing method based on the SAD-Net network of any embodiment shown in the embodiments.

[0117] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0118] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0121] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0122] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0123] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (program) are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0124] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program codes.

[0125] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0126] In summary, the present application provides a frequency domain image defogging method, system, medium, program product and terminal based on the SAD-Net network, including: obtaining a foggy image to be defogged, downsampling the foggy image to be defogged to obtain a reduced-size foggy image; inputting the reduced-size foggy image into a pre-trained SAD-Net image defogging model to output an initial defogged image; subtracting the reduced-size foggy image from the initial defogged image to obtain a first fog distribution map; upsampling the first fog distribution map to obtain a second fog distribution map; subtracting the foggy image to be defogged from the second fog distribution map to obtain a final defogged image corresponding to the foggy image to be defogged. The present application performs downsampling on the foggy image, which can reduce the amount of floating-point operations in the subsequent model operation by 75%. In addition, the use of the SAD-Net network model can increase high-frequency details in the defogging process, alleviating the loss of high-frequency details caused by downsampling. At the same time, directly using the original image to directly subtract the fog distribution map can reduce information loss. The image defogging method of the present application can make the target image more prominent in details, richer in high-frequency information, and more obvious in detection targets after defogging. Therefore, the present application effectively overcomes the various shortcomings of the existing technology and has high industrial application value.

[0127] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A frequency domain image dehazing method based on SAD-Net network, characterized in that: include: Obtaining a foggy image to be defogged, and performing downsampling processing on the foggy image to be defogged to obtain a foggy image with a reduced size; Input the downsized foggy image into the pre-trained SAD-Net image dehazing model and output the initial dehazed image; Subtracting the reduced-size foggy image from the initial defogging image to obtain a first fog distribution map; performing upsampling processing on the first fog distribution map to obtain a second fog distribution map; A subtraction process is performed on the foggy image to be defogged and the second fog distribution map to obtain a final defogged image corresponding to the foggy image to be defogged.

2. The frequency domain image defogging method based on the SAD-Net network according to claim 1, characterized in that: The construction process of the pre-trained SAD-Net image dehazing model includes: Establish SAD-Net network based on U-Net architecture; Introducing a frequency domain detail enhancement convolution module and a frequency domain detail enhancement attention module into the SAD-Net network to obtain a SAD-Net image dehazing model; The SAD-Net image dehazing model is trained to obtain a pre-trained SAD-Net image dehazing model.

3. The frequency domain image defogging method based on the SAD-Net network according to claim 2, characterized in that: The process of introducing the frequency domain detail enhancement convolution module and the frequency domain detail enhancement attention module into the SAD-Net network to obtain the SAD-Net image dehazing model includes: The structure of the SAD-Net network includes an encoder, an adaptive feature transformation module and a decoder; a frequency domain detail enhancement convolution module is introduced into the encoder and decoder of the SAD-Net network, and a frequency domain detail enhancement attention module is introduced into the adaptive feature transformation module of the SAD-Net network to obtain a SAD-Net image dehazing model.

4. The frequency domain image defogging method based on the SAD-Net network according to claim 2, characterized in that: The frequency domain detail enhancement convolution module includes: a first wavelet transform processing layer, a first difference convolution processing layer, a first splicing processing layer, a first inverse wavelet transform processing layer, a first 3×3 convolution layer, a first residual connection layer, a second 3×3 convolution layer and a second residual connection layer; Among them, the first wavelet transform processing layer, the first differential convolution processing layer, the first splicing processing layer, the first inverse wavelet transform processing layer, the first 3×3 convolution layer, the first residual connection layer, the second 3×3 convolution layer, and the second residual connection layer are connected in sequence; the first residual connection layer is connected to the input layer of the frequency domain detail enhancement convolution module; the second residual connection layer is connected to the input layer of the frequency domain detail enhancement convolution module.

5. The frequency domain image defogging method based on the SAD-Net network according to claim 2, characterized in that: The frequency domain detail enhancement attention module includes: a second wavelet transform processing layer, a second differential convolution processing layer, a second splicing processing layer, a second inverse wavelet transform processing layer, a third 3×3 convolution layer, a third residual connection layer, a frequency domain guided attention layer and a fourth 3×3 convolution layer; Among them, the second wavelet transform processing layer, the second differential convolution processing layer, the second splicing processing layer, the second inverse wavelet transform processing layer, the third 3×3 convolution layer, the third residual connection layer, the frequency domain guided attention layer, the fourth residual connection layer and the fourth 3×3 convolution layer are connected in sequence; the third residual connection layer is connected to the input layer of the frequency domain detail enhancement attention module; the fourth residual connection layer is connected to the input layer of the frequency domain detail enhancement attention module.

6. The frequency domain image defogging method based on the SAD-Net network according to claim 5, characterized in that: The frequency-domain guided attention layer includes: The third wavelet transform processing layer, the third splicing processing layer, the first average pooling layer, the first one-dimensional convolution layer, the first Sigmoid function layer, the first 1×1 convolution layer, the second average pooling layer, the second one-dimensional convolution layer, the second Sigmoid function layer, the first point product operation layer, the third average pooling layer, the normalization layer, the second 1×1 convolution layer, the third 1×1 convolution layer, the fourth 1×1 convolution layer, the CA-MHSA layer, the fourth average pooling layer, the third Sigmoid function layer, the second point product operation layer, and the third inverse wavelet transform processing layer; Among them, the third wavelet transform processing layer is connected to the third splicing processing layer; the third splicing processing layer is respectively connected to the first average pooling layer, the first 1×1 convolution layer and the second average pooling layer; the first average pooling layer, the first one-dimensional convolution layer, and the first Sigmoid function layer are connected in sequence; the second average pooling layer, the second one-dimensional convolution layer, and the second Sigmoid function layer are connected in sequence; the first Sigmoid function layer, the first 1×1 convolution layer and the second Sigmoid function layer are respectively connected to the first dot product operation layer; The first dot product operation layer, the third average pooling layer, and the normalization layer are connected in sequence; the normalization layer is respectively connected to the second 1×1 convolution layer, the third 1×1 convolution layer, and the fourth 1×1 convolution layer; the second 1×1 convolution layer, the third 1×1 convolution layer, and the fourth 1×1 convolution layer are respectively connected to the CA-MHSA layer; the CA-MHSA layer, the fourth average pooling layer, the third sigmoid function layer, the second dot product operation layer, and the third inverse wavelet transform processing layer are connected in sequence; the first dot product operation layer is connected to the second dot product operation layer.

7. A frequency domain image defogging system based on SAD-Net network, characterized in that: include: A downsampling processing module is used to obtain a foggy image to be defogged, and perform downsampling processing on the foggy image to be defogged to obtain a foggy image with a reduced size; The model processing module inputs the downsized foggy image into the pre-trained SAD-Net image defogging model and outputs the initial defogging image; a subtraction processing module, configured to perform subtraction processing on the reduced-size foggy image and the initial defogging image to obtain a first fog distribution map; an upsampling processing module, configured to perform upsampling processing on the first fog distribution map to obtain a second fog distribution map; The defogging image acquisition module is used to perform subtraction processing on the foggy image to be defogged and the second fog distribution map to obtain a final defogging image corresponding to the foggy image to be defogged.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the frequency domain image defogging method based on the SAD-Net network according to any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that The computer program product includes computer program code, and when the computer program code is run on a computer, the computer implements the frequency domain image dehazing method based on the SAD-Net network according to any one of claims 1 to 6.

10. An electronic terminal comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the frequency domain image dehazing method based on the SAD-Net network as described in any one of claims 1 to 6.

Images