Image defogging method based on multi-scale feature fusion

Abstract

The invention discloses an image defogging method based on multi-scale feature fusion. The method comprises: obtaining a foggy image and dividing the foggy image into a training set and a test set; constructing an original multi-scale feature fusion image defogging model, and performing training evaluation by using the training set and the test set to obtain a multi-scale feature fusion image defogging model; and inputting a to-be-processed foggy image into the multi-scale feature fusion image defogging model, and outputting a defogged image. According to the method, the problem that the feature information is easy to lose in the traditional image defogging process is effectively relieved, so that the accuracy of the defogging effect and the detail reduction capability are remarkably improved. A multi-scale feature fusion mechanism is introduced, and image semantic and texture information of different scales is cooperatively integrated. Compared with a single-scale feature dependence scheme, the method has the advantage that the information expression integrity and the detail retention capability are remarkably improved. In addition, the completeness and robustness of feature representation are further enhanced through the synergistic effect of the modules, so that the model shows higher performance and stability in dealing with a defogging task in a complex scene.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an image defogging method based on multi-scale feature fusion. BACKGROUND

[0002] The suspended particles in the haze environment can seriously interfere with the propagation of light, resulting in a decrease in the intensity of reflected light, thereby causing degradation phenomena such as insufficient image contrast, color distortion, and detail blurring. This decrease in visual quality directly weakens the authenticity of the scene expression and increases the difficulty of target recognition and positioning. Modern intelligent systems such as video monitoring, traffic management, autonomous driving, aerial reconnaissance, and remote sensing applications all rely on high-quality images for environmental perception and decision analysis. Therefore, eliminating haze interference and restoring the original quality of the image has become a core problem in computer vision research.

[0003] According to the processing mechanism, the existing defogging techniques can be divided into two categories: traditional algorithms and deep learning methods. Traditional algorithms rely on physical modeling and empirical rules to describe the optical propagation process under haze conditions by constructing an atmospheric scattering model. This method needs to estimate the transmission medium parameters and the atmospheric lighting conditions, and then perform image restoration operations. However, traditional methods are limited by specific assumptions such as the dark channel prior, and have poor adaptability when facing non-uniform haze distribution, which can easily cause problems such as detail loss, over-correction, or artificial artifacts.

[0004] The development of deep learning technology has promoted the rise of data-driven defogging methods. Convolutional neural networks automatically learn the feature mapping relationship between degradation and restoration by analyzing a large number of paired haze images and clear images. This method does not rely on physical assumptions and can handle diverse haze scenes, with excellent performance in color restoration and detail reconstruction. However, deep learning methods face the challenge of large data requirements, and high-quality performance requires rich labeled data support, as well as complex network architecture design for multi-scale information processing.

[0005] The current research focus is to develop efficient network design schemes that optimize the model structure while ensuring defogging effects. Multi-scale feature fusion technology has become an important means of performance optimization because it can integrate image information at different levels, significantly improving feature expression and detail recovery capabilities. This technology not only enhances the environmental adaptability of the model, but also provides a more robust and accurate technical path for image restoration. SUMMARY

[0006] To solve the above problems, the purpose of the present application is to provide an image defogging method based on multi-scale feature fusion.

[0007] In order to achieve the above purpose, the image defogging method based on multi-scale feature fusion provided by the present application comprises the following steps in sequence:

[0008] 1) Select a public standard dataset, then preprocess the original foggy image in the standard dataset, including invalid pixel filtering, normalization processing and physical model prior fusion, to enhance the generalization ability of the image, obtain the preprocessed foggy image, and then divide all the preprocessed foggy images into training set and test set according to the proportion;

[0009] 2) Construct an original multi-scale feature fusion image defogging model, the original multi-scale feature fusion image defogging model includes an encoding module, a multi-scale feature extraction module, a feature fusion module, a decoding module and an enhancement network module; then input the training set into the original multi-scale feature fusion image defogging model for training, and use the test set for evaluation, finally obtain the multi-scale feature fusion image defogging model;

[0010] 3) Use the camera to process the foggy image to be processed according to the preprocessing method of step 1), and then input the multi-scale feature fusion image defogging model obtained in step 2), and finally the multi-scale feature fusion image defogging model outputs a clear defogging image.

[0011] In step 1), the standard dataset uses the RESIDE standard dataset; the proportion of the training set and the test set is 8:2.

[0012] In step 2), the encoding module adopts a three-level convolution structure, each level is composed of a convolution layer, a batch normalization layer and a ReLU activation function; the convolution layer is used to capture the significant local features in the preprocessed foggy image; the batch normalization layer is used to standardize the feature map of each batch; the ReLU activation function introduces a nonlinear transformation mechanism, and finally outputs the initial feature map.

[0013] In step 2), the multi-scale feature extraction module is composed of three multi-scale expansion blocks connected in series, which are MDB-1 module, MDB-2 module and MDB-3 module; each module is connected through residual connection; wherein the MDB-1 module receives the initial feature map output by the encoding module, and extracts image features of different scales through multi-scale expansion convolution operation; the MDB-2 module is specially designed for optimization processing of medium-scale haze features; the MDB-3 module focuses on processing large-scale haze patterns.

[0014] The multi-scale dilated block comprises three convolutional branches operating in parallel with different dilation rates. The first convolutional branch uses a 3×3 convolutional layer with a dilation rate of 1; the second convolutional branch uses a 3×3 convolutional layer with a dilation rate of 2; and the third convolutional branch uses a 3×3 convolutional layer with a dilation rate of 3. The internal structure of each dilated block and the different dilated blocks are connected by residual connections. The first convolutional branch is mainly responsible for extracting near-field and local detail features from the initial feature map; the second convolutional branch focuses on extracting medium-scale features; and the third convolutional branch is responsible for extracting large-scale and global features. The three convolutional branches employ an adaptive fusion strategy based on an attention mechanism to extract feature maps, ultimately outputting a multi-scale feature map.

[0015] In step 2), the feature fusion module includes a channel attention module and a spatial attention module; the channel attention module includes an average pooling layer, a global max pooling layer, and a parameter-shared multilayer perceptron network; the multilayer perceptron network includes two fully connected layers; the spatial attention module includes an average pooling layer, a global max pooling layer, and a 7×7 convolutional layer; the two fully connected layers on the multilayer perceptron network undergo a nonlinear transformation via a ReLU activation function; the decoding module is a mirror image of the encoding module; the enhancement network module consists of a detail enhancement branch and a 3×3 convolutional layer; the detail enhancement branch includes three structural units M1, M2, and M3; each structural unit uses a structure of alternating 1×1 and 3×3 convolutions, specifically: 1×1 convolution - 3×3 convolution - 1×1 convolution - 3×3 convolution - 1×1 convolution - 3×3 convolution, with a ReLU activation function added after each convolutional layer; and a second skip connection is introduced in structural units M2 and M3.

[0016] The feature fusion module employs a stacking operation, merging the multi-scale feature maps output by the multi-scale feature extraction module and inputting them into the channel attention module. The channel attention module extracts global statistical information from the multi-scale feature maps using global average pooling and global max pooling layers. The pooled feature maps are then input into a parameter-shared multilayer perceptron network. A non-linear transformation is performed between two fully connected layers on the multilayer perceptron network using the ReLU activation function. First, the first fully connected layer reduces the dimension of the pooled feature map to 1 / 16 of its original size; then, the second fully connected layer restores it to its original dimension. The output is normalized using the Sigmoid activation function to generate attention weights for each channel. Finally, these channel attention weights are combined with the input multi-scale feature maps. The graphs are multiplied to obtain a channel-refined feature map. Then, a spatial attention module is used to perform spatial pooling operations on the channel-refined feature map, including average pooling and max pooling along the channel dimension, generating two single-channel feature maps. The two aggregated single-channel feature maps are merged into a single dual-channel feature map through a concatenation operation, and then input into a 7×7 convolutional layer for processing. The output of the convolutional layer is normalized using a sigmoid activation function to generate attention weights for each spatial location. Finally, the attention weights at the spatial locations are multiplied with the channel-refined feature map to obtain a spatially refined feature map. A residual connection is added to add the input multi-scale feature map to the spatially refined feature map for feature fusion, and the final output is a fused feature map.

[0017] In step 2), the decoding module is a mirror image of the encoding module, used to process the fused feature map output by the feature fusion module, and finally output a haze density map reflecting the distribution of haze.

[0018] In step 2), the enhanced network module and the output of the decoding module are deeply fused. This fusion consists of a detail enhancement branch and a 3×3 convolutional layer. The detail enhancement branch comprises three structural units: M1, M2, and M3. Each structural unit uses an alternating structure of three 1×1 and 3×3 convolutions, specifically: 1×1 convolution - 3×3 convolution - 1×1 convolution - 3×3 convolution - 1×1 convolution - 3×3 convolution, with a ReLU activation function added after each convolutional layer. Each structural unit transforms the haze density map output from the decoding module through a 1×1 convolutional layer (serving as a standard branch) and combines it with the haze density map via a skip connection. In structural units M2 and M3, a second skip connection is introduced to form a richer feature fusion path. Finally, a 3×3 convolutional layer fuses the output of the decoding module and the output of the detail enhancement branch.

[0019] The image dehazing method based on multi-scale feature fusion provided by this invention has the following beneficial effects:

[0020] 1. This method effectively alleviates the problem of feature information loss during traditional image dehazing, thus significantly improving the accuracy and detail restoration capabilities of the dehazing effect. It introduces a multi-scale feature fusion mechanism to collaboratively integrate semantic and textural information from different scales. Compared to single-scale feature-dependent schemes, this significantly improves the completeness of information representation and detail preservation. Furthermore, the synergistic effect between modules further enhances the completeness and robustness of feature representation, enabling the model to exhibit stronger performance and stability in dehazing tasks under complex scenarios.

[0021] 2. Multi-scale feature extraction utilizes multiple sets of parallel convolutional operations with different dilation rates to effectively capture rich information about near-field details and far-field structures in images. This strategy breaks through the limitation of traditional methods that extract features only at a single scale, significantly enhancing the model's adaptability and generalization performance in the face of complex and varied scenes;

[0022] 3. The combination of channel attention and spatial attention mechanisms enables the model to dynamically enhance key feature channels and important spatial regions in the image. This adaptive focusing mechanism improves the ability to express the features of hazy areas, thereby significantly improving the overall dehazing accuracy.

[0023] 4. By incorporating residual connections into the model, the vanishing gradient problem, a common issue in deep neural networks during training, is effectively mitigated. This mechanism not only accelerates the model's training convergence speed but also improves the stability and robustness of the final model to some extent. Attached Figure Description

[0024] Figure 1 The flowchart of the image dehazing method based on multi-scale feature fusion provided by the present invention is shown below.

[0025] Figure 2 This is a structural diagram of the multi-scale feature fusion image dehazing model constructed in this invention;

[0026] Figure 3 This is a structural diagram of the MFEM module in this invention;

[0027] Figure 4 This is a diagram showing the internal structure of the MDB submodule within the MFEM module of this invention.

[0028] Figure 5 This is a structural diagram of the FFM module in this invention;

[0029] Figure 6 This is a structural diagram of the EN module in this invention. Detailed Implementation

[0030] The present invention will now be described in detail with reference to the accompanying drawings.

[0031] like Figure 1 As shown, the image dehazing method based on multi-scale feature fusion provided by the present invention includes the following steps performed in sequence:

[0032] 1) Select a publicly available standard dataset, and then perform preprocessing on the original foggy images in the standard dataset, including invalid pixel filtering, normalization, and fusion of physical model priors, to enhance the generalization ability of the images and obtain preprocessed foggy images. Then, divide all preprocessed foggy images into training and test sets according to the proportion.

[0033] In this invention, the standard dataset used is the RESIDE standard dataset; the ratio of the training set to the test set is 8:2.

[0034] 2) Constructing such Figure 2 The original multi-scale feature fusion image dehazing model shown includes an encoding module, a multi-scale feature extraction (MFEM) module, a feature fusion (FFM) module, a decoding module, and an enhancement network (EN) module. The training set is then input into the original multi-scale feature fusion image dehazing model for training, and evaluated using a test set to finally obtain the multi-scale feature fusion image dehazing model.

[0035] The encoding module employs a three-stage convolutional structure for deep feature extraction. Each stage consists of a convolutional layer, a batch normalization (BN) layer, and a ReLU activation function. The convolutional layer, as the core component of feature extraction, primarily captures significant local features in the preprocessed hazy image, such as edge information, corner features, and texture patterns. The batch normalization layer normalizes the distribution of intermediate features. By standardizing the feature maps of each batch, it not only effectively improves the convergence speed of training but also significantly enhances the stability of the training process, preventing gradient vanishing or exploding problems. The ReLU activation function, by introducing a non-linear transformation mechanism, greatly improves the model's ability to recognize complex patterns and abstract features, enabling the model to learn richer and more effective feature representations. Furthermore, the encoding module effectively compresses the spatial dimension of the image through a layer-by-layer downsampling strategy, while systematically extracting feature information at different scales, thus providing a richer and more comprehensive information foundation for subsequent feature processing and image reconstruction stages. Finally, the initial feature map is output.

[0036] like Figure 3As shown, the multi-scale feature extraction module, as the core component of the entire multi-scale feature fusion image dehazing model, is designed based on the multi-scale feature representation theory in deep learning and the receptive field expansion technology in computer vision. It consists of three cascaded multi-scale dilatation blocks: MDB-1, MDB-2, and MDB-3. This module replaces standard convolutional layers, effectively expanding the receptive field without increasing the number of parameters, thus enhancing the model's ability to perceive haze features at different scales. The MDB-1 module, as the first-level processing unit, undertakes the important tasks of initial feature extraction and multi-scale information decoupling. This module receives the initial feature maps output by the encoding module, which contain key visual elements such as the basic structural information, edge features, and texture patterns of the original hazy image. The MDB-1 module systematically extracts image features at different scales (local details, medium-scale structure, and global context) through multi-scale dilatation convolution operations. This multi-level feature extraction lays a rich information foundation for subsequent processing, significantly improving the model's understanding and processing capabilities for complex haze distribution patterns. The MDB-2 and MDB-3 modules further deepen and enhance the perception capabilities of medium- and large-scale haze patterns. These two modules, through a progressive processing approach, form a complete and efficient multi-scale feature extraction system. The MDB-2 module is specifically optimized for medium-scale haze features, using increased dilation rates and adjusted convolutional kernel sizes to better capture haze distribution features within a medium range. This module also introduces a feature enhancement mechanism, significantly improving the expressive power and discriminative power of medium-scale features through residual learning and feature recombination techniques. The MDB-3 module focuses on processing large-scale haze patterns, effectively capturing haze distribution information over long distances and large areas through larger dilation rates and more complex convolutional structures. Furthermore, each module is connected via residual connections, allowing shallow features to be directly transferred to deeper layers, reducing information attenuation and improving the model's feature extraction capabilities and training stability.

[0037] like Figure 4 As shown, each multi-scale dilation block includes three convolutional branches that operate in parallel with different dilation rates. These branches specialize in extracting and processing features from the foreground, midground, and background of the initial feature map, respectively. The first convolutional branch uses a 3×3 convolutional layer with a dilation rate of 1; the second uses a 3×3 convolutional layer with a dilation rate of 2; and the third uses a 3×3 convolutional layer with a dilation rate of 3.

[0038] The first convolutional branch is primarily responsible for extracting near-field and local detail features from the initial feature map, including edge information, texture details, and local contrast variations. This branch has a relatively small receptive field but retains more high-frequency information and detail features, playing a crucial role in subsequent image reconstruction and detail restoration. The second convolutional branch focuses on extracting medium-scale features, capturing structural information and regional features within a certain range. This branch acts as a vital bridge between near-field and far-field features, effectively integrating local and global information through its moderate receptive field. The third convolutional branch is responsible for extracting large-scale and global features. With a larger receptive field, it captures long-distance contextual information and the overall haze distribution pattern. This branch is key to understanding the overall structure of the image and global haze characteristics.

[0039] The three convolutional branches described above employ an adaptive fusion strategy based on an attention mechanism to extract feature maps. By learning the importance weights of features at different scales, the contribution level of each convolutional branch is dynamically adjusted, thereby obtaining a more optimized feature representation. Feature recalibration technology is also introduced during the fusion process. By analyzing the correlation and complementarity between features at different scales, the fusion weights are dynamically adjusted to ensure that the final feature representation can fully utilize the effective information at each scale. The formula is as follows:

[0040]

[0041] Where U1, M1, and D1 represent the features obtained by the first convolution in the first to third convolution branches of the input initial feature map, respectively; U2, M2, and D2 represent the features obtained by the second convolution in the first to third convolution branches of the input initial feature map, respectively; w and b represent the weights and biases, respectively, where the superscript indicates the layer number and the subscript indicates the kernel size and dilation rate; σ(x) represents the ReLU activation function; [] represents the concatenation operation.

[0042] Furthermore, residual connections are introduced in the multi-scale feature extraction module. These residual connections are not only applied to the internal structure of each expansion block but also establish cross-layer connections between different expansion blocks. This improves the training stability of the model. Simultaneously, residual connections help preserve important information from the original features, preventing the loss of crucial details during deep processing. The final output is a multi-scale feature map.

[0043] like Figure 5As shown, the core function of the feature fusion module is to efficiently and intelligently integrate the multi-scale feature maps output by the multi-scale feature extraction module, thereby further improving the model's ability to recover image details and its processing effect. It comprises two key components: a channel attention module and a spatial attention module. The channel attention module includes an average pooling layer, a global max pooling layer, and a parameter-sharing multilayer perceptron (MLP) network; the MLP network includes two fully connected layers. The spatial attention module includes an average pooling layer, a global max pooling layer, and a 7×7 convolutional layer. This module is designed based on attention mechanism theory and feature importance evaluation techniques, achieving optimal feature fusion results through a dual attention architecture.

[0044] First, the feature fusion module employs a stacking operation, merging the multi-scale feature maps output by the multi-scale feature extraction module and inputting them into the channel attention module. The channel attention module extracts global statistical information from the multi-scale feature maps using global average pooling and global max pooling layers. These two pooling operations capture the global characteristics of the feature channels from different perspectives: global average pooling obtains the average activation intensity of each channel, reflecting the overall importance of that channel; global max pooling captures the maximum activation value of each channel, reflecting the saliency of that channel. The feature maps after pooling are then input into a parameter-shared multilayer perceptron network. A non-linear transformation using the ReLU activation function is performed between the two fully connected layers in the multilayer perceptron network. This module employs a dimensionality reduction-increase strategy: first, the first fully connected layer reduces the dimension of the pooled feature map to 1 / 16 of its original size, and then the second fully connected layer restores it to its original dimension. The output of the multilayer perceptron network is normalized using the Sigmoid activation function to generate the attention weight M for each channel. c (F). These attention weights, ranging from 0 to 1, reflect the importance of the corresponding channels. Important channels have attention weights close to 1, while less important channels have attention weights close to 0, thus enhancing important features and suppressing redundant features. The formula is as follows:

[0045] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0046] Where F represents the input multi-scale feature map, AvgPool(F) and MaxPool(F) represent global average pooling and global max pooling, respectively, MLP(·) represents a multilayer perceptron with shared parameters, and σ(x) represents the Sigmoid activation function.

[0047] Next, the attention weights of the aforementioned channels are multiplied by the input multi-scale feature map to obtain the channel-refined feature map. Then, the spatial attention module performs spatial-dimensional pooling on the channel-refined feature map. Similar to the channel attention module, the spatial attention module also employs a dual pooling strategy, including average pooling and max pooling along the channel dimension. This operation aggregates information from all channels at each spatial location, generating two single-channel feature maps. Average pooling captures the average feature response at each spatial location, reflecting the overall importance of that location; max pooling captures the strongest feature response at each spatial location, reflecting the saliency of that location. This dual spatial aggregation strategy ensures that the model can evaluate the importance of each spatial location from multiple perspectives. The two aggregated single-channel feature maps are merged into a single dual-channel feature map through a concatenation operation, and then input into a 7×7 convolutional layer for processing. This convolutional layer uses a large kernel size, which can capture a wider range of spatial context information, thereby generating more accurate attention weights for spatial locations. The output of the convolutional layer is normalized using a sigmoid activation function to generate the attention weight M for each spatial location. s (F). These attention weights reflect the importance of different spatial locations, with higher attention weights for important areas and lower attention weights for less important areas. The formula is as follows:

[0048] M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)]))

[0049] Where AvgPool(F) and MaxPool(F) represent global average pooling and global max pooling, respectively, f 7×7 () represents a 7×7 convolution operation, and σ(x) represents the Sigmoid activation function;

[0050] Finally, the attention weights for the spatial locations are multiplied by the channel-refined feature map to obtain the spatially refined feature map. A residual connection is added to perform feature fusion by adding the input multi-scale feature map to the spatially refined feature map to prevent the loss of high-frequency details. The final output is the fused feature map F', as shown in the following formula:

[0051]

[0052] Where F represents the input multi-scale feature map, M c M represents the channel attention module. s Represents spatial attention module, This indicates element-wise multiplication.

[0053] The decoding module is structurally designed as a mirror image of the encoding module. Its main task is to gradually restore the spatial resolution of the image through deconvolution and further process the fused feature map to finally output a haze density map reflecting the distribution of haze. In this process, to better preserve detail information and improve reconstruction quality, a skip connection mechanism is introduced in the model design, directly passing the key features extracted at each scale during the encoding stage to the corresponding decoding layer. This not only enhances the utilization efficiency of low-level information during reconstruction but also helps in the restoration of detail textures, thereby effectively improving the overall dehazing effect and visual quality of the image.

[0054] like Figure 6 As shown, the enhancement network module, as the final key component of the entire multi-scale feature fusion image dehazing model, is deeply fused with the output of the decoding module to form an end-to-end image optimization system. The enhancement network module consists of a detail enhancement branch and a 3×3 convolutional layer, using a multi-layered skip connection structure to improve the efficiency and effectiveness of feature learning. The detail enhancement branch is designed with reference to the residual concept of ResNet, containing three structural units: M1, M2, and M3. Each structural unit uses an alternating structure of three 1×1 and 3×3 convolutions, specifically: 1×1 convolution - 3×3 convolution - 1×1 convolution - 3×3 convolution - 1×1 convolution - 3×3 convolution, with a ReLU activation function added after each convolutional layer. Each structural unit transforms the haze density map output by the decoding module through a 1×1 convolutional layer (as a standard branch) and combines it with the haze density map through skip connections. Specifically, a second skip connection is introduced in the M2 and M3 structural units to form richer feature fusion paths. This multi-level skip connection structure significantly improves the feature learning efficiency and expressive power of the modules. The formula is as follows:

[0055] y = x + f(x) + g(x)

[0056] Where x represents the skip branch, f(x) represents the continuous branch consisting of a 3×3 convolutional layer and the ReLU activation function, and g(x) represents the standard branch.

[0057] Furthermore, the ReLU activation function enhances the non-linear expressive power of the module. The detail enhancement branch extracts detailed information from the haze density map through multi-layer convolution operations, effectively restoring the image's texture and edge information. Finally, a 3×3 convolutional layer fuses the output of the decoding module and the output of the detail enhancement branch, effectively compensating for any detail information that may be lost during decoding and further improving the quality and detail representation of the dehazed image. This results in a multi-scale feature fusion image dehazing model.

[0058] The core design of the aforementioned multi-scale feature fusion image dehazing model employs an encoder-decoder network architecture. Specifically, through the constructed multi-scale feature extraction module, the model can capture feature information at different spatial scales and abstraction levels in hazy images in parallel and collaboratively. Subsequently, the designed feature fusion module efficiently integrates multi-level feature representations extracted from different paths. To further improve the restoration quality, a feature enhancement network module is also embedded in the model to optimize and strengthen the fused feature representation at key stages. This mechanism of multi-scale feature collaborative extraction, deep fusion, and targeted enhancement significantly strengthens the model's robust feature learning and representation capabilities when facing haze interference of varying degrees and structures.

[0059] 3) Use the camera to take the foggy image to be processed and process it according to the preprocessing method in step 1). Then input it into the multi-scale feature fusion image dehazing model obtained in step 2). Finally, the multi-scale feature fusion image dehazing model outputs a clear dehazing image.

[0060] To verify the effectiveness of the present invention, the inventors conducted the following experiments:

[0061] Tested on the SOTS subset of the RESIDE dataset, the method of this invention demonstrates significant advantages in terms of objective evaluation metrics. On the SOTS Indoor test set, the peak signal-to-noise ratio (PSNR) reaches 30.17 dB and the structural similarity (SSIM) reaches 0.960; on the SOTS Outdoor test set, the PSNR reaches 27.03 dB and the SSIM reaches 0.946.

[0062] Compared with existing technologies, the method of this invention improves the PSNR by 13.8% in indoor scenes compared to the USID-Net method (26.51dB) and by 25.3% compared to the EPDN method (24.08dB); in outdoor scenes, the PSNR is improved by 12.5% ​​compared to the DehazeNet method (24.03dB) and by 28.5% compared to the AOD-Net method (21.04dB). Regarding structural similarity metrics, the SSIM value in indoor scenes is improved by 11.4% compared to the DehazeNet method (0.862), and the SSIM value in outdoor scenes is improved by 14.3% compared to the USID-Net method (0.828).

[0063] Experimental data show that the method of the present invention is significantly superior to existing methods such as AOD-Net, DehazeNet, USID-Net and EPDN in key indicators, demonstrating the technical advantages of the present invention in image dehazing.

Claims

1. An image defogging method based on multi-scale feature fusion, characterized in that: The image defogging method comprises the following steps in sequence: 1) Select a public standard data set, then perform preprocessing on the original foggy images in the standard data set, including invalid pixel filtering, normalization processing and physical model prior fusion, to enhance the generalization ability of the images, obtain the preprocessed foggy images, and then divide all the preprocessed foggy images into a training set and a test set in proportion; 2) Construct an original multi-scale feature fusion image defogging model, the original multi-scale feature fusion image defogging model comprising an encoding module, a multi-scale feature extraction module, a feature fusion module, a decoding module and an enhancement network module; then input the training set into the original multi-scale feature fusion image defogging model for training, and evaluate it with the test set, and finally obtain the multi-scale feature fusion image defogging model; 3) Use a camera to process the foggy images to be processed according to the preprocessing method of step 1), and then input the multi-scale feature fusion image defogging model obtained in step 2), and finally the multi-scale feature fusion image defogging model outputs clear defogging images.

2. The image defogging method based on multi-scale feature fusion according to claim 1, characterized in that: In step 1), the standard data set uses the RESIDE standard data set; the proportion of the training set and the test set is 8:

2.

3. The image defogging method based on multi-scale feature fusion according to claim 1, characterized in that: In step 2), the encoding module adopts a three-level convolution structure, each level being composed of a convolution layer, a batch normalization layer and a ReLU activation function; the convolution layer is used to capture the significant local features in the preprocessed foggy images; the batch normalization layer standardizes each batch of feature maps; and the ReLU activation function introduces a nonlinear transformation mechanism, and finally outputs the initial feature map.

4. The image defogging method based on multi-scale feature fusion according to claim 1, characterized in that: In step 2), the multi-scale feature extraction module is composed of three multi-scale expansion blocks connected in series, namely MDB-1 module, MDB-2 module and MDB-3 module; each module is connected through residual connection; wherein the MDB-1 module receives the initial feature map output by the encoding module, and systematically extracts image features of different scales through multi-scale expansion convolution operation; the MDB-2 module is specially designed for optimization processing of medium-scale haze features; and the MDB-3 module focuses on processing of large-scale haze patterns.

5. The image defogging method based on multi-scale feature fusion according to claim 4, characterized in that: The multi-scale expansion block comprises three convolution branches with different expansion rates and running in parallel, wherein the first convolution branch uses a 3x3 convolution layer with an expansion rate of 1; the second convolution branch uses a 3x3 convolution layer with an expansion rate of 2; and the third convolution branch uses a 3x3 convolution layer with an expansion rate of 3; and the internal structure of each expansion block and different expansion blocks are connected through residual connection; the first convolution branch is mainly responsible for extracting near-range and local detail features of the initial feature map; the second convolution branch focuses on extracting medium-scale features; and the third convolution branch is responsible for extracting large-scale and global features, and the three convolution branches adopt an adaptive fusion strategy based on an attention mechanism to extract feature maps, and finally output multi-scale feature maps.

6. The image defogging method based on multi-scale feature fusion according to claim 1, characterized in that: In step 2), the feature fusion module comprises a channel attention module and a spatial attention module; the channel attention module comprises an average pooling layer, a global max pooling layer and a parameter-shared multi-layer perceptron network; the multi-layer perceptron network comprises two fully connected layers; the spatial attention module comprises an average pooling layer, a global max pooling layer and a 7*7 convolution layer; the two fully connected layers in the multi-layer perceptron network are connected by a ReLU activation function for nonlinear transformation; The structure of the decoding module is a mirror structure of the above-mentioned encoding module; the enhancement network module is composed of a detail enhancement branch and a 3*3 convolution layer; the detail enhancement branch comprises three structure units M1, M2 and M3; each structure unit adopts an alternating connection structure of three 1*1 convolution layers and three 3*3 convolution layers, specifically: 1*1 convolution-3*3 convolution-1*1 convolution-3*3 convolution-1*1 convolution-3*3 convolution, and a ReLU activation function is added after each convolution layer; and a second jump connection is introduced in the M2 structure unit and the M3 structure unit; The feature fusion module adopts a stacking operation to combine the multi-scale feature maps output by the multi-scale feature extraction module and input into the channel attention module; The channel attention module extracts global statistical information of the multi-scale feature maps through a global average pooling layer and a global max pooling layer; the feature maps after the pooling operation are input into a parameter-shared multi-layer perceptron network; the two fully connected layers in the multi-layer perceptron network are connected by a ReLU activation function for nonlinear transformation; first, the feature maps after the pooling operation are reduced to 1 / 16 of the original dimension through the first fully connected layer, and then restored to the original dimension through the second fully connected layer; the output is normalized by a Sigmoid activation function to generate attention weights of each channel; then, the attention weights of the above-mentioned channels are multiplied by the input multi-scale feature maps to obtain channel-refined feature maps; Then, the spatial attention module is used to perform a spatial dimension pooling operation on the channel-refined feature maps, including average pooling and max pooling operations along the channel dimension, to generate two single-channel feature maps; The two single-channel feature maps after aggregation are combined into a double-channel feature map through a splicing operation, and then input into a 7*7 convolution layer for processing; the output of the convolution layer is normalized by a Sigmoid activation function to generate attention weights of each spatial position; finally, the attention weights of the above-mentioned spatial positions are multiplied by the channel-refined feature maps to obtain spatial-refined feature maps; a residual connection is added to add the input multi-scale feature maps and the above-mentioned spatial-refined feature maps to perform feature fusion, and finally output a fusion feature map.

7. The image defogging method based on multi-scale feature fusion according to claim 1, characterized in that: In step 2), the decoding module is a mirror of the encoding module, used to process the fusion feature map output by the feature fusion module, and finally output a haze density map reflecting the haze distribution.

8. The image defogging method based on multi-scale feature fusion according to claim 1, characterized in that: In step 2), the enhanced network module is deeply fused with the output end of the decoding module, and is composed of a detail enhancement branch and a 3*3 convolution layer, wherein the detail enhancement branch comprises three structural units M1, M2 and M3; each structural unit adopts the structure of three 1*1 convolution and 3*3 convolution connected alternately, specifically 1*1 convolution-3*3 convolution-1*1 convolution-3*3 convolution-1*1 convolution-3*3 convolution, and a ReLU activation function is added after each convolution layer. Each structural unit transforms the haze density map output by the decoding module through a 1*1 convolution layer as a standard branch, and combines with the haze density map through a jump connection; in the M2 structural unit and the M3 structural unit, a second jump connection is additionally introduced to form a more rich feature fusion path; finally, the output of the decoding module and the output of the detail enhancement branch are fused through a 3*3 convolution layer.

Images