Image defogging method based on multi-scale depth fusion

Abstract

The invention discloses an image defogging method based on multi-scale depth fusion, and belongs to the technical field of computer vision. Based on an atmospheric scattering model, image pairs with different fog concentrations are randomly generated by using known depth-of-field information to construct a training data set. A network architecture adopts an encoder-decoder structure, an encoder extracts multi-scale fog related features through a multi-branch down-sampling module, a decoder introduces a full-scale jump connection mechanism, and feature information of all encoding layers and a preceding-stage decoding layer is fused to enhance the detail reconstruction capability. And a feature map output by the decoder is processed by a progressive multi-scale image recovery network to generate a high-quality defogging result. L1 norm loss, perception loss and multi-scale structure similarity loss are jointly optimized in the training process, and pixel-level precision and visual perception quality are effectively balanced. According to the method, fog with different concentrations can be adaptively processed, and the edge sharpness, texture details and color authenticity of the defogged image are remarkably improved.

Description

Technical Field

[0001] This invention discloses an image dehazing method based on multi-scale depth fusion, belonging to the field of computer vision technology. Background Technology

[0002] With the widespread application of computer vision and intelligent image processing technologies, high-quality images have become a fundamental guarantee for critical tasks such as object detection, autonomous driving, and video surveillance. However, during the shooting of natural scenes, particles such as water vapor and dust suspended in the atmosphere can cause light scattering, resulting in problems such as reduced contrast, color shift, and blurred details in the acquired images, which seriously weakens the reliability and accuracy of subsequent high-level vision tasks. Especially under complex weather conditions, fog distribution has spatial non-uniformity and variable concentration, which places higher demands on the robustness and generalization ability of dehazing algorithms.

[0003] Image dehazing technology aims to recover a clear and realistic fog-free scene from a single foggy image. Traditional methods are mainly divided into two categories: one is based on atmospheric scattering physics models, which achieves restoration by estimating the transmission map and global atmospheric light, such as dark channel prior (DCP), but it relies on strong statistical assumptions and is prone to halo artifacts in sky areas or low-texture areas; the other uses image enhancement strategies, such as histogram equalization or contrast stretching, which can improve visual quality but are difficult to reconstruct the real scene structure and lighting information. In recent years, deep learning-driven end-to-end dehazing networks (such as AOD-Net and DehazeNet) have broken free from explicit prior constraints, but they are limited by their single-scale feature extraction capabilities and often perform poorly in terms of edge sharpness preservation and detail texture restoration.

[0004] While some existing technologies have introduced encoder-decoder architectures and attempted multi-scale feature fusion (such as UNet3+), significant shortcomings remain: First, the downsampling process generally employs standard convolution, resulting in a fixed receptive field and a lack of adaptive modeling capabilities for regions with varying fog concentrations. Second, skip connections only achieve feature transfer between local layers, failing to fully integrate full-scale encoding information and cross-stage semantic associations. Third, the loss function design focuses on pixel-level errors, neglecting the collaborative optimization of high-level perceptual quality and structural similarity. These issues lead to frequent loss of detail, color distortion, or artifact retention in existing methods when processing high-density fog images, making it difficult to meet the dual requirements of image realism and usability in practical applications. Therefore, there is an urgent need for an image dehazing method that can deeply fuse multi-scale contextual information, possess strong expressive power, and balance perceptual quality. Summary of the Invention

[0005] The purpose of this invention is to provide an image dehazing method based on multi-scale depth fusion, which can effectively solve the problems mentioned in the background art.

[0006] An image dehazing method based on multi-scale depth fusion includes the following specific steps: S1: Based on the atmospheric scattering model, using known depth information, randomly select global atmospheric light values ​​and atmospheric scattering coefficient values ​​to generate foggy images from fog-free images and establish a training dataset; S2: Construct a dehazing network based on an encoder and a decoder. The encoder includes a first-level to a fifth-level encoder, and the decoder includes a first-level to a fourth-level decoder. The network structure of the dehazing network uses a multi-scale learning module to extract multi-scale feature information related to dehazing. S3: Input the image to be dehazed into the encoder and decoder to obtain the dehazed feature map, and input the dehazed feature map into the image recovery network to generate the final dehazed image; S4: The dehazing network is trained by using a linear weighted combination of the L1 norm loss function, the perceptual loss function, and the multi-scale structural similarity loss function as the total loss function. After training, a single hazy image is input and the corresponding dehazed image is output.

[0007] Furthermore, the encoder consists of five downsampling modules that progressively reduce the size of the input feature map, and the decoder consists of four upsampling modules, with the deconvolution operation stride in each upsampling module progressively increasing the feature map size. The S3 further includes: S31, in each level encoder, a multi-branch downsampling module is used to extract fog map features, the feature map scale is reduced through downsampling operation, and the processed features are input to the next level encoder; S32 introduces a full-scale skip connection mechanism in each level of decoder, which merges the feature maps output by all five levels of encoders and the output feature maps of the previous level decoder, and splices them after spatial and channel alignment to generate the dehazing feature map of the current decoding layer. S33, the feature map output by the first-level decoder is input into the image restoration network, which adopts a progressive multi-scale convolutional structure to generate the final dehazed image.

[0008] Furthermore, in S1, the global atmospheric light value is randomly selected within the range of 0.7 to 1.0, and the atmospheric scattering coefficient value is randomly selected within the range of 0.6 to 1.8. A foggy image is generated based on the atmospheric scattering model I(x) = J(x)t(x) + A(1 - t(x)), where t(x) is the transmittance, J(x) is the fog-free image, I(x) is the foggy image, and A is the global atmospheric light.

[0009] Furthermore, the method also includes: The multi-branch downsampling module contains three parallel branches: the first branch contains one convolutional unit, the second branch contains two convolutional units, and the third branch contains three convolutional units. The outputs of each branch are concatenated in the channel dimension to form a comprehensive feature representation. The convolutional unit is composed of pointwise convolution, depthwise convolution and further pointwise convolution connected in sequence, with batch normalization and linear rectified activation function following each convolutional layer. The first depthwise convolution in the first branch uses a 3×3 kernel with a stride of 2 and a padding parameter of 1 to achieve feature extraction. Figure 2 The subsamples are downsampled by 1 / 2, and the dilation coefficients of the subsequent depth convolutions are 1, 2, and 4 respectively, with a stride of 1.

[0010] Furthermore, the full-scale skip connection mechanism performs max pooling on the shallow encoder feature map and bilinear interpolation on the deep encoder feature map in each decoding layer. It then unifies the spatial size and number of channels of all feature maps, concatenates them along the feature dimension, and processes them with batch normalization and linear rectified activation function to obtain the dehazing feature map of the current decoding layer.

[0011] Furthermore, the image restoration network comprises a five-layer convolutional structure, with the first layer using a 1×1 convolutional kernel to compress channels and extract point-level features; The second layer uses a 3×3 convolution kernel, and the third layer uses a 5×5 convolution kernel. The outputs of the two layers are integrated through a bidirectional fusion mechanism. The fourth layer uses a 7×7 convolutional kernel to model the global fog concentration distribution, and its output is fed back to the fusion layer to form enhanced features; The fifth layer uses a 3×3 convolution kernel to refine and fuse the results, generating the final dehazed image.

[0012] Furthermore, the L1 norm loss function calculates the sum of the absolute values ​​of the differences between corresponding pixel values ​​in the hazy image and the dehazed image; The perceptual loss function calculates semantic differences based on the feature map of the l-th layer of a pre-trained deep neural network. The multi-scale structural similarity loss function calculates the similarity of brightness, contrast, and structure at multiple scales and then sums them by weight.

[0013] Furthermore, it also includes: In the first-level encoder, the first multi-branch downsampling module is used to extract fog map features. The feature map scale is reduced by downsampling operation to obtain the first downsampled feature map, and the processed first downsampled feature map is input into the second-level encoder. In the second-level encoder, the second multi-branch downsampling module is used to downsample the first downsampled feature map to reduce the feature map scale and obtain the second downsampled feature map. The processed second downsampled feature map is then input into the third-level encoder. In the third-level encoder, the third multi-branch downsampling module is used to downsample the second downsampled feature map to reduce the feature map scale and obtain the third downsampled feature map. The processed third downsampled feature map is then input into the fourth-level encoder. In the fourth-level encoder, the fourth multi-branch downsampling module is used to downsample the third downsampling feature map to reduce the feature map scale and obtain the fourth downsampling feature map. The processed fourth downsampling feature map is then input into the fifth-level encoder. In the fifth-level encoder, the fifth multi-branch downsampling module is used to downsample the fourth downsampled feature map to reduce the feature map scale and obtain the fifth downsampled feature map. The processed fifth downsampled feature map is then input into the fourth-level decoder.

[0014] Furthermore, S32 also includes: The first downsampled feature map, the second downsampled feature map, the third downsampled feature map, the fourth downsampled feature map, and the fifth downsampled feature map are input into the fourth-level decoder to obtain the fourth upsampled feature map; The fourth upsampled feature map, the first downsampled feature map, the second downsampled feature map, and the third downsampled feature map are input into the third-level decoder to obtain the third upsampled feature map; The third upsampled feature map, the first downsampled feature map, and the second downsampled feature map are input into the second-level decoder to obtain the second upsampled feature map. The second upsampled feature map is input into the first-level decoder to obtain the dehazed feature map.

[0015] Compared with existing technologies, this invention has the following advantages: by constructing a dehazing network architecture based on multi-scale deep fusion, using a multi-branch downsampling module to extract fog-related features at different scales, combining a full-scale skip connection mechanism to achieve seamless fusion of shallow fine information and deep semantic information, and using a multi-loss function joint optimization strategy to balance pixel-level accuracy and perceptual quality, a high-quality dehazed image is finally generated through a progressive multi-scale image restoration network. This effectively solves the shortcomings of traditional methods in edge sharpness preservation, detail texture restoration, and color authenticity, and significantly improves the visual effect of image dehazing and the reliability of subsequent visual tasks. Attached Figure Description

[0016] Figure 1 A flowchart illustrating the workflow of an image dehazing method based on multi-scale depth fusion, as claimed in an embodiment of the present invention. Figure 2 A schematic diagram of the PDConv convolutional module of an image dehazing method based on multi-scale depth fusion claimed in an embodiment of the present invention; Figure 3A schematic diagram of a PDDB downsampling module for an image dehazing method based on multi-scale depth fusion claimed in an embodiment of the present invention; Figure 4 A schematic diagram of the scale-connected image dehazing algorithm of an image dehazing method based on multi-scale depth fusion claimed in an embodiment of the present invention; Figure 5 The diagram shows the AOD-Net network model used in the multi-scale depth fusion-based image dehazing method claimed in the embodiments of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] The terms "first," "second," and "third" in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0019] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0021] Reference Figure 1 The first embodiment of this invention claims protection for an image dehazing method based on multi-scale depth fusion, comprising the following specific steps: S1: Based on the atmospheric scattering model, using known depth information, randomly select global atmospheric light values ​​and atmospheric scattering coefficient values ​​to generate foggy images from fog-free images and establish a training dataset; S2: Construct a dehazing network based on an encoder and a decoder. The encoder includes a first-level to a fifth-level encoder, and the decoder includes a first-level to a fourth-level decoder. The network structure of the dehazing network uses a multi-scale learning module to extract multi-scale feature information related to dehazing. S3: Input the image to be dehazed into the encoder and decoder to obtain the dehazed feature map, and input the dehazed feature map into the image recovery network to generate the final dehazed image; S4: The dehazing network is trained by using a linear weighted combination of the L1 norm loss function, the perceptual loss function, and the multi-scale structural similarity loss function as the total loss function. After training, a single hazy image is input and the corresponding dehazed image is output.

[0022] Furthermore, the encoder consists of five downsampling modules that progressively reduce the size of the input feature map, and the decoder consists of four upsampling modules, with the deconvolution operation stride in each upsampling module progressively increasing the feature map size. The S3 further includes: S31, in each level encoder, a multi-branch downsampling module is used to extract fog map features, the feature map scale is reduced through downsampling operation, and the processed features are input to the next level encoder; S32 introduces a full-scale skip connection mechanism in each level of decoder, which merges the feature maps output by all five levels of encoders and the output feature maps of the previous level decoder, and splices them after spatial and channel alignment to generate the dehazing feature map of the current decoding layer. S33, the feature map output by the first-level decoder is input into the image restoration network, which adopts a progressive multi-scale convolutional structure to generate the final dehazed image.

[0023] Furthermore, in S1, the global atmospheric light value is randomly selected within the range of 0.7 to 1.0, and the atmospheric scattering coefficient value is randomly selected within the range of 0.6 to 1.8. A foggy image is generated based on the atmospheric scattering model I(x) = J(x)t(x) + A(1 - t(x)), where t(x) is the transmittance, J(x) is the fog-free image, I(x) is the foggy image, and A is the global atmospheric light.

[0024] Furthermore, the method also includes: The multi-branch downsampling module contains three parallel branches: the first branch contains one convolutional unit, the second branch contains two convolutional units, and the third branch contains three convolutional units. The outputs of each branch are concatenated in the channel dimension to form a comprehensive feature representation. The convolutional unit is composed of pointwise convolution, depthwise convolution and further pointwise convolution connected in sequence, with batch normalization and linear rectified activation function following each convolutional layer. The first depthwise convolution in the first branch uses a 3×3 kernel with a stride of 2 and a padding parameter of 1 to achieve feature extraction. Figure 2 The subsamples are downsampled by 1 / 2, and the dilation coefficients of the subsequent depth convolutions are 1, 2, and 4 respectively, with a stride of 1.

[0025] In this embodiment, the PDConv unit includes a convolutional combination structure of pointwise convolution, depthwise convolution, and then pointwise convolution, with batch normalization (BN) and ReLU activation functions added after each layer. The PDDB module contains three branches, stacking 1, 2, and 3 PDConv structures respectively. In the first branch, the first depthwise convolutional kernel has a size of 3×3 and a stride of 2, while the stride of the remaining layers is 1, and the dilation coefficients are 1, 2, and 4.

[0026] Furthermore, the full-scale skip connection mechanism performs max pooling on the shallow encoder feature map and bilinear interpolation on the deep encoder feature map in each decoding layer. It then unifies the spatial size and number of channels of all feature maps, concatenates them along the feature dimension, and processes them with batch normalization and linear rectified activation function to obtain the dehazing feature map of the current decoding layer.

[0027] Furthermore, the image restoration network comprises a five-layer convolutional structure, with the first layer using a 1×1 convolutional kernel to compress channels and extract point-level features; The second layer uses a 3×3 convolution kernel, and the third layer uses a 5×5 convolution kernel. The outputs of the two layers are integrated through a bidirectional fusion mechanism. The fourth layer uses a 7×7 convolutional kernel to model the global fog concentration distribution, and its output is fed back to the fusion layer to form enhanced features; The fifth layer uses a 3×3 convolution kernel to refine and fuse the results, generating the final dehazed image.

[0028] Furthermore, the L1 norm loss function calculates the sum of the absolute values ​​of the differences between corresponding pixel values ​​in the hazy image and the dehazed image; The perceptual loss function calculates semantic differences based on the feature map of the l-th layer of a pre-trained deep neural network. The multi-scale structural similarity loss function calculates the similarity of brightness, contrast, and structure at multiple scales and then sums them by weight.

[0029] Furthermore, it also includes: In the first-level encoder, the first multi-branch downsampling module is used to extract fog map features. The feature map scale is reduced by downsampling operation to obtain the first downsampled feature map, and the processed first downsampled feature map is input into the second-level encoder. In the second-level encoder, the second multi-branch downsampling module is used to downsample the first downsampled feature map to reduce the feature map scale and obtain the second downsampled feature map. The processed second downsampled feature map is then input into the third-level encoder. In the third-level encoder, the third multi-branch downsampling module is used to downsample the second downsampled feature map to reduce the feature map scale and obtain the third downsampled feature map. The processed third downsampled feature map is then input into the fourth-level encoder. In the fourth-level encoder, the fourth multi-branch downsampling module is used to downsample the third downsampling feature map to reduce the feature map scale and obtain the fourth downsampling feature map. The processed fourth downsampling feature map is then input into the fifth-level encoder. In the fifth-level encoder, the fifth multi-branch downsampling module is used to downsample the fourth downsampled feature map to reduce the feature map scale and obtain the fifth downsampled feature map. The processed fifth downsampled feature map is then input into the fourth-level decoder.

[0030] Furthermore, S32 also includes: The first downsampled feature map, the second downsampled feature map, the third downsampled feature map, the fourth downsampled feature map, and the fifth downsampled feature map are input into the fourth-level decoder to obtain the fourth upsampled feature map; The fourth upsampled feature map, the first downsampled feature map, the second downsampled feature map, and the third downsampled feature map are input into the third-level decoder to obtain the third upsampled feature map; The third upsampled feature map, the first downsampled feature map, and the second downsampled feature map are input into the second-level decoder to obtain the second upsampled feature map. The second upsampled feature map is input into the first-level decoder to obtain the dehazed feature map.

[0031] In this embodiment, the multi-branch downsampling module PDDB contains 3 parallel branches, each branch consisting of 1-3 stacked PDConv units, wherein the first layer of PDConv uses a depthwise convolution with kernel size=3 and stride=2.

[0032] A dehazing network is built based on an encoder-decoder structure. The encoder contains multiple PDDB modules, each PDDB module contains multiple parallel branches, and each branch is composed of several stacked PDConv units. The encoding part extracts effective feature information from the fog map dataset. The decoder structure introduces a full-scale skip connection mechanism to generate a decoded feature map by fusing the feature maps of all encoding layers and the previous decoding layer. The feature map output from the final decoding layer is input into AOD-Net to generate a dehazed image; like Figure 2 and 3 As shown, pointwise convolution, depthwise convolution, and pointwise convolution are treated as a single convolutional block PDConv. Each PDConv pass through batch normalization and the ReLU activation function. The module employs a parallel and serial hybrid architecture, with each branch containing a different number of stacked PDConv-BN-ReLU sub-modules to extract feature information at different processing depths. The output features of all branches are fused in the middle, and the input features are directly passed to the fusion position, thereby achieving multi-level, multi-path feature enhancement. This structure effectively integrates shallow and deep features, improving the model's expressive power and training stability. This module is called PDDB (PDConv Downsampling Block). Each branch in the PDDB module consists of multiple stacked PDConvs. Branch 1 contains 1 PDConv, Branch 2 contains 2 PDConvs, and Branch 3 contains 3 PDConvs. The Depthwise convolution of Branch 1's PDConv uses kernel size=3, stride=2, and padding=1 to achieve 1 / 2 downsampling of the feature map size. Subsequent layers maintain stride=1, and the dilation coefficient is set to [1,2,4] to achieve multi-scale receptive fields.

[0033] Specifically, the defogging network is: Specifically, the first data processing, the second data processing, the third data processing, the fourth data processing, and the fifth data processing all involve: using the PDDB downsampling module to extract feature information from the fog map dataset, performing downsampling operations on the fog map dataset to reduce its scale, and then inputting it into the next level encoder.

[0034] The following section uses the downsampling module as an example to illustrate the network processing flow. The input feature map size of the multi-scale learning network is N×N, denoted as F∈R. N×N .

[0035] Input size: N×N. After the first downsampling: N / 2×N / 2; after the second downsampling: N / 4×N / 4; after the third downsampling: N / 8×N / 8; after the fourth downsampling: N / 16×N / 16; after the fifth downsampling: N / 32×N / 32.

[0036] To seamlessly integrate shallow, fine-grained information with deep semantic information, a feature aggregation mechanism was implemented on the concatenated feature maps at five scales. Taking the third decoder as an example: during upsampling, the decoder performs max pooling on the feature maps of the first and second encoders to pass on low-level semantic information. After unifying the feature maps, 64 3×3 filters are used for convolution, resulting in a 64-channel feature map to unify the number of feature maps and reduce redundant information. The fourth and fifth encoders undergo bilinear interpolation and convolution operations to unify the size and number of channels, and are finally concatenated according to dimensions to form a 320-channel feature map. Finally, batch normalization and the ReLU activation function are used to obtain the feature map of the third decoder.

[0037] The feature map of the first decoder is fed into the AOD-Net network, referring to... Figure 4 The AOD-Net network is specifically as follows: AOD-Net employs a convolutional neural network framework, with its core structure comprising five convolutional layers. Each layer is equipped with three parallel filters of varying sizes, designed to fuse multi-scale image features.

[0038] AOD-Net's five convolutional kernels employ a progressive multi-scale design: Conv1 uses a 1×1 kernel to compress channels and extract point-level features; Conv2 (3×3 kernel) and Conv3 (5×5 kernel) capture local details and mesoscale structures respectively, and their features are fused bidirectionally through Concat1 and Concat2; Conv4 uses a 7×7 large kernel to model the global fog density distribution, and its output is fed back to Concat2 after convolution to form enhanced features; Conv5 finally refines the fusion result of Concat3 with a 3×3 kernel to generate core parameters. K ( x ), directly outputs the dehazed image using physical formulas, such as Figure 5 As shown.

[0039] The model is trained using a linear weighted combination of three loss functions: L1 loss, VGG perceptual loss, and MS-SSIM. By using the trained model parameters and inputting a foggy image, a defogging image can be obtained.

[0040] Thanks to full-scale feature fusion on each decoder, this invention can fully capture and preserve the key details of the original image from all coding levels during the restoration process. Therefore, the generated dehazed images are significantly improved in terms of detail richness and realism, better matching the observation of the original hazy scene.

[0041] Example 1 To achieve realistic image dehazing, this invention proposes an image dehazing method based on multi-scale depth fusion. (See [link to relevant documentation]). Figure 1 See the description below for details:

[0042] Based on an atmospheric scattering model, using known depth of field, and randomly selecting global atmospheric light and atmospheric scattering coefficients, foggy images are generated from fog-free images to establish a training set. The specific steps are: selecting an image dataset with known depth of field information, and randomly selecting global atmospheric light... A Given a value between [0.7, 1.0], and a randomly selected atmospheric scattering coefficient β between [0.6, 1.8], the results are obtained from the haze-free map. J ( x Obtain the fog map I ( x ).

[0043] A dehazing network is constructed based on an encoder and a decoder. The network structure is built using a multi-scale learning module. The multi-scale learning module uses receptive fields and feature maps of different scales to extract fog-related features. The network is then connected in series with the dehazing network AOD-Net to obtain the final dehazing map.

[0044] The specific steps for establishing the training set in step 4 are as follows: Eight hundred images were selected from the known image dataset as fog-free images. For each fog-free image, eight fog images with different fog concentrations were generated based on the atmospheric scattering model and the atmospheric scattering coefficient, resulting in 6400 fog images. These 6400 pairs of fog-free and fog-within-fog images were used to form the training set.

[0045] A dehazing network is trained using a linear combination of the L1 norm loss function, the VGG perceptual loss function, and the MS-SSIM loss function. Using the trained model parameters, a dehazed image can be obtained by inputting a hazy image.

[0046] Before model training, there is a dataset preprocessing step: Standardize all images to the same size. .

[0047] Normalize all RGB pixel values ​​of the training set images to [-1, 1].

[0048] The training batch size was set to 8, the learning rate was set to 0.0001, and cosine annealing was used for training.

[0049] Compared with existing technologies, this invention has the following advantages: by constructing a dehazing network architecture based on multi-scale deep fusion, using a multi-branch downsampling module to extract fog-related features at different scales, combining a full-scale skip connection mechanism to achieve seamless fusion of shallow fine information and deep semantic information, and using a multi-loss function joint optimization strategy to balance pixel-level accuracy and perceptual quality, a high-quality dehazed image is finally generated through a progressive multi-scale image restoration network. This effectively solves the shortcomings of traditional methods in edge sharpness preservation, detail texture restoration, and color authenticity, and significantly improves the visual effect of image dehazing and the reliability of subsequent visual tasks.

[0050] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

[0051] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0052] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0053] The specific embodiments of the invention have been described in detail above, but they are only examples, and this application is not limited to the specific embodiments described above. For those skilled in the art, any equivalent modifications or substitutions to the invention are also within the scope of this application. Therefore, all equivalent changes, modifications, and improvements made without departing from the spirit and principles of this application should be covered within the scope of this application.

Claims

1. An image dehazing method based on multi-scale depth fusion, characterized in that: The specific steps include the following: S1: Based on the atmospheric scattering model, using known depth information, randomly select global atmospheric light values ​​and atmospheric scattering coefficient values ​​to generate foggy images from fog-free images and establish a training dataset; S2: Construct a dehazing network based on an encoder and a decoder. The encoder includes a first-level to a fifth-level encoder, and the decoder includes a first-level to a fourth-level decoder. The network structure of the dehazing network uses a multi-scale learning module to extract multi-scale feature information related to dehazing. S3: Input the image to be dehazed into the encoder and decoder to obtain the dehazed feature map, and input the dehazed feature map into the image recovery network to generate the final dehazed image; S4: The dehazing network is trained by using a linear weighted combination of the L1 norm loss function, the perceptual loss function, and the multi-scale structural similarity loss function as the total loss function. After training, a single hazy image is input and the corresponding dehazed image is output.

2. The image dehazing method based on multi-scale depth fusion according to claim 1, characterized in that, The encoder consists of five downsampling modules that progressively reduce the size of the input feature map, and the decoder consists of four upsampling modules that progressively increase the size of the feature map by setting the stride of the deconvolution operation in each upsampling module. The S3 further includes: S31, in each level encoder, a multi-branch downsampling module is used to extract fog map features, the feature map scale is reduced through downsampling operation, and the processed features are input to the next level encoder; S32 introduces a full-scale skip connection mechanism in each level of decoder, which merges the feature maps output by all five levels of encoders and the output feature maps of the previous level decoder, and splices them after spatial and channel alignment to generate the dehazing feature map of the current decoding layer. S33, the feature map output by the first-level decoder is input into the image restoration network, which adopts a progressive multi-scale convolutional structure to generate the final dehazed image.

3. The image dehazing method based on multi-scale depth fusion according to claim 1, characterized in that, In S1, the global atmospheric light value is randomly selected within the range of 0.7 to 1.0, and the atmospheric scattering coefficient value is randomly selected within the range of 0.6 to 1.

8. A foggy image is generated based on the atmospheric scattering model I(x) = J(x)t(x) + A(1 - t(x)), where t(x) is the transmittance, J(x) is the fog-free image, and A is the global atmospheric light.

4. The image dehazing method based on multi-scale depth fusion according to claim 1, characterized in that, Also includes: The multi-branch downsampling module contains three parallel branches: the first branch contains one convolutional unit, the second branch contains two convolutional units, and the third branch contains three convolutional units. The outputs of each branch are concatenated in the channel dimension to form a comprehensive feature representation. The convolutional unit is composed of pointwise convolution, depthwise convolution and further pointwise convolution connected in sequence, with batch normalization and activation function following each convolutional layer. The first depthwise convolution in the first branch uses a 3×3 convolution kernel with a stride of 2 and a padding parameter of 1 to achieve half-sampling of the feature map. The dilation coefficients of the subsequent depthwise convolutions are 1, 2, and 4, respectively, with a stride of 1.

5. The image dehazing method based on multi-scale depth fusion according to claim 1, characterized in that, The full-scale skip connection mechanism performs max pooling on the feature map of the shallow encoder and bilinear interpolation on the feature map of the deep encoder in each decoding layer. It then unifies the spatial size and number of channels of all feature maps, splices them along the feature dimension, and processes them with batch normalization and linear rectified activation function to obtain the dehazing feature map of the current decoding layer.

6. The image dehazing method based on multi-scale depth fusion according to claim 1, characterized in that, The image restoration network contains a five-layer convolutional structure, with the first layer using a 1×1 convolutional kernel; The second layer uses a 3×3 convolutional kernel, and the third layer uses a 5×5 convolutional kernel. The outputs of the two layers are integrated through a bidirectional fusion mechanism. The fourth layer uses a 7×7 convolutional kernel to model the global fog concentration distribution, and its output is fed back to the fusion layer to form enhanced features; The fifth layer uses a 3×3 convolution kernel to generate the final dehazed image.

7. The image dehazing method based on multi-scale depth fusion according to claim 1, characterized in that, The L1 norm loss function calculates the sum of the absolute values ​​of the differences between corresponding pixel values ​​in the hazy image and the dehazed image; The perceptual loss function calculates semantic differences based on the feature map of the l-th layer of a pre-trained deep neural network. The multi-scale structural similarity loss function calculates the similarity of brightness, contrast, and structure at multiple scales and then sums them by weight.

8. The image dehazing method based on multi-scale depth fusion according to claim 2, characterized in that, S31 further includes: In the first-level encoder, the first multi-branch downsampling module is used to extract fog map features. The feature map scale is reduced by downsampling operation to obtain the first downsampled feature map, and the processed first downsampled feature map is input into the second-level encoder. In the second-level encoder, the second multi-branch downsampling module is used to downsample the first downsampled feature map to reduce the feature map scale and obtain the second downsampled feature map. The processed second downsampled feature map is then input into the third-level encoder. In the third-level encoder, the third multi-branch downsampling module is used to downsample the second downsampled feature map to reduce the feature map scale and obtain the third downsampled feature map. The processed third downsampled feature map is then input into the fourth-level encoder. In the fourth-level encoder, the fourth multi-branch downsampling module is used to downsample the third downsampling feature map to reduce the feature map scale and obtain the fourth downsampling feature map. The processed fourth downsampling feature map is then input into the fifth-level encoder. In the fifth-level encoder, the fifth multi-branch downsampling module is used to downsample the fourth downsampled feature map to reduce the feature map scale and obtain the fifth downsampled feature map. The processed fifth downsampled feature map is then input into the fourth-level decoder.

9. The image dehazing method based on multi-scale depth fusion according to claim 2, characterized in that, The S32 further includes: The first downsampled feature map, the second downsampled feature map, the third downsampled feature map, the fourth downsampled feature map, and the fifth downsampled feature map are input into the fourth-level decoder to obtain the fourth upsampled feature map; The fourth upsampled feature map, the first downsampled feature map, the second downsampled feature map, and the third downsampled feature map are input into the third-level decoder to obtain the third upsampled feature map; The third upsampled feature map, the first downsampled feature map, and the second downsampled feature map are input into the second-level decoder to obtain the second upsampled feature map. The second upsampled feature map is input into the first-level decoder to obtain the dehazed feature map.

Images