Real scene image defogging method based on fusion of large kernel convolution and physical perception

Abstract

The invention discloses a real scene image defogging method based on large kernel convolution and physical perception fusion, and relates to the field of image defogging, and the method comprises the following steps: building a defogging model which comprises a large kernel strip convolution module and a double-flow physical perception fusion convolution network; collecting a training data set; a multi-scale physical defogging network is trained; defogging the real scene image; a large-kernel stripe convolution module in the convolution network adopts a multi-branch structure design, and the multi-branch structure comprises a local branch, a global branch and an identity mapping branch; the local branches are used for extracting fine-grained textures; the global branch is used for modeling structural features in horizontal and vertical directions; the identity mapping branches are used for transmitting original features. According to the method, multi-branch collaboration and direction perception are set in large-kernel strip convolution, so that the combined modeling capability of the network to local textures and long-distance directivity features is improved, and the adaptability to complex non-uniform haze is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of image dehazing technology, and in particular relates to a method for dehazing real-scene images based on large kernel convolution and physical perception fusion. Background Technology

[0002] In applications such as agricultural assessment, disaster monitoring, and environmental modeling, the quality of real-world images directly impacts the reliability of the task. However, dust, smoke, and water vapor particles in the air cause light scattering, leading to decreased image contrast, color distortion, and blurred details. These degradation phenomena not only weaken the visualization effect of the image but also significantly reduce the accuracy and stability of subsequent computer vision tasks. Therefore, image dehazing, as a key preprocessing technique, aims to restore the sharpness and natural perceptual characteristics of images, thereby enhancing their application value in various downstream tasks.

[0003] Traditional dehazing methods often rely on manually designed physical prior models, which can achieve certain results under specific conditions, but often perform poorly in complex and ever-changing real-world environments. With the rapid development of deep learning, data-driven methods have gradually shown significant advantages in dehazing tasks, not only learning complex fog distribution patterns but also effectively restoring obscured details and structures. However, convolutional networks are limited by their local receptive fields, making it difficult to capture long-distance dependencies and global structures; while networks with stronger global modeling capabilities improve dehazing quality, they are usually accompanied by high computational and memory overhead, limiting their implementation in real-time applications and resource-constrained devices. These challenges indicate that how to maintain dehazing effectiveness while balancing efficiency and generalization ability remains a key problem to be solved in this field. Based on this, this invention designs a real-scene image dehazing method based on large-kernel convolution and physical perception fusion. Summary of the Invention

[0004] The purpose of this invention is to solve the problems in the prior art, and to propose a method and apparatus for dehazing real scene images based on large kernel convolution and physical perception fusion.

[0005] This invention first discloses a method for dehazing real-scene images based on large kernel convolution and physical perception fusion, including the following steps: Includes the following steps: S1: Establish a dehazing model, wherein the dehazing model uses a fusion convolutional network including a large kernel strip convolutional module and a physical perception module to fit the mapping relationship between a foggy image and a clear image.

[0006] S2: Collect the training dataset; S3: Training a dehazing network for multi-scale physics; S4: Realistic scene image dehazing; In step S1, the large kernel strip convolution module in the convolutional network adopts a multi-branch structure design, which aims to simultaneously capture the local texture, global directional information and original features of the image, thereby achieving a higher quality dehazing effect. The multi-branch structure includes local branches, global branches, and identity mapping branches; The local branches are used to extract fine-grained textures; The global branch effectively models the structural features in the horizontal and vertical directions using large kernel strip convolution; The identity mapping branch is used to transmit the original features.

[0007] In the above method, the dual-stream physical sensing fusion module explicitly introduces an atmospheric scattering model during the feature fusion process of the encoder and decoder, and fuses features from the upsampling and encoding ends of the skip connections.

[0008] In the above method, the dual-stream physical sensing fusion module includes a transmittance branch and an illumination branch. These two branches respectively simulate an atmospheric scattering model and feature weighting before fusion, enhancing the complementarity of cross-layer features. The transmittance branch is used to simulate transmittance in atmospheric scattering models, highlighting local details and edge structures; The illumination branch is used to simulate atmospheric light and capture global brightness and color information.

[0009] In the above method, the local branch includes a 3×3 depth separable convolution, which is used to extract local fine-grained textures, and the response intensity of each channel is dynamically adjusted by setting a channel attention mechanism to highlight texture features that are sensitive to haze.

[0010] In the above method, the global branch includes large-scale horizontal and vertical strip convolution kernels to model the distribution pattern of haze in the horizontal and vertical directions. The output of the global branch is adaptively weighted by combining channel attention to ensure the coordination between directional features and local details.

[0011] In the above method, in the transmissivity branch, the features of the skip connections are first subjected to depthwise separable convolution to extract spatial structure information, and then a pixel-wise transmissivity weight is generated by 1×1 convolution and sigmoid activation function, thereby adaptively emphasizing detailed features such as edges and textures.

[0012] In the above method, the illumination branch obtains the overall semantics and context of the scene through global average pooling, and then uses lightweight 1×1 convolution and sigmoid activation function to generate illumination perception weights, which are used to adjust the overall brightness and color distribution, so that the restoration result is more natural and conforms to the real imaging law.

[0013] In the above method, the upsampling and skip connection features of the transmittance branch and illumination branch outputs are fused, and then spliced ​​and projected with a 1×1 convolution to generate high-quality fused features.

[0014] In the above method, the training dataset is based on the real-world scene datasets NH-HAZE, NH-HAZE2, and DENSE-HAZE.

[0015] The beneficial effects of this invention are as follows: 1. In this solution, the present invention sets up multi-branch collaboration and direction awareness in the large kernel strip convolution, which significantly improves the network's ability to jointly model local texture and long-distance directional features, thereby enhancing its adaptability to complex non-uniform haze. Experimental results show that this module can effectively improve the texture restoration, color restoration and overall visual consistency of dehazed images, and provide a more robust feature representation for the entire network.

[0016] 2. In this scheme, the dual-stream physical perception fusion module integrates multi-scale information and directly embeds physical priors into the feature learning process, significantly improving the interpretability and generalization ability of the network in real-world dehazing tasks. Simultaneously, this module demonstrates superior performance in detail restoration, global consistency, and model robustness, providing crucial support for achieving clearer and more natural dehazing results. Attached Figure Description

[0017] Figure 1 This is a model diagram of a real-scene image dehazing method based on large kernel convolution and physical perception fusion disclosed in this invention.

[0018] Figure 2 This is a flowchart illustrating the workflow of a real-scene image dehazing method based on large kernel convolution and physical perception fusion disclosed in this invention.

[0019] Figure 3 This is a visual comparison of images from the NH-HAZE dataset in a real-scene image dehazing method based on large kernel convolution and physical perception fusion disclosed in this invention.

[0020] Figure 4 This is a visual comparison of images from the DENSE-HAZE dataset in a real-scene image dehazing method based on large kernel convolution and physical perception fusion disclosed in this invention.

[0021] Figure 5 This is a visual comparison image of images from the NH-HAZE2 dataset in a real-scene image dehazing method based on large kernel convolution and physical perception fusion disclosed in this invention. Detailed Implementation

[0022] To facilitate understanding of this application and to make the aforementioned objectives, features, and advantages of this application more apparent, a detailed description of specific embodiments of this application is provided below in conjunction with the accompanying drawings. Numerous specific details are set forth in the following description to provide a thorough understanding of this application, and preferred embodiments are shown in the accompanying drawings. However, this application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application. This application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified. In the description of this application, "several" means at least one, such as one, two, etc., unless otherwise explicitly specified. It should be noted that when an element is referred to as being "fixed to" another element, it can be directly attached to the other element or there may be an intervening element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementations. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is only for describing particular implementations and is not intended to limit the scope of this application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0023] To better understand the technical solution of the present invention, the embodiments of the present invention will be further described below with reference to the accompanying drawings: refer to Figures 1-5 This invention designs a real-scene image dehazing method based on large kernel convolution and physical perception fusion. The hardware configuration includes an Intel(R) Xeon(R) Siler 4210R CPU processor, an Nvidia RTX A5000 graphics processor with a clock speed of 2.40GHz, 256GB of memory, and an Ubuntu 20.04 operating system. The dehazing method is implemented based on the PyTorch framework and specifically includes the following steps: Step 1: Build a dehazing model Let h represent the hazy image, r represent the restored clear image, and F represent the mapping relationship between the hazy image and the corresponding clear image. Then the dehazing problem is modeled in the following form (i.e., the dehazing model):

[0024] According to the above formula, once the mapping relationship F is obtained, given a foggy image h, a clear image can be obtained through the functional relationship mapping, thereby achieving image dehazing. This invention employs a convolutional network that fuses large-kernel strip convolution and physical perception to fit the mapping relationship F(h) between foggy and clear images. In this structure, the overall network framework is based on U-Net, and each layer consists of a feature extraction block for either upsampling or downsampling. Within the extraction block structure, large-kernel strip convolution blocks extract local texture details and long-range directional structural features in parallel, which are then fed into an atmospheric scattering model unit for effective dehazing. The atmospheric scattering model is described as follows:

[0025] Where I(x) is the image affected by haze; J(x) is the target image to be recovered; t(x) is the transmittance of light through the atmospheric medium; x represents the pixel position; and A is the global atmospheric light constant.

[0026] To better recover the foggy image, the original foggy image is fed back into the atmospheric scattering model unit at the final output of the network to guide the network to recover the image based on the original image structure.

[0027] Step 2: Design a dehazing network that fuses large-kernel strip convolution and physical perception. Based on the dehazing model established in step 1, design as follows: Figure 1 The dehazing network shown combines large-kernel strip convolution and two-stream physical awareness fusion. The large-kernel strip convolution module employs a multi-branch structure to simultaneously capture local texture, global directional information, and the original features of the input image, thereby achieving higher-quality dehazing results. The motivation behind this module's design is that haze is typically non-uniformly distributed in images, obscuring local details and causing structural attenuation along specific directions. Therefore, relying solely on local convolution or single-scale features is insufficient to fully recover damaged information, while introducing directional awareness and multi-scale feature fusion can better combat the degradation caused by haze.

[0028] Specifically, the local branches utilize 3×3 depthwise separable convolutions to extract local fine-grained textures, and dynamically adjust the response intensity of each channel through a channel attention mechanism to highlight texture features sensitive to haze. This effectively recovers fine edges and texture structures obscured by haze without significantly increasing computational cost.

[0029] The global branch introduces large-scale horizontal and vertical stripe convolutional kernels (1×2¹ and 2¹×1) to model the distribution patterns of haze in the horizontal and vertical directions. This design not only enhances the network's ability to perceive large-scale haze but also effectively extracts linear and structural information from the scene, such as building outlines and road directions, thus maintaining geometric consistency during the reconstruction process. The output of this branch is further adaptively weighted using channel attention to ensure coordination between directional features and local details.

[0030] The identity mapping branch directly passes the input features. Its existence can both preserve the undamaged details in the original image and alleviate gradient vanishing during training, thereby improving the overall stability and convergence speed of the model.

[0031] Finally, the results from the three branches are concatenated along the channel dimension and compressed and reintegrated through 1×1 convolution. This process achieves a dynamic balance between local texture and directional feature modeling, enabling the large-kernel strip convolution module to emphasize both fine-grained information and large-scale structural features in the feature space.

[0032] Thanks to this multi-branch collaborative and orientation-aware design, the large-kernel strip convolution module significantly improves the network's ability to jointly model local textures and long-range directional features, thereby enhancing its adaptability to complex and non-uniform haze. Experimental results show that this module can effectively improve texture restoration, color reproduction, and overall visual consistency of dehazed images, providing a more robust feature representation for the entire network.

[0033] In the dual-stream physical sensing fusion module, the main design idea is to explicitly introduce an atmospheric scattering model during the feature fusion process of the encoder and decoder. The classic atmospheric scattering model can be expressed as: I(x) = J(x)T(x) + A(1-T(x)) Where T(x) represents transmittance and A represents atmospheric light, this formula shows that the degradation of haze in the image is related to both transmittance distribution and global illumination conditions. Inspired by this, the dual-stream physical sensing fusion module decouples the fusion process of features from skip connections and upsampled features into two physically guided branches: the transmittance branch and the illumination branch.

[0034] In the transmittance branch, features are first extracted for spatial structure information through depthwise separable convolution, and then pixel-wise transmittance weights are generated using 1×1 convolution and a sigmoid activation function, thus adaptively emphasizing details such as edges and textures. This design can simulate the uneven occlusion of different areas by haze at the local level, significantly improving the ability to restore details. The illumination branch obtains the overall semantics and context of the scene through global average pooling, and then uses lightweight 1×1 convolution and a sigmoid activation function to generate illumination-aware weights to adjust the overall brightness and color distribution, making the restoration results more natural and consistent with the laws of real imaging.

[0035] It is worth noting that these two branches simulate atmospheric scattering models separately: the transmittance branch emphasizes local details, while the illumination branch focuses on global consistency. Their complementary effect not only enhances the effectiveness of cross-layer feature fusion but also establishes a dynamic balance between structure restoration and color correction. Finally, the fusion of the upsampled and skip connection features output from the two branches is concatenated and projected using a 1×1 convolution to generate high-quality fused features.

[0036] This dual-stream modeling approach effectively integrates multi-scale information and directly embeds physical priors into the feature learning process, significantly improving the network's interpretability and generalization ability in real-world dehazing tasks. Furthermore, the module demonstrates superior performance in detail restoration, global consistency, and model robustness, providing crucial support for achieving clearer and more natural dehazing results. The network takes a foggy image as input and outputs a corresponding clear image; the entire network has an end-to-end structure. Each layer of the network extracts features, which are then processed by an atmospheric scattering model unit. The final layer of the network maps the feature maps back to the original image dimensions.

[0037] Step 3: Collect the training dataset This invention utilizes three real-world haze datasets: DENSE-HAZE, NH-HAZE, and NH-HAZE2. Both DENSE-HAZE and NH-HAZE datasets contain 45 training images and 5 test images, while the NH-HAZE2 dataset is larger, containing 70 training images and 5 test images. Unlike DENSE-HAZE, which exhibits relatively uniform fog distribution and overall density, NH-HAZE and NH-HAZE2 more closely resemble natural scenes, displaying significant non-uniformity in fog distribution. Not only is there substantial variation in fog concentration across different regions, but it is also accompanied by complex background textures and lighting variations. This characteristic makes dehazing more challenging, as the model needs to recover occluded details while maintaining the naturalness of the overall structure and color. In particular, the NH-HAZE2 dataset further enhances the handling of scene diversity and fog complexity, better reflecting the difficulties encountered in real-world applications.

[0038] Step 4: Train a multi-scale physics-based dehazing network Learning-based dehazing methods require labeled fog samples for training. In this step, the network is trained using collected foggy image samples to learn the mapping relationship between foggy and clear images. To achieve high-quality dehazing, we employ a composite loss function, Ltotal, which combines three complementary terms. The Ltotal loss function is as follows:

[0039]

[0040]

[0041]

[0042] in, This is the actual output of the network. This is the true value image.

[0043] Where J represents the actual output of the network, and GT represents the ground truth image. This invention was trained using the PyTorch framework on a system equipped with two NVIDIA GeForce RTX A5000 GPUs. We used the Adam optimizer (β1 = 0.5, β2 = 0.999) and cosine annealing to optimize the algorithm, setting the initial learning rate to 0.00008 for the NH-HAZE and NH-HAZE 2 datasets, and 0.00002 for the DENSE-HAZE dataset. During training, images were randomly cropped to 800 × 600 pixels, and the original image size was used for testing. This invention chose stochastic gradient descent to optimize the loss function, using hazy images to iteratively learn the network and update the network parameters. Training ended when the network loss value stabilized, and the saved network parameters at this point constituted the trained dehazed network model.

[0044] Step 5: Dehazing of Realistic Scene Images The real-world image dehazing method designed in this invention is end-to-end. Once the network model is trained, the image to be dehazed is simply input into the network, and through the network's forward propagation, the restored image can be obtained at the output. Clear images. This invention uses a test set from the dataset for performance testing. To demonstrate the competitiveness of this dehazing method, the network is compared with eight state-of-the-art dehazing methods: DCP, GridDehaze-Net, Dehamer, Dehazeformer-L, OK-Net, Two-Branch-Dehazing, MixDehaze-Net, and AdaIR. To ensure a fair comparison, we used the official implementations of these deep learning models, provided by their respective authors, during training. Intuitive results can be found in the appendix. Figures 3-5 The data can be found in the table below.

[0045] Table 1 compares our method with eight other methods on the NH-HAZE and DENSE-HAZE datasets. Table 2 compares our method with eight other methods on the NH-HAZE2 dataset and model complexity. In the tables, PSNR represents peak signal-to-noise ratio, SSIM represents result similarity, LPIPS is used to evaluate the perceptual difference between two images, FLOPs represents model complexity (the smaller the FLOPs value, the lower the model complexity), Hazy Image represents a foggy image, and GT represents a ground truth image.

[0046] Table 1. Quantitative Comparison Results of NH-HAZE and DENSE-HAZE Datasets

[0047] Table 1 (Continued) shows the quantitative comparison results between the NH-HAZE and DENSE-HAZE datasets.

[0048] Table 2. Quantitative comparison results and model complexity comparison of the NH-HAZE2 dataset.

[0049] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative in all respects and are not the only ones. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. A method for dehazing real-scene images based on large kernel convolution and physical perception fusion, characterized in that, Includes the following steps: S1: Establish a dehazing model, wherein the dehazing model uses a convolutional network including a large kernel strip convolutional module and a two-stream physical perception fusion to fit the mapping relationship between the hazy image and the clear image; S2: Collect the training dataset; S3: Training a dehazing network for multi-scale physics; S4: Realistic scene image dehazing; In step S1, the large kernel strip convolution module in the convolutional network adopts a multi-branch structure design, which aims to simultaneously capture the local texture, global directional information and original features of the image, thereby achieving a higher quality dehazing effect. The multi-branch structure includes local branches, global branches, and identity mapping branches; The local branches are used to extract fine-grained textures; The global branch is used to model structural features in the horizontal and vertical directions; The identity mapping branch is used to transmit the original features.

2. The defogging method according to claim 1, characterized in that, The dual-stream physical sensing fusion module explicitly introduces an atmospheric scattering model during the feature fusion process between the encoder and decoder, and fuses features from the upsampling and encoding ends via skip connections.

3. The defogging method according to claim 1, characterized in that, The dual-stream physical sensing fusion module includes a transmittance branch and an illumination branch. These two branches respectively simulate an atmospheric scattering model and feature weighting, enhancing the complementarity of cross-layer features. The transmittance branch is used to simulate transmittance in atmospheric scattering models, highlighting local details and edge structures; The illumination branch is used to simulate atmospheric light and capture global brightness and color information.

4. The defogging method according to claim 1, characterized in that, The local branches include 3×3 depth-separable convolutions, which are used to extract local fine-grained textures, and dynamically adjust the response intensity of each channel by setting a channel attention mechanism to highlight texture features that are sensitive to haze.

5. The method for dehazing real-scene images based on large kernel convolution and physical perception fusion according to claim 1, characterized in that, The global branch includes large-scale horizontal and vertical strip convolution kernels to model the distribution patterns of haze in the horizontal and vertical directions. The output of the global branch is adaptively weighted by combining channel attention to ensure the coordination between directional features and local details.

6. The defogging method according to claim 2, characterized in that, In the transmissivity branch, the features of the skip connections are first subjected to depthwise separable convolution to extract spatial structure information, and then a pixel-wise transmissivity weight is generated by 1×1 convolution and sigmoid activation function, thereby adaptively emphasizing detailed features such as edges and textures.

7. In the dehazing method according to claim 2, the illumination branch obtains the overall semantics and context of the scene through global average pooling, and then uses lightweight 1×1 convolution and sigmoid activation function to generate illumination perception weights to adjust the overall brightness and color distribution, so that the restoration result is more natural and conforms to the real imaging rules.

8. The defogging method according to claim 7, characterized in that: The upsampling and skip connection features of the transmittance branch and illumination branch outputs are fused, and then spliced ​​and projected with a 1×1 convolution to generate high-quality fused features.

9. The defogging method according to claim 7, characterized in that: In step S2, the training dataset is based on the real-world scene datasets NH-HAZE, NH-HAZE2, and DENSE-HAZE.

Images