An end-to-end defogging network fused with mamba and multi-scale context enhancement

Abstract

The application discloses an end-to-end defogging network fusing Mamba and multi-scale context enhancement, and belongs to the technical field of computer vision and image restoration. The existing defogging technology has problems of complex global modeling calculation, poor local detail recovery and insufficient feature fusion adaptation. Four core structure designs are adopted to solve the problems: 1. a Mamba state space model is introduced to realize global fog long-range dependence modeling with linear complexity; 2. a multi-scale hollow convolution module and a local context enhancement unit are designed to cover multi-scale local features; 3. an enhanced adaptive fusion module is constructed to realize dynamic adaptation of global and local features; and 4. a'multi-scale input-cross-scale fusion-multi-scale output' architecture is adopted to gradually recover details. The network can efficiently process high-resolution foggy images and is suitable for scenes requiring real-time high-precision defogging such as automatic driving, security monitoring, remote sensing imaging and the like.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image restoration technology, specifically an end-to-end dehazing network that integrates a state-space model (Mamba) with multi-scale context enhancement. This network can efficiently process high-resolution foggy images, accurately restoring local texture details while maintaining the accuracy of global fog distribution modeling. It is suitable for scenarios requiring real-time, high-precision dehazing, such as autonomous driving, security monitoring, remote sensing imaging, and outdoor monitoring equipment. It is particularly effective in addressing image quality degradation caused by non-uniform fog in complex environments, providing high-quality image input for subsequent advanced vision tasks such as object detection and image recognition. Background Technology

[0002] With the widespread application of outdoor vision equipment, the impact of severe weather conditions such as fog and mist on image acquisition is becoming increasingly prominent. Foggy images generally suffer from reduced contrast, blurred details, and color distortion, which not only affects human visual perception but also severely restricts the performance of critical tasks such as road condition recognition in autonomous driving, target tracking in security monitoring, and ground feature analysis in remote sensing imaging. Therefore, image dehazing technology, as a core component of image preprocessing, has become an important research direction in the field of computer vision.

[0003] Early image dehazing methods were mostly based on atmospheric scattering physics models. They estimated key parameters such as atmospheric light and transmittance in the model by designing manual priors (e.g., dark channel priors, color attenuation priors), thereby achieving dehazing. However, these methods rely on strong prior assumptions and are prone to parameter estimation biases in non-uniform haze and complex lighting scenarios, leading to problems such as color blocks and loss of detail in dehazed images, and limiting their generalization ability. With the development of deep learning technology, dehazing methods based on convolutional neural networks (CNNs) have become mainstream. CNNs, with their powerful feature extraction capabilities, can directly map hazy images to clear images through end-to-end learning, or more accurately predict atmospheric scattering model parameters. However, traditional CNNs are limited by a fixed receptive field, making it difficult to simultaneously capture the multi-scale features of non-uniform haze—they lack sufficient modeling of the global distribution of large-scale dense fog areas and are less sensitive to the recovery of local details such as fine textures and edges, easily leading to overall blurring and loss of detail in the dehazed image. In recent years, Transformer-type models, with their global modeling capabilities through self-attention mechanisms, have been introduced into dehazing tasks, effectively capturing the long-range dependence of fog distribution across the entire image. However, self-attention mechanisms suffer from quadratic computational complexity (O(n²)). When processing high-resolution images (such as 4K images in autonomous driving scenarios), the computational load increases exponentially, failing to meet real-time requirements and making deployment difficult on edge devices with limited computing power (such as in-vehicle terminals and outdoor surveillance cameras). Furthermore, existing methods have significant shortcomings in fusing global and local features: most employ simple stitching or fixed-weight fusion, failing to dynamically adjust feature weights based on the spatial heterogeneity of fog concentration—over-reliance on local features in dense fog regions leads to incomplete defogging, while over-reliance on global features in light fog regions results in smoothing of details. In summary, current defogging technology faces three core challenges: low global modeling efficiency, poor local detail recovery, and insufficient adaptive feature fusion. An end-to-end solution that balances computational efficiency and defogging accuracy while optimizing global-local feature collaborative modeling is urgently needed. Summary of the Invention

[0004] To address the shortcomings of existing dehazing methods in terms of global modeling efficiency, local detail restoration, and adaptive feature fusion, this invention provides an end-to-end dehazing network that integrates Mamba and multi-scale context enhancement. Through an architecture design of "dual-branch collaborative modeling + dynamic fusion mechanism," this network achieves accurate global fog distribution modeling and efficient local detail restoration while maintaining linear computational complexity, ultimately outputting high-quality, real-time dehazing results.

[0005] The core design idea of ​​this invention is to decompose the dehazing task into two major sub-tasks: "global fog distribution modeling" and "local detail feature extraction". The global branch of MambaModel solves the problem of low computational efficiency of traditional Transformer, and the local branch of Multi-Scale Dilated Convolution Module (MDCM) makes up for the limited receptive field of CNN. The Enhanced Adaptive Fusion Module (EAF) realizes the dynamic adaptation and fusion of features from the two branches. Combined with multi-objective loss function, pixel accuracy and visual quality are optimized to form a complete end-to-end dehazing process.

[0006] The specific technical solution is as follows, and the implementation of this network includes the following steps:

[0007] Step 1: Construct an end-to-end architecture of "multi-scale input - cross-scale fusion - multi-scale output" to achieve progressive restoration from foggy images to clear images, providing a basic framework for collaborative modeling of global and local features.

[0008] Input Processing: For a single hazy image of its original size (H×W×3, where H is height, W is width, and 3 is the number of RGB channels), two downsampled input images are generated: one half (H / 2×W / 2×3) and one quarter (H / 4×W / 4×3) of the original size. Multi-scale input can cover the feature requirements of different fog concentration scales—small-scale input facilitates rapid capture of the global fog distribution, while large-scale input helps preserve local details, avoiding the problems of "incomplete global modeling" or "loss of local details" caused by single-scale input.

[0009] Encoding Process: A layered convolutional encoder is employed to extract features from the foggy input at three different scales. The encoder maps the input image from pixel space to feature space through layer-by-layer convolution stacking and downsampling operations, simultaneously extracting features at different levels—shallow features focus on local details such as texture and edges, while deeper features capture macroscopic information such as global fog distribution and scene structure. Furthermore, convolutional layers with specific structures (such as 3×3 ordinary convolutions) are introduced to extract contextual features from the downsampled input. These contextual features are then fused with the encoder features at the corresponding scales, enhancing the correlation between global and local features and providing richer foundational features for subsequent bi-branch modeling.

[0010] Decoding Process: A progressive upsampling decoder structure is adopted, gradually upsampling the deep features output by the encoder to the original image size. During the restoration process, the features of the global branch of MambaModel and the local branch of MDCM are integrated in conjunction with the enhanced adaptive fusion module (detailed in step 6) to ensure that the features at each level can fuse global fog distribution information and local detail information. Finally, the decoder outputs the dehazing results corresponding to the three input scales, where the dehazing result at the original size is used as the final output, and the small-scale dehazing result is used to assist training and improve the model's adaptability to scenes with different fog concentrations.

[0011] Step 2: MambaModel global modeling branch design. Based on the state space model, an efficient global branch is constructed to break through the bottleneck of the complexity of the second-order calculation of Transformer. While ensuring the efficiency of linear calculation, it accurately captures the long-range dependency of fog distribution in the whole image and provides global fog distribution prior for the defogging task.

[0012] Feature preprocessing: The basic features output by the encoder (denoted as X) are enhanced using convolutional residual blocks. This residual block consists of two 3×3 convolutional layers and one non-linear activation function (such as ReLU): First, the features undergo dimensionality transformation and preliminary feature extraction through the first 3×3 convolutional layer. After the activation function introduces non-linear expressive power, the features are further enhanced through the second 3×3 convolutional layer. Finally, the initial features X are directly superimposed onto the processed features through residual connections, avoiding the loss of feature information during transmission and mitigating the gradient vanishing problem in deep networks. The preprocessed features (denoted as X_conv) retain the local structural information of the basic features while enhancing their discriminative power, laying the foundation for subsequent global modeling.

[0013] Sequence Transformation and Normalization: The Mamba model natively adapts to one-dimensional sequence inputs. Therefore, the two-dimensional feature map X_conv (dimensions B×C×H×W, where B is the batch size and C is the number of channels) needs to be converted into a one-dimensional sequence. Specifically, the spatial dimensions (H×W) of the feature map are first flattened to one dimension (HW), resulting in features of dimension B×C×HW. Then, the dimensional order is adjusted to B×HW×C (HW is the sequence length L) to fit the Mamba input format (B×L×C). After the conversion, layer normalization (LayerNorm) is applied to the channel dimension (C) of the sequence to stabilize numerical fluctuations during model training and avoid training instability caused by excessive differences in feature values.

[0014] Global Dependency Modeling: Mamba's selective state-space mechanism dynamically captures key information in the sequence (such as regions of abrupt fog concentration changes and scene boundary regions). The core of this mechanism is the combination of linear state updates and gating: in each step of sequence processing, Mamba dynamically adjusts the contribution ratio of historical states to the current input based on the importance of the current input features through a gating function. For regions with gradual fog concentration changes, the gating function reduces the weight of the current input, relying more on historical states to maintain global consistency; for regions with abrupt fog concentration changes, the gating function increases the weight of the current input, strengthening the capture of local abrupt features. Compared to the Transformer's self-attention mechanism, the computational complexity of this process is only linear (O(B×L×C), L=HW), which can significantly reduce computational load and improve real-time performance in high-resolution image processing.

[0015] Spatial Feature Reconstruction: The one-dimensional sequence (B×L×C) output by Mamba is converted back into a two-dimensional feature map to adapt to the subsequent feature fusion process. The specific operation is the reverse of the sequence transformation: first, the dimensional order is adjusted to B×C×L, then the sequence length L is restored to the spatial dimension H×W, resulting in features of dimension B×C×H×W; finally, a 3×3 convolution layer is used to smooth spatial noise in the feature map (such as spatial structure shifts caused by Mamba sequence processing), enhancing the consistency of global features in spatial dimensions and ensuring spatial alignment during local feature fusion. The reconstructed features (denoted as F_g) are the output of the global modeling branch, containing long-range dependency information on the fog distribution across the entire image.

[0016] Step 3: Multi-scale parallel dilated convolution branch design. As the core component of the local perception branch, it achieves comprehensive extraction of local features at different scales through multiple sets of parallel dilated convolutions with differentiated parameters, adapts to the feature distribution requirements of non-uniform haze, and provides multi-scale feature support for local detail recovery.

[0017] Branch parameter design: Three independent parallel convolutional branches are set up. Each branch covers local features at near, medium and far scales through different combinations of dilation rate and convolutional kernel size, ensuring multi-scale adaptation to non-uniform haze.

[0018] Short-range feature branch: Employs a 3×3 convolution with a dilation rate of 1, resulting in the smallest receptive field (3×3). This branch focuses on capturing subtle texture details in the image, such as leaf textures, road markings, and object edges, avoiding detail loss due to an excessively large receptive field. In foggy scenes, it can accurately recover slightly obscured fine structures, enhancing the detail richness of dehazed images.

[0019] Mid-range feature branch: Employs a 3×3 convolution with a dilation rate of 3, resulting in a medium receptive field (7×7). This branch is used to extract mid-scale fog distribution features in the image, adapting to scenes with gradually changing fog concentrations in local areas, such as dense fog around buildings. By expanding the receptive field, the overall shape and distribution range of the fog can be captured, avoiding the omission of small, localized dense fog areas.

[0020] Long-range feature branch: Employs a 5×5 convolution with a dilation rate of 2, resulting in the largest receptive field (9×9). This branch can cover large-scale, non-uniform fog areas, such as large areas of dense fog and distant fog scenes, capturing features of large-scale fog concentration differences. By using a larger convolution kernel and a moderate dilation rate, it expands the receptive field while avoiding feature dilution, ensuring the integrity of feature extraction for large areas of dense fog.

[0021] Feature output processing: After convolution, each branch is sequentially connected to a batch normalization layer and a non-linear activation function. The batch normalization layer reduces feature differences between different branches by standardizing the numerical distribution of features, providing a unified numerical basis for subsequent feature fusion. The non-linear activation function introduces non-linear expressive power, enhancing the model's ability to capture complex local features and avoiding the problem of insufficient expression caused by linear features. The three processed branch features (denoted as B1, B2, and B3) correspond to local features at near, medium, and far scales, respectively, providing high-quality basic features for subsequent local context enhancement.

[0022] Step 4: Local Context Enhancement Unit Design. To address the mesh artifact problem that is easily generated by dilated convolution, attention mechanism and lightweight convolution are combined to enhance the discriminativeness of local features, improve the perception accuracy of non-uniform haze and fine textures, and optimize the quality of local features.

[0023] Channel Attention Enhancement: For the features output by each parallel branch (B1, B2, B3), a channel attention mechanism is introduced. This mechanism adaptively generates weight coefficients for each channel by calculating the mean of the features along the channel dimension. Larger weight coefficients are generated for channels that significantly contribute to the dehazing task, such as fog density difference channels and texture detail channels, to enhance their representation; smaller weight coefficients are generated for irrelevant channels to suppress their interference. Specifically, the process involves: first, performing channel-dimensional average pooling on the branch features to obtain global statistical information for each channel; then, mapping the statistical information to channel weights using two 1×1 convolution layers; finally, multiplying the weights point-by-point with the original branch features to obtain the channel-weighted features. This process highlights the expression of key feature channels, providing more discriminative channel features for subsequent detail recovery.

[0024] Depthwise separable convolution optimization: 3×3 depthwise separable convolution is used to process the channel-weighted features. Depthwise separable convolution breaks down traditional convolution into "depthwise convolution" and "pointwise convolution": depthwise convolution performs convolution on each channel individually, strengthening the local spatial relationships within each channel; pointwise convolution integrates features from different channels through 1×1 convolution, enabling information exchange between channels. Compared to traditional convolution, this method significantly reduces computational cost while effectively suppressing mesh artifacts caused by dilated convolution—by strengthening the local spatial relationships through depthwise convolution, it fills the gaps in the feature space caused by the dilation rate, making the spatial continuity of the feature map stronger and avoiding detail distortion caused by artifacts during subsequent fusion.

[0025] Residual connection integration: Features processed by depthwise separable convolution are residually connected to the original branch features before channel attention weighting. This direct stacking preserves the basic information of the initial local features, avoiding over-smoothing or loss of detail during enhancement, such as blurring of fine textures by convolution operations. Simultaneously, residual connections alleviate the gradient vanishing problem of local context enhancement units, ensuring the stability of the training process. The integrated features (denoted as B1', B2', B3') represent the outputs of each parallel branch after local context enhancement, possessing stronger local feature discriminativeness and spatial continuity.

[0026] Step 5: Cross-modal gating interaction design, as the core mechanism of dual-branch collaboration, dynamically allocates the fusion ratio of global and local features through gating scores, so as to achieve the collaborative effect of global guidance of local and local correction of global, and avoid the problem of insufficient feature adaptation caused by fixed weight fusion.

[0027] Feature Sequence Transformation: The two-dimensional local features output from the local branches of the two-dimensional global features output by MambaModel are converted into one-dimensional sequence forms. The specific operation is the same as the sequence transformation in step 2: first flatten the spatial dimension to one dimension, then adjust the dimensional order to B×L×C (L=HW). This transformation can break the spatial dimension limitation, making it easier to model the fine-grained relationship between global and local features—compared to directly fusing on the two-dimensional feature map, the sequence form can more flexibly capture the correspondence between global fog distribution and local details (such as the global fog concentration level corresponding to a certain local detail area), avoiding fusion deviations caused by differences in spatial structure.

[0028] Gated score generation: The transformed global feature sequence and local feature sequence are concatenated by channels (the dimension becomes B×L×2C), and then input into a multilayer perceptron for nonlinear transformation to generate a gated score. The MLP consists of two linear transformation layers and one activation function layer: the first linear transformation layer maps the number of channels from 2C to 4C, increasing the model's ability to express complex associations by adding dimension; after introducing nonlinearity through the activation function, the second linear transformation layer maps the number of channels back from 4C to C, ensuring the output dimension is consistent with the original features; finally, the Sigmoid activation function compresses the output value to the 0-1 range, obtaining the gated score (denoted as g). The value of the gated score g directly represents the weight ratio of the global features in the fusion result, and the remaining ratio (1-g) is automatically allocated to local features—in dense fog areas, the value of g automatically increases, and in light fog / clear areas, the value of g automatically decreases, achieving dynamic weight adaptation.

[0029] Feature weighted fusion and reconstruction: Based on the gating score g, the global feature sequence and the local feature sequence are weighted and summed to obtain the fused feature sequence. The fused sequence is then converted back into a two-dimensional feature map, with the operation consistent with the spatial reconstruction in step 2, ensuring that subsequent processing meets the spatial structure requirements of the image features and providing a fusion basis for channel-space collaborative attention. The fused features initially integrate the advantages of global and local features, providing a foundation for subsequent optimization.

[0030] Step 6: Channel-space collaborative attention design, which performs secondary optimization on the features after cross-modal gating fusion, strengthens key information from both channel and spatial dimensions, further improves the discriminativeness and dehazing adaptability of the fused features, and ensures a precise balance between global fog removal and local detail preservation.

[0031] Channel Dimension Optimization: First, the global features output by MambaModel and the local features output by local branches are concatenated along the channel dimension. Adaptive average pooling is then applied to the concatenated features—compressing the feature map to a 1×1 spatial dimension. This aggregates global statistical information at the channel level, avoiding interference from spatial dimension noise in channel weight calculation. Subsequently, two 1×1 convolutions transform the aggregated information: the first 1×1 convolution reduces the number of channels from 2C to C, lowering computational complexity and compressing channel information; the second 1×1 convolution restores the number of channels to 2C, consistent with the number of channels in the concatenated features, ultimately generating a channel-dimensional weight map. Multiplying this weight map point-by-point with the original concatenated features strengthens the expression of channels crucial for dehazing, suppresses interference from irrelevant channels, and lays a high-quality channel foundation for spatial dimension optimization.

[0032] Spatial dimension optimization: Channel-weighted features are input into a 3×3 depthwise separable convolution. Depthwise convolution strengthens feature associations within the local space, and pointwise convolution integrates channel information. Then, a 1×1 convolution layer is used to adjust the number of feature channels to C, generating a spatial dimension weight map. This weight map adaptively highlights features in locally clear areas of the image while suppressing redundant information in foggy areas, ensuring that key details in the spatial dimension are not obscured by fog features.

[0033] Collaborative attention integration: The spatial weight map is multiplied point-by-point with the cross-modal gating fused features to obtain spatially weighted features; then, these features are residually concatenated with the original features to form channel-spatial dual-optimized features. This integration method retains the basic fusion information of F_fused while strengthening the feature representation of key channels and key regions through the synergistic effect of channel and spatial attention, effectively resolving the contradiction between incomplete global dehazing and loss of local details, and providing high-quality features for subsequent multi-scale feature enhancement.

[0034] The invention is further characterized by:

[0035] Step 1 Implementation Process: By covering features of different fog concentrations through multi-scale input, strengthening global-local correlation through cross-scale feature fusion, and achieving progressive detail restoration through multi-scale output. Compared with the traditional single-scale architecture, this framework can optimize global modeling and local perception at different scales, avoiding the contradiction of not being able to achieve both global and local goals at a single scale, while providing an adapted feature input environment for subsequent dual-branch and dynamic fusion mechanisms.

[0036] Step 2 Implementation Process: The linear computational complexity of Mamba addresses the inefficiency of the Transformer. Residual preprocessing and spatial reconstruction preserve the spatial structure of features, while a selective state-space mechanism accurately captures key fog distribution information. This branch satisfies the real-time processing requirements of high-resolution images and provides a global fog distribution prior for subsequent local detail restoration, avoiding over- or under-dehazing in certain areas.

[0037] Step 3 Implementation Process: By combining differentiated dilation rates and convolutional kernel sizes, multi-scale feature extraction from fine textures to large-area fog masses is achieved, overcoming the limitations of traditional CNNs with their fixed receptive fields. This branch provides a rich multi-scale feature foundation for local perception, ensuring that subsequent processing can accurately adapt to the different scale feature requirements of non-uniform fog.

[0038] Step 4 implementation process: Key features are enhanced through channel attention, artifacts are suppressed through depthwise separable convolution, and detailed information is preserved through residual connections. This unit addresses the inherent defects of dilated convolution, providing high-quality local features for subsequent weighted fusion and ensuring the accuracy of local detail recovery.

[0039] Step 5 Implementation Process: Adaptive weight allocation of global and local features is achieved through gating scores, avoiding the limitations of fixed-weight fusion. This mechanism enables the model to dynamically adjust feature dependencies based on the spatial heterogeneity of fog concentration, achieving optimal feature fusion in different fog conditions and providing high-quality initial fused features for subsequent channel-space collaborative attention.

[0040] Step 6 Implementation Process: Channel attention focuses on key fog feature and detail feature channels, while spatial attention highlights clear and detailed areas. The combined effect of these two approaches achieves refined optimization of the fused features. This module enables a better balance between global dehazing and local detail preservation in the fused features, providing crucial support for improving the quality of the final dehazing result.

[0041] In summary, the main technical contributions of this invention are as follows:

[0042] (1) The Mamba state space model is introduced into the defogging task. The linear computational complexity (O(n)) replaces the quadratic complexity (O(n²)) of the Transformer. While ensuring the long-range dependence of global fog distribution on the capture accuracy, the processing efficiency of high-resolution images is greatly improved, and the contradiction between global modeling and real-time performance is resolved.

[0043] (2) Design multi-scale parallel dilated convolution branches and local context enhancement units. By using differentiated dilation rate and convolution kernel size, local features at near, medium and far scales are covered. Combined with channel attention and depth-separable convolution, artifacts are suppressed and details are enhanced, thus making up for the limited receptive field of traditional CNN.

[0044] (3) A two-level fusion mechanism of cross-modal gating interaction and channel-space collaborative attention is proposed. Gating interaction realizes the adaptive weight allocation of global and local features, and collaborative attention realizes the enhancement of key information in two dimensions. The combination of the two solves the problem of insufficient adaptability of fixed fusion methods.

[0045] (4) Construct an end-to-end architecture of multi-scale input-cross-scale fusion-multi-scale output. Expand the feature coverage through multiple inputs and achieve progressive detail restoration through multiple outputs, avoiding the problem of incomplete global modeling or loss of local details caused by single-scale input. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the overall structure of the proposed end-to-end defogging network that integrates Mamba and multi-scale context enhancement.

[0047] Figure 2 This is a schematic diagram of the proposed end-to-end defogging network training and evaluation testing process that integrates Mamba and multi-scale context enhancement.

[0048] Figure 3 The image shows a visual comparison of the proposed end-to-end dehazing network that integrates Mamba and multi-scale context enhancement with other dehazing methods. Detailed Implementation

[0049] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the described embodiments are intended only to facilitate understanding of the invention and are not intended to limit it in any way. The drawings are all in a very simplified form and use non-precise proportions, only to facilitate and clarify the illustration of the embodiments of the invention; the structures shown in the drawings are part of the actual structures. Furthermore, the technical features involved in the various embodiments of the invention described below can be combined with each other as long as they do not conflict with each other.

[0050] The end-to-end dehazing network proposed in this invention, which integrates Mamba and multi-scale contextual enhancement, is a breakthrough solution addressing the three core pain points of current dehazing technologies: low global modeling efficiency, poor local detail recovery, and insufficient adaptive feature fusion. This network employs a collaborative architecture of "Mamba global modeling branch + multi-scale dilated convolutional local perception branch + enhanced adaptive fusion module," achieving accurate modeling of the global fog distribution and efficient recovery of local details while maintaining linear computational complexity. The final output is a dehazing result with both high accuracy and high real-time performance, providing crucial image preprocessing technology support for fields such as autonomous driving, security monitoring, and remote sensing imaging.

[0051] As attached Figure 1 As shown, the overall structure of an end-to-end defogging network integrating Mamba and multi-scale context enhancement is presented. This structure mainly includes the following six key component stages:

[0052] (1) Multi-scale input and dual-branch feature extraction: Multi-scale (1 / 2 and 1 / 4 of the original size) inputs are generated for the original foggy image. Local multi-scale features and global long-range dependency features are extracted in parallel by the multi-scale dilated convolution branch and the global branch of the Mamba module, respectively, laying a dual-dimensional feature foundation for subsequent collaborative modeling.

[0053] (2) Multi-scale dilated convolution branch design: This branch covers the local features of non-uniform haze at the near, medium and far scales through a three-parallel dilated convolution structure of near, medium and far ranges, and adapts to the detailed perception needs of different scenarios such as leaf texture, local fog, and large area of ​​dense fog.

[0054] (3) Global branch design of Mamba module: The Mamba state space model is introduced to convert 2D features into 1D sequences and capture the long-range dependency of fog distribution in the whole image with linear computational complexity (O (n)), replacing the quadratic complexity (O (n²)) of the traditional Transformer. While ensuring the accuracy of global modeling, the processing efficiency of high-resolution images is greatly improved.

[0055] (4) Context enhancement unit design: The output features of multi-scale dilated convolution are enhanced, and the fog features, detail features and depth separable convolution are combined with channel attention to enhance the discriminativeness and spatial continuity of local features, providing high-quality local input for feature fusion.

[0056] (5) Gating weight and attention fusion design: The fusion ratio of global and local features is dynamically allocated by the gating weight unit, with the fog area emphasizing the global and the fog area emphasizing the local; combined with spatial attention to highlight clear areas and pixel attention to strengthen detail gradients, the fusion features are optimized from two dimensions to solve the problem of insufficient adaptability of fixed weight fusion.

[0057] (6) Multi-scale output and progressive restoration design: The decoder adopts a progressive upsampling strategy to gradually restore the fused features from the downsampling scale to the original size, outputting a clear image at multiple scales. The original size result is used as the final dehazing output, and the small-scale result assists in training, realizing the restoration of detail consistency under multi-scale supervision.

[0058] Ultimately, through the collaborative work of the above six modules, the network framework achieves end-to-end optimization of efficient global modeling, accurate local perception, dynamic feature fusion, and progressive detail restoration. The final output is a dehazing result that combines pixel accuracy, visual quality, and real-time performance, and can be widely used in scenarios such as autonomous driving and security monitoring that require high-precision real-time dehazing.

[0059] In the dual-branch feature extraction stage, the Mamba module and the multi-scale dilated convolution module respectively handle the extraction of global and local features. As the core of global modeling, the Mamba module's technological innovation lies in introducing a state-space model into the dehazing task, breaking through the secondary computational complexity bottleneck of the traditional Transformer self-attention mechanism. Specifically, Mamba flattens the two-dimensional feature map into a one-dimensional sequence with length L = H × W, where H is the image height and W is the image width. Then, it uses a selective state-space mechanism to linearly process the sequence, reducing the computational complexity from O(L²) of the Transformer to O(L). This improvement can increase computational efficiency by more than an order of magnitude when processing high-resolution images, perfectly resolving the contradiction between global modeling and real-time performance. Simultaneously, the Mamba module preprocesses the input features through convolutional residual blocks. These residual blocks contain two layers of 3×3 convolutions and ReLU activation functions. Residual connections ensure the complete transmission of feature information, providing fundamental features with both local structure and discriminative power for subsequent global modeling.

[0060] The multi-scale dilated convolution module focuses on multi-scale perception of local details. Its design includes three parallel convolutional branches, each achieving full coverage of local features at near, medium, and far scales through differentiated combinations of dilation rates and kernel sizes. Specifically, the near-range branch uses a 3×3 convolution with a dilation rate of 1 and a receptive field of 3×3, focusing on capturing fine details such as leaf textures and road markings; the medium-range branch uses a 3×3 convolution with a dilation rate of 3, expanding the receptive field to 7×7, adapting to feature extraction from medium-scale fog patches such as localized dense fog around buildings; and the far-range branch uses a 5×5 convolution with a dilation rate of 2, achieving a receptive field of 9×9, effectively covering the feature distribution of large, non-uniform fog areas. The advantage of this multi-scale design is that it simulates the human visual system's perception mechanism of objects at different scales, ensuring the network's comprehensive adaptation to non-uniform fog scenes and overcoming the inherent limitations of traditional CNNs with their fixed receptive fields.

[0061] The context enhancement unit is a crucial link connecting dual-branch feature extraction and dynamic fusion. Its core function is to enhance the discriminative power of local features and suppress dilated convolution artifacts. This unit achieves its function through the synergy of channel attention and depthwise separable convolution: The channel attention module first performs channel-dimensional average pooling on the output features of multi-scale dilated convolutions to generate global statistical information for each channel. Then, through two layers of 1×1 convolutions with a sigmoid activation function inserted in between, the statistical information is mapped to channel weights. The magnitude of the weight directly reflects the importance of the corresponding channel to the dehazing task. For example, channels with fog density differences and texture detail channels are given higher weights, while noise channels are suppressed. The depthwise separable convolution module uses a combination of 3×3 depthwise convolutions and 1×1 pointwise convolutions. While significantly reducing the computational cost, it enhances the local spatial correlation of each channel through depthwise convolution, effectively filling the feature space gaps caused by the dilation rate, thereby eliminating the mesh artifacts easily generated by dilated convolutions and significantly improving the spatial continuity of local features.

[0062] The dynamic fusion process consists of a gating weight unit, spatial attention, and pixel attention, enabling adaptive coordination between global and local features. The gating weight unit, acting as an intelligent regulator for the dual-branch fusion, first converts the global features output from the Mamba module and the context-enhanced local features into a one-dimensional sequence. This sequence is then concatenated through channels and input into a multilayer perceptron to generate a gating score. This gating score ranges from 0 to 1, and its physical meaning represents the weight ratio of global features in the fusion result. In dense fog areas, the gating score automatically increases, making the fusion result more dependent on the prior global fog distribution; in light fog or clear areas, the gating score automatically decreases, allowing local detail features to dominate the fusion. This adaptive weight allocation mechanism completely solves the problem of poor fog adaptability in traditional fixed-weight fusion.

[0063] Spatial attention and pixel attention refine the fused features from different dimensions. The spatial attention module generates a spatial weight map through a combination of 3×3 depthwise separable convolutions and 1×1 convolutions. This weight map adaptively highlights features in locally sharp areas of the image while suppressing redundant information in foggy regions. The pixel attention module focuses on pixel-level gradient information, calculating the gradient magnitudes of image edges and textures to enhance the gradient continuity in detailed areas, preventing blurred or broken edges after dehazing. The synergistic effect of these two modules achieves a precise balance between global dehazing and local detail preservation in the fused features.

[0064] In the final multi-scale output stage, the decoder employs a progressive upsampling strategy, gradually restoring the fused features from the downsampling scale to the original size. During this process, feature information from the corresponding scale is incorporated at each upsampling stage, ensuring the consistency of detail in the dehazing result across different scales. In the output multi-scale clear image, the original-size result serves as the final dehazed output, while the smaller-scale result is used to aid training, improving the model's generalization ability to different foggy scenarios through multi-scale supervision signals.

[0065] At the detailed design level, the global modeling process of the Mamba module can be further broken down into four steps: feature preprocessing, sequence transformation, state space computation, and feature reconstruction. In the feature preprocessing stage, the number of convolutional kernels in the convolutional residual blocks is consistent with the number of input feature channels, ensuring that the feature dimension remains unchanged. The residual connections achieve an identity mapping by directly superimposing input and output features, effectively mitigating the gradient vanishing problem in deep networks. The sequence transformation stage strictly follows the order of flattening and transposing, converting the two-dimensional feature map into a one-dimensional sequence. This conversion provides a suitable input format for Mamba's linear processing.

[0066] State-space computation is the core of the Mamba module. It processes the sequence element-wise using a linear transformation matrix and a gating vector. The linear transformation matrix captures long-range dependencies between sequence elements, while the gating vector dynamically adjusts the contribution ratio between historical states and the current input. This mechanism enables Mamba to achieve global modeling accuracy comparable to the Transformer in O(L) time complexity. The feature reconstruction stage uses a transpose operation to restore the one-dimensional sequence to a two-dimensional feature map, and then performs spatial smoothing on the feature map using a 3×3 convolution layer. This eliminates spatial structure shifts that may occur during sequence processing, ensuring precise alignment between global and local features in the spatial dimension, laying the foundation for subsequent fusion stages.

[0067] Detailed design analysis of multi-scale dilated convolution and context enhancement unit. Each branch of the multi-scale dilated convolution branch is followed by a batch normalization (BN) layer and a ReLU activation function after the convolution operation. The BN layer normalizes features, reducing the difference in feature distribution between different branches and providing a unified numerical basis for subsequent fusion. The ReLU activation function introduces non-linear expressive power, enhancing the model's ability to capture complex local features. The channel attention module of the context enhancement unit uses a compression-expansion structure for its two 1×1 convolution layers, which reduces computational cost while enhancing information interaction between channels, making weight allocation more accurate. In the depthwise separable convolution module, the number of convolution kernels is the same as the number of input channels, with each kernel responsible for only one channel's convolution operation. This design significantly reduces the number of parameters and strengthens local spatial relationships through channel-wise convolution, effectively suppressing dilated convolution artifacts.

[0068] The implementation details of the gating weights and attention mechanism demonstrate the technical depth of this network in dynamic fusion. The MLP structure of the gating weight unit consists of two linear layers and a GELU activation function. The first linear layer expands the input dimension from 2C to 4C, improving the model's ability to express complex relationships by increasing the dimension. The GELU activation function introduces a smooth nonlinear transformation in the form of a Gaussian error linear unit, making the generation of gating scores more stable. The second linear layer compresses the dimension from 4C back to C and restricts the output value to between 0 and 1 through the Sigmoid activation function, ensuring that the physical meaning of the gating scores is clear.

[0069] The spatial attention module employs a 3×3 depthwise separable convolution, with a stride of 1 in the depthwise convolution part to maintain the spatial resolution of the feature map. The pointwise convolution part integrates feature information from different channels through 1×1 convolutions, enabling information exchange between channels. The pixel attention module calculates the gradient magnitude of the image to generate a gradient weight map. This weight map is multiplied pointwise with the fused features and then residually connected to the original fused features, thereby enhancing the gradient information in detailed regions.

[0070] Based on the aforementioned network architecture and module design, to fully verify the effectiveness and advancement of the proposed end-to-end dehazing network integrating Mamba and multi-scale context enhancement in practical applications, this paper conducts experimental evaluations of the designed model and performs multiple comparative experiments with current mainstream image dehazing methods. The training, testing, and evaluation process of the proposed dehazing network is attached. Figure 2 As shown. Specifically, 1000 haze images were collected in real natural foggy scenes as the data basis, and divided into training set, validation set and test set in a ratio of 8:1:1, which were used for model training, parameter tuning and final performance evaluation, respectively. During the training phase, 80% of the images were used to train the network end-to-end; during the validation phase, 10% of the images were selected for intermediate performance monitoring and model selection; during the testing phase, the remaining 10% of the images were used to independently evaluate the generalization ability and actual defogging effect of the model. To ensure the fairness and authority of the experimental comparison, three representative defogging methods were selected as comparison objects: (1) traditional atmospheric physics model methods based on image priors, dark channel prior method DCP; (2) typical supervised learning method GFN; (3) attention-based defogging method GFN. Qualitative comparison indicators include, but are not limited to, image sharpness, detail preservation ability, color restoration naturalness and overall visual quality. See attached Figure 3As shown, the proposed dehazing network can effectively remove large areas of non-uniform dense fog while preserving fine details such as building edges, without obvious color blocks or blurring. The experimental results fully demonstrate that the proposed Mamba architecture, which features efficient global modeling, multi-scale local perception, and dynamic feature fusion, is highly effective and advanced in image dehazing tasks. It not only achieves comprehensive superiority in pixel accuracy, visual quality, and real-time performance, but also provides a practical and feasible technical path for image enhancement and dehazing applications in real-world scenarios.

[0071] To comprehensively verify the effectiveness, stability, and technological innovation of the end-to-end dehazing network integrating Mamba and multi-scale context enhancement proposed in this invention in practical application scenarios, this study conducted a series of experimental evaluations based on the design of its overall network architecture and core functional modules, and carried out multi-dimensional comparisons with current mainstream image dehazing methods, aiming to objectively evaluate the performance of the network model of this invention from a qualitative perspective.

[0072] As attached Figure 2 As shown, the dehazing network training-validation-testing process adopted in this invention includes five stages: data acquisition, dataset partitioning, model training, validation and parameter tuning, and final testing. Specifically, experimental data is collected from real-world foggy scenes, accumulating 1000 foggy image samples covering various weather conditions, shooting angles, and land cover types to ensure scene diversity and fog coverage. The dataset is divided into training, validation, and testing sets in an 8:1:1 ratio, used for end-to-end training of the network model, hyperparameter tuning, and independent validation of final performance, respectively. During the training phase, each sub-module is collaboratively optimized to achieve accurate modeling of global fog distribution and local detailed features; during the validation phase, the loss curve is tracked in real-time and intermediate feature visualization is used to select the model parameters with optimal generalization ability; during the testing phase, performance evaluation is conducted on a sample set not used in training to ensure the objectivity and reliability of the results.

[0073] To ensure the fairness and authority of the comparative experiment, three representative image dehazing methods were selected as references: (1) Dark Channel Prior (DCP) image prior algorithm based on atmospheric physical model, which can represent the traditional prior-based dehazing method; (2) GFN, a typical supervised learning method, which reflects the typical performance of deep supervised learning in dehazing tasks; and (3) GFN, an attention-based dehazing method, which is used to compare the efficiency and effect differences of attention-based methods.

[0074] In terms of performance evaluation metrics, a comprehensive comparison was conducted using multiple qualitative evaluation standards, including image sharpness, detail and texture retention, color fidelity, and edge structure integrity. (See attached image.) Figure 3As shown, the proposed end-to-end dehazing network that integrates Mamba and multi-scale context enhancement not only achieves high-quality image restoration, but also performs well in multiple visual dimensions. It significantly outperforms the comparison methods in terms of edge structure integrity and color reproduction authenticity, fully demonstrating the effectiveness and adaptability of the proposed method in complex haze scenes.

[0075] Experimental results fully verify the technical advantages of this invention's Mamba architecture, which features efficient global modeling, multi-scale local perception, and dynamic feature fusion. It achieves a dual improvement in image clarity and visual quality with relatively low computational complexity, and its inference efficiency meets the requirements of real-time scenarios. Compared to existing methods, this network demonstrates superior robustness and detail recovery capabilities under complex fog conditions, providing a technologically advanced and engineering-practical solution for real-time defogging applications in fields such as autonomous driving and security monitoring.

[0076] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An end-to-end defogging network integrating Mamba and multi-scale context enhancement, characterized in that, Includes the following steps: Step 1: Multi-scale encoding-decoding overall framework design. Two downsampled scale inputs are generated for the hazy input image. Multi-scale features are extracted by the hierarchical encoder and fused with contextual information. Then, the decoder progressively recovers the high-resolution features and outputs the corresponding scale dehazing results. Step 2: Design of Mamba global modeling branch. The input features are preprocessed through convolutional residual blocks, converted into 1D sequences, and then input into Mamba to capture the long-range dependence of global fog. Finally, they are reconstructed into 2D global features. Step 3: Design of multi-scale parallel dilated convolution branches, setting up 3 parallel dilated convolution branches with differentiated parameters to cover local feature extraction at near, medium and far scales; Step 4: Design of local context enhancement units, combining channel attention and depthwise separable convolution to enhance the discriminative power of local features and suppress dilated convolution artifacts; Step 5: Cross-modal gating interaction design. By dynamically allocating the fusion ratio of global and local features through gating scores, dual-branch collaboration is achieved. Global and local features are converted into 1D sequences and spliced ​​together. A gating score in the range of 0-1 is generated through a multilayer perceptron. The two sequences are weighted and fused according to the scores and reconstructed into 2D features, which dynamically adapts to the global-local feature requirements of different regions. Step 6: Channel-space collaborative attention design, optimizing fusion features from the channel and space dimensions to enhance the expression of key information.

2. The end-to-end defogging network fused Mamba with multiscale context enhancement of claim 1, wherein: The multi-scale input-cross-scale fusion-multi-scale output process generates downsampling scales of 1 / 2 and 1 / 4 of the original image size, respectively. Features are extracted by encoder downsampling and details are restored by decoder upsampling. Combined with the dynamic fusion mechanism in steps 5-6, global and local features are integrated to ensure the dehazing effect of high-resolution images.

3. The end-to-end defogging network fused Mamba with multiscale context enhancement of claim 1, wherein: Local features are enhanced by convolutional residual blocks, which contain two layers of 3×3 convolutions and nonlinear activation functions, while residual connections are introduced to avoid information loss. After the 2D features are converted into 1D sequences and input into Mamba, global dependencies are captured through linear state updates and gating mechanisms, with computational complexity approaching linear levels. Finally, 2D global features are reconstructed through 3×3 convolutions, balancing global modeling accuracy and computational efficiency.

4. The end-to-end defogging network integrating Mamba and multi-scale context enhancement as described in claim 1, characterized in that: It contains three independent branches: the near-range branch uses a 3×3 convolution with a dilation rate of 1 to capture fine textures, the mid-range branch uses a 3×3 convolution with a dilation rate of 3 to capture medium-sized fog, and the far-range branch uses a 5×5 convolution with a dilation rate of 2 to capture large-scale non-uniform fog. Each branch is connected to a batch normalization layer and a non-linear activation function to stabilize the feature distribution and enhance the expressive power.

5. The end-to-end defogging network fused Mamba with multiscale context enhancement of claim 1, wherein: Channel weights are generated through channel attention to enhance key feature channels; 3×3 depthwise separable convolution is used to suppress grid artifacts and enhance spatial correlation; finally, the original branch features are superimposed through residual connections to avoid loss of details and improve the discriminative power of local features.

6. The end-to-end defogging network fusing Mamba with multi-scale context enhancement of claim 1, wherein: Channel weights are generated by adaptive average pooling and 1×1 convolution, and spatial weights are generated by depthwise separable convolution and 1×1 convolution. The two types of weights are weighted with the fusion features respectively, and then the original fusion features are superimposed to achieve dual enhancement of key information in both channel and spatial dimensions.

Images