Non-uniform fog image defogging method and system based on multi-scale feature constraint

Abstract

The invention provides a non-uniform fog image defogging method and system based on multi-scale feature constraint, relates to the field of image processing and computer vision, and aims to solve the problems of unbalanced defogging, detail loss and the like caused by non-uniform and complex haze distribution in a non-uniform fog image. According to the method, a feature extraction and reconstruction module and a feature enhancement module are constructed, the feature extraction and reconstruction module is combined with a Laplacian pyramid up-down sampling and self-attention mechanism, and multi-scale features of an image are utilized to realize high-precision defogging; and a state space model is introduced through a feature enhancement module to extract and fuse shallow features and deep semantic features of the image so as to enhance and supplement detail and structure information of the defogged image. According to the method, the problems of unbalanced defogging, detail loss and the like are solved, the problem that the model only depends on single spatial domain characteristics or single physical priori knowledge is solved, and the defogging precision and generalization ability of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the fields of image processing and computer vision, and in particular relates to a method and system for defogging a non-uniform fog image based on multi-scale feature constraints. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In computer vision applications such as autonomous driving, remote sensing monitoring, and image recognition, haze can significantly degrade image visual quality and impact system performance. However, deep learning-based end-to-end image dehazing methods have achieved remarkable results, particularly in processing uniformly fogged images.

[0004] However, images captured in complex, real-world scenes, particularly those with multiple fog layers, varying depths, or varying concentrations, often exhibit non-uniform fog. Existing methods for dehazing uniform fog images ignore the variations in fog concentration and regional complexity within the image. Consequently, when processing non-uniform fog images with complex distribution and uneven fog concentration, they are prone to problems such as uneven dehazing, insufficient detail recovery, and color distortion, limiting their effectiveness in real-world scenarios.

[0005] In addition, due to the complex and uneven distribution of fog in non-uniform fog images, and the scarcity of non-uniform fog image data in real scenes, the existing non-uniform fog image dehazing models are still not ideal in dehazing accuracy and generalization ability. There are still obvious deficiencies in restoring image structure and details and reconstructing true colors. Moreover, the models only rely on single spatial domain features or single physical prior knowledge, and cannot meet the dehazing requirements of complex non-uniform fog images. Summary of the Invention

[0006] To overcome the shortcomings of the aforementioned prior art, the present invention provides a method and system for defogging non-uniform foggy images based on multi-scale feature constraints. By combining a Laplacian pyramid with a self-attention mechanism through a feature extraction and reconstruction module, the system effectively extracts and fuses multi-scale features of the image. This system, constrained by a dual prior loss, adapts to the defogging needs of complex, non-uniform foggy images. Furthermore, a feature enhancement module introduces a state-space model to enhance and supplement image detail and structural information, addressing issues such as uneven defogging and detail loss, while improving the model's defogging accuracy and generalization capabilities.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of the present invention discloses a method for defogging a non-uniform fog image based on multi-scale feature constraints, comprising: The foggy image is input into the feature extraction and reconstruction module, and features are extracted layer by layer through two layers of Laplacian pyramid downsampling layers, respectively obtaining the detail features and global features of the corresponding layers; Based on the detailed features and global features of the corresponding layer, layer-by-layer fusion reconstruction is performed through two layers of Laplacian pyramid upsampling layers in sequence; Based on the dual prior loss, the features after fusion and reconstruction of each layer are constrained to obtain the optimized dehazed image features; The foggy image is input into the feature enhancement module, mapped to a high-dimensional feature space through the block embedding module, and multi-scale features are extracted layer by layer through multiple state space models and block fusion modules; Align multi-scale features through downsampling layers, and perform dense multi-scale feature aggregation through upsampling layers; The aggregated multi-scale features are fused with deep features through the state space model and reconstructed through the upsampling layer to obtain enhanced dehazing features. The optimized dehazed image features and the enhanced dehazed features are weightedly fused to obtain the final dehazed image.

[0008] As a further technical solution, we sequentially extract features layer by layer through two layers of Laplacian pyramid downsampling layers to obtain the detail features and global features of the corresponding layers. The specific process is as follows: Input the foggy image into the convolution layer for feature extraction to obtain the initial image features; Through the first Laplacian pyramid downsampling layer, the initial image features are separated into the first layer of detail features and the first layer of global features; Through the second Laplacian pyramid downsampling layer, the first global feature is separated into the second layer of detail features and the second layer of global features.

[0009] As a further technical solution, the initial image features are separated into the first layer of detail features and the first layer of global features through the first Laplacian pyramid downsampling layer, wherein each Laplacian pyramid downsampling layer includes a first Gaussian filter module, a downsampling layer, an upsampling layer and a second Gaussian filter module. The specific process is as follows: Input the initial features into the first Gaussian filter module for Gaussian blur to generate blurred image features; Down-sampling the blurred image features through a down-sampling layer to obtain a first global feature; The first global feature is passed through the upsampling layer and Gaussian filter module in sequence, and after the resolution is restored, it is subtracted from the initial feature to obtain the first detail feature.

[0010] As a further technical solution, based on the detail features and global features of the corresponding layer, two layers of Laplacian pyramid upsampling layers are sequentially used for layer-by-layer fusion reconstruction. The specific process is as follows: The second-layer global features are reconstructed through the first attention module and weighted fused with the second-layer global features to obtain enhanced second-layer global features; Input the second layer detail features and the enhanced second global features into the first Laplacian pyramid upsampling layer for fusion reconstruction to obtain the first reconstructed features; Reconstructing the first reconstructed features through the second attention module to obtain enhanced first reconstructed features; The first-layer global features and the enhanced first reconstruction features are weightedly fused, and the fused features and the first-layer detail features are input into the second Laplacian pyramid upsampling layer for fusion reconstruction to obtain the second reconstruction features.

[0011] As a further technical solution, the dual prior loss includes multi-scale dark channel prior loss and multi-scale color attenuation prior loss. The specific formula is: ; ; in, is the multi-scale dark channel prior loss; is the multi-scale color attenuation prior loss; S is the total number of upsampling layers, that is, the total number of scales; For the t Feature maps at different scales, Indicates the number of channels, is the size of the feature map at this scale; It is a feature map Dark channel value of It is a feature map The weight of the dark channel prior loss; It is a feature map The color attenuation value of It is a feature map The weight of the color attenuation prior loss.

[0012] As a further technical solution, multi-scale features are extracted layer by layer through multiple state space models and block fusion modules. The specific process is as follows: Perform feature extraction through the first state space model to obtain first state space multi-scale features; Performing feature fusion on the multi-scale features of the first state space through a first fusion module to obtain a first fused feature; Perform feature extraction on the first fusion feature through the second state space model to obtain the second state space multi-scale feature; The second state space multi-scale features are fused through a second fusion module to obtain a second fused feature; Perform feature extraction on the second fusion feature through the third state space model to obtain the third state space multi-scale feature; The third state space multi-scale features are fused through the third fusion module to obtain the third fused features.

[0013] As a further technical solution, feature extraction is performed through the first state space model to obtain the first state space multi-scale features, wherein each state space includes a first layer of normalization and two parallel branches, and the two parallel branches include a first linear layer, a first activation function, a second linear layer, a deep convolutional layer, a second activation function, a two-dimensional selective scanning module, a second layer of normalization and a linear layer. The specific process is as follows: Normalization is performed through the first layer normalization to obtain normalized features; The normalized features are input into the first linear layer and the first activation function for feature transformation to obtain the first branch changed features; Perform feature transformation on the normalized features through the second linear layer to obtain the second initial transformation features; Perform feature extraction and nonlinear mapping on the second initial transformation features through a deep convolution layer and a second activation function to obtain a two-dimensional feature map; The two-dimensional feature map is expanded into a one-dimensional sequence along four directions for selective scanning through a two-dimensional selective scanning module; The scanning results are merged into a new two-dimensional feature map, and the transformed features of the second branch are obtained through layer normalization. Multiply the transformed features of the first branch and the transformed features of the second branch element by element, and adjust the channel dimension through the third linear layer to obtain the adjusted features; The adjusted features and input features are weightedly fused through residual connections to obtain the first state space multi-scale features.

[0014] As a further technical solution, we align multi-scale features through the downsampling layer and perform dense multi-scale feature aggregation through the upsampling layer. The specific process is as follows: Aligning the first state space multi-scale features through a first downsampling layer to obtain aligned first multi-scale features; Aligning the second state space multi-scale features through a second downsampling layer to obtain aligned second multi-scale features; Performing weighted fusion on the third state space multi-scale feature, the aligned first multi-scale feature, and the aligned second multi-scale feature to obtain a first weighted fusion feature; Performing feature aggregation on the first weighted fusion feature through a first upsampling layer to obtain a first densely aggregated feature; Performing weighted fusion on the first densely aggregated feature, the aligned first multi-scale feature, and the second state space multi-scale feature to obtain a second weighted fusion feature; The second weighted fusion features are aggregated through a second upsampling to obtain a second densely aggregated feature.

[0015] The second densely aggregated features and the first state space multi-scale features are weightedly fused to obtain a third densely aggregated feature.

[0016] As a further technical solution, the aggregated multi-scale features are subjected to deep feature fusion through the state space model and reconstructed through the upsampling layer to obtain enhanced dehazing features. The specific process is as follows: Inputting the first weighted fusion feature into the fourth state space model for deep feature fusion to obtain a first deep feature; Inputting the second weighted fusion feature into the fifth state space model for deep feature fusion to obtain a second deep feature; Inputting the third weighted fusion feature into the sixth state space model for deep feature fusion to obtain a third deep feature; Reconstruct the first depth feature through a third upsampling layer to obtain a first reconstructed feature; The first reconstructed feature and the second depth feature are weightedly fused, and the depth feature is reconstructed through the fourth upsampling layer to obtain the second reconstructed feature; The second reconstruction feature and the third depth feature are weightedly fused, and feature extraction is performed through a convolutional layer to obtain enhanced dehazing features.

[0017] The second aspect discloses a non-uniform fog image defogging system based on multi-scale feature constraints, comprising: The feature extraction and reconstruction module is used to input the foggy image into the feature extraction and reconstruction module, and perform layer-by-layer feature extraction through two layers of Laplacian pyramid downsampling layers in sequence to obtain the detailed features and global features of the corresponding layers respectively; Based on the detailed features and global features of the corresponding layer, layer-by-layer fusion reconstruction is performed through two layers of Laplacian pyramid upsampling layers in sequence; Based on the dual prior loss, the features after fusion and reconstruction of each layer are constrained to obtain the optimized dehazed image features; The feature enhancement module is used to input foggy images into the feature enhancement module, map them to high-dimensional feature space through the block embedding module, and extract multi-scale features layer by layer through multiple state space models and block fusion modules; Align multi-scale features through downsampling layers, and perform dense multi-scale feature aggregation through upsampling layers; The aggregated multi-scale features are fused with deep features through the state space model and reconstructed through the upsampling layer to obtain enhanced dehazing features. The feature fusion module is used to perform weighted fusion of the optimized dehazed image features and the enhanced dehazed features to obtain the final dehazed image.

[0018] One or more of the above technical solutions have the following beneficial effects: In this embodiment, the feature extraction module and reconstruction module combine Laplacian pyramid downsampling / upsampling and self-attention mechanism to effectively extract and fuse multi-scale features to achieve high-precision dehazing. The multi-scale dual prior loss is used for constraint to further improve the dehazing quality and meet the dehazing requirements of complex clearing of non-uniform foggy images. At the same time, the state space model is introduced through the feature enhancement module to enhance and supplement image details and structural information, solving problems such as unbalanced dehazing and detail loss. It overcomes the problem that the model relies only on single spatial domain features or single physical prior knowledge, further improving the dehazing accuracy and generalization ability of the model.

[0019] In this embodiment, a multi-scale dual prior loss is used to constrain multi-scale feature reconstruction, effectively capturing the illumination attenuation effect and image structural information caused by haze, thereby improving the dehazing effect. The edge information of the image generated by the constructed edge consistency loss constraint model can effectively enhance the model's recovery of image structural information, improving the edge and texture clarity of the dehazed image. This achieves a "staged" guided optimization from coarse to fine, allowing for more refined differentiation and meeting the dehazing intensity requirements of different areas.

[0020] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0022] Figure 1 This is a framework flow chart of a method for defogging a non-uniform fog image based on multi-scale feature constraints in the first embodiment of the present invention; Figure 2 This is a flow chart of Laplacian pyramid downsampling and Laplacian pyramid upsampling in the first embodiment; Among them, Figure (a) is the Laplacian pyramid downsampling, and Figure (b) is the Laplacian pyramid upsampling; Figure 3 This is a schematic diagram of the state space model framework of the first embodiment; Figure 4 Schematic diagram comparing the dehazing effects of different models on the NH-HAZE dataset in Example 1; Figure 5 Schematic diagram comparing the dehazing effects of different models on the NH-HAZE2 dataset in Example 1; Figure 6 This is a schematic diagram comparing the dehazing effects of different models in Example 1 on the HD-NH-HAZE dataset. DETAILED DESCRIPTION

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0024] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0025] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0026] Example 1 This embodiment discloses a method for defogging a non-uniform fog image based on multi-scale feature constraints.

[0027] To more clearly illustrate this embodiment, a non-uniform fog image dehazing process based on multi-scale feature constraints can be specifically described as follows: This embodiment provides a method for defogging a non-uniform foggy image based on multi-scale feature constraints, including: S1: Input the foggy image into the feature extraction and reconstruction module, and perform feature extraction layer by layer through two layers of Laplacian pyramid downsampling layers in sequence to obtain the detail features and global features of the corresponding layers respectively; Based on the detailed features and global features of the corresponding layer, layer-by-layer fusion reconstruction is performed through two layers of Laplacian pyramid upsampling layers in sequence; Based on the dual prior loss, the features after fusion and reconstruction of each layer are constrained to obtain the optimized dehazed image features; S2: The foggy image is input into the feature enhancement module, mapped to a high-dimensional feature space through the block embedding module, and multi-scale features are extracted layer by layer through multiple state space models and block fusion modules; Align multi-scale features through downsampling layers, and perform dense multi-scale feature aggregation through upsampling layers; The aggregated multi-scale features are fused with deep features through the state space model and reconstructed through the upsampling layer to obtain enhanced dehazing features. S3. Perform weighted fusion on the optimized defogging image features and the enhanced defogging features to obtain the final defogging image.

[0028] To address the problems of uneven dehazing and detail loss caused by the uneven and complex distribution of haze in non-uniform fog images, this embodiment constructs a non-uniform fog image dehazing method model based on multi-scale feature constraints. The model consists of a feature extraction and reconstruction module and a feature enhancement module. The two modules use different methods to extract and fuse the multi-scale features of the image.

[0029] like Figure 1 、 Figure 2 As shown, in step S1, the foggy image is input into the feature extraction and reconstruction module to obtain optimized defogging image features.

[0030] S1-1. Obtain a foggy image.

[0031] In this embodiment, three non-uniform fog image datasets, NH-HAZE, NH-HAZE2, and HD-NH-HAZE, are used.

[0032] Specifically, (1) the NH-HAZE dataset contains 55 pairs of real non-uniform fog images and their corresponding fog-free images, of which 40 pairs are used for training sets, 5 pairs are used for validation sets, and 5 pairs are used for testing sets. The resolution of each image is 1200×1600.

[0033] (2) The NH-HAZE2 dataset contains 35 pairs of real non-uniform fog images and their corresponding fog-free images, of which 25 pairs are used for training sets, 5 pairs are used for validation sets, and 5 pairs are used for testing sets. The resolution of each image is 1200×1600.

[0034] (3) The HD-NH-HAZE dataset contains 45 pairs of real non-uniform fog images and their corresponding fog-free images, of which 40 pairs are used for training sets, 5 pairs are used for validation sets, and 5 pairs are used for testing sets. The resolution of each image is 6000×4000.

[0035] S1-2, perform feature extraction layer by layer through two layers of Laplacian pyramid downsampling layers in turn, and obtain the detail features and global features of the corresponding layers respectively.

[0036] In this embodiment, the feature extraction and reconstruction module uses an encoder-decoder architecture to learn the feature mapping relationship between foggy and clear images. This module enhances modeling of haze distribution at different scales through multi-scale feature extraction. Combined with feature fusion and reconstruction, it improves the dehazing effect on non-uniformly fogged images, enhancing image detail fidelity and structural integrity.

[0037] The feature extraction module and reconstruction module mainly include convolutional layers, multiple Laplacian pyramid downsampling layers, multiple Laplacian pyramid upsampling layers, and multiple self-attention modules.

[0038] like Figure 1 As shown, in this embodiment, the input foggy image first passes through the encoder's convolutional layer and two Laplacian pyramid downsampling layers to extract multi-scale features layer by layer. Each Laplacian pyramid downsampling layer separates image features into detail features (such as edges and textures) and global features (such as the global haze distribution). The specific process is as follows: (1) Input the foggy image into the convolutional layer for feature extraction to obtain the initial image features.

[0039] The input foggy image first passes through the convolutional layer of the encoder to obtain multi-scale initial features.

[0040] (2) Through the first Laplacian pyramid downsampling layer, the initial image features are separated into the first layer of detail features and the first layer of global features.

[0041] In this embodiment, each downsampling layer of the Laplacian pyramid includes a first Gaussian filter module, a downsampling layer, an upsampling layer, and a second Gaussian filter module. The specific process is as follows: 1) Input the initial features into the first Gaussian filter module for Gaussian blur to generate blurred image features.

[0042] 2) Downsample the blurred image features through the downsampling layer to obtain the first global features.

[0043] Input features After Gaussian downsampling (Gaussian filtering + downsampling), a low-resolution global information feature map is obtained , which retains global information and removes detail information.

[0044] 3) The first global feature is passed through the upsampling layer and the Gaussian filter module in sequence, and after the resolution is restored, it is subtracted from the initial feature to obtain the first detail feature.

[0045] Then, Through Gaussian upsampling (upsampling + Gaussian filtering), a global information feature map with the same resolution as the original input feature is obtained , subtract the feature map from the original input feature , get the detail information feature map with the same resolution as the input feature .

[0046] The Laplacian pyramid realizes the decoupling of global features and detail features, and converts the original resolution detail feature map into With low-resolution global feature map It is transmitted as output to subsequent processing.

[0047] (3) Through the second Laplacian pyramid downsampling layer, the first global feature is separated into the second layer of detail features and the second layer of global features.

[0048] 1) Input the first global feature into the first Gaussian filter module for Gaussian blur to generate blurred image features.

[0049] 2) Downsample the blurred image features through the downsampling layer to obtain the second global features.

[0050] 3) The second global feature is passed through the upsampling layer and Gaussian filter module in sequence, and after the resolution is restored, it is subtracted from the initial feature to obtain the second detail feature.

[0051] After the above steps, Laplacian pyramid downsampling can effectively capture multi-scale contextual information and enhance the modeling ability of non-uniform fog by reducing the size of feature maps layer by layer and extracting features of different scales.

[0052] S1-3, based on the detail features and global features of the corresponding layer, layer-by-layer fusion reconstruction is performed through two layers of Laplacian pyramid upsampling layers.

[0053] In this embodiment, global features are combined with local features through the decoder's Laplacian pyramid upsampling layer to gradually restore the haze-free image features, thus achieving multi-scale feature reconstruction. The specific process is as follows: (1) The second-layer global features are reconstructed through the first attention module and weighted fused with the second-layer global features to obtain the enhanced second global features.

[0054] After each layer of the decoder, a self-attention mechanism is used to achieve efficient fusion of multi-scale features. The self-attention mechanism can effectively capture the long-range dependencies of features and retain the details of features. (2) The second layer detail features and the enhanced second global features are input into the first Laplacian pyramid upsampling layer for fusion reconstruction to obtain the first reconstructed features.

[0055] The first Laplacian pyramid upsampling layer is fused and reconstructed, specifically, the low-resolution global feature map (the enhanced second global feature) After Gaussian upsampling (upsampling + Gaussian filtering) and high-resolution detail feature map (second layer detail feature) Add together and restore the image features with non-uniform fog features removed.

[0056] (3) The first reconstructed feature is reconstructed through the second attention module to obtain the enhanced first reconstructed feature.

[0057] (4) The first-layer global features and the enhanced first reconstruction features are weightedly fused, and the fused features and the first-layer detail features are input into the second Laplacian pyramid upsampling layer for fusion reconstruction to obtain the second reconstruction features.

[0058] S1-4. Based on the dual prior loss, the features after fusion and reconstruction of each layer are constrained to obtain the optimized dehazed image features.

[0059] In the research and application of image dehazing, fully leveraging the inherent prior information of the image is crucial for restoring a clear, haze-free image. Two priors, the dark channel prior and the color attenuation prior, can effectively reflect the foggy state of the image. Specifically, foggy images generally have higher dark channel and color attenuation values, while clear images generally have lower dark channel and color attenuation values.

[0060] In this embodiment, each layer of features in the feature fusion reconstruction process is constrained by a dual prior loss based on the dark channel prior and the color attenuation prior. This multi-scale dual prior loss function, based on the dark channel prior and the color attenuation prior, is used to constrain the multi-scale features reconstructed during the upsampling process, further improving the model's dehazing quality for non-uniformly fogged images.

[0061] (1) Construct multi-scale dual prior loss.

[0062] This embodiment proposes a multi-scale dual-prior loss function, which is composed of two constraints: a multi-scale dark channel prior loss and a multi-scale color attenuation prior loss.

[0063] 1) Construct multi-scale dark channel prior loss.

[0064] In a natural image without fog, the pixel value of at least one color channel is close to zero in a local area. Furthermore, through analysis of dark channel priors, it is found that the dark channel values ​​of foggy images are generally higher, while those of clear images are generally lower. Therefore, it can be concluded that the higher the dehazing accuracy of the model, the clearer the generated image should be, that is, the smaller the dark channel value of the generated image should be. Therefore, by constraining the dark channel value of the generated image to be as small as possible, the clarity of the generated image can be improved. However, for non-uniform fog images, simply constraining the dark channel value of the final generated image is difficult to adapt to the different concentrations of haze in the foggy image, which may lead to insufficient detail recovery or uneven dehazing. Therefore, in order to more accurately restore non-uniform fog images,

[0065] This embodiment proposes a multi-scale dark channel prior loss function, which constrains each layer of features reconstructed by the feature extraction and reconstruction modules during the upsampling process to make its dark channel value as small as possible. This multi-scale joint constraint can enhance the recovery of details at different scales and improve the network's adaptive defogging ability for non-uniform fog. Moreover, the closer the defogged image is to the final output, the stricter the constraint on its dark channel value should be. Therefore, different weights are assigned to the dark channel prior losses at different scales, guiding the network to repair haze information of different scales layer by layer from coarse to fine, where the closer the feature scale is to the final output, the higher the loss weight.

[0066] Specifically, during the upsampling process, the dark channel value of the feature map reconstructed at each scale is calculated, and a total loss function is constructed to globally optimize the dark channel values ​​of all scales, so that the defogging results at different scales can better conform to the dark channel characteristics of the haze-free image. The multi-scale dark channel prior loss function formula is: (1) in, S is the total number of upsampling layers, i.e. the total number of scales; For the t Feature maps at different scales, Indicates the number of channels, is the size of the feature map at this scale; It is a feature map Dark channel value of It is a feature map The weight of the dark channel prior loss.

[0067] Feature Map Dark channel value The calculation process is to first calculate the minimum eigenvalue of each channel, and then average the minimum eigenvalues ​​of all channels to obtain its dark channel value. The formula is: (2) Among them, point Representation feature map In the channel The point on For channel At the point The eigenvalue at For channel The minimum eigenvalue of .

[0068] After the above steps, by constructing a multi-scale dark channel prior loss constraint, the detail recovery at different scales can be enhanced, and the network's adaptive dehazing capability for non-uniform fog can be improved.

[0069] 2) Construct multi-scale dark channel prior loss.

[0070] In foggy images, the difference between brightness and saturation (called color attenuation) is positively correlated with fog concentration. Specifically, as fog concentration increases, the color attenuation also increases. This color attenuation phenomenon is more pronounced in images produced under severe haze conditions. Therefore, by constraining the color attenuation of the generated image to be as small as possible, we can improve the clarity of the generated image, better restore true colors, and reduce color distortion.

[0071] For non-uniform haze images, due to the uneven distribution of haze, the spatial variation of color attenuation is large. Simply constraining the color attenuation value of the resulting image is difficult to effectively address the characteristics of different haze areas, and may lead to insufficient restoration of local areas or uneven dehazing. Therefore, this embodiment constructs a multi-scale color attenuation prior loss function.

[0072] By applying color attenuation prior constraints to all feature layers of different scales reconstructed during the upsampling process of the feature extraction and reconstruction modules, and assigning different weights to the color attenuation prior losses at different scales, the closer the feature scale is to the final output, the greater the loss weight is. This guides the model to gradually correct the color degradation phenomenon from coarse to fine, enhances the adaptability to non-uniform fog images, and thus improves the dehazing performance. The multi-scale color attenuation prior loss function is formulated as follows: (3) in, S is the total number of upsampling layers, i.e. the total number of scales; For the t Feature maps at different scales, Indicates the number of channels, is the size of the feature map at this scale; It is a feature map The color attenuation value of It is a feature map The weight of the color attenuation prior loss.

[0073] Feature Map Color attenuation value The calculation formula is: (4) Among them, point Representation feature map In the channel The point on For channel At the point The eigenvalue at Indicates all channels At the point The maximum eigenvalue of Indicates all channels At the point The minimum eigenvalue of C' Represents the difference between the maximum eigenvalue and the minimum eigenvalue of all channels c at point (i, j).

[0074] After the above steps, by constructing a multi-scale dark channel prior loss constraint, the detail recovery at different scales can be enhanced, and the network's adaptive dehazing capability for non-uniform fog can be improved.

[0075] By constraining the color attenuation value of the generated image to be as small as possible, the clarity of the generated image can be improved, and the generated image can better restore the true color and reduce color loss. Combining the multi-scale dark channel prior loss and using these two prior information to constrain the feature channels of the generated image can effectively capture the light attenuation effect and image structure information caused by haze, thereby improving the dehazing effect.

[0076] S2. Input the foggy image into the feature enhancement module for feature extraction and fusion to obtain enhanced defogging features.

[0077] The feature enhancement module draws on the encoder-decoder architecture of U-Net3+ and adopts a pre-trained state-space model based on a large-scale dataset as the encoder. It leverages its deep feature extraction and layer-by-layer high-resolution feature reconstruction capabilities to enhance dehazing performance.

[0078] The feature enhancement module mainly includes a block embedding module, multiple state space models, multiple block fusion modules, multiple downsampling layers, multiple upsampling layers and convolutional layers.

[0079] S2-1, mapped to high-dimensional feature space through the block embedding module, and extracted multi-scale features layer by layer through multiple state space models and block fusion modules.

[0080] In this example, the input foggy image is embedded into a high-dimensional feature space through a block embedding module.

[0081] Then, through the combination of three state space models and block fusion modules, multi-scale features are extracted layer by layer, providing shallow features and deep semantic features for subsequent dehazing and reconstruction. The specific process is as follows: (1) Feature extraction is performed through the first state space model to obtain the first state space multi-scale features.

[0082] In this embodiment, each state space includes a first layer of normalization and two parallel branches, the two parallel branches are a first gating branch and a second selective scanning branch, and the two parallel branches include a first linear layer, a first activation function, a second linear layer, a depth convolution layer, a second activation function, a two-dimensional selective scanning module, a second layer of normalization and a linear layer.

[0083] The state-space model uses gated linear layers and two-dimensional selective scanning to implement a global attention mechanism with linear complexity. The main process is: 1) Perform normalization through the first layer normalization to obtain normalized features.

[0084] The input features are first normalized by LN (Layer Normalization) to obtain normalized features.

[0085] 2) The normalized features are input into the first linear layer and the first activation function for feature transformation to obtain the changed features of the first branch.

[0086] The gating mechanism is used to divide the processing into two parallel branches. In the first gating branch, feature transformation is performed through a linear layer (Linear) and an activation function (SiLU).

[0087] 3) The normalized features are transformed through the second linear layer to obtain the second initial transformed features.

[0088] 4) The second initial transformation features are extracted and nonlinearly mapped through the deep convolution layer and the second activation function to obtain a two-dimensional feature map.

[0089] The second selective scanning branch first passes through the linear layer, depthwise separable convolution (Depthwise Conv) and activation function (SiLU) in sequence, and obtains a two-dimensional feature map through steps 3) and 4).

[0090] 5) The two-dimensional selective scanning module is used to expand the two-dimensional feature map into a one-dimensional sequence along four directions for selective scanning.

[0091] Then, the second initial transformation feature is input into the core two-dimensional selective scanning module, and cross scanning is used to expand the 2D feature map into a 1D sequence along four directions (upper left → lower right, lower right → upper left, upper right → lower left, lower left → upper right). After scanning in each direction, sequence modeling is performed through the state space model to dynamically capture long-range dependencies.

[0092] 6) The scanning results are merged into a new two-dimensional feature map and processed by layer normalization to obtain the transformed features of the second branch.

[0093] The scan results are merged into a new 2D feature map and layer normalized, thus maintaining linear computational complexity while preserving the global receptive field.

[0094] 7) Multiply the transformed features of the first branch and the transformed features of the second branch element by element, and adjust the channel dimension through the third linear layer to obtain the adjusted features.

[0095] The output results of the two branches are multiplied element by element and then pass through a linear layer to adjust the channel dimension.

[0096] 8) The adjusted features and input features are weightedly fused through residual connections to obtain the first state space multi-scale features.

[0097] Finally, the final result is connected with the input feature residual to obtain the multi-scale feature of the first state space.

[0098] (2) The multi-scale features of the first state space are fused through the first fusion module to obtain the first fused features.

[0099] (3) The first fusion feature is extracted through the second state space model to obtain the second state space multi-scale feature.

[0100] In the second state space model, the specific implementation process is implemented according to step (1) to obtain the second state space multi-scale features.

[0101] (4) The second state space multi-scale features are fused through the second fusion module to obtain the second fused features.

[0102] (5) The second fusion feature is extracted through the third state space model to obtain the third state space multi-scale feature.

[0103] In the third state space model, the specific implementation process is implemented according to step (1) to obtain the multi-scale features of the third state space.

[0104] (6) The third state space multi-scale features are fused through the third fusion module to obtain the third fusion feature.

[0105] Through the above steps, shallow features (such as image edge texture and local structures obscured by fog), mid-level features (fog concentration in different local areas), and deep features (global fog distribution patterns across the entire image) of the foggy image are extracted layer by layer. Furthermore, when extracting features at each layer, the multi-directional scanning strategy of the state-space model effectively integrates global and local features, addressing the shortcomings of traditional models in modeling long-range dependencies and computational efficiency.

[0106] S2-2, align multi-scale features through the downsampling layer, and perform dense multi-scale feature aggregation through the upsampling layer.

[0107] In this embodiment, the specific process is: (1) Align the first state space multi-scale features through the first downsampling layer to obtain the aligned first multi-scale features.

[0108] (2) Align the second state space multi-scale features through the second downsampling layer to obtain the aligned second multi-scale features.

[0109] (3) The third state space multi-scale feature, the aligned first multi-scale feature and the aligned second multi-scale feature are weightedly fused to obtain a first weighted fusion feature.

[0110] The first weighted fusion features are aggregated through the first upsampling layer to obtain first densely aggregated features.

[0111] (4) The first densely aggregated features, the aligned first multi-scale features, and the second state space multi-scale features are weightedly fused to obtain a second weighted fusion feature.

[0112] (5) The second weighted fusion features are aggregated through the second upsampling to obtain the second dense aggregation features.

[0113] (6) The second densely aggregated features and the first state space multi-scale features are weightedly fused to obtain the third densely aggregated features.

[0114] After the above steps, dense jump connections across scales can be achieved, which effectively enhances the fusion of features at different scales. This solves the defect that traditional methods easily lose shallow information, achieves feature enhancement, and is more suitable for the situation where the available data set is small in the study of dehazing non-uniform fog images.

[0115] S2-2. The aggregated multi-scale features are subjected to deep feature fusion through the state space model and reconstructed through the upsampling layer to obtain enhanced dehazing features.

[0116] In this embodiment, the aggregated multi-scale features are subjected to deep feature fusion through a state-space model to adaptively assign weights to important regions in the feature map, enhance the modeling capabilities of key detail features, and capture the correlation between non-uniform fog. Finally, the fused multi-scale features are gradually passed through upsampling layers to reconstruct features and are ultimately output through a convolutional layer. The specific process is as follows: (1) Input the first weighted fusion feature into the fourth state space model for deep feature fusion to obtain the first deep feature.

[0117] (2) Input the second weighted fusion feature into the fifth state space model for deep feature fusion to obtain the second deep feature.

[0118] (3) The third weighted fusion feature is input into the sixth state space model for deep feature fusion to obtain the third deep feature.

[0119] (4) The first depth feature is reconstructed through the third upsampling layer to obtain the first reconstructed feature.

[0120] (5) The first reconstructed feature and the second depth feature are weightedly fused, and the depth feature is reconstructed through the fourth upsampling layer to obtain the second reconstructed feature.

[0121] (6) The second reconstruction feature and the third depth feature are weightedly fused and feature extracted through the convolution layer to obtain the enhanced dehazing feature.

[0122] After the above steps, the aggregated multi-scale features are learned more deeply through the state-space model, the features that contribute most to the final dehazing result are extracted, noise or irrelevant information is suppressed, and the weights of important areas in the feature map are adaptively assigned. Through upsampling, finer local details are gradually integrated into the global structure, achieving spatially adaptive feature reconstruction from coarse to fine.

[0123] S3. Perform weighted fusion on the optimized defogging image features and the enhanced defogging features to obtain the final defogging image.

[0124] The feature information obtained by the feature extraction and reconstruction module (i.e., optimized dehazed image features) is weighted and fused with the deep features obtained by the feature enhancement module (i.e., enhanced dehazed features) to achieve efficient modeling of global and local features. The fused features are processed by the convolutional layer to ultimately generate a clear and natural dehazed image.

[0125] S4. Construct edge consistency loss.

[0126] The spatial distribution of haze in non-uniform fog images is significantly non-uniform, with different regions exhibiting varying degrees of blur and detail loss. This distribution makes it difficult for traditional dehazing methods to recover details in heavily fogged areas, especially edge information. Traditional methods often rely on global or local pixel-level optimization, often ignoring the structural features of the image when processing non-uniform haze areas. To overcome this problem, this embodiment constructs an edge consistency loss function (EDL) based on a pre-trained edge detection model for dehazing non-uniform fog images. This is different from traditional dehazing methods based on edge features.

[0127] Specifically, a lightweight, pre-trained edge detection model based on deep learning is used to detect edges in images. Pre-trained on a large-scale edge dataset using a multi-layer convolutional neural network, this model can more accurately capture image details and edge features, demonstrating high robustness and adaptability in areas with dense and unevenly distributed haze. By using the output of this model as the target value for the edge consistency loss function, the dehazing network's sensitivity to image structure is enhanced, effectively preventing the loss of edge information and further improving the dehazing effect on images with non-uniform haze.

[0128] The edge consistency loss function uses the edge information obtained by edge detection of the image based on the pre-trained edge detection model as the target to optimize the training process of the network model. This loss function restores the details and edge information in the image to the greatest extent by minimizing the difference between the edge information of the generated image and the real image. The edge consistency loss function is formulated as: (5) in, Represents the edge information of the dehazed image generated by the model, Represents the edge information of a real haze-free image.

[0129] After the above steps, the constructed edge consistency loss function is used to optimize the network, which can effectively restore the edge details in the image, especially in areas that are more significantly affected by haze.

[0130] S5. Constructing the total loss function In this embodiment, the total loss function is constructed based on multi-scale dual prior loss, edge consistency loss, and commonly used smooth L1 loss, perceptual loss, multi-scale structure loss and adversarial loss. (1) Smooth L1 loss.

[0131] Smooth L1 loss is a loss function that combines the advantages of L1 loss and L2 loss. The formula is: (6) in, Represents the total number of pixels in the image, and represent the real image and the model generated image respectively. The first channel pixel values; (7).

[0132] The first The first channel The difference between pixel values ​​is used to control the smoothing range.

[0133] (2) Perceptual loss Perceptual loss uses the feature extraction capability of the VGG16 pre-trained network to calculate the difference between the generated image and the real image in the high-level feature space, thereby optimizing the image for a more visually perceptual effect. The formula is: (8) in, and represent the model generated image and the real image respectively. Channel features, Indicates the first The feature map of the layer, 、 and Respectively represent The number of channels, width, and height of the layer feature map.

[0134] (3) Multi-scale structure loss Multi-scale Structure Similarity Loss (MS-SSIM) is a metric used for image quality assessment. It introduces a multi-scale analysis mechanism based on the classic Structure Similarity Index (SSIM). By evaluating multi-level similarities between images at multiple resolutions, such as structure, brightness, and contrast, it provides a more comprehensive and robust image quality measurement. Specifically, the image is downsampled multiple times, the SSIM value is calculated at each scale, and the results at each scale are weighted and fused. The formula is: (9) in, and Represent the model generated image and the real image respectively; , , Represents the image downsampling The brightness similarity of the layer, Contrast similarity and structural similarity of layers; is the weight coefficient at each scale, Indicates the total number of scales to which the image is downsampled.

[0135] The optimization goal of the scale structure loss is to maximize the structural similarity between the generated image and the real image at multiple scales. The formula is: (10).

[0136] (4) Fighting Losses To improve the realism and detail restoration of generated images, this paper introduces a generative adversarial network structure during model training and uses a local discriminator based on PatchGAN as the discriminant network. The adversarial loss is used to constrain the generated images to be consistent with the distribution of real images at the perceptual level, thereby further improving the model's generalization ability. The formula is: (11) in, The generator generates the images, Represents the local discriminator's first The authenticity of the image patch is judged. They represent the feature map sizes output by the local discriminator.

[0137] The total loss function is: (12) in, for smoothing L1; is the loss of multi-scale structure loss; for perceived loss; To combat losses; is the multi-scale dark channel prior loss, is the multi-scale color attenuation prior loss, It is a multi-scale dual prior loss; is the edge consistency loss; , , , and They are the weights of multi-scale structure loss, perceptual loss, adversarial loss, multi-scale dual prior loss and edge consistency loss respectively.

[0138] After the above steps, through multi-loss joint optimization, in the task of dehazing non-uniform fog images, each loss function works together to improve the dehazing effect: the smooth L1 loss stabilizes training by suppressing outliers; the multi-scale structure loss maintains the global structural consistency of the image at different resolutions; the perceptual loss enhances the authenticity of texture details based on deep feature matching; the adversarial loss constrains the visual naturalness of the generated image by using the discriminator; the multi-scale dark channel prior loss and the color attenuation prior loss respectively model the degradation laws from the perspectives of fog concentration distribution and color shift, and the two are combined into a multi-scale dual prior loss to strengthen the model's physical prior learning of complex fog conditions; the edge consistency loss preserves the sharpness of object contours through gradient alignment. This effectively solves the problems of local residue, color distortion, and blurred details caused by non-uniform fog, achieving the unity of high-fidelity dehazing and cross-domain generalization capabilities.

[0139] Through steps S1-S3, the model of the non-uniform fog image dehazing method based on multi-scale feature constraints was trained and tested. The experimental environment configuration was: Ubuntu 18.04; NVIDIA A100; CUDA 12.0; PyTorch 1.12.1; Pycharm 2022; Python 3.7.16. The model optimizer was selected as Adam, the initial learning rate was set to 0.0001, the model was trained for 3000 iterations, the batch size was 4, and the weight parameters were adjusted. , , , and Set to 0.5, 0.01, 0.0005, 0.05 and 0.001 respectively.

[0140] In this embodiment, the comparative experimental results are as follows: (1) Visual comparison results In order to demonstrate the superiority of the model in this embodiment in dehazing non-uniform fog images, the model in this embodiment is compared with the DCP, AOD, GCANet, FFA, TDN, Two-branch and ITBdehaze dehazing models on three non-uniform fog image datasets: NH-HAZE, NH-HAZE2 and HD-NH-HAZE. The results are shown in the figure below. Figure 4 、 Figure 5 and Figure 6 shown.

[0141] in, Figure 4-Figure 6 In the figure, the first row is the input foggy images, the second to ninth rows are the dehazed images generated by DCP, AOD, GCANet, FFA, TDN, Two-branch, ITBdehaze and the model in this paper, respectively, and the last row is the real clear image.

[0142] As can be seen from the figure, the DCP model has serious color distortion and a large amount of fog residue after dehazing. The AOD model has a very poor dehazing effect and a large amount of fog residue. The GCANet model has color distortion and blurred texture details after dehazing. The FFA model is insufficient in restoring image texture details and colors. The TDN model has problems with color distortion and blurred texture details. Although the Two-branch model has improved in dehazing effect, it still has problems with fog residue and color difference. The ITBdehaze model performs poorly in color processing and texture restoration. Compared with the above models, the model of this embodiment performs better in image color restoration, texture detail restoration and overall visual consistency, especially in non-uniform fog environments, it can more effectively restore the true structure and color information of the image.

[0143] (2) Quantitative comparison results.

[0144] This example conducted a quantitative analysis experiment comparing various objective metrics of the proposed model with other dehazing models to further validate the effectiveness of the proposed model in dehazing non-uniform fog images. This paper used peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics. Table 1 shows the quantitative evaluation results of the proposed model and other dehazing models on three non-uniform fog image datasets: NH-HAZE, NH-HAZE2, and HD-NH-HAZE. As can be seen, the proposed model outperformed the other methods on all three datasets.

[0145] Table 1 Quantitative evaluation results of different models on NH-HAZE, NH-HAZE2 and HD-NH-HAZE datasets

[0146] In this example, the model consists of a feature extraction and reconstruction module and a feature enhancement module. The feature extraction and reconstruction module combines the Laplacian pyramid with a self-attention mechanism to leverage the multi-scale features of the image for high-precision dehazing. Laplacian pyramid downsampling is used to extract multi-scale feature information, the self-attention mechanism is used for multi-scale feature reconstruction, and Laplacian pyramid upsampling is used for multi-scale feature fusion and reconstruction. Furthermore, a multi-scale dual-prior loss based on a dark channel prior and a color attenuation prior effectively improves the model's dehazing quality. The feature enhancement module draws on the principles of the U-Net3+ model, constructing a U-shaped encoder-decoder architecture for multi-layer feature extraction and fusion. It also introduces a state-space model to extract and fuse shallow and deep semantic features of the image, enhancing and supplementing the details and structure of the dehazed image. An edge consistency loss effectively improves edge and texture clarity in the dehazed image. This effectively addresses the problems of uneven dehazing and loss of detail caused by the uneven and complex distribution of haze in non-uniform haze images.

[0147] Example 2 This embodiment discloses a non-uniform fog image defogging system based on multi-scale feature constraints, including: The feature extraction and reconstruction module is used to input the foggy image into the feature extraction and reconstruction module, and perform layer-by-layer feature extraction through two layers of Laplacian pyramid downsampling layers in sequence to obtain the detailed features and global features of the corresponding layers respectively; Based on the detailed features and global features of the corresponding layer, layer-by-layer fusion reconstruction is performed through two layers of Laplacian pyramid upsampling layers in sequence; Based on the dual prior loss, the features after fusion and reconstruction of each layer are constrained to obtain the optimized dehazed image features; The feature enhancement module is used to input foggy images into the feature enhancement module, map them to high-dimensional feature space through the block embedding module, and extract multi-scale features layer by layer through multiple state space models and block fusion modules; Align multi-scale features through downsampling layers, and perform dense multi-scale feature aggregation through upsampling layers; The aggregated multi-scale features are fused with deep features through the state space model and reconstructed through the upsampling layer to obtain enhanced dehazing features. The feature fusion module is used to perform weighted fusion of the optimized dehazed image features and the enhanced dehazed features to obtain the final dehazed image.

[0148] Based on providing a non-uniform fog image defogging system based on multi-scale feature constraints, the method steps in embodiment 1 are implemented. Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0149] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A method for defogging non-uniform fog images based on multi-scale feature constraints, characterized by: include: The foggy image is input into the feature extraction and reconstruction module, and features are extracted layer by layer through two layers of Laplacian pyramid downsampling layers, respectively obtaining the detail features and global features of the corresponding layers; Based on the detailed features and global features of the corresponding layer, layer-by-layer fusion reconstruction is performed through two layers of Laplacian pyramid upsampling layers in sequence; Based on the dual prior loss, the features after fusion and reconstruction of each layer are constrained to obtain the optimized dehazed image features; The foggy image is input into the feature enhancement module, mapped to a high-dimensional feature space through the block embedding module, and multi-scale features are extracted layer by layer through multiple state space models and block fusion modules; Align multi-scale features through downsampling layers, and perform dense multi-scale feature aggregation through upsampling layers; The aggregated multi-scale features are fused with deep features through the state space model and reconstructed through the upsampling layer to obtain enhanced dehazing features. The optimized dehazed image features and the enhanced dehazed features are weightedly fused to obtain the final dehazed image.

2. The method for defogging a non-uniform fog image based on multi-scale feature constraints according to claim 1, characterized in that: The feature extraction is performed layer by layer through two layers of Laplacian pyramid downsampling layers in sequence to obtain the detail features and global features of the corresponding layers respectively. The specific process is as follows: Input the foggy image into the convolution layer for feature extraction to obtain the initial image features; Through the first Laplacian pyramid downsampling layer, the initial image features are separated into the first layer of detail features and the first layer of global features; Through the second Laplacian pyramid downsampling layer, the first global feature is separated into the second layer of detail features and the second layer of global features.

3. The method for defogging a non-uniform fog image based on multi-scale feature constraints according to claim 2, characterized in that: The initial image features are separated into the first layer of detail features and the first layer of global features through the first Laplacian pyramid downsampling layer. The Laplacian pyramid downsampling layer includes the first Gaussian filter module, the downsampling layer, the upsampling layer, and the second Gaussian filter module. The specific process is as follows: Input the initial features into the first Gaussian filter module for Gaussian blur to generate blurred image features; Down-sampling the blurred image features through a down-sampling layer to obtain a first global feature; The first global feature is passed through the upsampling layer and Gaussian filter module in sequence, and after the resolution is restored, it is subtracted from the initial feature to obtain the first detail feature.

4. The method for defogging a non-uniform fog image based on multi-scale feature constraints according to claim 1, wherein: Based on the detail features and global features of the corresponding layer, layer-by-layer fusion reconstruction is performed sequentially through two layers of Laplacian pyramid upsampling layers. The specific process is as follows; The second-layer global features are reconstructed through the first attention module and weighted fused with the second-layer global features to obtain enhanced second-layer global features; Input the second layer detail features and the enhanced second global features into the first Laplacian pyramid upsampling layer for fusion reconstruction to obtain the first reconstructed features; Reconstructing the first reconstructed features through the second attention module to obtain enhanced first reconstructed features; The first-layer global features and the enhanced first reconstruction features are weightedly fused, and the fused features and the first-layer detail features are input into the second Laplacian pyramid upsampling layer for fusion reconstruction to obtain the second reconstruction features.

5. The method for defogging a non-uniform fog image based on multi-scale feature constraints according to claim 1, wherein: The dual prior loss includes multi-scale dark channel prior loss and multi-scale color attenuation prior loss. The specific formula is: ； ； in, is the multi-scale dark channel prior loss; is the multi-scale color attenuation prior loss; S is the total number of upsampling layers, i.e. the total number of scales; For the t Feature maps at different scales, Indicates the number of channels, is the size of the feature map at this scale; It is a feature map Dark channel value of It is a feature map The weight of the dark channel prior loss; It is a feature map The color attenuation value of It is a feature map The weight of the color attenuation prior loss.

6. The method for defogging a non-uniform fog image based on multi-scale feature constraints according to claim 1, wherein: Multi-scale features are extracted layer by layer through multiple state space models and block fusion modules. The specific process is as follows: Perform feature extraction through the first state space model to obtain first state space multi-scale features; The multi-scale features of the first state space are fused through the first fusion module; Get the first fusion feature; Perform feature extraction on the first fusion feature through the second state space model to obtain the second state space multi-scale feature; The second state space multi-scale features are fused through the second fusion module; Obtain the second fusion feature; Perform feature extraction on the second fusion feature through the third state space model to obtain the third state space multi-scale feature; The third state space multi-scale features are fused through the third fusion module; Get the third fusion feature.

7. The method for defogging a non-uniform fog image based on multi-scale feature constraints according to claim 6, characterized in that: Feature extraction is performed through the first state space model to obtain the first state space multi-scale features, where each state space includes a first layer of normalization and two parallel branches. The two parallel branches include a first linear layer, a first activation function, a second linear layer, a deep convolutional layer, a second activation function, a two-dimensional selective scanning module, a second layer of normalization and a linear layer. The specific process is as follows: Normalization is performed through the first layer normalization to obtain normalized features; The normalized features are input into the first linear layer and the first activation function for feature transformation to obtain the first branch changed features; Perform feature transformation on the normalized features through the second linear layer to obtain the second initial transformation features; Perform feature extraction and nonlinear mapping on the second initial transformation features through a deep convolution layer and a second activation function to obtain a two-dimensional feature map; The two-dimensional feature map is expanded into a one-dimensional sequence along four directions for selective scanning through a two-dimensional selective scanning module; The scanning results are merged into a new two-dimensional feature map, and the transformed features of the second branch are obtained through layer normalization. Multiply the transformed features of the first branch and the transformed features of the second branch element by element, and adjust the channel dimension through the third linear layer to obtain the adjusted features; The adjusted features and input features are weightedly fused through residual connections to obtain the first state space multi-scale features.

8. The method for defogging a non-uniform fog image based on multi-scale feature constraints according to claim 1, wherein: The multi-scale features are aligned through the downsampling layer, and dense multi-scale feature aggregation is performed through the upsampling layer. The specific process is as follows: Aligning the first state space multi-scale features through a first downsampling layer to obtain aligned first multi-scale features; Aligning the second state space multi-scale features through a second downsampling layer to obtain aligned second multi-scale features; Performing weighted fusion on the third state space multi-scale feature, the aligned first multi-scale feature, and the aligned second multi-scale feature to obtain a first weighted fusion feature; Performing feature aggregation on the first weighted fusion feature through a first upsampling layer to obtain a first densely aggregated feature; Performing weighted fusion on the first densely aggregated feature, the aligned first multi-scale feature, and the second state space multi-scale feature to obtain a second weighted fusion feature; Performing feature aggregation on the second weighted fusion features through a second upsampling to obtain a second densely aggregated feature; The second densely aggregated features and the first state space multi-scale features are weightedly fused to obtain a third densely aggregated feature.

9. The method for defogging a non-uniform fog image based on multi-scale feature constraints according to claim 1, wherein: The aggregated multi-scale features are fused with deep features through the state space model and reconstructed through the upsampling layer to obtain the enhanced dehazing features. The specific process is as follows: Inputting the first weighted fusion feature into the fourth state space model for deep feature fusion to obtain a first deep feature; Inputting the second weighted fusion feature into the fifth state space model for deep feature fusion to obtain a second deep feature; Inputting the third weighted fusion feature into the sixth state space model for deep feature fusion to obtain a third deep feature; Reconstruct the first depth feature through a third upsampling layer to obtain a first reconstructed feature; The first reconstructed feature and the second depth feature are weightedly fused, and the depth feature is reconstructed through the fourth upsampling layer to obtain the second reconstructed feature; The second reconstruction feature and the third depth feature are weightedly fused, and feature extraction is performed through a convolutional layer to obtain enhanced dehazing features.

10. A non-uniform fog image defogging system based on multi-scale feature constraints, characterized by: include: The feature extraction and reconstruction module is used to input the foggy image into the feature extraction and reconstruction module, and perform layer-by-layer feature extraction through two layers of Laplacian pyramid downsampling layers in sequence to obtain the detailed features and global features of the corresponding layers respectively; Based on the detailed features and global features of the corresponding layer, layer-by-layer fusion reconstruction is performed through two layers of Laplacian pyramid upsampling layers in sequence; Based on the dual prior loss, the features after fusion and reconstruction of each layer are constrained to obtain the optimized dehazed image features; The feature enhancement module is used to input foggy images into the feature enhancement module, map them to high-dimensional feature space through the block embedding module, and extract multi-scale features layer by layer through multiple state space models and block fusion modules; Align multi-scale features through downsampling layers, and perform dense multi-scale feature aggregation through upsampling layers; The aggregated multi-scale features are fused with deep features through the state space model and reconstructed through the upsampling layer to obtain enhanced dehazing features. The feature fusion module is used to perform weighted fusion of the optimized dehazed image features and the enhanced dehazed features to obtain the final dehazed image.

Images