Snow and fog degraded image restoration method based on adaptive multi-scale enhancement

Abstract

The invention discloses a snow and fog degraded image restoration method based on adaptive multi-scale enhancement, and the method comprises the following steps: carrying out the standardization processing of a snow and fog degraded image, and obtaining a standardized image; according to the low-order statistical characteristics of the standardized image, generating a condition prompt tensor for describing a weather interference type; according to the condition prompt tensor, performing feature extraction on the standardized image through a multi-scale Transform backbone network to obtain multi-scale feature information; generating a noise component guide tensor by adopting a composite noise prompt mechanism according to the multi-modal distribution characteristics of noise in the standardized image; and reconstructing according to the multi-scale feature information and the noise component guide tensor to obtain a restored clear image. According to the method, the quality of image restoration can be remarkably improved, and the accuracy of image restoration in a severe weather environment can be improved.

Description

Technical Field

[0001] This invention belongs to the field of snow and fog image restoration technology, and particularly relates to a method for restoring snow and fog degraded images based on adaptive multi-scale enhancement. Background Technology

[0002] Images serve as a crucial medium for computer vision systems to perceive external environmental information, and their quality directly impacts the accuracy and stability of downstream tasks such as object detection, scene recognition, autonomous driving, and railway inspection. In various practical applications, especially in outdoor scenarios, images are frequently affected by adverse weather conditions, such as rain, snow, and fog. These natural factors can cause significant image degradation, leading to a decline in the performance of the visual perception system and even affecting its normal operation.

[0003] Snow and fog weather is a typical complex meteorological condition, combining spatial occlusion and light scattering characteristics, belonging to a coupled composite degradation problem that has not yet been effectively solved in the field of image restoration. In this environment, images typically face the following degradation characteristics: First, snow occlusion causes random information loss in local areas and exhibits strong time-varying properties; second, fog reduces overall contrast through multipath scattering, accompanied by color shift. This type of image degradation process exhibits nonlinearity, multi-scale, and non-uniform coupling characteristics, greatly increasing the technical difficulty of image enhancement or restoration. Existing research shows that traditional image enhancement methods struggle to simultaneously address dynamic occlusion and scattering degradation problems, and suffer from limitations such as insufficient modeling capabilities and poor generalization performance. Summary of the Invention

[0004] This invention proposes a snow and fog degradation image restoration method based on adaptive multi-scale enhancement to solve the problems existing in the prior art.

[0005] To achieve the above objectives, this invention provides a method for restoring snow and fog-degraded images based on adaptive multi-scale enhancement, comprising the following steps:

[0006] The snow and fog degraded image is standardized to obtain a standardized image;

[0007] Based on the low-order statistical features of the standardized image, a conditional cue tensor describing the type of weather disturbance is generated.

[0008] Based on the conditional cue tensor, feature extraction is performed on the standardized image through a multi-scale Transformer backbone network to obtain multi-scale feature information;

[0009] Based on the multimodal distribution characteristics of noise in standardized images, a composite noise cueing mechanism is used to generate noise component guiding tensors.

[0010] Based on the multi-scale feature information and noise components, the tensor is reconstructed to obtain a restored clear image.

[0011] Optionally, the standardization process includes:

[0012] The input image is resized to meet preset requirements;

[0013] The resized image is then subjected to brightness normalization.

[0014] The image after brightness normalization is converted to a color space to a preset color space.

[0015] Optionally, the generation of the conditional cue tensor describing the type of weather disturbance includes:

[0016] A lightweight CNN structure is used to extract low-order statistical features of normalized images;

[0017] The extracted low-order statistical features are mapped to a weather dictionary to generate a conditional cue tensor describing the type of weather disturbance.

[0018] Optionally, the multi-scale Transformer backbone network includes several scale branches, and the extraction steps for each scale branch include:

[0019] Fine-grained modeling of local pixel regions is performed using an in-window attention mechanism to capture high-frequency details;

[0020] The decay pattern of the image is modeled globally using a multi-head self-attention mechanism;

[0021] Features of different resolutions are fused through cross-scale residual connections and feature alignment operations.

[0022] Optionally, the multi-scale Transformer backbone network also includes an Intra-PT module, which is used to perform fine-grained modeling of local regions of the image during the encoding stage, thereby enhancing the ability to capture local occlusion features.

[0023] Optionally, the Transformer module in the multi-scale Transformer backbone network includes a multi-head self-attention mechanism and a feedforward network module, wherein the feedforward network module adopts a combination structure of depthwise separable convolution and multilayer perceptron.

[0024] Optionally, the obtained restored clear image includes:

[0025] High-resolution reconstruction of multi-scale feature information is performed based on convolutional upsampling operations;

[0026] Edge loss and structural similarity loss are introduced as optimization objectives to improve the visual quality and structural consistency of the restored image.

[0027] The present invention also proposes a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0028] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0029] The present invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the steps of the method.

[0030] Compared with the prior art, the present invention has the following advantages and technical effects:

[0031] This invention discloses a method for restoring snow and fog-degraded images based on adaptive multi-scale enhancement, comprising the following steps: standardizing the snow and fog-degraded image to obtain a standardized image; generating a conditional cue tensor describing the type of weather interference based on the low-order statistical features of the standardized image; extracting features from the standardized image using a multi-scale Transformer backbone network based on the conditional cue tensor to obtain multi-scale feature information; generating a noise component guiding tensor using a composite noise cue mechanism based on the multimodal distribution characteristics of noise in the standardized image; and reconstructing the image based on the multi-scale feature information and the noise component guiding tensor to obtain a restored clear image. This invention significantly enhances the quality of image restoration, especially its noise suppression capability under complex weather conditions, improving the robustness of image restoration systems in practical applications. This technology not only has technical advantages but also helps improve the accuracy of image restoration under adverse weather conditions, possessing high social value and the potential for widespread application in fields such as traffic monitoring and environmental monitoring. Attached Figure Description

[0032] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0033] Figure 1 This is a schematic diagram illustrating the restoration implementation of an embodiment of the present invention;

[0034] Figure 2 This is a diagram of the noise compensation module according to an embodiment of the present invention;

[0035] Figure 3 This is a structural diagram of the encoder module according to an embodiment of the present invention;

[0036] Figure 4 This is a structural diagram of the feedforward network module according to an embodiment of the present invention;

[0037] Figure 5 This is a structural diagram of the Intra-PT module according to an embodiment of the present invention;

[0038] Figure 6 This is a structural diagram of the Transformer module in the decoder of an embodiment of the present invention;

[0039] Figure 7 This is a diagram illustrating the defogging effect of an embodiment of the present invention. Detailed Implementation

[0040] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0042] Example 1

[0043] This embodiment provides a method for restoring snow and fog-degraded images based on adaptive multi-scale enhancement. Specifically applied to image sharpening processing in UAV visual perception systems for railway infrastructure inspection, it includes the following steps:

[0044] The snow and fog degraded image is standardized to obtain a standardized image;

[0045] Based on the low-order statistical features of the standardized image, a conditional cue tensor describing the type of weather disturbance is generated.

[0046] Based on the conditional cue tensor, feature extraction is performed on the standardized image through a multi-scale Transformer backbone network to obtain multi-scale feature information;

[0047] Based on the multimodal distribution characteristics of noise in standardized images, a composite noise cueing mechanism is used to generate noise component guiding tensors.

[0048] Based on the multi-scale feature information and noise components, the tensor is reconstructed to obtain a restored clear image.

[0049] Furthermore, the standardization process includes:

[0050] The input image is resized to meet preset requirements;

[0051] The resized image is then subjected to brightness normalization.

[0052] The image after brightness normalization is converted to a color space to a preset color space.

[0053] Furthermore, the generation of the conditional cue tensor describing the type of weather disturbance includes:

[0054] A lightweight CNN structure is used to extract low-order statistical features of normalized images;

[0055] The extracted low-order statistical features are mapped to a weather dictionary to generate a conditional cue tensor describing the type of weather disturbance.

[0056] Furthermore, the multi-scale Transformer backbone network includes several scale branches, and the extraction steps for each scale branch include:

[0057] Fine-grained modeling of local pixel regions is performed using an in-window attention mechanism to capture high-frequency details;

[0058] The decay pattern of the image is modeled globally using a multi-head self-attention mechanism;

[0059] Features of different resolutions are fused through cross-scale residual connections and feature alignment operations.

[0060] Furthermore, the multi-scale Transformer backbone network also includes an Intra-PT module, which is used to perform fine-grained modeling of local regions of the image during the encoding stage, thereby enhancing the ability to capture local occlusion features.

[0061] Furthermore, the Transformer module in the multi-scale Transformer backbone network includes a multi-head self-attention mechanism and a feedforward network module. The feedforward network module adopts a combination structure of depthwise separable convolution and multilayer perceptron.

[0062] Furthermore, the obtained restored clear image includes:

[0063] High-resolution reconstruction of multi-scale feature information is performed based on convolutional upsampling operations;

[0064] Edge loss and structural similarity loss are introduced as optimization objectives to improve the visual quality and structural consistency of the restored image.

[0065] like Figure 1As shown, this embodiment is based on the NoiseWeather image restoration neural network structure. The NoiseWeather network adopts a multi-scale Transformer architecture, combining a local detail enhancement module and a weather type indication mechanism to achieve collaborative modeling and accurate restoration of local occlusion (such as snow spots) and global attenuation (such as fog). The following sections, in conjunction with the illustrated structure, will describe the composition and workflow of this network in detail in stages.

[0066] First, the network input is a degraded image of size H×W×3. This image may be affected by weather factors such as rain, snow, and fog, manifesting as local occlusion, blurring, and brightness reduction. In addition, the input also includes weather type information or noise labels corresponding to the image, serving as supplementary auxiliary information to guide the model's perception direction. All input data first enters the noise analysis module, which models and classifies the image degradation type, providing prior knowledge support for subsequent processing.

[0067] Next, the image enters the encoder stage. First, the Patch Embedding module divides the entire image into multiple fixed-size image patches, for example, each patch is 16×16 pixels. Each patch is mapped to a high-dimensional vector through a linear transformation, thus forming the input sequence that the Transformer encoder can process. The input sequence then enters the multi-stage Transformer encoder module, which consists of multiple layers. Figure 3 This is a diagram of the encoder module structure, illustrating the architecture of the WeatherClean network. This network receives a noisy image processed by the noise supplementation module, divides it into multiple patches, and progressively inputs them into the encoder of the transformer module. At each stage, the image resolution is gradually reduced, allowing the network to learn coarse and fine information about the image at different levels, thus effectively removing noise and restoring the image. The encoding module consists of multiple Transformer Blocks, each containing a multi-head self-attention mechanism, a feedforward network, layer normalization, and residual connections, collectively building a powerful feature modeling capability. The structure of the feedforward network module is shown below. Figure 4As shown, the improved feedforward network module structure is illustrated. This module effectively reduces computational complexity by introducing a reduction ratio R when calculating self-attention, and simplifies the key representation through dimensionality reshaping and linear transformation. Subsequently, the obtained self-attention features are fed into a feedforward network containing deep convolutions, enhancing the local feature extraction capability and providing positional information to the Transformer, thereby improving the model's expressive power and denoising performance.

[0068] Figure 6 The Transformer module structure in the decoder is demonstrated. A learnable embedding of weather type is used as the query, and together with the features output by the encoder, attention is computed to extract task-related information, which is ultimately used for image reconstruction.

[0069] Following each Transformer encoding stage, the network introduces an Intra-PT (Intra-Patch Transformer) module, which is one of the key technologies in this embodiment. The Intra-PT module further divides each patch into smaller sub-blocks and performs attention modeling between these sub-blocks. This not only preserves detailed information in local regions (such as snow spots, raindrops, and other disturbing areas) but also improves the model's ability to perceive and reconstruct small-scale perturbations. Through this backbone + local enhancement structure design, this embodiment achieves simultaneous perception and joint modeling of local and global image information.

[0070] As the network depth increases, the Transformer encoder gradually downsamples the spatial dimensions of the image while continuously enhancing semantic information, ultimately outputting multi-scale hierarchical features. These features are then passed to the decoder module. The decoder adopts a task-aware Transformer decoder structure, with its core innovation being the introduction of Weather Type Queries. This part first generates a set of learnable query vectors based on the image's weather type (e.g., "snow," "fog," or "snow + fog"), which serve as the input to the decoder. These query vectors, along with the feature maps generated by the encoder (as keys and values), enter the multi-head attention mechanism in the decoder, thereby achieving precise focus and differentiated modeling of specific weather-affected areas in the feature space.

[0071] The Transformer decoder outputs a set of task-aware feature maps, which integrate local and global semantic information, weather perception information, and image structure information. These feature maps are then mapped back to the spatial dimensions of the original image through a convolutional projection module. This module, composed of multiple convolutional layers, effectively restores the spatial structure and color texture of the image, outputting a clear, restored image with dimensions H×W×3.

[0072] Through the above comprehensive structural design, the multi-scale Transformer architecture ensures global structural modeling capabilities, the Intra-PT module enhances the modeling of local details, and the weather type cue mechanism guides the model to perform differentiated reconstruction based on task perception. The overall network structure possesses both good restoration accuracy and end-to-end trainability and deployment efficiency, making it suitable for image sharpening preprocessing tasks in various practical scenarios such as autonomous driving, smart security, and remote sensing image analysis.

[0073] Transformer module:

[0074] In this embodiment, the Transformer module is the basic unit for constructing the entire encoder and decoder structure. Its function is to perform global modeling and context awareness of image features at different scales. This module mainly consists of two parts: a multi-head attention (MSA) mechanism and a feed-forward network (FFN) module, supplemented by layer normalization and residual connections to enhance training stability and expressive power.

[0075] In each Transformer Block, the input features are first normalized through a LayerNorm layer to mitigate the impact of distributional differences between different feature dimensions on the training process. Subsequently, the input is fed into a Multi-Head Self-Attention (MSA) mechanism for feature modeling. This module generates query (Q), key (K), and value (V) vectors from the input vector sequence and calculates attention weights to capture long-distance dependencies between different image regions. Its core calculation can be expressed by the following two formulas:

[0076] Forward propagation process of Transformer Block:

[0077] T i (I i ) = FFN(MSA(I i )+(I i )

[0078] Where Ti represents the output of the stage Transformer, Ii is the input feature, MSA represents the multi-head self-attention module, and FFN represents the feedforward network module. Residual connections (plus signs) ensure gradient stability.

[0079] Raw attention score calculation:

[0080]

[0081] Where d is the dimension of the key vector, used for scaling to prevent gradient explosion, and softmax ensures the normalization of attention weights.

[0082] This embodiment introduces an important efficiency optimization mechanism—the Reduction Ratio (R)—to reduce the computational complexity of the original self-attention. The computational complexity of traditional self-attention is O(N²), where N is the length of the input sequence (i.e., the number of patches). By reshaping the dimensions of the keys (K) and values ​​(V) from (N, C) to (N / R, CR), and then mapping the number of channels back from CR to C through a linear transformation, the overall computational complexity of attention can be reduced to O(N² / R), significantly improving operational efficiency, especially suitable for high-resolution image processing tasks.

[0083] After attention calculation, the output features are passed through another LayerNorm layer and fed into an improved feedforward neural network module (FFN Block). Unlike the standard fully connected FFN used in traditional Vision Transformer (ViT), this embodiment introduces depth-wise convolution into the FFN. This design enhances the model's ability to perceive local regions, incorporates positional information, and effectively compensates for the lack of spatial awareness in the original Transformer. Specifically, depth-wise convolution operates independently within each channel, helping to capture local texture features while preserving global context modeling capabilities.

[0084] The output of the entire Transformer Block is finally added to the input through residual connections to form the output features of this stage, providing context-enhanced representations for subsequent encoding or decoding stages.

[0085] Intra-PT module:

[0086] The design of this module aims to solve the problem that traditional Transformer encoders are prone to losing minute structural information and have difficulty capturing fine-grained local features during the downsampling process. Especially under the conditions of snow and fog coupling perturbation, there are a lot of small occlusions in the image (such as snow spots and fog spots). If they are not modeled, the restoration result is very easy to be blurry or lack details.

[0087] like Figure 5 The Intra-PT module shown consists of two key sub-modules: PatchEmbedding and Transformer Block (self-attention encoding module).

[0088] Structurally, the Intra-PT module is inserted between each stage of the main Transformer encoder as an auxiliary enhancement module. Its core idea is that after the input feature map at each stage is divided into standard patches, these patches are further subdivided into smaller sub-patches. These sub-patches are half the size of the original patches (i.e., halved in both width and height). For example, if the original patch is 16×16, the sub-patches are 8×8. This partitioning method allows the model to capture finer-grained texture information within a smaller receptive field, making it particularly suitable for recovering microstructural regions such as snowflakes, texture breaks, and local occlusions.

[0089] After the sub-patches are divided, the Intra-PT module first embeds the sub-patches into vector sequences through a lightweight Patch Embedding layer, constructing an input format suitable for Transformer processing. These embedded vectors are then fed into a separate Transformer Block for processing. This Block is structurally similar to the main encoding module, containing conventional components such as multi-head self-attention, feedforward networks (FFN), normalization, and residual connections. However, to improve processing efficiency, the Intra-PT module employs a higher compression ratio (i.e., a larger scaling factor R) to reduce computational resource consumption and adapt to sub-patch-level modeling tasks.

[0090] From a functional perspective, the main advantage of the Intra-PT module lies in its ability to effectively supplement the shortcomings of the backbone Transformer module in local modeling without significantly increasing computational overhead. Because it operates on smaller sub-patches, the module can automatically capture minute but crucial interference information in the image, such as snowflake edges and haze transition areas, thus providing richer and more nuanced information at the feature level.

[0091] During the forward propagation process, the output of each stage can be formally represented as:

[0092] Y i =MT i (X i )+IntraPT i (P(X i ))

[0093] Where Yi represents the output of the i-th stage, Mti() represents the backbone Transformer encoder module, IntraPTi() represents the Intra-PT module, P() is the function for sub-block partitioning, and Xi is the input feature of this stage. It can be seen that this embodiment adds the output features of the Intra-PT module to the output features of the main module element-wise, thereby achieving the fusion modeling of local and global features.

[0094] In addition, it should be noted that in the first encoding stage, the sub-block division of the Intra-PT module directly applies to the original image patch; while in subsequent stages, the module operates on intermediate feature maps rather than the original image, so its modeling object is more abstract and can adapt to different levels of semantic expression.

[0095] Furthermore, this embodiment also includes noise supplementation. Figure 2 The structure of the noise enhancement module is shown. This module adds two Gaussian noise images of equal intensity to generate an image with a more complex noise distribution. By increasing the complexity of the noise, it effectively masks its interference with training, enhances the model's ability to extract useful information, and thus improves the image restoration effect.

[0096] The final dehazing visualization result is as follows: Figure 7 As shown. (Through) Figure 7 It is evident that the recovery effect of this application is significant.

[0097] This embodiment also proposes an image restoration system, including:

[0098] Image preprocessing module: This module performs standardization operations on the input image, including image resizing, brightness normalization, and color space conversion, to ensure input consistency and computational efficiency for subsequent modules.

[0099] Weather type cue generation module: Based on the low-order statistical features of the input image, this module generates a conditional cue tensor describing the current weather disturbance type (such as snowfall density and fog concentration gradient). This module can use a lightweight CNN structure to extract features and combine them with a weather dictionary mapping to generate cue vectors, which guide subsequent networks to focus on specific degraded areas.

[0100] Multi-scale Transformer backbone network: The backbone network consists of multiple scale branches, each responsible for extracting image feature information at different spatial resolutions. Each scale contains the following key structures:

[0101] Local detail enhancement module: Based on window-based attention, it focuses on high-frequency details within local pixel regions, such as snowflake edges and texture patterns. It enhances image detail preservation by combining positional encoding and channel attention.

[0102] Global Context Modeling Module: This module utilizes the standard multi-head self-attention mechanism to model decay patterns (such as fog gradation) across the entire image, enhancing the network's ability to express long-range information dependencies.

[0103] Inter-scale fusion mechanism: Through cross-scale residual connection and feature alignment operations, features of different resolutions are effectively fused in terms of spatial and semantic information, thereby improving the overall restoration effect.

[0104] Noise Modeling and Guidance Module: Employs a composite noise cue mechanism to generate a noise component guidance tensor based on the multimodal distribution characteristics of noise in the image. This guides the network to more accurately identify and reduce structural noise in snow and fog images, avoiding artifact residue.

[0105] Image restoration module: Combining output features from multiple scales, the module generates the final restored image. This module can perform high-resolution reconstruction based on convolutional upsampling operations and introduces edge loss and structural similarity loss (SSIM) as optimization objectives to improve the visual quality and structural consistency of the restored image.

[0106] The role of each module in the system:

[0107] 1. The image preprocessing module ensures the consistency of data input;

[0108] 2. The weather type prompt module provides environmental priors, which helps the network to focus on modeling interference mechanisms;

[0109] 3. The multi-scale Transformer network structure processes local and global features simultaneously, improving the detail and overall consistency of the restored data;

[0110] 4. The noise warning module enhances the model's noise resistance.

[0111] 5. The image restoration module completes the final high-quality image reconstruction.

[0112] Furthermore, the multi-scale Transformer network can be replaced with a pyramid-structured U-Net or residual pyramid network (such as Res2Net) to achieve similar multi-scale modeling results;

[0113] Furthermore, the noise alert module can be replaced by a noise estimation network, which uses an independent noise prediction model to model different types of noise.

[0114] Furthermore, weather type prompts can generate conditional embedding vectors through meta-learning or few-shot learning, enabling more adaptive external environment prompts.

[0115] This embodiment also proposes a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0116] This embodiment also proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method.

[0117] This embodiment also proposes a computer program product, including a computer program that, when executed by a processor, implements the steps of the method.

[0118] The present invention achieves the following effects:

[0119] Technical Effect 1: This invention introduces an Intra-PT (Intra-Graph Transformer) module to perform fine-grained modeling of local regions of the image during the encoding stage, solving the problem of insufficient capture of local occlusion features in traditional convolutional structures when processing complex weather-degraded images. Based on a self-attention mechanism within a local window, this module can accurately capture the spatial distribution features of unstructured occlusions such as snowflakes and raindrops, enhancing the expression of local texture and edge information. Experimental data shows that in the task of synthesizing snow scene image restoration, the model using this module improves the PSNR by an average of 1.27 dB compared to the traditional convolutional structure (from 28.6 dB to 29.87 dB). This indicates that the invention can significantly enhance the quality of image restoration, especially the noise suppression capability under complex weather conditions, improving the robustness of the image restoration system in practical applications. This technical effect not only has technical advantages but also helps improve the accuracy of image restoration under severe weather conditions, possessing high social value and the potential for widespread application in fields such as traffic monitoring and environmental monitoring.

[0120] Technical Effect 2: This invention proposes a task-guided Transformer decoder structure. By introducing a weather type query vector during the decoding stage, it guides the decoder to perform differential feature recovery, solving the problem that existing methods cannot perform differential recovery based on degradation type under various weather conditions. This structure embeds the degradation type as a priori condition into the feature recovery process, enabling the model to perform targeted image reconstruction based on different weather types (such as snow, fog, and rain). Experimental results show that for different types of image degradation scenarios such as snow, fog, and rain, the guided mechanism reduces the model's average reconstruction error by approximately 12.4% (from 2.33 to 2.04), and improves the PSNR by 0.84 dB in snowy scenarios (from 29.12 dB to 29.96 dB). This improvement significantly enhances image recovery accuracy and strengthens the model's adaptability to different weather conditions. This technology has broad application potential in fields such as intelligent transportation and autonomous driving, possessing strong market prospects and social value, and can significantly improve safety and reliability in complex environments.

[0121] Technical Effect 3: This invention constructs a multi-scale feature extraction structure, combining a hierarchical Transformer encoder, a local attention mechanism, and skip connections to effectively fuse the global semantic structure and local texture details of an image. This structure enhances the collaborative modeling ability between multi-scale features, demonstrating a significant advantage, particularly in restoring complex traffic scene images such as buildings and roads. Experimental data shows that the model's mIoU metric improved by 3.5% on the Cityscapes real-world rain and fog dataset (from 69.1% to 71.6%), indicating that this structure can effectively maintain semantic consistency in complex scenes while enhancing the ability to reconstruct details. This improvement not only solves the problem of detail loss in image restoration but also optimizes multi-scale feature fusion, enabling the model to more accurately restore details in complex traffic scenes. This technology helps improve the accuracy of real-time image analysis in intelligent transportation systems, promotes the development of urban traffic management and autonomous driving technology, and has significant technical and economic value.

[0122] Technical Effect 4: This invention optimizes the feedforward network (FFN) structure in the Transformer, employing a combination of depthwise separable convolution and multilayer perceptron (MLP). This significantly reduces the number of model parameters and computational cost while maintaining or even improving model performance. Experimental data shows that, while maintaining essentially the same recovery quality, the number of model parameters is reduced by approximately 28.6% (from 26.4M to 18.8M), and the inference time is shortened by approximately 31% (from 1.25 seconds / frame to 0.86 seconds / frame). This optimization not only improves inference speed but also significantly reduces energy consumption and computational resource requirements, enhancing the model's deployment capability on resource-constrained devices. The technical effects demonstrate that, without sacrificing recovery quality, the optimized model has lower deployment costs and faster inference speeds, possessing broad engineering application potential. It is particularly competitive in scenarios such as smart devices and edge computing.

[0123] Technical Effect 5: This invention introduces a noise or weather type cue mechanism, providing the model with prior information on image degradation categories during the inference phase. This enables the model to perform efficient image restoration based on semantic guidance. Experimental data shows that, under both cue-free and cue-enabled conditions, the model improves the average restored PSNR by 0.94 dB (from 29.6 dB to 30.54 dB) and the SSIM index by approximately 0.012 (from 0.880 to 0.892) when processing complex weather images. This mechanism significantly improves the model's robustness and generalization ability under different weather conditions, especially for the restoration of complex weather images. The technical effects demonstrate that the weather type cue mechanism can effectively improve the model's performance in real-world complex scenarios, possessing broad application prospects, particularly suitable for fields such as intelligent monitoring and UAV image analysis, and has high economic and social value.

[0124] Technical Effect 6: This invention constructs a cross-domain image restoration training mechanism, combining multimodal self-supervised learning with cross-modal consistency constraint strategies to guide the model to learn universal image restoration knowledge in unlabeled real weather images. This mechanism significantly improves the model's cross-domain adaptability, enabling it to maintain good restoration performance in multiple weather environments. Experimental data shows that, under the settings of synthetic domain training and real domain testing, compared with the baseline model without self-supervised strategies, the proposed method improves the PSNR by an average of 1.52dB in real snow and fog scenes (from 28.4dB to 29.92dB). This mechanism can effectively improve the model's adaptability in different weather scenarios and has better cross-domain transfer capabilities. This technical effect can significantly reduce the dependence on labeled data, improve the universality of image restoration, and has significant social benefits and broad market prospects, suitable for applications in various devices and scenarios.

[0125] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for restoring snow and fog-degraded images based on adaptive multi-scale enhancement, characterized in that, Includes the following steps: The snow and fog degraded image is standardized to obtain a standardized image; Based on the low-order statistical features of the standardized image, a conditional cue tensor describing the type of weather disturbance is generated. Based on the conditional cue tensor, feature extraction is performed on the standardized image through a multi-scale Transformer backbone network to obtain multi-scale feature information; Based on the multimodal distribution characteristics of noise in standardized images, a composite noise cueing mechanism is used to generate noise component guiding tensors. Based on the multi-scale feature information and noise components, the tensor is reconstructed to obtain a restored clear image.

2. The method according to claim 1, characterized in that, The standardization process includes: The input image is resized to meet preset requirements; The resized image is then subjected to brightness normalization. The image after brightness normalization is converted to a color space to a preset color space.

3. The method according to claim 1, characterized in that, The generated conditional cue tensor describing the type of weather disturbance includes: A lightweight CNN structure is used to extract low-order statistical features of normalized images; The extracted low-order statistical features are mapped to a weather dictionary to generate a conditional cue tensor describing the type of weather disturbance.

4. The method according to claim 1, characterized in that, The multi-scale Transformer backbone network includes several scale branches, and the extraction steps for each scale branch include: Fine-grained modeling of local pixel regions is performed using an in-window attention mechanism to capture high-frequency details; The decay pattern of the image is modeled globally using a multi-head self-attention mechanism; Features of different resolutions are fused through cross-scale residual connections and feature alignment operations.

5. The method according to claim 1, characterized in that, The multi-scale Transformer backbone network also includes an Intra-PT module, which is used to perform fine-grained modeling of local regions of the image during the encoding stage, thereby enhancing the ability to capture local occlusion features.

6. The method according to claim 1, characterized in that, The Transformer module in the multi-scale Transformer backbone network includes a multi-head self-attention mechanism and a feedforward network module. The feedforward network module adopts a combination structure of depthwise separable convolution and multilayer perceptron.

7. The method according to claim 1, characterized in that, The restored clear image includes: High-resolution reconstruction of multi-scale feature information is performed based on convolutional upsampling operations; Edge loss and structural similarity loss are introduced as optimization objectives to improve the visual quality and structural consistency of the restored image.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-7.

Images