Multi-degraded image restoration method based on semantic guidance

Abstract

The invention discloses a multi-degraded image restoration method based on semantic guidance. According to the method, for degraded images captured by a vehicle-mounted camera under severe weather conditions, potential features of the images are extracted by adopting a trunk network based on Transform, multi-modal semantic information is extracted in combination with a CLIP visual language model, and semantic guidance is provided for different degradation types through a dynamic text prompt generation mechanism. A self-adaptive feature fusion module is designed, channel attention and space attention mechanisms are combined to realize effective integration of multi-modal features, and a degradation feature extraction and fusion module is introduced to enhance the generalization ability of the model. According to the semantic guidance multi-type image recovery network SGIRN provided by the invention, the global modeling capability of the Transform and the cross-modal representation capability of the CLIP visual language model are combined, so that high-quality recovery of various weather degradation types is realized.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, specifically to a semantically guided multi-degradation image restoration method based on deep learning, which is particularly suitable for handling image degradation problems under various weather conditions such as rain, snow, and fog. Background Technology

[0002] With the rapid development of intelligent transportation and autonomous driving technologies, vehicle-mounted cameras play a crucial role in road driving, responsible for capturing real-time road environment information and providing key data support for vehicle target detection, path planning, and decision-making. However, in real-world driving environments, images captured by vehicle-mounted cameras are often affected by various degradation factors, including interference from adverse weather conditions such as rain, snow, and fog. These degradation factors severely impact image quality, thereby adversely affecting subsequent tasks such as image analysis, target detection, and path planning.

[0003] Traditional image restoration methods mainly focus on processing single degradation types, such as specialized deraining, defogging, and denoising algorithms. These methods are usually based on specific physical models or prior assumptions, and perform well in specific tasks, but lack the ability to generalize across degradation types and are difficult to cope with the complex and ever-changing weather conditions in real-world scenarios.

[0004] In recent years, deep learning-based image restoration methods have made significant progress, especially convolutional neural networks (CNNs) and Transformer architectures, which have performed exceptionally well in single degradation restoration tasks. However, these methods still face the following challenges when dealing with multiple degradation types: the feature representations of different degradation types vary greatly, making unified processing difficult; semantic information is difficult to preserve during restoration, and important details are easily lost; the model's generalization ability to unseen degradation types is limited; and there is a lack of effective mechanisms for utilizing cross-modal information.

[0005] Recently, the development of visual language models (such as CLIP) has provided new ideas for image restoration tasks. These models, through contrastive learning of large-scale image-text pairs, can map images and text to a unified embedding space, providing rich semantic information for image understanding. However, how to effectively integrate multimodal features to achieve optimal image restoration results, especially in dynamically adapting to different degradation conditions, still has room for research. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a semantically guided multi-degraded image restoration method that can simultaneously handle image degradation problems under various weather conditions, and has good generalization ability and restoration effect.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a semantically guided method for restoring multiple degraded images, comprising the following steps:

[0008] Collect images, perform preprocessing and data augmentation, and create a multimodal degraded image dataset;

[0009] A semantically guided multi-degraded image restoration network model, SGIRN, is established, including a multimodal feature extraction module, a degradation feature extraction and fusion module, and an improved Transformer backbone network. The network is trained using a dataset and backpropagated to adjust the parameters to obtain the ideal model.

[0010] The multimodal feature extraction module uses the CLIP visual language model to extract the input degraded image I. d Visual semantic features F semantic It identifies degradation types, dynamically generates text prompts, and extracts corresponding text features F. text_adapted ;

[0011] The degradation feature extraction and fusion module extracts the input degradation image I through multi-scale convolution and feature pyramid structure. d Degradation characteristics F at different scales multi and with latent feature F latent Perform dynamic fusion to generate fusion feature F final ;

[0012] The improved Transformer backbone network adopts an encoder-decoder structure, and the encoder extracts the input degraded image I through a multi-head self-attention mechanism. d Potential features F latent The backbone network of the Transformer also embeds an adaptive feature fusion module, which fuses latent features F through learnable weight parameters. latent Visual semantic features F semantic and text features F text_adapted It also combines channel attention and spatial attention mechanisms to enhance the output key feature F. _out ;F final F _out The image Ir is recovered by the decoder output of the backbone network;

[0013] After acquiring and preprocessing the degraded image to be processed, it is input into the SGIRN ideal model, which automatically outputs the corresponding restored image.

[0014] The preprocessing includes, but is not limited to, random cropping, flipping, rotation, brightness and contrast adjustment, size adjustment, and normalization; the multimodal degradation image pair consists of complex background images of multiple weather scenes and their corresponding clear images.

[0015] The multimodal feature extraction module includes:

[0016] (1) Visual feature extraction submodule, which uses CLIP's image encoder to extract the input degraded image I. d Visual semantic features F semantic And adjust the feature dimensions through a semantic feature adapter;

[0017] (2) Degradation type recognition submodule: calculates the similarity between the input image and the preset degradation type text template to determine the most likely degradation type;

[0018] (3) Text feature extraction submodule: Based on the identified degradation type, dynamic text prompts are dynamically generated, and text features F are extracted through CLIP text encoder. text_adapted .

[0019] The generation of dynamic text prompts includes:

[0020] For rainwater degradation, generate a text prompt of the type "clear image, no rain, high visual clarity";

[0021] For fog degradation, generate a text prompt of the type "fog-free image, high visibility, and saturated colors";

[0022] For snow scene degradation, generate a text prompt of the type "Clear image without snowflakes, rich in detail".

[0023] The degradation feature extraction and fusion module includes:

[0024] (1) Multi-scale feature extraction submodule: Uses three different sizes of convolution kernels (3×3, 5×5, and 7×7) to extract the input degraded image I in parallel. d Multi-scenario weather features at different scales F multi ;

[0025] (2) Feature Pyramid Submodule: A feature pyramid is constructed through max pooling and upsampling operations to enhance feature representation at different levels;

[0026] First convolutional layer extracts features F pyramid1 =Conv(F multi );

[0027] Max pooling is used to reduce the feature map size, followed by convolution to capture a wider range of contextual information, resulting in the first layer of convolutional feature extraction F. pyramid2 =Conv(MaxPool(F pyramid1 ));

[0028] The features from the two feature pyramid layers are combined to generate a degenerate feature representation F. deg=Concat(F pyramid1 Upsample(F pyramid2 (3) Dynamic fusion submodule, which generates dynamic weights through convolutional network to adaptively adjust the fusion ratio of image features and degradation features;

[0029] Image latent features F latent and degradation characteristics F deg The input is fed into a convolutional layer to generate two weights w1 and w2: w1, w2 = Split(Conv(Concat(F)) latent ,F deg )))

[0030] Weighted fusion yields the fused feature F fused =w1·F latent +w2·F deg

[0031] Further processing through the channel attention layer yields the important channel enhanced representation features F. ca =F fused ·σ(MLP(Concat(AvgPool(F fused ),MaxPool(F fused ))));

[0032] (3) Introduce residual connections: Output fusion feature F final =F ca +F latent .

[0033] The adaptive feature fusion module fuses latent features F latent Visual semantic features F semantic and text features F text_adapted Represented as:

[0034] F m_fused =w latent ·F latent +w text ·F text_adapted +w semantic ·F semantic

[0035] Among them, w latent w text w semantic These are the corresponding learnable scalar weight parameters.

[0036] Loss functions include:

[0037] Calculate the reconstruction loss, and use L1 loss to measure the pixel-level difference between the restored image and the real image;

[0038] Calculate the CLIP loss and use the CLIP model to calculate the cosine similarity between the restored image and the real image in the CLIP feature space;

[0039] A total loss function is constructed by combining reconstruction loss and CLIP loss, which is used for network training optimization.

[0040] The method supports processing image degradation types including: rain degradation, snow degradation, fog degradation, low light degradation, and motion blur degradation.

[0041] The present invention has the following beneficial effects and advantages:

[0042] 1. This invention employs an encoder-decoder architecture in its Transformer-based backbone network design, utilizing a multi-head self-attention mechanism to capture global information and long-range dependencies in images. Compared to traditional CNNs, Transformers offer significant advantages in handling non-local features and understanding complex scenes.

[0043] 2. In its multimodal feature extraction mechanism, this invention utilizes a pre-trained CLIP model to extract visual semantic features from the input image. Simultaneously, a degradation type recognition module is designed to dynamically generate targeted text prompts based on the recognition results and extract corresponding text features. This design enables the model to provide semantic guidance according to different degradation types.

[0044] 3. In the adaptive feature fusion strategy, this invention designs an adaptive feature fusion module, which fuses features from different modalities through learnable weight parameters, and combines channel attention and spatial attention mechanisms to dynamically learn and strengthen key feature representations.

[0045] 4. In the process of degradation feature extraction and fusion, this invention extracts degradation features at different scales through multi-scale convolution and feature pyramid structure, and uses a dynamic weight generation mechanism to achieve effective fusion with image features, thereby enhancing the model's generalization ability for different degradation types.

[0046] 5. This invention combines reconstruction loss and CLIP loss to ensure pixel-level accuracy while maintaining semantic consistency. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the overall architecture of SGIRN.

[0048] Figure 2 This is a structural diagram of the adaptive multimodal feature fusion module.

[0049] Figure 3 This is a structural diagram of the degradation feature extraction and fusion module.

[0050] Figure 4This is a visual comparison of SGIRN with other advanced methods under different weather conditions. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0052] I. Constructing a Network Model

[0053] like Figure 1 The diagram shown illustrates the overall SGIRN architecture of the method of this invention. The semantically guided multi-type image restoration method of this invention includes the following steps:

[0054] Step 1: Construct a Transformer-based backbone network, adopting an encoder-decoder structure, and extract latent features of the input degraded image through a multi-head self-attention mechanism;

[0055] Step 2: Construct a multimodal feature extraction module, use the CLIP visual language model to extract the visual semantic features of the input image, determine the degradation type of the image through degradation type recognition, dynamically generate text prompts based on the recognition results and extract the corresponding text features;

[0056] Step 4: Construct a degradation feature extraction and fusion module, extract degradation features at different scales through multi-scale convolution and feature pyramid structure, and dynamically fuse them with image features;

[0057] Step 3: Construct an adaptive feature fusion module, which fuses latent features, visual semantic features, and text features through learnable weight parameters, and strengthens the representation of key features by combining channel attention and spatial attention mechanisms;

[0058] Step 5: Output the restored high-quality image through the decoder of the backbone network.

[0059] Furthermore, in step 1, the Transformer-based backbone network adopts a U-shaped encoder-decoder structure similar to Restormer, but optimized for multiple types of degradation recovery tasks. The encoder consists of multiple Transformer blocks. Specifically, the Transformer-based backbone network includes:

[0060] (1) The encoder part consists of multiple Transformer blocks, each containing a multi-head self-attention layer and a feedforward network. Self-attention computation is performed on the feature dimension to reduce computational complexity; the input degraded image I is extracted. d Potential features F latent .

[0061] The Transformer backbone network also embeds an adaptive feature fusion module, which fuses latent features F through learnable weight parameters. latent Visual semantic features F semantic and text features F text_adapted It also combines channel attention and spatial attention mechanisms to enhance the output key feature F. _out ; Describe in detail in step 3.

[0062] (2) The decoder section is similar in structure to the encoder but adds cross-layer connections to integrate features from each layer of the encoder into the decoding process to preserve detailed information. Specifically, the decoder section processes F through the decoder of the backbone network. final F _out Output the restored high-quality image Ir.

[0063] Step 2: Multimodal feature extraction, the structure of this module is as follows: Figure 2 As shown.

[0064] (1) Visual Feature Extraction: CLIP's VIT-B / 16 was used as the visual encoder, and the model parameters were frozen to preserve pre-trained knowledge. Visual semantic features F were extracted by adjusting the dimensions using a semantic feature adapter. semantic ;

[0065] (2) Degradation type recognition: Calculate the similarity between the input image and the preset text template to determine the most likely degradation type;

[0066] (3) Text Feature Extraction: Based on the recognition results, dynamic text prompts are generated, and text features F are extracted using the CLIP text encoder. text_adapted :

[0067] Rainwater degradation: "A clear image with no rainwater and high visual clarity";

[0068] Fog degradation: "A fog-free image has high visibility and saturated colors";

[0069] Snow scene degradation: "A clear image without snowflakes, rich in detail."

[0070] Step 3: Adaptive feature fusion, the structure of this module is as follows: Figure 2 As shown.

[0071] (1) Learnable Weight Fusion Module: To ensure the effectiveness of each feature in different contexts, learnable weight parameters are assigned to latent features, text features, and visual semantic features. These weight parameters are automatically learned by the network, allowing the network to adaptively adjust the importance of each feature based on different inputs. During the fusion process, for latent features F... latent Text features F text_adapted Visual semantic features F extracted by CLIP semantic Perform weighted calculations to generate multi-modal fusion state features F. m_fused , can be represented as:

[0072] F m_fused =w latent ·F latent +w text ·F text_adapted +w semantic ·F semantic

[0073] Among them, w latent w text w semantic The learnable scalar weights are initialized to 1 / 3. This flexible weight adjustment strategy enhances the model's adaptability under different input conditions. Through training, the model learns to assign appropriate weights to images with different degradation types; for example, it may rely more on visual semantic features for foggy images, while it may focus more on textual guidance features for rainy images.

[0074] (2) Channel attention mechanism SE module: for F m_fused Channel feature descriptions are obtained through max pooling and average pooling, and channel weights are generated to enhance the representation of important channels.

[0075] (3) Spatial attention mechanism module, which combines max pooling and average pooling operations to generate a spatial attention graph and focus on important regions in space;

[0076] (4) Residual connection: The fused features are connected to the original latent features F. latent Add them together to output the key feature F. _out .

[0077] Step 4: Degradation feature extraction and fusion, the structure of this module is as follows: Figure 3 As shown.

[0078] (1) Multi-scale feature extraction: Convolutional kernels of three different sizes (3×3, 5×5, and 7×7) are used in parallel to extract features at different scales. d These features are then concatenated along the channel dimension to form a multi-channel feature map F. multiThis multi-scale design allows the network to simultaneously capture degradation features at different scales, such as fine linear degradation like rain lines and large-scale smooth degradation like fog. Furthermore, receptive fields at different scales also help handle regions with varying degrees of degradation.

[0079] (2) Feature Pyramid: Introducing a feature pyramid structure enhances the representational power of different feature levels.

[0080] F pyramid1 =Conv(F multi ): First-layer convolution extracts features

[0081] F pyramid2 =Conv(MaxPool(F pyramid1 ): Minimum pooling is used to reduce the feature map size, and then convolution is performed to capture a wider range of contextual information;

[0082] F deg =Concat(F pyramid1 Upsample(F pyramid2 ): Combining the features from two feature pyramid layers to form the final degenerate feature representation;

[0083] (3) Dynamic Fusion: This module generates two dynamic weights through a convolutional network to adjust the fused features according to different input features. Input image features F latent and degradation characteristics F deg It is fed into a convolutional layer, generating two weights w1 and w2:

[0084] w1,w2=Split(Conv(Concat(F latent ,F deg )))

[0085] Based on the generated weights, perform weighted fusion: F fused =w1·F latent +w2·F deg

[0086] The fused feature F fused Further processing via a channel attention layer significantly enhances the feature representation capabilities of important channels. The specific implementation is as follows:

[0087] F ca =F fused ·σ(MLP(Concat(AvgPool(F fused ),MaxPool(F fused ))))

[0088] In this way, the model can more comprehensively assess the importance of channels, more accurately identify and emphasize the channels most valuable to the recovery process, thereby further improving recovery outcomes.

[0089] To enhance the network's learning ability and maintain smooth information transmission, we introduced a residual connection F at the end of the fusion process. final =F ca +F latent

[0090] By adding the original latent features to the fused features, the network can retain information from the original image during the learning process, preventing the loss of important information. This design is widely used in deep network training to help the model converge faster and improve recovery performance. Especially when dealing with regions rich in detail, residual connections can effectively prevent over-smoothing and loss of detail.

[0091] Step 5: Specifically, the decoder part processes F through the decoder of the backbone network. final F _out Output the restored high-quality image Ir.

[0092] Loss function design

[0093] Reconstruction loss: L1 loss is used to calculate the pixel-level differences between the restored image and the real image;

[0094]

[0095] Where N is the total number of pixels, I ri and I gti These represent the i-th pixel value in the restored image and the original image, respectively.

[0096] CLIP loss: Calculates the cosine similarity between the restored image and the ground image in the CLIP feature space using the CLIP model.

[0097]

[0098] Among them, E clip This refers to the CLIP image encoder.

[0099] A total loss function is constructed by combining reconstruction loss and CLIP loss, which is used for network training optimization.

[0100]

[0101] Where λ1 and λ2 are weight hyperparameters;

[0102] II. Training Implementation Details

[0103] 1. Data preprocessing:

[0104] This invention performs the following image processing operations on original complex background images with various weather scenes such as rain, snow, and fog in order to unify the size and expand the dataset.

[0105] Randomly crop to 256×256 size; randomly flip and rotate horizontally (±10 degrees); randomly adjust brightness / contrast (±0.1).

[0106] 2. Training strategy:

[0107] During model training, the Adam optimizer (β1 = 0.9, β2 = 0.999) was used, with an initial learning rate of 1e-5, cosine annealing scheduling, a batch size of 16, 120 training epochs, gradient clipping (maximum norm 1.0), and mixed precision training.

[0108] 3. Dataset configuration:

[0109] The following publicly available scenario-specific datasets can be used: Rain100L / Rain100H: rain removal task; RESIDE-SOTS: fog removal task; Snow100K: snow removal task.

[0110] III. Implementation

[0111] The present invention includes the following modules:

[0112] Image input module: Responsible for receiving degraded images to be processed, and supports multiple image format inputs.

[0113] Preprocessing module: Performs preprocessing operations such as resizing and normalization on the input image.

[0114] Feature extraction module: Includes Transformer backbone network and CLIP multimodal feature extraction unit to extract latent and semantic features of images.

[0115] Feature fusion module: Enables adaptive multimodal feature fusion and degenerate feature extraction and fusion.

[0116] Image reconstruction module: Outputs a high-quality restored image through a decoder network.

[0117] Post-processing module: performs quality assessment and format conversion on the recovery results.

[0118] The semantically guided multi-type image restoration method of the present invention effectively solves the image degradation problem under various weather conditions through innovative network architecture design and multimodal feature fusion strategy. While ensuring restoration quality, it has good generalization ability and provides reliable technical support for practical applications such as intelligent transportation and autonomous driving.

[0119] Visual comparison results of SGIRN with other advanced methods under different weather conditions, as shown below Figure 4 As shown in the visual comparison results, all methods achieved good results in image restoration tasks. Overall, the restoration results of different methods are visually similar, reflecting the overall maturity of current image restoration technology.

[0120] Defog removal mission ( Figure 4 (First row): For foggy images, all methods significantly improve image visibility and contrast. Although SGIRN is visually similar to other methods, its PSNR value reaches 31.58dB and SSIM reaches 0.96, making it the best performing method in quantitative evaluation.

[0121] Rain-removing mission ( Figure 4 (Third line): All methods can effectively remove rain line interference and restore image clarity. Although the visual differences are relatively subtle, quantitative indicators show that SGIRN achieved the highest PSNR value, indicating its advantage in pixel-level restoration accuracy.

[0122] Go to the Snow Mission ( Figure 4 (Fifth row): In snow scene processing, all comparison methods can effectively remove the snowflake effect. SGIRN achieves a SSIM score of 0.93 while maintaining the overall visual effect, which is a significant improvement over other methods, indicating that it performs better in preserving structural similarity.

[0123] The above descriptions are merely specific embodiments of this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A semantically guided method for restoring multiple degraded images, characterized in that, Includes the following steps: Collect images, perform preprocessing and data augmentation, and create a multimodal degraded image dataset; A semantically guided multi-degraded image restoration network model, SGIRN, is established, including a multimodal feature extraction module, a degradation feature extraction and fusion module, and an improved Transformer backbone network. The network is trained using a dataset and backpropagated to adjust the parameters to obtain the ideal model. The multimodal feature extraction module uses the CLIP visual language model to extract the input degraded image I. d Visual semantic features F semantic It identifies degradation types, dynamically generates text prompts, and extracts corresponding text features F. text_adapted ; The degradation feature extraction and fusion module extracts the input degradation image I through multi-scale convolution and feature pyramid structure. d Degradation characteristics F at different scales multi and with latent feature F latent Perform dynamic fusion to generate fusion feature F final ; The improved Transformer backbone network adopts an encoder-decoder structure, and the encoder extracts the input degraded image I through a multi-head self-attention mechanism. d Potential features F latent The backbone network of the Transformer also embeds an adaptive feature fusion module, which fuses latent features F through learnable weight parameters. latent Visual semantic features F semantic and text features F text_adapted It also combines channel attention and spatial attention mechanisms to enhance the output key feature F. _out ;F final F _out The image Ir is recovered by the decoder output of the backbone network; After acquiring and preprocessing the degraded image to be processed, it is input into the SGIRN ideal model, which automatically outputs the corresponding restored image.

2. The semantically guided multi-degradation image restoration method according to claim 1, characterized in that, The preprocessing includes, but is not limited to, random cropping, flipping, rotation, brightness and contrast adjustment, size adjustment, and normalization; the multimodal degradation image pair consists of complex background images of multiple weather scenes and their corresponding clear images.

3. The semantically guided multi-degradation image restoration method according to claim 1, characterized in that, The multimodal feature extraction module includes: (1) Visual feature extraction submodule, which uses CLIP's image encoder to extract the input degraded image I. d Visual semantic features F semantic And adjust the feature dimensions through a semantic feature adapter; (2) Degradation type recognition submodule: calculates the similarity between the input image and the preset degradation type text template to determine the most likely degradation type; (3) Text feature extraction submodule: Based on the identified degradation type, dynamic text prompts are dynamically generated, and text features F are extracted through CLIP text encoder. text_adapted .

4. The semantically guided multi-degradation image restoration method according to claim 3, characterized in that, The generation of dynamic text prompts includes: For rainwater degradation, generate a text prompt of the type "clear image, no rain, high visual clarity"; For fog degradation, generate a text prompt of the type "fog-free image, high visibility, and saturated colors"; For snow scene degradation, generate a text prompt of the type "Clear image without snowflakes, rich in detail".

5. The semantically guided multi-degradation image restoration method according to claim 1, characterized in that, The degradation feature extraction and fusion module includes: (1) Multi-scale feature extraction submodule: Uses three different sizes of convolution kernels (3×3, 5×5, and 7×7) to extract the input degraded image I in parallel. d Multi-scenario weather features at different scales F multi ; (2) Feature Pyramid Submodule: A feature pyramid is constructed through max pooling and upsampling operations to enhance feature representation at different levels; First convolutional layer extracts features F pyramid1 =Conv(F multi ); Max pooling is used to reduce the feature map size, followed by convolution to capture a wider range of contextual information, resulting in the first layer of convolutional feature extraction F. pyramid2 =Conv(MaxPool(F pyramid1 )); The features from the two feature pyramid layers are combined to generate a degenerate feature representation F. deg =Concat(F pyramid1 Upsample(F pyramid2 (3) Dynamic fusion submodule, which generates dynamic weights through convolutional network to adaptively adjust the fusion ratio of image features and degradation features; Image latent features F latent and degradation characteristics F deg The input is fed into a convolutional layer to generate two weights w1 and w2: w1, w2 = Split(Conv(Concat(F)) latent ,F deg ))) Weighted fusion yields the fused feature F fused =w1·F latent +w2·F deg Further processing through the channel attention layer yields the important channel enhanced representation features F. ca =F fused ·σ(MLP(Concat(AvgPool(F fused ),MaxPool(F fused )))); (3) Introduce residual connections: Output fusion feature F final =F ca +F latent .

6. The semantically guided multi-degradation image restoration method according to claim 1, characterized in that, The adaptive feature fusion module fuses latent features F latent Visual semantic features F semantic and text features F text_adapted Represented as: F m_fused =w latent ·F latent +w text ·F text_adapted +w semantic ·F semantic Among them, w latent w text w semantic These are the corresponding learnable scalar weight parameters.

7. The semantically guided multi-degradation image restoration method according to claim 1, characterized in that, Loss functions include: Calculate the reconstruction loss, and use L1 loss to measure the pixel-level difference between the restored image and the real image; Calculate the CLIP loss and use the CLIP model to calculate the cosine similarity between the restored image and the real image in the CLIP feature space; A total loss function is constructed by combining reconstruction loss and CLIP loss, which is used for network training optimization.

8. The semantically guided multi-degradation image restoration method according to claim 1, characterized in that, The method supports processing image degradation types including: rain degradation, snow degradation, fog degradation, low light degradation, and motion blur degradation.

Images