Recovery method for multi-weather mixed degraded image

Abstract

The invention discloses a method for recovering a multi-weather mixed degraded image, and belongs to the technical field of image processing. The method comprises the following steps: mapping each pixel point of a multi-weather mixed degraded image to a high-dimensional feature space by using a shallow feature extraction module to obtain shallow features; performing high and low frequency decomposition on the shallow layer features by using a wavelet convolution coding module, and obtaining deep layer features based on low frequency features; processing the deep features by using a prompt generation module to obtain weather prompt deep features including weather prompt information; gradually recovering the weather prompt deep features into a high-dimensional feature map with the same size as the multi-weather mixed degraded image input by using an up-sampling convolution decoding module; and mapping the high-dimensional feature map into a three-dimensional RGB recovery map by using an image reconstruction module, and taking the three-dimensional RGB recovery map as the output of the image processing system. The method is wide in coverage and high in expansibility, is not limited to specific weather scenes, and can effectively restore the multi-weather mixed degraded image.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method for restoring multi-weather mixed degraded images. Background Art

[0002] Scenes in the real world are often affected by various adverse weather conditions, such as fog, haze, rain, snow, etc. These factors can distort images and reduce visibility. Restoring degraded images under different weather conditions is crucial for enhancing the perception ability of autonomous systems. With the development of deep learning technology, significant progress has been made in the field of image restoration. However, most image restoration algorithms are only applicable to single weather conditions, which limits their effectiveness in real-world scenarios where multiple weather conditions often coexist. Single-weather condition restoration methods usually perform poorly, as Figure 1 shown.

[0003] To address the impact of multi-weather degradation conditions, some meaningful work has emerged.

[0004] For example, All-in-One first introduced the multi-weather restoration task and also proposed a unified framework that uses task-specific encoders and decoders, further optimized through neural architecture search to achieve effective feature fusion without the need to create specific methods for each type of weather restoration task. However, using neural architecture search and multiple encoders incurs high computational costs.

[0005] Another example is TransWeather, which uses the Transformer model and adopts a Transformer-based encoder and decoder structure to improve performance. In the encoder stage, the attention mechanism can be used to extract weather features, and in the decoder stage, the weather features are combined for adjustment. However, the Transformer-based restoration method may have difficulty capturing the global context and dependencies of degraded images due to the need to divide the image into blocks, which has a negative impact on the restoration results.

[0006] Yet another example is WeatherDiff, which introduces the diffusion model into the multi-weather restoration task and proposes a "block"-based conditional diffusion method. By using smooth noise estimation during the inference process to guide the denoising process, image restoration is achieved. However, using a multi-step sampling diffusion model significantly increases the model parameters and requires a large amount of computational resources, leading to challenges in terms of training complexity and inference time.

[0007] Existing restoration models mostly target the degradation factors of a single weather. However, in real-world scenarios, images are usually affected by a combination of multiple degradation factors, and there is currently little research on enhancing such mixed degradation. There are still technical problems in existing research, such as large parameter numbers, low interpretability, and low scalability in the methods for enhancing mixed degradation. Summary of the Invention

[0008] To address the above technical problems, starting from the goal of improving the recognition accuracy of the AI restoration algorithm under multi-weather mixed degradation, the present invention proposes a restoration scheme for multi-weather mixed degradation images, which aims to enhance the performance of the AI restoration algorithm under multi-weather mixed degradation. This scheme has a wide coverage and strong scalability, is no longer limited to specific weather scenarios, and can effectively restore multi-weather mixed degradation images.

[0009] The first aspect of the present invention discloses a method for restoring multi-weather mixed degradation images. The method uses an image processing system to restore multi-weather mixed degradation images. The image processing system includes a shallow feature extraction module, a wavelet convolutional coding module, a prompt generation module, an upsampling convolutional decoding module, and an image reconstruction module. The method includes:

[0010] Step S1: After the multi-weather mixed degradation image is input into the image processing system, the shallow feature extraction module maps each pixel point of the multi-weather mixed degradation image to a high-dimensional feature space to obtain shallow features.

[0011] Step S2: The wavelet convolutional coding module performs high-frequency and low-frequency decomposition on the shallow features to obtain deep features based on the low-frequency features.

[0012] Step S3: The prompt generation module processes the deep features to obtain weather prompt deep features containing weather prompt information.

[0013] Step S4: The upsampling convolutional decoding module gradually restores the weather prompt deep features to a high-dimensional feature map with the same size as the input of the multi-weather mixed degradation image.

[0014] Step S5: The image reconstruction module maps the high-dimensional feature map into a three-dimensional RGB restoration image, and uses the three-dimensional RGB restoration image as the output of the image processing system.

[0015] According to the method of the first aspect of the present invention, in step S1, the shallow feature extraction module maps each pixel point of the multi-weather mixed degradation image to a high-dimensional feature space; specifically, it includes: the multi-weather mixed degradation image is X, the feature size of X is 3×H×W, H is the height of the image, and W is the width of the image. A convolutional layer with a 3×3 convolutional kernel in the shallow feature extraction module is used to perform the mapping operation to extract shallow features.

[0016] According to the method of the first aspect of the present invention, in step S2, the wavelet convolutional coding module is a wavelet convolutional encoder for processing shallow features; wherein, for the wavelet convolutional encoder, based on the U-shaped network coding structure, Haar wavelet convolution is added after each coding block for frequency separation to obtain low-frequency features and high-frequency features, and the low-frequency features are sent to the next coding block for continued processing, while the high-frequency features are subjected to skip connections.

[0017] According to the method of the first aspect of the present invention, in step S3, the prompt generation module receives the deep features from the wavelet convolutional coding module, obtains weather prompts through the weather prompt network, and obtains weather prompt deep features that can guide the encoder for progressive recovery; wherein:

[0018] For the deep features obtained by the wavelet convolutional coding module, its size is d×H×W, where d represents the number of channels. After the deep features pass through the global average pooling operation, a vector of size d is obtained. Subsequently, after passing through a 1×1 convolutional layer, a weather prompt vector is obtained, which is then sent to a fully connected network and the Softmax function, and a vector of size N is generated through mapping, corresponding to N weather types;

[0019] Using the CrossEntropy loss as the loss function to constrain the learning of weather prompts, so that the weather prompt vector conforms to the actual degraded weather scene of the input image; the weather prompt deep features are obtained by fusing the weather prompt with the deep features.

[0020] According to the method of the first aspect of the present invention, in step S4, the upsampling convolutional decoding module is used to decode and recover the weather prompt deep features to the same size as the multi-weather mixed degraded image; wherein:

[0021] Based on the structure of the U-shaped network, the upsampling convolutional decoding module performs skip connections between the high-frequency features obtained by the wavelet convolutional encoder and the decoding features of the same size after each decoding block, uses the high-frequency features to improve the texture information, uses a 1×1 convolutional layer to fuse the high-frequency features and the decoding block features, and then uses the upsampling block to double the size and halve the dimension of the fused features, and sends them to the next decoding block. The high-dimensional feature map obtained by the last decoding block has the same size as the multi-weather mixed degraded image.

[0022] According to the method of the first aspect of the present invention, in step S5, through the spanned long connection, the high-frequency information in the shallow features and the low-frequency information in the deep features are utilized and fused, so that the image reconstruction module learns more information to reconstruct the image; wherein:

[0023] The image reconstruction module splices and fuses the high-dimensional feature map output by the upsampling convolutional decoding module and the shallow features output by the shallow feature extraction module to form cross-connected residual features; the residual features are sequentially input into two convolutional layers with convolutional kernel sizes of 3×3 and 3×3 respectively, and then a restored image with the same size and dimension as the multi-weather mixed degraded image is output.

[0024] The second aspect of the present invention discloses a restoration system for multi-weather mixed degraded images. The system includes a shallow feature extraction module, a wavelet convolutional encoding module, a hint generation module, an upsampling convolutional decoding module, and an image reconstruction module, wherein:

[0025] The shallow feature extraction module is configured to map each pixel point of the multi-weather mixed degraded image to a high-dimensional feature space to obtain shallow features;

[0026] The wavelet convolutional encoding module is configured to perform high-low frequency decomposition on the shallow features and obtain deep features based on the low-frequency features;

[0027] The hint generation module is configured to process the deep features to obtain weather hint deep features containing weather hint information;

[0028] The upsampling convolutional decoding module is configured to gradually restore the weather hint deep features to a high-dimensional feature map with the same size as the input of the multi-weather mixed degraded image;

[0029] The image reconstruction module is configured to map the high-dimensional feature map into a three-dimensional RGB restored image, and use the three-dimensional RGB restored image as the output of the image processing system.

[0030] The third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, it implements the method for restoring a multi-weather mixed degraded image described in the first aspect of the present disclosure.

[0031] The fourth aspect of the present invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the method for restoring a multi-weather mixed degraded image described in the first aspect of the present disclosure.

[0032] In summary, the technical solution proposed by the present invention is used to restore the mixed degraded images caused by the superposition of multiple weather factors. Since this solution is completely based on a convolutional neural network, the inference speed is greatly improved. This solution includes a shallow feature extraction module, a wavelet convolutional encoding module, a hint generation module, an upsampling convolutional decoding module, and an image reconstruction module. Among them, the wavelet convolutional encoder can utilize the characteristics of wavelet transform to separate the high and low frequencies of feature information, which can not only effectively extract the key information of the image but also retain the texture information of the image, facilitating the improvement of the image restoration quality. The hint generation module can adaptively extract the degradation hint vector of the degraded weather and learn the degraded weather type of the image, thereby enhancing the sensitivity of the model to the degraded features and guiding the decoder to better perform reconstruction and restoration according to the degraded weather type. The upsampling convolutional decoder is used to restore the deep features to the size of the original degraded image. During the process of restoring the deep features of the weather hint, the decoder utilizes the high-frequency texture features extracted in the encoding stage, which is beneficial to improving the image restoration quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 It is a comparison schematic diagram of single-weather restoration and multi-weather restoration in the prior art;

[0035] Figure 2 It is a schematic structural diagram of an image processing system according to an embodiment of the present invention;

[0036] Figure 3 It is a schematic diagram of a wavelet convolutional encoder according to an embodiment of the present invention;

[0037] Figure 4 It is a schematic diagram of a hint generation module according to an embodiment of the present invention.

[0038] Figure 5 It is a schematic diagram of the fusion of weather hint and deep features according to an embodiment of the present invention.

[0039] Figure 6 It is a schematic diagram of the process of restoring multi-weather mixed degraded images according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0041] The first aspect of the present invention discloses a method for restoring a multi-weather mixed degraded image. The method uses an image processing system to restore the multi-weather mixed degraded image. The image processing system includes a shallow feature extraction module, a wavelet convolutional coding module, a hint generation module, an upsampling convolutional decoding module, and an image reconstruction module. The method includes:

[0042] Step S1: After the multi-weather mixed degraded image is input into the image processing system, the shallow feature extraction module maps each pixel point of the multi-weather mixed degraded image to a high-dimensional feature space to obtain shallow features.

[0043] Step S2: The wavelet convolutional coding module performs high-frequency and low-frequency decomposition on the shallow features to obtain deep features based on the low-frequency features.

[0044] Step S3: The hint generation module processes the deep features to obtain weather hint deep features containing weather hint information.

[0045] Step S4: The upsampling convolutional decoding module gradually restores the weather hint deep features to a high-dimensional feature map of the same size as the input of the multi-weather mixed degraded image.

[0046] Step S5: The image reconstruction module maps the high-dimensional feature map into a three-dimensional RGB restored image, and uses the three-dimensional RGB restored image as the output of the image processing system.

[0047] According to the method of the first aspect of the present invention, in step S1, the shallow feature extraction module maps each pixel point of the multi-weather mixed degraded image to a high-dimensional feature space; specifically, the multi-weather mixed degraded image is X, the feature size of X is 3×H×W, H is the height of the image, and W is the width of the image. A convolutional layer with a 3×3 convolutional kernel in the shallow feature extraction module is used to perform the mapping operation to extract shallow features.

[0048] According to the method of the first aspect of the present invention, in step S2, the wavelet convolutional coding module is a wavelet convolutional encoder for processing shallow features; wherein, for the wavelet convolutional encoder, on the basis of the U-shaped network coding structure, Haar wavelet convolution is added after each coding block for frequency separation to obtain low-frequency features and high-frequency features, and the low-frequency features are sent to the next coding block for further processing, while the high-frequency features are subjected to skip connections.

[0049] According to the method of the first aspect of the present invention, in step S3, the prompt generation module receives the deep features from the wavelet convolutional coding module, obtains weather prompts through the weather prompt network, and obtains weather prompt deep features that can guide the encoder for progressive recovery; wherein:

[0050] For the deep features obtained by the wavelet convolutional coding module, its size is d×H×W, where d represents the number of channels. After the deep features pass through the global average pooling operation, a vector of size d is obtained. Subsequently, after passing through a 1×1 convolutional layer, a weather prompt vector is obtained, which is then sent to a fully connected network and the Softmax function, and a vector of size N is generated through mapping, corresponding to N weather types;

[0051] Using the CrossEntropy loss as the loss function to constrain the learning of weather prompts, so that the weather prompt vector conforms to the actual degraded weather scene of the input picture; the weather prompt deep features are obtained by fusing the weather prompts with the deep features.

[0052] According to the method of the first aspect of the present invention, in step S4, the upsampling convolutional decoding module decodes and restores the weather prompt deep features to the same size as the multi-weather mixed degraded image; wherein:

[0053] On the basis of the structure of the U-shaped network, the upsampling convolutional decoding module performs skip connections between the high-frequency features obtained by the wavelet convolutional encoder and the decoding features of the same size after each decoding block, uses the high-frequency features to improve the texture information, uses a 1×1 convolutional layer to fuse the high-frequency features and the decoding block features, and then uses the upsampling block to double the size and halve the dimension of the fused features, and sends them to the next decoding block. The high-dimensional feature map obtained by the last decoding block has the same size as the multi-weather mixed degraded image.

[0054] According to the method of the first aspect of the present invention, in step S5, through the spanned long connection, the high-frequency information in the shallow features and the low-frequency information in the deep features are utilized and fused, so that the image reconstruction module can learn more information to reconstruct the image; wherein:

[0055] The image reconstruction module splices and fuses the high-dimensional feature map output by the upsampling convolutional decoding module and the shallow features output by the shallow feature extraction module to form cross-connected residual features; the residual features are sequentially input into two convolutional layers with convolutional kernel sizes of 3×3 and 3×3 respectively, and then a restored image with the same size and dimension as the multi-weather mixed degraded image is output.

[0056] The first embodiment

[0057] The hybrid degradation enhancement method proposed by the present invention includes five parts: a shallow feature extraction module, a wavelet convolutional encoding module, a hint generation module, an upsampling convolutional decoding module, and an image reconstruction module. As Figure 2 shown, after the degraded image is input into the system, first, the shallow feature extraction module maps each pixel point of the input image to a high-dimensional feature space to obtain shallow features; then the wavelet convolutional encoding module performs high-frequency and low-frequency decomposition on the shallow features to obtain deep features; the low-frequency deep features pass through the hint generation module to obtain weather hint deep features containing weather hints; the weather hint deep features are gradually restored to a high-dimensional feature map with the same size as the input picture through the upsampling convolutional decoding module; finally, the image reconstruction module maps the high-dimensional feature map into a three-dimensional RGB restored image.

[0058] The second embodiment

[0059] Shallow feature extraction module: used to map each pixel of the input image to a high-dimensional feature space. Using a convolutional layer can greatly improve the optimization stability and peak performance. Therefore, given a mixed degraded image X, the feature size of X is 3×H×W, where H is the height of the image and W is the width of the image, and a convolutional kernel with a size of 3×3 is used to extract shallow features.

[0060] Wavelet convolutional encoding module: This patent designs a wavelet convolutional encoder for processing shallow features. Based on the U-shaped network encoding structure, Haar wavelet convolution is added after each encoding block to perform frequency separation to obtain low-frequency features and high-frequency features. The low-frequency features are sent to the next encoding block for continued processing, and the high-frequency features are subjected to skip connections, as Figure 3 shown.

[0061] Hint generation module: This module receives the deep features obtained from the wavelet convolutional encoding module and obtains weather hints through the designed weather hint network to obtain progressive restoration that can guide the encoder, as Figure 4 shown.

[0062] For the low-frequency deep features obtained by the wavelet convolution coding module, the input features (d×H×W) are subjected to global average pooling operation to obtain a vector of d. First, it passes through a 1×1 convolutional layer to obtain a weather hint vector, and then is fed into a fully connected network and mapped by the Softmax function to generate a vector of N, that is, corresponding to N weather types. Using the CrossEntropy loss as the loss function to constrain the learning of the weather hint, so that the weather hint vector can conform to the actual degraded weather scene of the input picture.

[0063] Fuse the described weather hint with the low-frequency deep features, as Figure 5 shown, and then input the deep features containing the weather hint after fusion into the upsampling convolutional decoding module to decode and restore the feature information until it is restored to the same size as the input degraded image.

[0064] Upsampling convolutional decoding module: This patent designs an upsampling convolutional decoder for restoring deep features to the size of the original degraded image. Based on the structure of the U-shaped network, after each decoding block, the high-frequency deep features obtained by the wavelet convolutional encoder are jump-connected with the decoding features of the same size to effectively utilize the high-frequency information to improve the texture information. After using a 1×1 convolutional layer to fuse the high-frequency deep features and the decoding block features, the upsampling block is used to double the size and halve the dimension of the features and send them to the next decoding block. The feature map obtained by the last decoding block has the same size as the original degraded image.

[0065] Image reconstruction module: The features output from the upsampling convolutional decoding module are concatenated and fused with the shallow features obtained by the shallow convolutional coding module to form a cross-connected residual feature. This residual feature is sequentially input into two convolutional layers with convolutional kernel sizes of 3×3 and 3×3 respectively, and a restored image with the same size and dimension as the input image is output. Through a spanning long connection, the high-frequency information in the shallow features and the low-frequency information in the deep features are effectively utilized and fused, enabling the image reconstruction module to learn more information to reconstruct the image.

[0066] The entire invention details of the proposed method are as Figure 6 shown.

[0067] The second aspect of the present invention discloses a system for restoring multi-weather mixed degraded images. The system includes a shallow feature extraction module, a wavelet convolution coding module, a hint generation module, an upsampling convolutional decoding module, and an image reconstruction module, where:

[0068] The shallow feature extraction module is configured to map each pixel point of the multi-weather mixed degraded image to a high-dimensional feature space, thereby obtaining shallow features;

[0069] The wavelet convolutional coding module is configured to: perform high-frequency and low-frequency decomposition on the shallow features, and obtain deep features based on the low-frequency features;

[0070] The hint generation module is configured to: process the deep features to obtain weather hint deep features containing weather hint information;

[0071] The upsampling convolutional decoding module is configured to: gradually restore the weather hint deep features to a high-dimensional feature map of the same size as the input of the multi-weather mixed degraded image;

[0072] The image reconstruction module is configured to: map the high-dimensional feature map into a three-dimensional RGB restored image, and use the three-dimensional RGB restored image as the output of the image processing system.

[0073] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, a method for restoring a multi-weather mixed degraded image according to the first aspect of the present disclosure is implemented.

[0074] A fourth aspect of the present invention discloses a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, a method for restoring a multi-weather mixed degraded image according to the first aspect of the present disclosure is implemented.

[0075] In summary, the technical solution proposed by the present invention is used to restore a mixed degraded image caused by the superposition of multiple weather factors. Since this solution is completely based on a convolutional neural network, the inference speed is greatly improved. This solution includes a shallow feature extraction module, a wavelet convolutional coding module, a hint generation module, an upsampling convolutional decoding module, and an image reconstruction module. Among them, the wavelet convolutional encoder can use the characteristics of wavelet transform to separate high-frequency and low-frequency feature information, which can not only effectively extract the key information of the image, but also retain the texture information of the image, which is beneficial to improving the restoration quality of the image. The hint generation module can adaptively extract the degradation hint vector of the degraded weather and learn the degraded weather type of the image, so as to enhance the sensitivity of the model to the degraded features, and guide the decoder to perform better reconstruction and restoration according to the degraded weather type. The upsampling convolutional decoder is used to restore the deep features to the size of the original degraded image. During the process of restoring the weather hint deep features, the decoder uses the high-frequency texture features extracted in the encoding stage, which is beneficial to improving the restoration quality of the image.

[0076] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope described in this specification. The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A method for restoring a mixed degraded image under multiple weather conditions, characterized in that: The method uses an image processing system to restore a multi-weather mixed degraded image, wherein the image processing system comprises a shallow feature extraction module, a wavelet convolution encoding module, a prompt generation module, an upsampling convolution decoding module and an image reconstruction module, and the method comprises: Step S1: After the multi-weather mixed degraded image is input into the image processing system, the shallow feature extraction module is used to map each pixel of the multi-weather mixed degraded image to a high-dimensional feature space, thereby obtaining shallow features; Step S2: using a wavelet convolutional coding module to decompose the shallow features into high and low frequencies, and obtaining deep features based on the low-frequency features; Step S3, using the prompt generation module to process the deep features to obtain weather prompt deep features containing weather prompt information; Step S4, using an upsampling convolutional decoding module to gradually restore the weather prompt deep features into a high-dimensional feature map of the same size as the multi-weather mixed degraded image input; Step S5: Use an image reconstruction module to map the high-dimensional feature map into a three-dimensional RGB restoration map, and use the three-dimensional RGB restoration map as the output of the image processing system.

2. The method for restoring a mixed degraded image under multiple weather conditions according to claim 1, characterized in that: In step S1, each pixel point of the multi-weather mixed degraded image is mapped to a high-dimensional feature space using a shallow feature extraction module; specifically, the multi-weather mixed degraded image is X, the feature size of X is 3×H×W, H is the height of the image, and W is the width of the image. The convolution layer with a 3×3 convolution kernel in the shallow feature extraction module is used to perform the mapping operation, thereby extracting shallow features.

3. The method for restoring a mixed degraded image under multiple weather conditions according to claim 2, characterized in that: In step S2, the wavelet convolution coding module is a wavelet convolution encoder, which is used to process shallow features; wherein, for the wavelet convolution encoder, based on the U-type network coding structure, Haar wavelet convolution is added after each coding block for frequency separation to obtain low-frequency features and high-frequency features, and the low-frequency features are sent to the next coding block for further processing, while the high-frequency features are jump-connected.

4. The method for restoring a multi-weather mixed degraded image according to claim 3, characterized in that: In step S3, the prompt generation module receives the deep features from the wavelet convolutional coding module, obtains the weather prompts through the weather prompt network, and obtains the deep features of the weather prompts that can guide the encoder to perform progressive recovery; wherein: The deep features obtained by the wavelet convolutional coding module have a size of d×H×W, where d represents the number of channels. The deep features are subjected to global average pooling to obtain a vector of size d, which is then subjected to a 1×1 convolutional layer to obtain a weather prompt vector, which is then sent to the fully connected network and the Softmax function to generate a vector of size N through mapping, corresponding to N types of weather. CrossEntropy loss is used as the loss function to constrain the learning of weather prompts, so that the weather prompt vector conforms to the actual degraded weather scene of the input image; the weather prompt deep features are obtained by fusing the weather prompts with the deep features.

5. The method for restoring a mixed degraded image under multiple weather conditions according to claim 4, characterized in that: In step S4, the upsampling convolutional decoding module is used to decode and restore the deep features of the weather prompt to the same size as the multi-weather mixed degraded image; wherein: Based on the structure of the U-type network, the upsampling convolution decoding module jumps the high-frequency features obtained by the wavelet convolution encoder with the decoding features of the same size after each decoding block, uses the high-frequency features to improve the texture information, and uses a 1×1 convolution layer to fuse the high-frequency features and the decoding block features. Then, the upsampling block is used to enlarge the size of the fused features by two times and reduce the dimension by two times, and then sent to the next decoding block. The high-dimensional feature map obtained by the last decoding block is the same size as the multi-weather mixed degraded image.

6. The method for restoring a mixed degraded image under multiple weather conditions according to claim 5, characterized in that: In step S5, through the long connection across, the high-frequency information in the shallow features and the low-frequency information in the deep features are utilized and fused, so that the image reconstruction module learns more information to reconstruct the image; wherein: The image reconstruction module concatenates and fuses the high-dimensional feature map output by the upsampling convolution decoding module with the shallow features output by the shallow feature extraction module to form a cross-connected residual feature; the residual feature is sequentially input into two convolutional layers with convolution kernel sizes of 3×3 and 3×3 respectively, and then outputs a restored image with the same size and dimension as the multi-weather mixed degraded image.

7. A restoration system for mixed degraded images in multiple weather conditions, characterized in that: The system includes a shallow feature extraction module, a wavelet convolution encoding module, a hint generation module, an upsampling convolution decoding module and an image reconstruction module, wherein: The shallow feature extraction module is configured to: map each pixel of the multi-weather mixed degraded image to a high-dimensional feature space, thereby obtaining shallow features; The wavelet convolutional coding module is configured to: decompose the shallow features into high and low frequencies, and obtain the deep features based on the low-frequency features; The prompt generation module is configured to: process the deep features to obtain weather prompt deep features containing weather prompt information; The upsampling convolutional decoding module is configured to: gradually restore the weather prompt deep features to a high-dimensional feature map of the same size as the multi-weather mixed degraded image input; The image reconstruction module is configured to: map the high-dimensional feature map into a three-dimensional RGB restoration map, and use the three-dimensional RGB restoration map as the output of the image processing system.

8. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the method for restoring a multi-weather mixed degraded image as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for restoring a multi-weather mixed degraded image according to any one of claims 1 to 6 is implemented.

Images