Image restoration method and system coping with various severe weather environment factors

Abstract

The invention discloses an image restoration method and system coping with various severe weather environment factors, and relates to the technical field of image restoration, and the method comprises the steps: recognizing an image weather type, and carrying out the test time self-adaption of a model; inputting an image into the adaptive model to obtain a feature vector of the image, and decoding the feature vector through a decoder; and transmitting the image to a soft reconstruction layer to obtain a restored image. According to the invention, a supervised learning mode is adopted to train a universal network capable of coping with multiple weathers. A soft prompt excitation model is introduced in the fine tuning stage to enhance the generalization ability to cope with different weathers. Explicit and implicit interaction is utilized to enhance the prompt effect, hidden information of prompts is explored through low-rank decomposition, and by adding contrast loss, prompts approach to similar tasks and leave away from opposite tasks, and the feature coding capacity is improved. According to the method, the influence of various severe weathers on the image can be effectively processed, the image quality is improved, and high-quality input is provided for subsequent defect identification and auxiliary analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of power system image restoration, and specifically to an image restoration method and system for coping with various adverse weather environmental factors. Background Art

[0002] Image acquisition technology is widely used in key scenarios such as transmission line inspection, substation monitoring, and distribution equipment status perception in the power system. The quality of the acquired images directly affects the accuracy and reliability of subsequent fault identification, defect detection, and risk assessment. However, in practical applications, most power facilities are deployed in outdoor environments and are easily interfered by various adverse weather factors, such as heavy rainfall, snow, thick fog, and dust. These meteorological conditions will significantly reduce the image clarity, resulting in problems such as blurring, occlusion, and low contrast, which severely restrict the performance of image-driven intelligent decision-making systems.

[0003] Although certain progress has been made in current research on image restoration under adverse weather conditions, there are still multiple key challenges. On the one hand, many existing methods are only designed for specific types of weather (such as only defogging or de-raining), lacking the ability to uniformly model multiple complex weather scenarios; on the other hand, some methods rely on additional physical priors or multi-modal sensors, resulting in increased system complexity and cost. In addition, due to the training data mostly being simulated degraded images, there are differences in the distribution of actual acquired images in the power scenario, and the generalization ability and stability of the restoration model in the real environment still need to be improved. Therefore, there is an urgent need for a highly robust image restoration method and system applicable to various adverse weather conditions in the power system, mainly to solve the problem that the acquired images are affected due to the influence of adverse weather in the power background, such as rain, snow, fog, raindrops, etc., so as to enhance the practicability and reliability of the perception system in complex environments. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: how to overcome the limitations of existing technologies that can only perform image restoration for a single type of weather, or require additional structures to increase costs, and the differences between training data and real images that affect the application ability of the model.

[0006] To solve the above technical problem, the present invention provides the following technical solution: An image restoration method for coping with various adverse weather environmental factors, which includes the following steps: Identify the image weather type and perform test-time adaptation on the model; Input the image into the adapted model, and obtain the feature vector of the image through the encoder, where the encoder includes a convolutional layer, three neural network layers, and two downsampling layers; According to the image feature vector, decoding is performed through a decoder, where the decoder includes two upsampling layers, two feature fusion layers, two neural network layers, and one convolutional layer; The obtained output enters the soft reconstruction layer to obtain the restored image.

[0007] As a preferred solution of the image restoration method for coping with various adverse weather environmental factors according to the present invention, wherein: the recognition of the image weather type and the test-time adaptation of the model include, For the input degraded image The specific degradation type of the input image is identified through a weather discriminator, and a secondary degraded image is constructed according to the degradation type The obtained paired images are input to the model, and As the input degraded image, and As the input clean image, the model is adaptively tested at the test time.

[0008] As a preferred solution of the image restoration method for coping with various adverse weather environmental factors according to the present invention, wherein: the obtaining of the image feature vector through the encoder includes refining and strengthening the features extracted from the input image through the first neural network layer, with a shape of , where represents the height, represents the width, represents the number of channels; According to the obtained output, through a downsampling layer and a neural network layer, the spatial dimension of the feature map is reduced through downsampling, The degraded image is input to the adaptively adjusted model, through a 3×3 convolutional layer for initial feature extraction to capture higher-level abstract features, and through the neural network layer to continue processing the feature information at a deeper level, and as the level deepens, more complex and more abstract features are captured, and the shape is , where is a real number; According to the output obtained from the previous layer, through the downsampling layer and the neural network layer again, the spatial dimension of the feature map is reduced through downsampling to capture higher-level abstract features, and through the neural network layer to continue processing the feature information at a deeper level, and as the level deepens, more complex and more abstract features are captured, and the shape is ; The neural network layer is composed of multiple attention network modules. Each attention network module first undergoes layer normalization, which is related to the mean and standard deviation of the image and brightness and contrast. The overall expression of layer normalization is: , Among them, x represents the input feature, respectively representing the mean and variance, , representing the scaling factor and the bias term, and representing the parameters of two linear layers, respectively adjusting the reserved mean and variance, and the final output is the normalization result; Calculate the multi-head attention, perform reflection padding and then calculate the attention for central cropping, and pass it through the MLP layer to the next attention network module.

[0009] As a preferred solution of an image restoration method for coping with various adverse weather environment factors according to the present invention, wherein: the decoding by the decoder includes, According to the finally obtained features, they pass through an upsampling layer, a selective convolution fusion layer and a neural network layer in sequence. The upsampling layer is used to increase the spatial size of the feature map and restore the details and size of the image; the selective convolution fusion layer fuses information from different levels by selectively weighting different features; the neural network layer processes and optimizes the fused features, and the current feature map shape is ; The feature map passes through an upsampling layer, a selective convolution fusion layer and a neural network layer again in sequence. The image details and resolution are further restored through upsampling, and different-level features are continuously fused by the selective convolution fusion layer, and are represented by the optimization of the neural network layer. The current feature map shape is ; Finally, it passes through a 3×3 convolution layer to generate the final output of the model .

[0010] As a preferred solution of an image restoration method for coping with various adverse weather environment factors according to the present invention, wherein: the soft reconstruction includes, Performing soft reconstruction on the finally obtained model output to obtain the estimated transmission map and the residual , and calculating to obtain the restored image.

[0011] As a preferred solution of an image restoration method for coping with various adverse weather environment factors according to the present invention, wherein: the calculating to obtain the restored image includes constructing a unified formula, and the expression is: , , , wherein, the weather-damaged image , the original clean image , represents the residual term, is the atmospheric light in the scene, is the transmission map generated by atmospheric scattering.

[0012] As a preferred solution of an image restoration method for coping with various adverse weather environmental factors according to the present invention, wherein: the obtained output enters the soft reconstruction layer, and the restored image further includes, In the model training stage, the model training is completed through a two-stage training strategy; In the pre-training stage, a shared backbone network is adopted, and a mixed dataset from various weather conditions is used to learn the general weather features in a supervised manner. The encoder and decoder are trained according to the loss function in the pre-training stage. In the fine-tuning stage, task-specific soft prompts are introduced to enhance the model's ability to handle specific weather degradations. In the fine-tuning stage, the parameters of the backbone are frozen, a contrastive loss is introduced, and the soft prompts are trained according to the loss function.

[0013] Another object of the present invention is to provide an image restoration system for coping with various adverse weather environmental factors, which can solve the problems of poor adaptability of the existing system to various adverse weather and insufficient restoration of image details through weather type recognition, adaptive model optimization and feature fusion technologies.

[0014] To solve the above technical problems, the present invention provides the following technical solution: an image restoration system for coping with various adverse weather environmental factors, including: a weather discriminator module, an encoder module, a decoder module, and a soft reconstruction module; The weather discriminator module is to identify the specific degradation type of the input image and generate adaptive initial conditions for the model. By analyzing the features of the input image, the weather type is classified and determined, and an additional secondary degraded image is generated according to the type; The encoder module includes a convolutional layer, multiple neural network layers and a downsampling layer, which are respectively used to capture the low-level and high-level features of the image, reduce the spatial dimension through downsampling, and normalize and calculate the attention of the features through a multi-level attention network module, and gradually extract key features from the input image; The decoder module includes an upsampling layer, an SKFusion feature fusion layer and a neural network layer. The upsampling increases the spatial dimension to restore the resolution of the image. The SKFusion layer enhances the detail performance of the output by weighted fusion of the features of different layers, and the neural network layer optimizes the fused features; The soft reconstruction module is to decompose the feature vector output by the decoder, calculate the residual layer map, further adjust and optimize the image content, combine the constructed degradation expression to complete the fine-grained image restoration effect, and integrate various weather degradation factors through a unified formula to complete the comprehensive restoration.

[0015] A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of an image restoration method for coping with multiple adverse weather environment factors as described above are implemented.

[0016] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of an image restoration method for coping with multiple adverse weather environment factors as described above are implemented.

[0017] Advantages of the present invention: The present invention provides an image restoration method for coping with multiple adverse weather environment factors, overcoming the problems in the prior art that only single-weather image restoration can be performed, additional structures are required to increase costs, and the application ability of the model is affected by the difference between training data and real images. The present invention can handle image restoration of multiple adverse weathers and has good competitiveness compared with the current state-of-the-art technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is the overall flowchart of an image restoration method for coping with multiple adverse weather environment factors provided by the first embodiment of the present invention; Figure 2 It is a comparison chart of the load balance rates of multiple algorithms under the same task request in an image restoration method for coping with multiple adverse weather environment factors provided by the first embodiment of the present invention; Figure 3 It is a running time graph under different repetition times of an image restoration method for coping with multiple adverse weather environment factors provided by the first embodiment of the present invention; Figure 4 It is a comparison of the load balance rates of an image restoration method for coping with multiple adverse weather environment factors provided by the third embodiment of the present invention under different numbers of tasks. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0021] Example 1, referring to Figures 1 to 3 One embodiment of the present invention provides an image restoration method for coping with various adverse weather environment factors, including: Step 1: Identify the weather type of the image and adapt the model for test time accordingly.

[0022] For the input degraded image The specific degradation type of the input image is identified by the weather discriminator, and a secondary degraded image is constructed according to the degradation type ; According to the obtained paired images Input to the model, taking as the input degraded image and as the input clean image, so as to adapt the model for test time.

[0023] Step 2: Input the image into the model adapted in Step 1, and obtain the feature vector of the image through the encoder, where the encoder includes a convolutional layer, three neural network layers, and two downsampling layers.

[0024] First, the degraded image , representing real numbers. Input to the model adapted in Step 1, passing through a 3×3 convolutional layer for initial feature extraction, and then passing through the refinement and enhancement of the first neural network layer to extract features from the input image. At this time, the shape is ; The specific content of feature extraction by the neural network layer is as follows: The neural network layer is composed of multiple attention network modules. For each attention network module, first perform layer normalization. Since the mean and standard deviation of the image are related to brightness and contrast, LayerNorm can generally be expressed as: , where represents the mean and variance, , represents the scaling factor and bias, and two linear layers are introduced, using weights and Adjust the retained mean and variance; calculate the multi-head attention, calculate the attention after reflection padding, and finally perform central cropping. The weights of MHSA in aggregating spatial information are dynamic, but since these weights are positive values, MHSA will sum the input features, playing a smoothing role during the operation and helping to retain the low-frequency information in the image. Therefore, an additional convolutional operation on V is added to supplement the possible limitations of MHSA in spatial information aggregation and enhance the model's ability to process spatial information. The specific calculation is expressed as: , where, respectively represent the query vector, key vector, and value vector of the attention mechanism, is V without window partitioning. Therefore, both the attention mechanism is used to aggregate the information within the window and convolution is used to aggregate the information within the neighborhood, thereby improving the model's spatial understanding ability; it is passed to the next attention network module through the MLP layer.

[0025] Next, according to the output obtained from the upper layer, it passes through the downsampling layer and the neural network layer. Through downsampling, the spatial dimension of the feature map is reduced to help the model capture higher-level abstract features. Through the neural network layer, the feature information is further processed at a deeper level, and as the level deepens, the model can capture more complex and abstract features. At this time, the shape is ; Finally, according to the obtained output, it passes through the downsampling layer and the neural network layer again. Through downsampling, the spatial dimension of the feature map is reduced to help the model capture higher-level abstract features. Through the neural network layer, the feature information is further processed at a deeper level, and as the level deepens, the model can capture more complex and abstract features. At this time, the shape is .

[0026] Step 3: Refer to Figure 2 , and according to the image feature vector obtained in Step 2, it is decoded through the decoder. The decoder includes two upsampling layers, two feature fusion layers, two neural network layers, and one convolutional layer.

[0027] First, the features obtained in Step 2 sequentially pass through an upsampling layer, an SK Fusion feature fusion layer, and a neural network layer. Through the upsampling layer, the spatial dimension of the feature map is increased to help restore the details and size of the image. Through the SK Fusion feature fusion layer, feature fusion is performed by selectively weighting different features and fusing the features from different layers. Through the neural network layer, the features fused by the SK Fusion layer are processed and optimized. At this time, the shape is ; Next, according to the obtained output, it passes through an upsampling layer, an SK Fusion feature fusion layer, and a neural network layer in sequence again. The upsampling layer increases the spatial dimension of the feature map to help restore the details and size of the image. The SK Fusion feature fusion layer performs feature fusion by selectively weighting different features and fuses features from different layers. The neural network layer processes and optimizes the features fused by the SK Fusion layer. At this time, the shape is ; Finally, according to the output obtained from the upper layer, it passes through a 3×3 convolutional layer to obtain the output .

[0028] Step Four: Refer to Figure 3 , and enter the soft reconstruction layer according to the output obtained in Step Three to obtain the restored image.

[0029] Perform soft reconstruction according to the final output obtained in Step Three, and then can be split to obtain and . Substitute them into the unified formula to obtain the repaired image.

[0030] The specific content of the construction of the unified formula is as follows: Let the weather-damaged image be , and the original clean image be .

[0031] To implement a unified image restoration method, the above image degradation formula can be integrated: , where , , where represents the residual term, T is the transmission map generated by atmospheric scattering, and A represents the global illumination.

[0032] Step Five: In the model training stage, complete the model training through a two-stage training strategy.

[0033] First, in the pre-training stage, adopt a shared backbone network and use a mixed dataset from various weather conditions to learn general weather features in a supervised manner; Secondly, according to the loss function in the pre-training stage: , where is the pre-training loss function, is the predicted value of the clear image, is the label image, It is the L1 distance.

[0034] Train the encoder and decoder mentioned in Step 2 and Step 3.

[0035] Next, introduce task-specific soft prompts in the fine-tuning stage to enhance the model's ability to handle specific weather degradations.

[0036] The specific content of introducing soft prompts is as follows: Introduce a set of task-specific prompts to enhance the model's ability to handle specific tasks. Introduce attention-level prompts at the input of the MSHA layer. Assuming the prompt length is m, the formula for calculating the attention mechanism can be expressed as: , where, respectively represent the query vector, key vector, and value vector of the attention mechanism; represents the prompt vector inserted into and The prompt vector contains task-specific soft prompts for encoding task-specific semantic knowledge, which are not shared between tasks. represents the vector dimension, represents the convolution operation.

[0037] Although the impacts of different weather conditions are visually different, there are both some common patterns and features as well as task-specific features in the process of image degradation. Therefore, the soft prompts are decomposed into two parts using low-rank decomposition, which can be expressed as: , where, represents matrix multiplication, represents the task-specific prompt for the i-th weather removal task, is the task-general prompt, is the rank of the parameter matrix. The task-specific prompt is specific to each task, while the general prompt is shared among all tasks. In this way, task-specific features are incorporated into the task-specific prompt, and general knowledge between tasks is incorporated into the task-general prompt.

[0038] Finally, in the fine-tuning stage, freeze the parameters of the backbone, introduce the contrastive loss, and train the mentioned soft prompts according to the loss function. The expression is: , where, is the loss in the fine-tuning stage, , are respectively the weight of the contrastive loss and the contrastive loss.

[0039] The specific content of introducing the contrastive loss is as follows: Based on the fact that the degradation of snow and raindrops is more similar, while rain and haze are relatively similar, contrastive learning is applied to task-specific prompts to encourage the prompts corresponding to similar degradation tasks to be more closely combined, achieving a higher similarity between the prompts corresponding to the degradation tasks. On the contrary, when corresponding to degradation tasks with less clear interaction relationships, a lower similarity is expected. The contrastive loss can be expressed as: , where is the index set of all tasks having a similar relationship with the i-th task, is the cardinality of the index set of all tasks having a similar relationship with the i-th task, is the temperature coefficient, is an indicator function that takes the value of 1 when and 0 otherwise, represents the similarity between two vectors, is the contrastive loss corresponding to the i-th task, is the cosine similarity between the prompt vector of the i-th task and the prompt vectors corresponding to similar tasks, is the cosine similarity between the prompt vector of the i-th task and the prompt vectors of non-similar tasks.

[0040] Embodiment 2, an embodiment of the present invention, provides a system for an image restoration method under multiple adverse weather environment factors, including: a weather discriminator module, an encoder module, a decoder module, and a soft reconstruction module; The weather discriminator module is to identify the specific degradation type of the input image and generate adaptive initial conditions for the model. By analyzing the features of the input image, it classifies and determines the weather type, and generates an additional secondary degraded image according to the type; The encoder module includes a convolutional layer, multiple neural network layers, and a downsampling layer, which are respectively used to capture the low-level and high-level features of the image, reduce the spatial dimension through downsampling, and gradually extract key features from the input image through a multi-level attention network module for feature normalization and attention calculation; The decoder module includes an upsampling layer, an SK Fusion feature fusion layer, and a neural network layer. The upsampling increases the spatial dimension to restore the resolution of the image. The SK Fusion layer enhances the detail performance of the output by weighted fusion of features from different layers, and the neural network layer optimizes the fused features; The soft reconstruction module is to further adjust and optimize the image content by decomposing the feature vector output by the decoder and calculating the residual layer, complete the fine-grained image restoration effect by combining the constructed degradation expression, and complete the comprehensive restoration by integrating multiple weather degradation factors through a unified formula.

[0041] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., which can store program codes of various kinds.

[0042] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or used in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0043] More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection parts with one or more wirings (electronic devices), portable computer disk cartridges (magnetic devices), random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fiber devices, and portable compact disc read-only memories (CDROMs). Additionally, a computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or otherwise processing it as necessary, and then storing it in a computer memory.

[0044] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0045] Example 3, referring to Figure 4 is another embodiment of the present invention. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments. In this embodiment, experiments are respectively carried out on the existing traditional method and the method of this embodiment.

[0046] The present invention uses unconstrained datasets: Outdoor Rain, Snow100K, RESIDE - outdoor, RainDrop.

[0047] The data pre - processing steps are as follows: sample the training data of different datasets and roughly align the sample numbers of different datasets. For the image inpainting model, based on the sample numbers of the four datasets, all samples (4500 pairs) of the OutdoorRain training set are selected, 7500 pairs of images are randomly sampled from the Snow100K training set, 5000 pairs of images are randomly sampled from the RESIDE dataset, and 1500 pairs of images are cyclically sampled from the RainDrop training set. For the weather discriminator, 12000 degraded images are randomly sampled from the OutdoorRain dataset, 18000 images are randomly sampled from the Snow100K training set, 15000 images are randomly sampled from the RESIDE dataset, and 8000 images are randomly sampled from the RainDrop training set. While sampling, data augmentation is performed on the data, such as random horizontal flipping, randomly cropping part of the size, randomly adjusting the contrast, randomly adjusting the brightness, randomly adjusting the saturation, etc.

[0048] The experimental settings are as follows: Use the Adam optimizer and combine it with the cosine annealing scheme to adjust the parameters and learning rate of the neural network. In the first pre - training stage, the model is trained for 200 epochs with a batch size of 4 and an initial learning rate of 0.0002. In the second fine - tuning stage, the model is trained for 250 cycles with a batch size of 8 and an initial learning rate of 0.00002. In this experiment, random flipping is applied to enhance the images, and the images are randomly cropped into blocks of size 256 x 256. In the adaptive stage during testing, the batch size is 8 and the learning rate used is 0.00002.

[0049] The present invention uses quantitative comparison and visual analysis to judge the model effect.

[0050] Quantitative comparison is carried out by comparing the Peak Signal to Noise Ratio (PSNR) and the Structural Similarity Index (SSIM).

[0051] Table 1: Quantitative comparison on the Outdoor Rain dataset , In the comparison of rain removal methods: As shown in Table 1, single rain removal methods and multi-weather restoration methods are presented. Among the single rain removal methods, the quantitative results of Pix2pix, HRGAN, and MPRNet are shown. For the multi-weather restoration methods, the quantitative results of All-in-one, TransWeather, Chen et al., and WGWS-Net are shown.

[0052] Table 2: Quantitative comparison on the Snow100K dataset , In the comparison of snow removal methods: As shown in Table 2, single snow removal methods and multi-weather restoration methods are presented. Among the single snow removal methods, the quantitative results of DetailsNet, DesnowNet, JSTASR, and DDMSNET are shown. For the multi-weather restoration methods, the quantitative results of All-in-one, TransWeather, Chen et al., and WGWS-Net are shown.

[0053] Table 3: Quantitative comparison on the RESIZE-outdoor dataset , In the comparison of fog removal methods: As shown in Table 3, single fog removal methods and multi-weather restoration methods are presented. Among the single fog removal methods, the quantitative results of EPDN, PFDN, KDDN, and MPRNet are shown. For the multi-weather restoration methods, the quantitative results of All-in-one, TransWeather, Chen et al., and WGWS-Net are shown.

[0054] Table 4: Quantitative comparison on the RainDrop dataset , Comparison of rain droplet removal methods: As shown in Table 4, single rain droplet removal methods and multi-weather restoration methods are presented. Among the single rain droplet removal methods, the quantitative results of Pix2pix, Attn.GAN, Quan et al., and CCN are shown. For the multi-weather restoration methods, the quantitative results of All-in-one, TransWeather, Chen et al., and WGWS-Net are shown.

[0055] It can be seen that the present invention performs excellently in tasks such as rain removal, snow removal, fog removal, and rain droplet removal.

[0056] In particular, to verify the restoration effect of the present method in the context of the power system, four image samples randomly selected from the real power operation and maintenance environment are used as inputs for visual analysis to check the visual effect of the model performance. As Figure 4 shown, the four degraded images are processed by the model and all perform excellently visually, further verifying the feasibility and engineering value of the present invention in the image processing of power industry inspections.

[0057] The results show that through quantitative comparison and visual analysis, an image restoration method proposed by the present invention for coping with various adverse weather environmental factors has good competitiveness compared with the existing top technologies.

[0058] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An image restoration method for coping with various adverse weather environmental factors, characterized in that, It includes: Identify the weather type of the image and perform test-time adaptation on the model; Input the image into the adapted model, and obtain the feature vector of the image through the encoder, where the encoder includes a convolutional layer, three neural network layers, and two downsampling layers; According to the image feature vector, perform decoding through the decoder, where the decoder includes two upsampling layers, two feature fusion layers, two neural network layers, and a convolutional layer; According to the obtained output, enter the soft reconstruction layer to obtain the restored image.

2. The image restoration method for coping with various adverse weather environment factors according to claim 1, characterized in that: The identifying the weather type of the image and performing test-time adaptation on the model includes, For the input degraded image The weather discriminator identifies the specific degradation type of the input image, and constructs a secondary degraded image according to the degradation type , and inputs the obtained paired images into the model, taking as the input degraded image, and taking as the input clean image to perform test-time adaptation on the model.

3. The image restoration method for coping with various adverse weather environment factors according to claim 2, characterized in that: The feature vector of the image obtained through the encoder includes the features extracted from the input image after being refined and enhanced by the first neural network layer, with a shape of , where represents the height, represents the width, represents the number of channels; According to the obtained output, pass through the downsampling layer and the neural network layer, and reduce the spatial dimension of the feature map through downsampling. Input the degraded image into the self - adapted model. After passing through a 3×3 convolutional layer for initial feature extraction to capture higher - level abstract features, the feature information is further processed at deeper levels through neural network layers. As the level deepens, more complex and abstract features are captured. At this time, the shape is , where is a real number; The output obtained from the previous layer is again passed through a downsampling layer and a neural network layer. The downsampling reduces the spatial dimension of the feature map to capture higher-level abstract features. The neural network layer further processes the feature information at a deeper level, and as the layer depth increases, more complex and abstract features are captured. At this time, the shape is ; The neural network layer is composed of multiple attention network modules. Each attention network module first undergoes layer normalization. Since the mean and standard deviation of the image are related to brightness and contrast, the overall expression of layer normalization is: ， Among them, F is a standard feedforward network, and x represents the input features. represent the mean and variance respectively. , represent the scaling factor and the bias term. and represent the parameters of two linear layers, which respectively adjust the reserved mean and variance, and the final output is the normalization result. Calculate multi-head attention, perform reflection padding and then calculate attention and perform central cropping, and pass it through the MLP layer to the next attention network module.

4. The image restoration method for coping with various adverse weather environmental factors according to claim 3, characterized in that: The performing decoding through the decoder includes, According to the finally obtained features, successively pass through an upsampling layer, a selective convolution fusion layer, and a neural network layer. The upsampling layer is used to increase the spatial size of the feature map and restore the details and size of the image; the selective convolution fusion layer fuses information from different levels by selectively weighting different features; The neural network layer processes and optimizes the fused features, and the current feature map shape is ; The feature map then sequentially passes through an upsampling layer, a selective convolution fusion layer, and a neural network layer. The image details and resolution are further restored through upsampling. Different levels of features are continuously fused through the selective convolution fusion layer, and the fused representation is optimized by the neural network layer. The shape of the current feature map is ; Finally, it passes through a 3×3 convolutional layer to generate the final output of the model .

5. The image restoration method for coping with various adverse weather environmental factors according to claim 4, wherein: The soft reconstruction includes, For the final output of the obtained model perform soft reconstruction to obtain the estimated transmission map and the residual , and calculate to obtain the repaired image.

6. The image restoration method for coping with various adverse weather environment factors as described in claim 5, characterized in that: The obtaining the restored image by calculation includes constructing a unified formula, and the expression is: ， ， ， Among them, the weather-damaged image , the original clean image , represents the residual term, is the atmospheric light in the scene, is the transmission map generated by atmospheric scattering.

7. The image restoration method for coping with various adverse weather environment factors according to claim 6, wherein: The entering the soft reconstruction layer according to the obtained output to obtain the restored image further includes, In the model training stage, complete the model training through a two-stage training strategy; In the pre-training stage, adopt a shared backbone network, use a mixed dataset from various weather conditions, learn the general features of the weather in a supervised manner, train the encoder and decoder according to the loss function in the pre-training stage. In the fine-tuning stage, introduce task-specific soft prompts to enhance the model's ability to handle specific weather degradations. In the fine-tuning stage, freeze the parameters of the backbone, introduce contrastive loss, and train the soft prompts according to the loss function.

8. An image restoration system for coping with various adverse weather environmental factors, which adopts an image restoration method for coping with various adverse weather environmental factors as described in any one of claims 1 to 7, and is characterized in that: It includes a weather discriminator module, an encoder module, a decoder module, and a soft reconstruction module; The weather discriminator module identifies the specific degradation type of the input image and generates adaptive initial conditions for the model. By analyzing the features of the input image, classify and determine the weather type, and generate additional secondary degraded images according to the type; The encoder module includes a convolutional layer, multiple neural network layers, and downsampling layers, which are respectively used to capture the low-level and high-level features of the image, reduce the spatial dimension through downsampling, and perform normalization and attention calculation on the features through multi-level attention network modules, and gradually extract key features from the input image; The decoder module includes an upsampling layer, an SKFusion feature fusion layer, and a neural network layer. The upsampling increases the spatial dimension to restore the resolution of the image. The SKFusion layer enhances the detail performance of the output by weighted fusion of features from different layers, and the neural network layer optimizes the fused features; The soft reconstruction module decomposes the feature vectors output by the decoder, calculates the residual layers, further adjusts and optimizes the image content, combines the constructed degradation expression to complete the fine-grained image restoration effect, and integrates various weather degradation factors through a unified formula to complete the comprehensive restoration.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for image restoration under multiple harsh weather environment factors according to any one of claims 1 to 7.

10. A computer-readable storage medium, having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for image restoration under multiple harsh weather environment factors according to any one of claims 1 to 7.

Images