An image restoration method and system for coping with various severe weather environment factors

Abstract

This invention discloses an image restoration method and system for handling various severe weather conditions, belonging to the field of image restoration technology. The method includes: identifying the weather type of the image and adapting the model to the testing time; inputting the image into the adapted model to obtain the image's feature vector, which is then decoded by a decoder; and finally transmitting the vector to a soft reconstruction layer to obtain the restored image. This invention employs supervised learning to train a general network capable of handling multiple weather conditions. During the fine-tuning stage, soft cues are introduced to enhance the model's ability to cope with different weather conditions and improve generalization. The effect of cues is enhanced by utilizing explicit and implicit interactions, exploring the hidden information of cues through low-rank decomposition, and adding contrastive loss to encourage cues to converge on similar tasks and diverge on opposite tasks, thereby improving the feature encoding ability. This invention can effectively handle the impact of various severe weather conditions on images, improve image quality, and provide high-quality input for subsequent defect identification and auxiliary analysis.

Description

Technical Field

[0001] This invention relates to the field of power system image restoration technology, specifically to an image restoration method and system for dealing with various severe weather and environmental factors. Background Technology

[0002] Image acquisition technology is widely used in power systems for critical scenarios such as transmission line inspection, substation monitoring, and distribution equipment status sensing. The quality of the acquired images directly affects the accuracy and reliability of subsequent fault identification, defect detection, and risk assessment. However, in practical applications, most power facilities are deployed outdoors, making them susceptible to interference from various severe weather conditions, such as heavy rain, snow accumulation, dense fog, and dust storms. These meteorological conditions significantly reduce image clarity, causing problems such as blurring, occlusion, and low contrast, severely limiting the performance of image-driven intelligent decision-making systems.

[0003] Despite some progress in image restoration research under severe weather conditions, several key challenges remain. Firstly, many existing methods are designed only for specific weather types (e.g., defogging or deraining only), lacking a unified modeling capability for diverse and complex weather scenarios. Secondly, some methods rely on additional physical priors or multimodal sensors, increasing system complexity and cost. Furthermore, since training data often consists of simulated degraded images, their distribution differs from actual images acquired in power system scenarios, hindering the generalization ability and stability of restoration models in real-world environments. Therefore, a robust image restoration method and system applicable to various severe weather conditions in power systems is urgently needed, primarily to address the impact of severe weather on acquired images in power system environments, such as rain, snow, fog, and raindrops. This will enhance the practicality and reliability of sensing systems in complex environments. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is: how to overcome the limitations of existing technologies that can only perform image restoration for single weather conditions, or require additional structures to increase costs, and how the difference between training data and real images affects the application capability of the model.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: an image restoration method for dealing with various severe weather environmental factors, comprising the following steps,

[0007] Identify the weather type in the image and adapt the model to the test time;

[0008] The image is input into the adaptive model, and the feature vector of the image is obtained through the encoder, which includes a convolutional layer, three neural network layers and two downsampling layers.

[0009] Based on the image feature vector, the image is decoded by a decoder, which includes two upsampling layers, two feature fusion layers, two neural network layers and one convolutional layer.

[0010] The output is fed into a soft reconstruction layer to obtain the restored image.

[0011] As a preferred embodiment of the image restoration method for dealing with various severe weather environmental factors described in this invention, the step of identifying the weather type of the image and adapting the model to the testing time includes:

[0012] For the input degraded image The weather discriminator identifies the specific degradation type of the input image, and a secondary degradation image is constructed based on the degradation type. The resulting paired images Input to the model, The degraded image as input will The clean image is used as input, and the model is tested at an adaptive time.

[0013] As a preferred embodiment of the image restoration method for dealing with various severe weather conditions described in this invention, the feature vector of the image obtained by the encoder includes features extracted from the input image after refinement and enhancement by the first neural network layer, with a shape of... ,in Indicates high, Indicates width. Indicates the number of channels;

[0014] The obtained output is processed through a downsampling layer and a neural network layer. The downsampling process reduces the spatial dimension of the feature map.

[0015] Degraded image The adaptive model is input and passed through a 3×3 convolutional layer for initial feature extraction, capturing higher-level abstract features. The neural network layers then continue processing the feature information at deeper levels, capturing more complex and abstract features with each layer. At this point, the shape is... ,in It is a real number;

[0016] The output from the previous layer passes through a downsampling layer and a neural network layer again. Downsampling reduces the spatial dimensionality of the feature map, capturing higher-level abstract features. The neural network layer continues to process feature information at a deeper level, capturing more complex and abstract features with each layer. At this point, the shape is... ;

[0017] The neural network layer consists of multiple attention network modules. Each attention network module first undergoes layer normalization. Based on the relationship between the image's mean and standard deviation and brightness and contrast, the overall expression for layer normalization is:

[0018] ,

[0019] Where x represents the input feature, These represent the mean and variance, respectively. , Indicates the scaling factor and bias term. and This represents the parameters of two linear layers, which are adjusted to retain the mean and variance respectively, and the final output is... This is the result of normalization;

[0020] The multi-head attention is calculated, reflection filling is performed, attention is calculated again, center clipping is performed, and then the result is passed through the MLP layer to the next attention network module.

[0021] As a preferred embodiment of the image restoration method for dealing with various severe weather conditions described in this invention, the decoding process includes:

[0022] The final features are sequentially processed through an upsampling layer, a selective convolutional fusion layer, and a neural network layer. The upsampling layer increases the spatial size of the feature map, restoring image details and dimensions. The selective convolutional fusion layer selectively weights different features, fusing information from different levels. The neural network layer processes and optimizes the fused features, resulting in a feature map with the following shape: ;

[0023] The feature map then passes through an upsampling layer, a selective convolutional fusion layer, and a neural network layer in sequence. Upsampling further restores image details and resolution. The selective convolutional fusion layer continues to fuse features from different levels, and the neural network layer optimizes the fused representation. The current feature map shape is... ;

[0024] Finally, a 3×3 convolutional layer is applied to generate the model's final output. .

[0025] As a preferred embodiment of the image restoration method for dealing with various severe weather conditions described in this invention, the soft reconstruction includes:

[0026] The final output of the obtained model Perform soft reconstruction to obtain the estimated transmission graph. and residual The repaired image is obtained through calculation.

[0027] As a preferred embodiment of the image restoration method for dealing with various severe weather conditions described in this invention, the step of obtaining the restored image through calculation includes constructing a unified formula, the expression of which is:

[0028] ,

[0029] , ,

[0030] Among them, weather-damaged images Original clean image , Represents the residual term. It is atmospheric light in the scene. This is a transport diagram resulting from atmospheric scattering.

[0031] As a preferred embodiment of the image restoration method for dealing with various severe weather conditions described in this invention, the step of entering the soft reconstruction layer based on the obtained output to obtain the restored image further includes...

[0032] During the model training phase, a two-stage training strategy is employed to complete the model training.

[0033] In the pre-training phase, a shared backbone network is used, and a mixed dataset from various weather conditions is used to learn general weather features in a supervised manner. The encoder and decoder are trained according to the loss function of the pre-training phase. In the fine-tuning phase, task-specific soft cues are introduced to enhance the model's ability to handle specific weather degradation. In the fine-tuning phase, the parameters of the backbone are frozen, a contrastive loss is introduced, and soft cues are trained according to the loss function.

[0034] Another objective of this invention is to provide an image restoration system that can cope with various severe weather conditions. This system can solve the problems of poor adaptability to various severe weather conditions and insufficient image detail restoration in existing systems by using weather type recognition, adaptive model optimization, and feature fusion technology.

[0035] To solve the above-mentioned technical problems, the present invention provides the following technical solution: an image restoration system for coping with various severe weather environmental factors, comprising: a weather discriminator module, an encoder module, a decoder module, and a soft reconstruction module;

[0036] The weather discriminator module identifies the specific degradation type of the input image and generates adaptive initial conditions for the model. By analyzing the features of the input image, it classifies and determines the weather type and generates additional secondary degradation images based on the type.

[0037] The encoder module includes a convolutional layer, multiple neural network layers, and a downsampling layer, which are used to capture low-level and high-level features of the image, respectively. The downsampling reduces the spatial dimension, and the multi-level attention network module normalizes and performs attention calculations on the features to gradually extract key features from the input image.

[0038] The decoder module includes an upsampling layer, an SKFusion feature fusion layer, and a neural network layer. Upsampling increases the spatial dimension to restore the image resolution. The SKFusion layer enhances the detail of the output by weighted fusion of features from different layers. The neural network layer optimizes the fused features.

[0039] The soft reconstruction module further adjusts and optimizes the image content by splitting the feature vector output by the decoder and calculating the residual layer. Combined with the constructed degradation expression, it achieves fine-grained image restoration. By integrating multiple weather degradation factors through a unified formula, it completes comprehensive restoration.

[0040] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of an image restoration method for dealing with various severe weather environmental factors as described above.

[0041] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an image restoration method for dealing with various severe weather environmental factors as described above.

[0042] The beneficial effects of this invention are as follows: This invention provides an image restoration method for handling various severe weather conditions, overcoming the problems of existing technologies that can only perform image restoration for single weather conditions, require additional structures increasing costs, and suffer from the impact of differences between training data and real images on the model's application capabilities. This invention can handle image restoration under various severe weather conditions and is highly competitive with current state-of-the-art technologies. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is an overall flowchart of an image restoration method for dealing with various severe weather environmental factors, provided in the first embodiment of the present invention.

[0045] Figure 2 This is a comparison chart of the load balancing rates of multiple algorithms under the same task request in an image restoration method for dealing with various severe weather conditions provided in the first embodiment of the present invention.

[0046] Figure 3 The first embodiment of the present invention provides a runtime diagram of an image restoration method for coping with various severe weather conditions under different repetition numbers;

[0047] Figure 4 The third embodiment of the present invention provides a method for image restoration under various severe weather conditions, which compares the load balancing rate under different task numbers. Detailed Implementation

[0048] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0049] Example 1, referring to Figures 1-3 As one embodiment of the present invention, an image restoration method for coping with various severe weather environmental factors is provided, including:

[0050] Step 1: Identify the weather type in the image and use this information to adapt the model to the testing time.

[0051] For the input degraded image The weather discriminator identifies the specific degradation type of the input image, and a secondary degradation image is constructed based on the degradation type. ;

[0052] Based on the obtained paired images Input to the model, The degraded image as input will Clean images are used as input to adapt the model to test time.

[0053] Step 2: Input the image into the model that has been adapted in Step 1, and obtain the feature vector of the image through the encoder, which includes a convolutional layer, three neural network layers and two downsampling layers.

[0054] First, degrade the image , Represents a real number. The input is fed into the adaptive model from step one, passing through a 3×3 convolutional layer for initial feature extraction. Then, the features extracted from the input image are refined and enhanced by the first neural network layer, resulting in a shape of... ;

[0055] The specific content of feature extraction performed by the neural network layer is as follows:

[0056] A neural network layer consists of multiple attention network modules. For each attention network module, layer normalization is first performed. Based on the relationship between the image's mean and standard deviation and brightness and contrast, the overall LayerNorm can be represented as follows:

[0057] ,

[0058] in, Represents the mean and variance. , This represents the scaling factor and bias, and introduces two linear layers using weights. and The retained mean and variance are adjusted; multi-head attention is calculated, and attention is calculated after reflection filling. Finally, center cropping is performed. The weights of MHSA when aggregating spatial information are dynamic, but because these weights are positive, MHSA will sum the input features, which plays a smoothing role in the operation and helps to retain low-frequency information in the image. Therefore, an additional convolution operation on V is added to supplement the possible limitations of MHSA in spatial information aggregation and enhance the model's ability to process spatial information. The specific calculation is expressed as follows:

[0059] ,

[0060] in, These represent the query vector, key vector, and value vector of the attention mechanism, respectively. Since V is not divided into windows, it uses both attention mechanisms to gather information within the window and convolution to gather information within the domain, thereby improving the model's spatial understanding ability; it is then passed through the MLP layer to the next attention network module.

[0061] Next, the output from the upper layer passes through a downsampling layer and a neural network layer. Downsampling reduces the spatial dimensionality of the feature map, helping the model capture higher-level abstract features. The neural network layer continues to process feature information at a deeper level. With increasing layer depth, the model can capture more complex and abstract features, at which point the shape becomes... ;

[0062] Finally, the output is passed through a downsampling layer and a neural network layer again. Downsampling reduces the spatial dimensionality of the feature map, helping the model capture higher-level abstract features. The neural network layer continues to process feature information at a deeper level. As the layers deepen, the model can capture more complex and abstract features, at which point the shape becomes... .

[0063] Step 3, refer to Figure 2 Based on the image feature vector obtained in step two, it is decoded by a decoder, which includes two upsampling layers, two feature fusion layers, two neural network layers and one convolutional layer.

[0064] First, the features obtained in step two are sequentially processed through an upsampling layer, an SK Fusion feature fusion layer, and a neural network layer. The upsampling layer increases the spatial dimension of the feature map, helping to recover image details and size. The SK Fusion feature fusion layer selectively weights different features to perform feature fusion, combining features from different layers. Finally, the neural network layer processes and optimizes the features fused from the SK Fusion layer, resulting in the following shape: ;

[0065] Next, the output is passed through an upsampling layer, an SK Fusion feature fusion layer, and a neural network layer. The upsampling layer increases the spatial dimension of the feature map, helping to recover image details and size. The SK Fusion feature fusion layer selectively weights different features to fuse features from different layers. The neural network layer processes and optimizes the features fused from the SK Fusion layer, resulting in a shape that is... ;

[0066] Finally, the output from the upper layer is passed through a 3×3 convolutional layer to obtain the final output. .

[0067] Step 4, Refer to Figure 3 Based on the output obtained in step three Entering the soft reconstruction layer, we obtain the restored image.

[0068] Based on the final output obtained in step three By performing soft refactoring, one can... To split, we get and Substitute it into the unified formula

[0069] The repaired image is obtained from this process.

[0070] The specific details of constructing the unified formula are as follows:

[0071] Image of weather damage Original clean image .

[0072] A unified image restoration method can be achieved by integrating the above image degradation formulas:

[0073] ,

[0074] in, , ,in Let T represent the residual term, T be the transport diagram generated by atmospheric scattering, and A be the global illumination.

[0075] Step 5: In the model training phase, the model training is completed through a two-stage training strategy.

[0076] First, in the pre-training phase, a shared backbone network is used to learn general weather features in a supervised manner using a mixed dataset from various weather conditions;

[0077] Secondly, based on the loss function during the pre-training phase:

[0078] ,

[0079] in, For the pre-training loss function, For the predicted value of a clear image, It is a label image. The distance is L1.

[0080] Train the encoder and decoder mentioned in steps two and three.

[0081] Next, task-specific soft cues are introduced during the fine-tuning phase to enhance the model's ability to handle specific weather degradation.

[0082] The specific content of introducing soft prompts is as follows:

[0083] A set of task-specific cues is introduced to enhance the model's ability to handle specific tasks. Attention-level cues are introduced into the input of the MSHA layer. Assuming the cue length is m, the formula for calculating the attention mechanism can be expressed as:

[0084] ,

[0085] in, These represent the query vector, key vector, and value vector of the attention mechanism, respectively. Indicates insertion into and The cue vectors in the dataset contain task-specific soft cues used to encode semantic knowledge specific to that task; these cue vectors are not shared between tasks. Represents the vector dimension. This indicates a convolution operation.

[0086] Although the visual effects of different weather conditions vary, they exhibit both common patterns and features in image degradation, as well as task-specific characteristics. Therefore, a low-rank decomposition is used to divide the soft cue into two parts, which can be represented as:

[0087] ,

[0088] in, Represents matrix multiplication. This indicates a task-specific hint for the i-th weather removal task. This is a general task hint. It is the rank of the parameter matrix. Task-specific hints are specific to each task, while general hints are shared across all tasks. This incorporates task-specific features into task-specific hints and general knowledge across tasks into task-general hints.

[0089] Finally, in the fine-tuning phase, the parameters of the backbone are frozen, and a contrastive loss is introduced. Based on the soft cues mentioned during loss function training, the expression is:

[0090] ,

[0091] in, For losses during the fine-tuning phase, , These are the weights of the contrast loss and the contrast loss itself.

[0092] The specific details of introducing contrast loss are as follows:

[0093] Based on the fact that snow and raindrops are more similar in degradation, while rain and haze are relatively similar, contrastive learning is applied to task-specific cues to encourage cues corresponding to similar degradation tasks to be more closely linked, achieving higher similarity between cues for corresponding degradation tasks. Conversely, lower similarity is expected when corresponding to degradation tasks with less clear interaction relationships. The contrastive loss can be expressed as:

[0094] ,

[0095] in, It is the set of indices of all tasks that have a similar relationship to the i-th task. It is the cardinality of the set of indices of all tasks that are similar to the i-th task. It is the temperature coefficient. It is an index function, which is when The value is 1 if the condition is met, and 0 otherwise. This represents the similarity between two vectors. Let i be the contrast loss corresponding to the i-th task. It is the cosine similarity between the cue vector of the i-th task and the cue vectors of similar tasks. It is the cosine similarity between the i-th task cue vector and the dissimilar task cue vectors.

[0096] Example 2, an embodiment of the present invention, provides a system for image restoration under various severe weather conditions, including: a weather discriminator module, an encoder module, a decoder module, and a soft reconstruction module;

[0097] The weather discriminator module identifies the specific degradation type of the input image and generates adaptive initial conditions for the model. By analyzing the features of the input image, it classifies and determines the weather type and generates additional secondary degradation images based on the type.

[0098] The encoder module includes a convolutional layer, multiple neural network layers, and a downsampling layer, which are used to capture low-level and high-level features of the image, respectively. The downsampling reduces the spatial dimension, and the multi-level attention network module normalizes and performs attention calculations on the features to gradually extract key features from the input image.

[0099] The decoder module includes an upsampling layer, an SK Fusion feature fusion layer, and a neural network layer. Upsampling increases the spatial dimension to restore the image resolution. The SK Fusion layer enhances the detail of the output by weighted fusion of features from different layers. The neural network layer optimizes the fused features.

[0100] The soft reconstruction module further adjusts and optimizes the image content by splitting the feature vector output by the decoder and calculating the residual layer. Combined with the constructed degradation expression, it achieves fine-grained image restoration. By integrating multiple weather degradation factors through a unified formula, it completes comprehensive restoration.

[0101] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0103] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0104] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0105] Example 3, referring to Figure 4 This is another embodiment of the present invention. To verify the beneficial effects of the present invention, scientific demonstration was conducted through economic benefit calculations and simulation experiments. This embodiment tested both the existing conventional method and the method of this embodiment.

[0106] This invention uses unconstrained datasets: Outdoor Rain, Snow100K, RESIDE-outdoor, and RainDrop.

[0107] The data preprocessing steps are as follows: Training data from different datasets are sampled to roughly align the sample counts across datasets. For the image inpainting model, based on the sample counts of the four datasets, all samples (4500 pairs) from the OutdoorRain training set are selected, 7500 pairs of images are randomly sampled from the Snow100K training set, 5000 pairs of images are randomly sampled from the RESIDE dataset, and 1500 pairs of images are cyclically sampled from the RainDrop training set. For the weather discriminator, 12000 degraded images are randomly sampled from the OutdoorRain dataset, 18000 images are randomly sampled from the Snow100K training set, 15000 images are randomly sampled from the RESIDE dataset, and 8000 images are randomly sampled from the RainDrop training set. Data augmentation is performed simultaneously with sampling, such as random horizontal flipping, random cropping of portion sizes, random adjustment of contrast, random adjustment of brightness, and random adjustment of saturation.

[0108] The experimental setup is as follows: The Adam optimizer, combined with cosine annealing, is used to adjust the parameters and learning rate of the neural network. In the first pre-training phase, the model is trained for 200 epochs with a batch size of 4 and an initial learning rate of 0.0002. In the second fine-tuning phase, the model is trained for 250 epochs with a batch size of 8 and an initial learning rate of 0.00002. This experiment applies random flipping to enhance the image and randomly crops the image into 256 x 256 blocks. In the adaptive phase during testing, the batch size is 8 and the learning rate is 0.00002.

[0109] This invention uses quantitative comparison and visual analysis to judge the model's effectiveness.

[0110] Quantitative comparisons are made by comparing the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM).

[0111] Table 1: Quantitative comparisons on the Outdoor Rain dataset

[0112] ,

[0113] A comparison of rain removal methods: Table 1 shows the results of single rain removal methods and multiple weather recovery methods. For single rain removal methods, quantitative results are presented for Pix2pix, HRGAN, and MPRNet. For multiple weather recovery methods, quantitative results are presented for All-in-one, TransWeather, Chen et al., and WGWS-Net.

[0114] Table 2: Quantitative comparisons on the Snow100K dataset

[0115] ,

[0116] A comparison of snow removal methods: Table 2 shows the results of single snow removal methods and multi-weather recovery methods. For single snow removal methods, quantitative results from DetailsNet, DesnowNet, JSTASR, and DDMSNET are presented. For multi-weather recovery methods, quantitative results from All-in-one, TransWeather, Chen et al., and WGWS-Net are presented.

[0117] Table 3: Quantitative comparisons on the RESIZE-outdoor dataset

[0118] ,

[0119] A comparison of defogging methods: Table 3 shows the results of single defogging methods and multi-weather recovery methods. For single defogging methods, quantitative results for EPDN, PFDN, KDDN, and MPRNet are presented. For multi-weather recovery methods, quantitative results for All-in-one, TransWeather, Chen et al., and WGWS-Net are presented.

[0120] Table 4: Quantitative comparisons on the RainDrop dataset

[0121] ,

[0122] A comparison of raindrop removal methods: Table 4 shows the results of single raindrop removal methods and multi-weather recovery methods. For single raindrop removal methods, quantitative results from Pix2pix, Attn.GAN, Quan et al., and CCN are presented. For multi-weather recovery methods, quantitative results from All-in-one, TransWeather, Chen et al., and WGWS-Net are presented.

[0123] It can be seen that the present invention performs excellently in tasks such as removing rain, snow, fog, and raindrops.

[0124] Specifically, to verify the repair effect of this method in the context of a power system, four image samples from a real power operation and maintenance environment were randomly selected as input for visual analysis to check the visual performance of the model. For example... Figure 4 As shown, the four degraded images, after model processing, all exhibit excellent visual performance, further verifying the feasibility and engineering value of this invention in power industry inspection image processing.

[0125] The results show that, through quantitative comparison and visual analysis, the image restoration method proposed in this invention, which addresses various severe weather conditions, is highly competitive with existing cutting-edge technologies.

[0126] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An image restoration method for dealing with various severe weather environmental factors, characterized in that, include: Identify the weather type in the image and adapt the model to the test time. The image is input into the adaptive model, and the feature vector of the image is obtained through the encoder, which includes a convolutional layer, three neural network layers and two downsampling layers. Based on the image feature vector, the image is decoded by a decoder, which includes two upsampling layers, two feature fusion layers, two neural network layers and one convolutional layer. The obtained output is fed into the soft reconstruction layer to obtain the restored image; For the input degraded image x t The weather discriminator identifies the specific degradation type of the input image, and a secondary degradation image x is constructed based on the degradation type. sd The resulting paired image {x sd ,x t Input x into the model sd The degraded image as input, x t The model is tested using a clean image as input; The feature vector of the image obtained by the encoder includes features extracted from the input image after being refined and enhanced by the first neural network layer, with a shape of H×W×C, where H represents height, W represents width, and C represents the number of channels; The obtained output is processed through a downsampling layer and a neural network layer. The downsampling process reduces the spatial dimension of the feature map. Degraded image The adaptive model is input and passed through a 3×3 convolutional layer for initial feature extraction, capturing higher-level abstract features. The neural network layers then continue processing the feature information at deeper levels, capturing more complex and abstract features with each layer. At this point, the shape is... in It is a real number; The output from the previous layer passes through a downsampling layer and a neural network layer again. Downsampling reduces the spatial dimensionality of the feature map, capturing higher-level abstract features. The neural network layer continues to process feature information at a deeper level, capturing more complex and abstract features with each layer. At this point, the shape is... The neural network layer consists of multiple attention network modules. Each attention network module first undergoes layer normalization. Based on the relationship between the image's mean and standard deviation and brightness and contrast, the overall expression for layer normalization is: Where F is a standard feedforward network, x represents the input features, μ and σ represent the mean and variance, respectively, γ and β represent the scaling factor and bias term, and W... γ B γ and W β B β The parameters of the two linear layers are used to adjust the retained mean and variance respectively, and the final output y is the normalized result. The multi-head attention is calculated, reflection filling is performed, attention is calculated again, center clipping is performed, and then the result is passed through the MLP layer to the next attention network module.

2. The image restoration method for dealing with various severe weather environmental factors as described in claim 1, characterized in that: The decoding process via the decoder includes, The final features are sequentially passed through an upsampling layer, a selective convolutional fusion layer, and a neural network layer. The upsampling layer is used to increase the spatial size of the feature map and restore the details and size of the image. The selective convolutional fusion layer fuses information from different levels by selectively weighting different features. The neural network layer processes and optimizes the fused features, and the current feature map shape is... The feature map then passes through an upsampling layer, a selective convolutional fusion layer, and a neural network layer in sequence. Upsampling further restores image details and resolution, while the selective convolutional fusion layer continues to fuse features from different levels. The neural network layer then optimizes the fused representation, resulting in a current feature map with a shape of H×. W×C; Finally, a 3×3 convolutional layer is applied to generate the model's final output.

3. The image restoration method for dealing with various severe weather environmental factors as described in claim 2, characterized in that: The soft reconstruction includes, The final output of the obtained model Perform soft reconstruction to obtain the estimated transmission graph. and residual The repaired image is obtained through calculation.

4. The image restoration method for dealing with various severe weather environmental factors as described in claim 3, characterized in that: The process of obtaining the repaired image through calculation includes constructing a unified formula, the expression of which is: B=O k ⊙I+O r +I Wherein, the weather-damaged image I, the original clean image B, r represents the residual term, A is the atmospheric light in the scene, and T is the transmission map generated by atmospheric scattering.

5. The image restoration method for dealing with various severe weather environmental factors as described in claim 4, characterized in that: The process of inputting the obtained output into the soft reconstruction layer to obtain the restored image also includes, During the model training phase, a two-stage training strategy is employed to complete the model training. In the pre-training phase, a shared backbone network is used, and a mixed dataset from various weather conditions is used to learn general weather features in a supervised manner. The encoder and decoder are trained according to the loss function of the pre-training phase. In the fine-tuning phase, task-specific soft cues are introduced to enhance the model's ability to handle specific weather degradation. In the fine-tuning phase, the parameters of the backbone are frozen, a contrastive loss is introduced, and soft cues are trained according to the loss function.

6. An image restoration system for coping with various severe weather environmental factors, employing the image restoration method for coping with various severe weather environmental factors as described in any one of claims 1 to 5, characterized in that: It includes a weather discriminator module, an encoder module, a decoder module, and a soft reconstruction module; The weather discriminator module identifies the specific degradation type of the input image and generates adaptive initial conditions for the model. By analyzing the features of the input image, it classifies and determines the weather type and generates additional secondary degradation images based on the type. The encoder module includes a convolutional layer, multiple neural network layers, and a downsampling layer, which are used to capture low-level and high-level features of the image, respectively. It reduces the spatial dimension through downsampling and normalizes and performs attention calculations on the features through a multi-level attention network module, thereby gradually extracting key features from the input image. The decoder module includes an upsampling layer, an SK Fusion feature fusion layer, and a neural network layer. Upsampling increases the spatial dimension to restore the image resolution. The SK Fusion layer enhances the detail of the output by weighted fusion of features from different layers. The neural network layer optimizes the fused features. The soft reconstruction module further adjusts and optimizes the image content by splitting the feature vector output by the decoder and calculating the residual layer. Combined with the constructed degradation expression, it achieves fine-grained image restoration. By integrating multiple weather degradation factors through a unified formula, it completes comprehensive restoration.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the image restoration method for dealing with various severe weather environmental factors as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image restoration method for dealing with various severe weather environmental factors as described in any one of claims 1 to 5.

Images