Continuous learning-based severe weather image sharpening method

Abstract

The invention discloses a severe weather image sharpening method based on continuous learning, and the method comprises the steps: 1, carrying out the preprocessing of an image data set under various severe weather conditions, including image pairing and multi-task division; 2, dividing the image restoration network into two parts of feature projection and output projection, designing a degradation sensing module for extracting degradation specific information output by the feature projection under different weather conditions, and establishing background supervision and degradation supervision by adopting a multi-teacher model for guiding learning of a student model; and 3, establishing respective optimization targets for the student model and the teacher model, inputting pairing data of different weather removal tasks, and optimizing the student network by a multilateral distillation normal form, thereby obtaining an optimal model with integrated image severe weather removal capability. According to the method, different types of severe weather images continuously acquired in a real scene can be dealt with, and the implementation effect of an image restoration network in a resource-limited scene is improved.

Description

Technical Field

[0001] The present invention relates to a method for sharpening a severe weather image based on continuous learning, and belongs to the technical field of computer vision and artificial intelligence. Technical Background

[0002] Inclement weather conditions, including rain, snow, and haze, significantly degrade image quality, hindering key vision tasks such as object tracking, detection, semantic segmentation, and face recognition. Existing image restoration techniques, such as rain, snow, and haze removal, primarily use deep neural networks to address specific types of degradation. However, for applications such as autonomous driving and surveillance, training dedicated networks for each weather type is inefficient and impractical.

[0003] To address the problem of multiple severe weather image removal, some all-in-one solutions have proposed using uniform network parameters to simultaneously handle various weather types. These models assume ideal training conditions, in which all possible degraded-to-clean mappings are contained in the training data domain and available at each training iteration. However, this assumption is unrealistic for practical applications, as weather variations often alter the distribution of degraded images, causing the mapping rules to be dispersed across multiple datasets. This fragmentation complicates the combination of diverse training datasets, and retraining all the data to adapt to new tasks is computationally expensive.

[0004] On the other hand, existing full-featured image restoration models only support one-time training and fine-tuning, retaining a large number of parameters to maintain good performance. Due to the information imbalance between degraded datasets acquired in different batches, this single hybrid training approach exacerbates overfitting on degradation types with larger datasets while increasing the cost, thereby hindering the model's ability to generalize to unknown degradation types.

[0005] Under continuously changing environmental conditions, the restoration network should also be able to continuously adapt and update through continuous learning without requiring extensive resources. Existing continuous learning methods rely on regularization constraints, sample replay, and model parameter expansion, lacking an effective continuous learning framework tailored to the underlying visual design. Furthermore, these methods introduce poor memory overhead and computational efficiency. How to achieve stable and cost-effective continuous learning with the small batches of data constantly acquired in the real world and achieve uniform removal of severe weather conditions in images has become an urgent challenge. Summary of the Invention

[0006] In order to address the shortcomings of the above-mentioned existing technologies, the present invention proposes a method for sharpening severe weather images based on continuous learning, in order to effectively combine the technical advantages of the continuous learning algorithm and the underlying visual framework, and fully explore the characteristics of image background information representation and degradation information representation, so as to achieve stable and low-cost continuous learning of the underlying visual system and effective removal of severe weather in the image.

[0007] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0008] The method for sharpening an image in severe weather based on continuous learning of the present invention is characterized in that it comprises the following steps:

[0009] Step 1: Obtain a set of image samples under T different severe weather conditions ,in, Indicates the A dataset of paired images of severe weather conditions, and , Indicates the The i-th degraded image under severe weather conditions, Indicates the The i-th clean image under bad weather conditions, , ; Indicates the total number of severe weather conditions, No. The total number of image sample pairs under severe weather conditions;

[0010] Step 2: Severe weather conditions Teacher Model and 1 student model ;in, represents the j-th teacher model;

[0011] Student Model right Process and obtain The i-th student clean image under bad weather conditions and the i-th student degradation feature map ;

[0012] The teacher model The i-th degraded image under severe weather conditions Process and obtain A degraded supervisory signal set under severe weather conditions and background supervision signal set ;

[0013] Student Model according to and ,right and Process and obtain The corresponding degradation supervision signal and Corresponding background supervision signal ;

[0014] Step 3: Build A basic loss function under severe weather conditions , degraded supervisory signal The loss function and background supervision signals The loss function , and calculate the student model using the multilateral distillation paradigm Total restoration losses and total degradation loss ;

[0015] Step 4: Order the paired data of T kinds of severe weather conditions in ascending order. Enter separately Teacher model and student model The student model is processed and the continuous learning algorithm of degradation-aware distillation is used to The training is performed to obtain the optimal student model, which is used to remove the bad weather in the input image.

[0016] The characteristic of the severe weather image clearing method based on continuous learning described in the present invention is that the j-th teacher model in step 2 Including: j-th image restoration model and the jth degradation-aware module ; Student Model Includes: Image restoration models , No. Degradation-aware modules Hedi Supervision Aggregation Module ; ;

[0017] Step 2.1: The student model Use the Feature Projection right Process and obtain The i-th student deep feature map under severe weather conditions , and then use the Output projections right Process and obtain The i-th student clean image predicted under severe weather conditions ;

[0018] The student model Use the Degradation-aware modules right Process and obtain The i-th student degradation feature map under severe weather conditions ;

[0019] Step 2.2, the j-th teacher model Using the jth feature projection For the first The i-th degraded image under severe weather conditions Processing is performed to obtain the i-th teacher deep feature map , and then use the j-th output projection right Process and obtain The i-th teacher clean image predicted under severe weather conditions , thus obtaining Teacher Model Output background supervision signal set , ;

[0020] The j-th teacher model Using the j-th degradation perception module For the i-th teacher deep feature map Process and get the jth teacher model Output of the i-th teacher degradation feature map ; thus obtaining Teacher Model The output degradation supervision signal set , ;

[0021] Step 2.3, the Supervision Aggregation Module For the degraded supervision signal set and background supervision signal set Processing is performed to obtain the degraded supervision signal and background supervision signals .

[0022] Furthermore, the first Degradation-aware modules Follow these steps to get :

[0023] Step 2.1.1, Perform channel average pooling operation to obtain Student average degradation feature map across channels ;

[0024] Step 2.1.2, respectively and Perform fast Fourier transform and get Amplitude spectrum of and Amplitude spectrum of , and thus use formula (1) to obtain the Information about student regression :

[0025] (1)

[0026] Step 2.1.3: Calculate using formula (2) The student weight tensor :

[0027] (2)

[0028] In formula (2), represents the linear unit activation function with Gaussian error, RDBS represents a series of dense residual connection operations and consists of dilated convolution blocks;

[0029] Step 2.1.4: Use formula (3) to obtain The i-th student degradation feature map under severe weather conditions :

[0030] (3)

[0031] In formula (3), The convolution kernel is The convolution operation, Represents tensor multiplication.

[0032] Furthermore, the j-th degradation perception module in step 2.2 Follow these steps to get :

[0033] Step 2.2.1, Perform channel average pooling operation to obtain Teacher average degradation feature map across channels ;

[0034] Step 2.2.2, respectively and Perform fast Fourier transform and get Amplitude spectrum of and Amplitude spectrum of , and then use formula (4) to obtain the Information about teacher degradation :

[0035] (4)

[0036] Step 2.2.3: Calculate using formula (5) The teacher weight tensor of :

[0037] (5)

[0038] Step 2.2.4: Use formula (6) to obtain The i-th teacher degradation feature map under severe weather conditions :

[0039] (6).

[0040] Furthermore, the step 2.3 includes:

[0041] Step 2.3.1. Calculation and KL divergence on the kth channel dimension and use The function processes the KL divergence to obtain the degenerate distance vector , thus obtaining and The degradation supervision importance matrix between ,in, is the total number of channels;

[0042] Step 2.3.2, calculation and KL divergence on the kth channel dimension and use Function processing KL divergence to obtain background distance , thus obtaining and The background supervision importance matrix between ;

[0043] Step 2.3.3: Use equations (11) and (12) to obtain the degraded supervision signal and background supervision signals :

[0044] (11)

[0045] (12).

[0046] Furthermore, the step 3 includes:

[0047] Step 3.1, build the A basic loss function under severe weather conditions ;

[0048] Step 3.1.1, use formula (13) to construct Content loss under severe weather conditions :

[0049] (13)

[0050] In formula (13), represents a constant;

[0051] Step 3.1.2: Use formula (13) to construct Perception loss under severe weather conditions :

[0052] (14)

[0053] In formula (14), represents exponential operation, Indicates extracting the first Layer hidden layer features, = {1,2,···,L}, L is the total number of hidden layers; > 0 is the temperature parameter that controls the output scale, and |·| distance, It is The weight coefficients of the hidden layers;

[0054] Step 3.1.3: Use formula (15) to construct Basic loss function under severe weather conditions :

[0055] (15)

[0056] In formula (15), It is a hyperparameter used to balance the content optimization goal and the perception optimization goal;

[0057] Step 3.2: Use formula (16) to construct Degraded supervision signals under severe weather conditions The loss function :

[0058] (16)

[0059] In formula (16), represents the KL divergence calculated along the channel dimension;

[0060] Step 3.3: Use formula (17) to construct Background supervision signals under severe weather conditions The loss function :

[0061] (17)

[0062] In formula (17), It is a hyperparameter used to balance the content optimization goal and the perception optimization goal;

[0063] Step 3.4: Integrate all loss functions using the multilateral distillation paradigm.

[0064] Step 3.4.1: Use formula (18) to construct Total restoration losses :

[0065] (18)

[0066] In formula (18), and is a hyperparameter used to balance degradation supervision and background supervision, Indicates the The background basic loss under severe weather conditions is as follows:

[0067] (19)

[0068] In formula (19), Used to balance fidelity and perceived effect;

[0069] Step 3.4.2: Use formula (20) to construct Total degradation loss :

[0070] (20)

[0071] In formula (20), Used to balance fidelity and perceived effect.

[0072] Furthermore, the step 4 includes:

[0073] when When there is no teacher model, the student model is trained end-to-end. Image restoration network in and degradation-aware modules Perform training and calculate formula (15) to update the image restoration network and degradation-aware modules When the loss error is less than the set threshold, the training is stopped and the tth optimal student model is obtained;

[0074] when When t-1 optimal student models are used as t-1 teacher models, the student models are frozen first. middle The parameters of are optimized using formula (18) Parameters, then freeze The parameters of are updated using formula (20) When the loss error is less than the set threshold, the training is stopped and the tth optimal student model is obtained; and the Tth optimal student model is As an optimal image restoration model with integrated image bad weather removal capabilities.

[0075] The electronic device of the present invention includes a memory and a processor, and is characterized in that the memory is used to store a program that supports the processor to execute the method for clearing severe weather images, and the processor is configured to execute the program stored in the memory.

[0076] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium. The characteristic is that the computer program executes the steps of the severe weather image clearing method when the computer program is executed by a processor.

[0077] Compared with the prior art, the present invention has the following beneficial effects:

[0078] 1. To address the high randomness and diversity of severe weather conditions and the difficulty of centralized training using small batches of data continuously acquired in real-world scenarios, this paper proposes a continuous learning algorithm based on degradation-aware distillation. This algorithm provides a stable and efficient solution tailored for the field of integrated image restoration, enabling continuous learning for severe weather removal in images. Experimental results demonstrate that the proposed method exhibits superior performance in severe weather removal.

[0079] 2. The present invention designs a degradation perception module that fully utilizes frequency domain knowledge to realize the representation learning of degradation information, effectively enhancing the image restoration model's knowledge accumulation of various different severe weather characteristics, thereby improving the model's ability to cope with complex and changeable severe weather conditions in real scenarios.

[0080] 3. The present invention proposes a simple and effective multilateral distillation method, which adopts an importance-guided supervision aggregation module to combine different degradation supervision and background supervision from multiple teacher models, integrating the restoration capabilities under various severe weather conditions into the image restoration model, thereby achieving accurate generation of clear images under various severe weather conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 This is a diagram of the implementation framework of the present invention in the reasoning stage;

[0082] Figure 2 Implementation framework diagram of the present invention in the training phase;

[0083] Figure 3 This is a structural diagram of the degradation perception module of the method of the present invention;

[0084] Figure 4 This is a structural diagram of the supervisory aggregation module of the method of the present invention. DETAILED DESCRIPTION

[0085] In this embodiment, a method for sharpening an image in severe weather conditions based on continuous learning is described. The implementation method of the image restoration model in the inference phase is shown in FIG. Figure 1 , input images of various severe weather conditions, and can infer the corresponding clean images end-to-end. The implementation details of the continuous learning algorithm in the training phase can be found in Figure 2 Specifically, the method is carried out in the following steps:

[0086] Step 1: Obtain a set of image samples under T different severe weather conditions , which simulates the small batch paired datasets continuously obtained in real scenarios; Indicates the A dataset of paired images of severe weather conditions, and , Indicates the The i-th degraded image under severe weather conditions, Indicates the The i-th clean image under bad weather conditions, , ; Indicates the total number of severe weather conditions, No. The total number of image sample pairs under severe weather conditions.

[0087] Step 2: Severe weather conditions Teacher Model , each teacher model Both include: Image restoration model and degradation-aware modules ;in Projected by features and output projection composition, ; For the Build a student model of severe weather conditions , including: image restoration model , degradation perception module and supervised aggregation modules ;in Projected by features and output projection composition;

[0088] The teacher model The i-th degraded image under severe weather conditions Processing to obtain the degraded supervisory signal set and background supervision signal set , and then use the supervised aggregation module right and Process and obtain The corresponding degradation supervision signal and Corresponding background supervision signal ;

[0089] Defining the image restoration model U-Net is used as the backbone network, and its framework details are as follows Figure 1 As shown, represents a series of convolutional layers and nonlinear activations after the last downsampling, express exist For the sake of simplicity, the composite function is omitted in the previous network layer. Figure 1 The global residual connection in , to degrade the image As input, after feature projection and output projection Two processes generate the final clean image , degradation perception module The structural details of Figure 3 shown.

[0090] Step 2.1, Student Model Using feature projection For the first The i-th degraded image under severe weather conditions Processing is performed to obtain the deep feature map of the i-th student model , and then use the output projection right Process and get the predicted clean image ; correspond Figure 3 Feature input in;

[0091] Student Model Using the degradation-aware module For the i-th student model deep feature map Processing is performed to obtain the student degradation feature map ;

[0092] Step 2.1.1, Perform channel average pooling operation to obtain the student model Average degradation feature map across channels ;

[0093] Step 2.1.2, respectively and Perform fast Fourier transform and get Amplitude spectrum of and Amplitude spectrum of , and thus use formula (1) to obtain the student model Degradation related information :

[0094] (1)

[0095] Step 2.1.3: Calculate using formula (2) The weight tensor of :

[0096] (2)

[0097] In formula (2), represents the linear unit activation function with Gaussian error, and RDBS represents a series of dense residual connection operations, which consists of dilated convolution blocks.

[0098] Step 2.1.4: Use formula (3) to obtain The i-th student degradation feature map under severe weather conditions :

[0099] (3)

[0100] In formula (3), The convolution kernel is The convolution operation, Represents tensor multiplication;

[0101] Step 2.2, j-th teacher model Using feature projection For the first The i-th degraded image under severe weather conditions Processing is performed to obtain the deep feature map of the i-th teacher model , and then use the output projection right Processing is performed to obtain the teacher prediction graph ;

[0102] The j-th teacher model Using the degradation-aware module For the i-th teacher model deep feature map Processing is performed to obtain the teacher degradation feature map .

[0103] Step 2.2.1, Perform channel average pooling operation to obtain the teacher model Average degradation feature map across channels ;

[0104] Step 2.2.2, respectively and Perform fast Fourier transform and get Amplitude spectrum of and Amplitude spectrum of , and then use formula (4) to obtain the teacher model Degradation related information :

[0105] (4)

[0106] Step 2.2.3: Calculate using formula (5) The weight tensor of :

[0107] (5)

[0108] Step 2.2.4: Use formula (6) to obtain The i-th teacher degradation feature map under severe weather conditions :

[0109] (6)

[0110] Step 2.3: Using equations (7) and (8), we can obtain Teacher Model The output degradation supervision signal set and background supervision signal set :

[0111] (7)

[0112] (8)

[0113] Step 2.4: Build a supervised aggregation module , supervised aggregation module The structural details of Figure 4 As shown, and the degradation supervision signal set and background supervision signal set Processing is performed to obtain the degraded supervision signal and background supervision signals .

[0114] Step 2.4.1. Calculate the Student Degradation Characteristic Map and the degraded supervisory signal set Degraded characteristic diagram of each teacher KL divergence in the channel dimension, and use Calculate the degenerate distance vector set ,in is the total number of channels;

[0115] Using formula (9) to get the degradation supervision importance matrix :

[0116] (9)

[0117] Step 2.4.2. Calculate the clean image predicted by the student model and background supervision signal set Each teacher prediction graph KL divergence in the channel dimension, and use Calculate the background distance vector ;

[0118] Using formula (10) to get the background supervision importance matrix :

[0119] (10)

[0120] Step 2.4.3: Use equations (11) and (12) to obtain the degraded supervision signal and background supervision signals :

[0121] (11)

[0122] (12)

[0123] Step 3: Construct the basic loss function for the bad weather removal task , degraded supervisory signal The loss function and background supervision signals The loss function , using the paradigm of multilateral distillation to calculate the image restoration network The total training loss and degradation-aware modules The total training loss .

[0124] Step 3.1: Construct the basic loss function for the bad weather removal task;

[0125] Step 3.1.1. Use Charbonnier loss as content loss :

[0126] (13)

[0127] In formula (11), Represents a constant, which is 1e−5 here to ensure the stability of convergence.

[0128] Step 3.1.2: Use contrast regularization loss as perceptual loss :

[0129] (14)

[0130] In formula (14), represents exponential operation, Pick = {1,2,···,L}, which means extracting the first Layer hidden layer features, L is the total number of hidden layers taken; > 0, is the temperature parameter that controls the output scale, |·| represents distance, is the weight coefficient; take , , .

[0131] Step 3.1.3: Use formula (15) to construct the basic loss function for the severe weather image restoration task. :

[0132] (15)

[0133] In formula (13), is a hyperparameter used to balance the content optimization objective and the perception optimization objective.

[0134] Step 3.2: Use formula (16) to construct the loss function under degradation supervision :

[0135] (16)

[0136] In formula (14), represents the KL divergence calculated along the channel dimension;

[0137] Step 3.3: Use formula (17) to construct the loss function under background supervision :

[0138] (17)

[0139] In formula (15), It is a hyperparameter used to balance the content optimization goal and the perception optimization goal. .

[0140] Step 3.4: Integrate all loss functions using the multilateral distillation paradigm.

[0141] Step 3.4.1. Use formula (18) to build the model through multilateral distillation aggregation The total training loss :

[0142] (18)

[0143] In formula (18), and It is a hyperparameter used to balance degradation supervision and background supervision, here we take , Same as the basic loss form:

[0144] (19)

[0145] In formula (19), Used to balance fidelity and perception, here we take ;

[0146] Step 3.4.2: Use formula (20) to build the perception module through multilateral distillation aggregation The total training loss :

[0147] (20)

[0148] In formula (20), Used to balance fidelity and perception, here we take .

[0149] Step 4: Build a continuous learning algorithm based on degradation-aware distillation and train the network:

[0150] Step 4.1: Enter the paired data of T types of severe weather conditions in ascending order. , continuous learning and training are performed in the form of T incremental tasks;

[0151] Step 4.2 When there is no teacher model, use formula (15) to optimize the image restoration network through end-to-end training and degradation-aware modules When the loss error is less than the set threshold, it enters the t+1 incremental task;

[0152] Step 4.3 When , the teacher model is taken from the optimal student model obtained from all previous incremental task training, first frozen Parameters of the student model are optimized using formula (18) Parameters, then freeze Parameters of , use formula (20) to update the student model When the loss error is less than the set threshold, it enters the t+1 incremental task;

[0153] Step 4.4 When the loss error is less than the set threshold, the training ends and the optimal student model with the ability to remove bad weather in the integrated image is obtained. ;

[0154] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0155] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

[0156] Example:

[0157] In order to verify the effectiveness of this method, the present invention uses three commonly used public datasets RESIDE, Rain100H and Snow100K for verification, which contain three severe weather conditions: fog, rain and snow, respectively.

[0158] The REISDE training set consists of 13,990 synthetic foggy images generated using 1,399 clean images from the existing indoor depth dataset NYU2 and the Middlebury Stereo Database. The test set consists of the Synthetic Object Test Set (SOTS) and the Hybrid Subjective Test Set (HSTS). The SOTS uses 500 indoor images from NYU2 (non-overlapping with the training images) to synthesize foggy images using the same process as the training data. The Rain100H training set consists of 1,800 images, each with four variants: rain, no-rain, rain-streak, and rain-region. The test set consists of 100 pairs of images, each with a clean version and a rain-streak version, for a total of 200 images. The Snow100K dataset consists of 100k synthetic snowy images, along with their corresponding snow-free ground truth images and snow masks, split into training and testing parts of varying sizes. It also contains 1,329 realistic snow images. It can be found that the data scale and weather type of these three sets of data are significantly different, which can accurately characterize the effect of the model in clarifying severe weather images.

[0159] When continuously learning multiple severe weather removals, the present invention adopts the most commonly used setting: fog → rain → snow, which corresponds to the streaming data in the three incremental tasks. , input the paired images of these three severe weather conditions into the image restoration network in sequence The training is completed in , and the final test will be implemented on all three severe weather conditions. The calculation of all indicators is based on the mean of all images in the test set.

[0160] Table 1 evaluates the quantitative results of the method of the present invention and the comparative method on the three tasks of fog→rain→snow. In the table, Individual Training means that three models are trained for evaluation under three weather conditions using an independent training method, which is considered to be the performance upper limit of the integrated weather removal task. Sequential Fine-tuning means that sequential training is performed on the three tasks of fog→rain→snow using a sequential fine-tuning method, which simulates the classic scenario of catastrophic forgetting caused by multiple training in real scenes. EWC represents a parameter regularization continuous learning method, LwF represents a memory-avoiding continuous learning method, MAS represents a parameter expansion continuous learning method, POD represents a knowledge cache continuous learning method, AFC represents an associative memory continuous learning method, TransWeather represents a Transformer-based integrated image restoration method, MutiTS represents a knowledge distillation-based integrated image restoration method, WGWS represents a hybrid expert-based integrated image restoration method, and RAM represents a masked autoencoder-based integrated image restoration method, which is used to compare the differences in the integrated image restoration effects. PIGWM represents a continuous learning method based on importance guidance, and CLAIO represents a continuous learning method based on sample playback. They belong to the same field as the ILAWR of the present invention and are used to compare the overall performance gap. Average represents the average value of the indicator, RESIDE represents the dehazing dataset mentioned above, Rain100H represents the deraining dataset mentioned above, and Snow100K represents the desnowing dataset mentioned above. PSNR represents the peak signal-to-noise ratio indicator, and SSIM represents the structural similarity indicator. It can be clearly seen from Table 1 that compared with the sequential fine-tuning method, the method proposed in the present invention (ILAWR) effectively alleviates catastrophic forgetting, and is superior to previous integrated and continuous learning methods in terms of objective indicators.

[0161] Table 1 Quantitative results of the proposed method and the comparative method on the fog→rain→snow tasks

[0162]

[0163] Comprehensively analyzing the three incremental tasks of fog→rain→snow, the proposed method (ILAWR) significantly outperforms many compared methods on all three datasets, thus proving the effectiveness of the proposed method.

Claims

1. A method for sharpening severe weather images based on continuous learning, characterized in that: The following steps are involved: Step 1: Obtain a set of image samples under T different severe weather conditions ,in, Indicates the A dataset of paired images of severe weather conditions, and , Indicates the The i-th degraded image under severe weather conditions, Indicates the The i-th clean image under bad weather conditions, , ; Indicates the total number of severe weather conditions, No. The total number of image sample pairs under severe weather conditions; Step 2: Severe weather conditions Teacher Model and 1 student model ;in, represents the j-th teacher model; Student Model right Process it and get The i-th student clean image under bad weather conditions and the i-th student degradation feature map ; The teacher model The i-th degraded image under severe weather conditions Process it and get A degraded supervisory signal set under severe weather conditions and background supervision signal set ; Student Model according to and ,right and Process and obtain The corresponding degradation supervision signal and Corresponding background supervision signal ; Step 3: Build A basic loss function under severe weather conditions , degraded supervisory signal The loss function and background supervision signals The loss function , and calculate the student model using the multilateral distillation paradigm Total restored losses and total degradation loss ; Step 4: Order the paired data of T kinds of severe weather conditions in ascending order. Enter separately Teacher model and student model The student model is processed and the continuous learning algorithm of degradation-aware distillation is used to The training is performed to obtain the optimal student model, which is used to remove the bad weather in the input image.

2. The method for sharpening an image in severe weather based on continuous learning according to claim 1, characterized in that: The j-th teacher model in step 2 Including: j-th image restoration model and the jth degradation-aware module ; Student Model Includes: Image restoration models , No. Degradation-aware modules Hedi Supervision Aggregation Module ; ; Step 2.1: The student model Use the Feature Projection right Process it and get The i-th student deep feature map under severe weather conditions , and then use the Output projections right Process it and get The i-th student clean image predicted under severe weather conditions ; The student model Use the Degradation-aware modules right Process it and get The i-th student degradation feature map under severe weather conditions ; Step 2.2, the j-th teacher model Using the jth feature projection For the first The i-th degraded image under severe weather conditions Processing is performed to obtain the i-th teacher deep feature map , and then use the j-th output projection right Process it and get The i-th teacher clean image predicted under severe weather conditions , thus obtaining Teacher Model Output background supervision signal set , ; The j-th teacher model Using the j-th degradation perception module For the i-th teacher deep feature map Process and get the jth teacher model Output of the i-th teacher degradation feature map ; thus obtaining Teacher Model The output degradation supervision signal set , ; Step 2.3, the Supervision Aggregation Module For the degraded supervision signal set and background supervision signal set Processing is performed to obtain the degraded supervision signal and background supervision signals .

3. The method for sharpening images in severe weather based on continuous learning according to claim 2, characterized in that: In step 2.1 Degradation-aware modules Follow these steps to get : Step 2.1.1, Perform channel average pooling operation to obtain the Student average degradation feature map across channels ; Step 2.1.2, respectively and Perform fast Fourier transform and get Amplitude spectrum of and Amplitude spectrum of , and thus use formula (1) to obtain the Information about student regression : (1) Step 2.1.3: Calculate using formula (2) The student weight tensor : (2) In formula (2), represents the linear unit activation function with Gaussian error, RDBS represents a series of dense residual connection operations and consists of dilated convolution blocks; Step 2.1.4: Use formula (3) to obtain The i-th student degradation feature map under severe weather conditions : (3) In formula (3), The convolution kernel is The convolution operation, Represents tensor multiplication.

4. The method for sharpening an image in severe weather based on continuous learning according to claim 3, characterized in that: The j-th degradation perception module in step 2.2 Follow these steps to get : Step 2.2.1, Perform channel average pooling operation to obtain the Teacher average degradation feature map across channels ; Step 2.2.2, respectively and Perform fast Fourier transform and get Amplitude spectrum of and Amplitude spectrum of , and then use formula (4) to obtain the Information about teacher degradation : (4) Step 2.2.3: Calculate using formula (5) The teacher weight tensor of : (5) Step 2.2.4: Use formula (6) to obtain The i-th teacher degradation feature map under severe weather conditions : (6)。 5. The method for sharpening images in severe weather based on continuous learning according to claim 4, characterized in that: The step 2.3 includes: Step 2.3.

1. Calculation and KL divergence in the kth channel dimension and use The function processes the KL divergence to obtain the degenerate distance vector , thus obtaining and The degradation supervision importance matrix between ,in, is the total number of channels; Step 2.3.2, calculation and KL divergence in the kth channel dimension and use Function processing KL divergence to obtain background distance , thus obtaining and The background supervision importance matrix between ; Step 2.3.3: Use equations (11) and (12) to obtain the degraded supervision signal and background supervision signals : (11) (12)。 6. The method for sharpening images in severe weather conditions based on continuous learning according to claim 5, characterized in that: The step 3 comprises: Step 3.1, build the A basic loss function under severe weather conditions ; Step 3.1.1, use formula (13) to construct Content loss under severe weather conditions : (13) In formula (13), represents a constant; Step 3.1.2: Use formula (13) to construct Perception loss under severe weather conditions : (14) In formula (14), represents exponential operation, Indicates extracting the first Layer hidden layer features, = {1,2,···,L}, L is the total number of hidden layers; > 0 is the temperature parameter that controls the output scale, and |·| distance, It is The weight coefficients of the hidden layers; Step 3.1.3: Use formula (15) to construct Basic loss function under severe weather conditions : (15) In formula (15), It is a hyperparameter used to balance the content optimization goal and the perception optimization goal; Step 3.2: Use formula (16) to construct Degraded supervision signals under severe weather conditions The loss function : (16) In formula (16), represents the KL divergence calculated along the channel dimension; Step 3.3: Use formula (17) to construct Background supervision signals under severe weather conditions The loss function : (17) In formula (17), It is a hyperparameter used to balance the content optimization goal and the perception optimization goal; Step 3.4: Integrate all loss functions using the multilateral distillation paradigm. Step 3.4.1: Use formula (18) to construct Total restoration losses : (18) In formula (18), and is a hyperparameter used to balance degradation supervision and background supervision, Indicates the The background basic loss under severe weather conditions is as follows: (19) In formula (19), Used to balance fidelity and perceived effect; Step 3.4.2: Use formula (20) to construct Total degradation loss : (20) In formula (20), Used to balance fidelity and perceived effect.

7. The method for sharpening images in severe weather conditions based on continuous learning according to claim 6, characterized in that: The step 4 comprises: when When there is no teacher model, the student model is trained end-to-end. Image restoration network in and degradation-aware modules Perform training and calculate formula (15) to update the image restoration network and degradation-aware modules When the loss error is less than the set threshold, the training is stopped and the tth optimal student model is obtained; when When t-1 optimal student models are used as t-1 teacher models, the student models are frozen first. middle The parameters of are optimized using formula (18) Parameters, then freeze The parameters of are updated using formula (20) When the loss error is less than the set threshold, the training is stopped and the tth optimal student model is obtained; and the Tth optimal student model is As an optimal image restoration model with integrated image bad weather removal capabilities.

8. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the severe weather image clearing method according to any one of claims 1 to 7, and the processor is configured to execute the program stored in the memory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the severe weather image clearing method according to any one of claims 1 to 7 are executed.

Images