Image rain removal method based on double-domain contrast learning

Abstract

The invention discloses an image rain removal method based on double-domain comparative learning, which comprises the following steps: step 1, establishing an unsupervised image rain removal model based on double-domain comparative learning, the model comprising a rain stripe extractor and a generator, the rain stripe extractor is used for extracting rain stripe features in an image with rain in a space domain and a frequency domain through comparative learning and reconstructing a rain stripe layer, and the generator is used for reconstructing a background layer on the input image with rain by using the rain stripe layer and outputting a background image; 2, performing adversarial training on the unsupervised image rain removal model on an unsupervised data set by using a discriminator, and obtaining a final pre-training model by comparing loss optimization model parameters such as a learning loss function and a frequency domain amplitude loss function; and a third step of using the pre-training model to carry out rain removal processing on an input image with rain to obtain a clear background image.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and image processing, and in particular to an image deraining method based on dual-domain contrast learning. Background Art

[0002] Image deraining technology aims to restore a clear background from rain-disturbed images, which is crucial for improving image quality and enhancing the performance of computer vision systems. In practical applications, image deraining technology can be widely used in traffic monitoring, security surveillance, autonomous driving, and other fields to ensure that systems can function properly in rainy conditions.

[0003] Existing image deraining methods are mainly divided into supervised learning and unsupervised learning methods. Supervised learning methods typically require a large amount of paired training data: rainy images and their corresponding clear background images. These methods achieve image deraining by learning the mapping relationship between rainy images and background images. However, in practical applications, obtaining a large amount of paired training data is very difficult, as the acquisition of rainy images is limited by weather conditions, and obtaining the corresponding clear background images also requires additional equipment and time. Therefore, supervised learning methods are limited in practical applications.

[0004] Unsupervised learning methods, however, do not require paired training data and instead leverage information from rainy images to remove rain. These methods typically achieve image deraining by learning intrinsic features of rainy images, such as the characteristics of rain streaks. However, existing unsupervised methods often focus solely on the differences between rainy images and background images, ignoring the intrinsic characteristics of rain streaks. Fully utilizing these features would significantly improve image deraining performance. Summary of the Invention

[0005] The main purpose of this invention is to propose an unsupervised image deraining method based on dual-domain contrastive learning, which extracts rain streak features and reconstructs the background layer by performing contrastive learning in the spatial domain and the frequency domain, thereby achieving high-quality image deraining.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] An unsupervised image deraining method based on dual-domain contrastive learning comprises the following steps: a first step, establishing an unsupervised image deraining model based on dual-domain contrastive learning, the model comprising a rain streak extractor and a generator, the rain streak extractor being used to extract rain streak features in a rainy image in the spatial domain and the frequency domain through contrastive learning and to reconstruct a rain streak layer, the generator being used to reconstruct a background layer on an input rainy image using the rain streak layer and output a background image; a second step, using a discriminator to perform adversarial training on the unsupervised image deraining model on an unsupervised dataset, optimizing model parameters through a contrastive learning loss function and a frequency domain amplitude loss function, and obtaining a final pre-trained model; and a third step, using the pre-trained model to derain the input rainy image to obtain a clear background image.

[0008] Furthermore, the rain streak extractor adopts an encoder-decoder architecture, wherein the encoder extracts the deep features of the rain streaks in the spatial domain and the frequency domain, and the decoder reconstructs and outputs the rain streak layer in the spatial domain.

[0009] Furthermore, the encoder in the rain streak extractor takes similar image blocks of the same scene as positive samples and image blocks of different scenes as negative samples, and extracts the depth features of rain streaks in the spatial domain and frequency domain through comparative learning; the decoder in the rain streak extractor takes the depth features of the rain streaks as input and reconstructs the rain streak layer.

[0010] Furthermore, the generator adopts an encoder-decoder architecture, wherein the encoder takes the rainy image and the corresponding rain streak layer as input to extract high-level features of the image, and the decoder uses the high-level features to restore the spatial resolution of the image to generate the background image.

[0011] Furthermore, the encoder in the generator includes multiple convolutional layers and activation functions, which gradually extract the high-level features of the image through downsampling operations; the decoder in the generator gradually restores the spatial resolution of the image through skip connections and upsampling operations to generate the background image.

[0012] Furthermore, the second step specifically includes: training the unsupervised image deraining model using rainy images and background images in the unsupervised dataset, wherein the training framework includes two interactive paths, the first path being rainy-no-rainy-rainy, and the second path being no-rainy-rainy-no-rainy;

[0013] The first half of the first path, rain-no rain, includes: the model receives the rain image R in the training set as input, uses the rain streak extractor to extract rain streak features from the input rain image R and reconstructs the first rain streak layer RS R , and then use the rain generator (G B) From the rainy image R and the first rain streak layer RS R Rebuilding the background layer The first discriminator (D B ) then receives the reconstructed background layer Conduct confrontation training;

[0014] The first half of the second path without rain and with rain includes: the model receives the background image B as input, uses the rain streak extractor to extract rain streak features from the input background image B and reconstructs the second rain streak layer RS B , and then use the rain generator (G R ) From the background image B and the first rain streak layer RS R Reconstructing fake rainy images The second discriminator (D R ) Receive fake rainy images Conduct adversarial training; the second half of the second path rain-no rain includes: using rain streak extractor to extract fake rainy images Extract the fake rain streak layer Using the rain removal generator (G B )From the fake rain streak layer and fake rainy images Restore the background image

[0015] The second half of the first path without rain and with rain includes: using the rain generator (G R ) from the reconstructed background layer and fake rain streak layer Restore the original rainy image

[0016] Furthermore, in the second step, the contrastive learning loss function is based on the rain streak layer output by the rain streak extractor and is implemented by the following steps: the rain streak extractor extracts rain streak features from the rainy image, including features in the spatial domain and the frequency domain; wherein: in the spatial domain, the rain streak extractor uses similar image blocks of the same scene as positive samples and image blocks of different scenes as negative samples; in the frequency domain, the rain streak extractor uses the amplitude spectrum features of the rain streak layer to enhance the feature extraction capability through contrastive learning; the contrastive learning loss function measures the similarity between positive samples and negative samples, and is implemented by calculating the difference between the similarity of the positive sample pair and the similarity of the negative sample pair; wherein the similarity is calculated by the inner product of the feature vector.

[0017] Furthermore, in the second step, the frequency domain amplitude loss function is based on the amplitude spectrum characteristics of the rain streak layer and the background layer, and is implemented by the following steps: performing fast Fourier transform on the rain streak layer and the background layer respectively to calculate their respective amplitude spectra; extracting stripe features from the calculated amplitude spectrum, wherein the stripe features are significant in the rain streak layer and absent in the background layer; calculating the frequency domain amplitude loss by comparing the amplitude spectrum characteristics of the rain streak layer and the background layer; the frequency domain amplitude loss function measures the presence of rain streak components in the background layer, and enhances the deraining effect by minimizing the frequency domain amplitude loss.

[0018] Furthermore, the third step specifically includes: inputting the rainy image to be processed into the trained rain streak extractor to reconstruct the rain streak layer; inputting the reconstructed rain streak layer and the rainy image to be processed into the trained generator together for rain removal processing, and outputting a clear background image.

[0019] The present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned image deraining method can be implemented.

[0020] The beneficial effects of the technical solution of the present invention are reflected in the following: the present invention makes full use of the characteristics of rain streaks and their specific morphology and distribution patterns in the image, and performs comparative learning in the spatial domain and the frequency domain. This not only improves the accuracy of feature extraction in the reconstruction process and enhances the deraining effect, thereby obtaining a clearer background image (derained image), but also enhances the robustness of the model, making it perform better when processing complex scenes.

[0021] At the same time, based on the mutual exclusivity of rain streaks and background images in the amplitude spectrum, the present invention optimizes model parameters using the frequency domain amplitude loss function, which can more effectively remove rain streaks and reconstruct high-quality background images.

[0022] In addition, the present invention does not require paired rain-preserving image-deraining image training data, which reduces the difficulty and cost of data acquisition and is more feasible and economical in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of an image deraining method based on dual-domain contrast learning according to an embodiment of the present invention.

[0024] Figure 2 Schematic diagram of a rain streak extractor of an unsupervised image rain removal model based on dual-domain contrastive learning in an embodiment of the present invention.

[0025] Figure 3 This is an architecture diagram of a rain streak extractor and generator of an unsupervised image deraining model based on dual-domain contrastive learning in an embodiment of the present invention.

[0026] Figure 4 3. It is a schematic diagram of the training process of the unsupervised image deraining model based on dual-domain contrastive learning in an embodiment of the present invention.

[0027] Figure 5 2 is a schematic diagram of frequency domain loss of an image deraining method based on dual-domain contrast learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below with reference to the accompanying drawings, specific implementation methods, and examples, which are provided for the purpose of illustration only and are not intended to be limiting.

[0029] Rain streaks have specific morphologies and distribution patterns in images. For example, they are typically distributed linearly and exhibit specific characteristics in the spatial and frequency domains. Therefore, fully leveraging these characteristics will help improve image deraining. Based on this, an embodiment of the present invention proposes an unsupervised image deraining method based on dual-domain contrastive learning. By performing contrastive learning in both the spatial and frequency domains, rain streak features are extracted and the background layer is reconstructed to obtain a clear background image, achieving high-quality image deraining.

[0030] Please refer to Figure 1 The image deraining method based on dual-domain contrast learning in this embodiment of the present invention mainly includes a training phase and a testing phase. The specific steps are as follows:

[0031] Training phase——

[0032] A1. Network Construction: An unsupervised image deraining model based on dual-domain contrastive learning is constructed using a rain streak extractor (RSE) with an encoder-decoder architecture and a generator. The rain streak extractor uses dual-domain features to extract rain streaks, which assist the generator in reconstructing a rain-free background image on a rainy image. Specifically, the rain streak extractor extracts rain streak features from rainy images in both the spatial and frequency domains through contrastive learning, reconstructing a rain streak layer. The generator uses the rain streak layer output by the rain streak extractor to reconstruct a background layer on the input rainy image, outputting a derained background image.

[0033] A2. Network training: Construct a discriminator to determine the authenticity of the background image output by the generator; use the discriminator to perform adversarial training on an unsupervised dataset; use contrastive learning loss function and frequency domain amplitude loss function to optimize model parameters and obtain the final pre-trained model.

[0034] Testing phase

[0035] A3. Use the model trained in step A2 to remove rain from the input rainy image to obtain a clear background image.

[0036] The unsupervised image deraining method proposed in the embodiments of the present invention performs image deraining in two steps: First, the rainy image is preprocessed, and a rain streak extractor (RSE) is used to extract the rain streak layer from the image. This layer is then enhanced by contrastive learning in the spatial and frequency domains. Second, a deraining generator is constructed. This network consists of a shallow feature extraction and stacked forward modules. The extracted rain streaks are used to extract the background layer of the rainy image. Finally, a network training process is constructed, with two interactive paths constructed and the intermediate results introduced into the discriminator for adversarial training. During the training process, model parameters are optimized using losses such as contrastive learning loss and frequency-domain amplitude loss. Ultimately, an ideal deraining model is obtained that can input rainy images and reconstruct high-quality background images.

[0037] A further detailed description is given below.

[0038] 1. Model: The rain removal model in this embodiment of the present invention primarily consists of a rain streak extractor (RSE) and a generator, both of which are implemented using an encoder-decoder architecture. For an input rainy image, the RSE first extracts rain streak features in both the spatial and frequency domains and reconstructs a rain streak layer. The generator then uses the RSE-extracted rain streak layer to reconstruct the background layer on the rainy image.

[0039] Specifically, in the rain streak extractor, the encoder uses similar image patches from the same scene as positive samples and image patches from different scenes as negative samples. Through comparative learning, it extracts deep features of rain streaks in both the spatial and frequency domains. The decoder, on the other hand, takes these deep features as input and reconstructs the rain streak layer through operations such as skip connections and convolutions. The generator uses a multi-layer convolutional network to extract features and reconstruct the background from the input image. The encoder, consisting of multiple convolutional layers and activation functions, takes the rain-staining image and its rain streak layer as input and gradually extracts high-level features through downsampling. The decoder utilizes these high-level features to gradually restore the spatial resolution of the image through skip connections and upsampling, generating a de-rained background image. In other words, the generator's input consists of the rain-staining image and its rain streak layer.

[0040] The structure of the rain streak extractor RSE is as follows Figure 2 As shown in Figure 1, an encoder-decoder architecture is used. Amp is a frequency domain processing layer that converts the image R into the frequency domain through Fourier transform and extracts the amplitude spectrum of the three channels. The encoder extracts deep features F from the rain stripes in the rain image R. R , the decoder uses the encoder's skip connection (Skip Connection) to gradually convert F RReconstructed into rain streak layer. The final function of RSE is to take the rain image R as input and generate its rain streak layer RS R .

[0041] In some specific embodiments, the rain streak extractor RSE adopts Figure 3 The encoder-decoder architecture shown in Figure 2. Downsampling is achieved using convolutions combined with pixel unscaling operations, where the convolution operation uses a 3×3 kernel with 1 padding and a stride of 1, reducing the number of channels by half; the pixel unscaling operation restores the original channel count while halving the spatial dimensions. Similarly, upsampling is performed using convolutions and pixel unscaling operations, where the number of channels is first increased through convolution, and then the pixel unscaling operation restores the original channel count while doubling the spatial dimensions. Figure 3 The 3×3 block in the example represents a two-dimensional convolutional block with a kernel size of 3, padding of 1, and stride of 1, while the 1×1 block represents a two-dimensional convolutional block with a kernel size of 1, padding of 0, and stride of 1. These blocks, together with the forward block (FB), form the encoder-decoder architecture.

[0042] For the rain streak extractor RSE, its forward block uses the residual block (RB), and the number of forward blocks from n1 to n7 is set to 1. Figure 3 As shown in Figure 2, each residual block RB contains two parallel paths. RB effectively extracts rain streak features through residual convolution operations, and the formula is as follows:

[0043] F out =GELU(Conv 1×1 ((LN(F in ))+F res ),

[0044] F res =Conv 3×3 (LN(GELU(Conv 3×3 (LN(F in ))))),

[0045] Among them, F in 、F out Represents the input and output of the residual block RB, F res represents the intermediate features extracted by the residual block, and F in 、F out and F res ∈R H×W×C , R H×W×C Represents a three-dimensional feature with a dimension of H×W×C, LN represents a LayerNorm module with a bias, and GELU is an activation function.

[0046] In the embodiment of the present invention, the generator also uses Figure 3 The encoder-decoder architecture shown in the figure has the following number of FBs specified in each stage: n1=1, n2=2, n3=2, n4=4, n5=2, n6=2, and n7=3. In the generator, the residual depth convolution block (RDB) is used as the forward block FB. In order to enhance the generator's ability to use the rain streak layer, as shown in Figure 3 , RDB is designed with two parallel processing paths: one performs residual addition; the other uses a deep convolution that mimics the channel attention mechanism. This operation can be described by the following formula:

[0047] F out =scale*LN(F in )+Conv 1×1 (F res ),

[0048] F res =Dconv 3×3 (Conv 1×1 (LN(F in )))*GeLU(Dconv 3×3 (Conv 1×1 (LN(F in )))),

[0049] Among them, F in 、F out Respectively represent the input and output of the residual depth convolution block RDB, F res represents the intermediate features extracted by the residual block, and F in 、F out and F res ∈R H×W×C ; scale∈R C is an adjustable factor, Dconv 3×3 It is similar to a 3×3 depthwise convolution.

[0050] 2. Training part:

[0051] Please refer to Figure 4 The training framework of the embodiment of the present invention consists of two interactive paths: with rain-no rain-with rain and no rain-with rain-no rain.

[0052] For the first half of the "rainy-no-rainy" process in the first path, the model accepts a rainy picture R as input, and the rain streak extractor RSE extracts the rain streak layer RS ​​from the rainy image R R , the generator uses the rain image R and the rain streak layer RS ​​extracted by RSE R Rebuilding the background layer Discriminator D B Then receive the reconstructed background layer Conduct adversarial training.

[0053] For the first half of the "no rain-with rain" process in the second path, the model receives the background image B as input, uses the rain streak extractor RSE to extract the rain streak features from the input background image B and reconstructs the corresponding rain streak layer RS B , and then use the rain generator G R From the background image B and the rain streak layer RS ​​of the first path R Reconstructing a fake rainy image Discriminator D R Receive fake rainy images Conduct adversarial training.

[0054] For the second half of the "rain-no rain" process in the second path, the rain streak extractor RSE is used to extract the false rainy image Extract the fake rain streak layer Using the deraining generator G B From the fake rain streak layer and fake rainy images Restore the background image The second half of the first path is the "no rain - with rain" process. On the one hand, the rain streak extractor RSE is used to extract the rain streak from the reconstructed background layer. Extract the fake rain streak layer On the other hand, using the rain generator G R From the reconstructed background layer And the fake rain streak layer obtained by the second path Restore the original rainy image

[0055] As can be seen, the second half of the first path, with rain, is the opposite of the first, forming a cyclical adversarial process. The second path is similar to the first, but with rain-with rain-no rain.

[0056] In the first path, RSE uses similar image patches from the same scene as positive samples and image patches from different scenes as negative samples. In the frequency domain, RSE leverages the amplitude spectrum features of the rain streak layer and enhances feature extraction through contrastive learning. The contrastive learning loss function measures the similarity between positive and negative samples by calculating the difference between the similarity of a positive sample pair and the similarity of a negative sample pair. The similarity is calculated using the inner product of the feature vectors.

[0057] At the output of the generator, the training process performs a fast Fourier transform (FFT) on the rain streak layer and the background layer to calculate their amplitude spectra. Stripe-like features are extracted from the amplitude spectra. These features are prominent in the rain streak layer and absent in the background layer. By comparing the amplitude spectral features of the rain streak layer and the background layer, a frequency domain amplitude loss is calculated. This frequency domain amplitude loss function measures the presence of rain streak components in the background layer. Minimizing this frequency domain amplitude loss enhances rain removal.

[0058] The rainy image R is modeled as the superposition of the background image B and the rain streak layer RS, that is:

[0059] R = B + RS;

[0060] like Figure 4 As shown, the first half of the two paths in training can be expressed as

[0061] RS R =RSE(R), RS B =RSE(B), where RS R and RS B Represent the rain streak layers extracted from the rainy image R and the background image B respectively. Generate RS R Finally, two generators G with the same structure but different responsibilities B and G R Will use RS R For rainwater removal and rainwater addition:

[0062]

[0063] Among them, G B and G R represents the rain removal generator and rain addition generator, Represents G B Using R and RS R The generated fake rain-free image, Represents G R Using B and RS R Then the two fake images are generated Input the discriminator D respectively R and D B Conduct adversarial training.

[0064] To construct a cycle-consistent transformation, the second half of the framework performs the inverse operation of the first half to recover the original rainy image and background image from the fake image:

[0065]

[0066] in, and Represent the reconstructed versions of R and B respectively.

[0067] like Figure 4 As shown, during the training process, RSE will perform contrastive learning. It compares R and R + (Two image patches cropped from rainy images of the same scene) are considered as positive sample pairs. At the same time, image patches R from different scenes - are used as negative samples. The RSE encoder extracts deep features F from these image patches. R 、F R+ and F R- From these deep features, a contrastive learning loss function is constructed to guide the learning process:

[0068]

[0069] Among them, fea R =MLR(F R ),and and And so on. τ and K are the hyperparameters set by the method, which are 0.07 and 8192 respectively in this example. MLP is a pooling linear layer used to transform the features F extracted by Encoder into R Convert to one-dimensional space for contrastive learning.

[0070] like Figure 5 As shown, the model uses the amplitude spectrum of the rain streak layer and the background layer to construct the frequency domain amplitude loss The specific implementation algorithm is as follows:

[0071]

[0072]

[0073] The core of the algorithm (PEOCESS) is to use a statistical method to extract the stripe area in the amplitude spectrum. For each point in the amplitude spectrum, the corresponding mean (mean) and standard deviation (std) are calculated within a fixed-size window centered on the point. Points with values ​​less than mean+std×std_th are considered to be outside the stripe area and are therefore filtered out. In addition, since the signals of natural images are mainly concentrated in the central circular and cross-shaped areas of the amplitude spectrum, the embodiment of the present invention excludes these areas from the loss calculation based on low_th and crs_th to maintain the integrity of the background layer. After filtering, any remaining overlap in the two layers of the amplitude spectrum indicates that the background layer still contains rain streak signals. The size of the loss reflects the degree of this overlap. low_th, crs_th, and std_th take the values ​​of 0.1, 0.03, and 1.5, respectively.

[0074] The total loss function during training is as follows:

[0075]

[0076] Among them, λ1=1, λ2=10, λ3=0.01, λ4=1, λ5=10, λ6=1, loss function and The definition of has been given above.

[0077] Fighting Losses The definition is as follows:

[0078]

[0079] Consistency loss The definition is as follows:

[0080]

[0081] Perceptual loss The definition is as follows:

[0082]

[0083] Among them, VGG16 is an open source model.

[0084] Rain streak pixel loss The definition is as follows:

[0085]

[0086] During model training, for all RSEs, the generator and the discriminator have their own Adam optimizer (lr = 0.001, β1 = 0.5, β1 = 0.999). After multiple iterative training, the optimal network parameters are obtained to obtain the final deraining model.

[0087] 3. Reasoning (Test) Section:

[0088] The trained rain removal model accepts the rainy image to be processed, first extracts its rain ripple layer, and then uses the rain ripple layer to reconstruct a clear background image on the rainy image.

[0089] Another embodiment of the present invention further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the image deraining method of the aforementioned embodiment. Based on this understanding, the image deraining method of the present invention can be embodied in the form of a software product, which can be stored on a non-volatile storage medium (such as a CD-ROM, USB flash drive, or removable hard drive) and includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute the method of the present invention in various implementation scenarios.

[0090] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art will recognize that several equivalent substitutions or obvious variations can be made without departing from the scope of the present invention, and that any equivalent performance or application should be considered to fall within the scope of protection of the present invention.

Claims

1. An image deraining method based on dual-domain contrastive learning, characterized in that: The following steps are involved: The first step is to establish an unsupervised image rain removal model based on dual-domain contrastive learning. The model includes a rain streak extractor and a generator. The rain streak extractor is used to extract rain streak features in rainy images in the spatial and frequency domains through contrastive learning and reconstruct the rain streak layer. The generator is used to reconstruct the background layer on the input rainy image using the rain streak layer and output the background image. The second step is to use the discriminator to perform adversarial training on the unsupervised image deraining model on the unsupervised dataset, and optimize the model parameters by comparing the learning loss function and the frequency domain amplitude loss function to obtain the final pre-trained model; The third step is to use the pre-trained model to perform rain removal processing on the input rainy image to obtain a clear background image.

2. The image deraining method according to claim 1, wherein: The rain streak extractor adopts an encoder-decoder architecture, wherein the encoder extracts the deep features of the rain streaks in the spatial domain and the frequency domain, and the decoder reconstructs and outputs the rain streak layer in the spatial domain.

3. The image deraining method according to claim 2, wherein: The encoder in the rain streak extractor uses similar image blocks of the same scene as positive samples and image blocks of different scenes as negative samples, and extracts the deep features of rain streaks in the spatial domain and frequency domain through comparative learning; the decoder in the rain streak extractor uses the deep features of the rain streaks as input to reconstruct the rain streak layer.

4. The image deraining method according to any one of claims 1 to 3, wherein: The generator adopts an encoder-decoder architecture, wherein the encoder takes the rainy image and the corresponding rain streak layer as input to extract high-level features of the image, and the decoder uses the high-level features to restore the spatial resolution of the image to generate the background image.

5. The image deraining method according to claim 4, wherein: The encoder in the generator includes multiple convolutional layers and activation functions, which gradually extract the high-level features of the image through downsampling operations; the decoder in the generator gradually restores the spatial resolution of the image through skip connections and upsampling operations to generate the background image.

6. The image deraining method according to any one of claims 1 to 5, characterized in that: The second step specifically includes: using rainy images and background images in the unsupervised dataset to train the unsupervised image deraining model, wherein the training framework includes two interactive paths, the first path is rainy-no-rainy-rainy, and the second path is no-rainy-rainy-no-rainy; The first half of the first path, rain-no rain, includes: the model receives the rain image R in the training set as input, uses the rain streak extractor to extract rain streak features from the input rain image R and reconstructs the first rain streak layer RS R , and then use the rain generator (G B ) From the rainy image R and the first rain streak layer RS R Rebuilding the background layer The first discriminator (D B ) then receives the reconstructed background layer Conduct confrontation training; The first half of the second path without rain and with rain includes: the model receives the background image B as input, uses the rain streak extractor to extract rain streak features from the input background image B and reconstructs the second rain streak layer RS B , and then use the rain generator (G R ) From the background image B and the first rain streak layer RS R Reconstructing fake rainy images The second discriminator (D R ) Receive fake rainy images Conduct adversarial training; the second half of the second path rain-no rain includes: using rain streak extractor to extract fake rainy images Extract the fake rain streak layer Using the rain removal generator (G B )From the fake rain streak layer and fake rainy images Restore the background image The second half of the first path without rain and with rain includes: using the rain generator (G R ) from the reconstructed background layer and fake rain streak layer Restore the original rainy image 7. The image deraining method according to claim 6, wherein: In the second step, the contrastive learning loss function is based on the rain streak layer output by the rain streak extractor and is implemented by the following steps: The rain streak extractor extracts rain streak features from rainy images, including features in the spatial domain and the frequency domain. In the spatial domain, the rain streak extractor uses similar image blocks of the same scene as positive samples and image blocks of different scenes as negative samples. In the frequency domain, the rain streak extractor utilizes the amplitude spectrum features of the rain streak layer and enhances feature extraction capabilities through contrastive learning. The contrastive learning loss function measures the similarity between positive and negative samples by calculating the difference between the similarity of positive sample pairs and the similarity of negative sample pairs. The similarity is calculated by the inner product of the feature vectors.

8. The image deraining method according to claim 6, wherein: In the second step, the frequency domain amplitude loss function is based on the amplitude spectrum characteristics of the rain streak layer and the background layer, and is implemented by the following steps: Perform fast Fourier transform on the rain streak layer and the background layer respectively to calculate their respective amplitude spectra; Extracting stripe features from the calculated amplitude spectrum, wherein the stripe features are significant in the rain streak layer but absent in the background layer; By comparing the amplitude spectrum characteristics of the rain streak layer and the background layer, the frequency domain amplitude loss is calculated; The frequency domain amplitude loss function measures the presence of rain streak components in the background layer, and the rain removal effect is enhanced by minimizing the frequency domain amplitude loss.

9. The image deraining method according to claim 1, wherein: The third step specifically includes: inputting the rainy image to be processed into the trained rain streak extractor to reconstruct the rain streak layer; inputting the reconstructed rain streak layer and the rainy image to be processed into the trained generator for rain removal processing, and outputting a clear background image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image deraining method according to any one of claims 1 to 9 can be implemented.

Images