A Single Image De-raining Method and System Based on a Multi-stage Network

Abstract

The present invention proposes a single-image de-raining method and system based on a multi-stage network, which obtains an image to be de-rained; constructs a first image de-raining network and a second image de-raining network based on a channel attention mechanism; constructs a third image de-raining network based on a channel attention mechanism and a spatial attention mechanism; inputs the image to be de-rained into the first image de-raining network, and the first image de-raining network outputs a first de-rained feature map; inputs the image to be de-rained and the first de-rained feature map into the second image de-raining network, and the second image de-raining network outputs a second de-rained feature map; inputs the image to be de-rained and the second de-rained feature map into the third image de-raining network, and the third image de-raining network outputs a de-rained image. The present invention improves the accuracy of de-raining while enhancing the generalization ability of the network and the ability to protect and restore the background details of the image.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more specifically, to a single image rain removal method and system based on a multi-stage network. Background Art

[0002] Rainy days not only affect human vision, but also affect artificial intelligence image processing tasks. On the one hand, rain streaks will block the image, and on the other hand, the superposition of rain streaks will also cause blur and deformation of the image background. Therefore, removing rain from rainy images to obtain rain-free images has become a necessary step in image processing tasks.

[0003] There is a single image deraining method based on an autoencoder convolutional neural network. The method decomposes the rainy image to be derained into a low-frequency base layer and a high-frequency detail layer, and uses a deraining model based on an autoencoder convolutional neural network to derain the high-frequency detail layer to generate a high-frequency detail layer after deraining. The high-frequency detail layer after deraining is then enhanced, and the image after deraining is determined based on the low-frequency base layer and the enhanced high-frequency detail layer.

[0004] The shapes of rain are complex and varied, including rain lines, raindrops, etc. The influence of light can also cause problems such as different colors and blurring of rain. In addition, the different angles of the falling direction of rain can also lead to deviations in the recognition of rain. Therefore, removing rain from images is a highly complex problem. The above method uses a single-stage convolutional neural network to remove rain from a single image. The accuracy of rain removal is limited, and it is difficult to repair the rain image in one go. In addition, in the process of removing rain streaks, some background details close to the shape of rain may be removed together, and the restored rain image may also be distorted. Summary of the invention

[0005] The present invention provides a single image rain removal method and system based on a multi-stage network to improve the accuracy of image rain removal and the protection and restoration capabilities of image background details.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] In a first aspect, the present invention proposes a single image deraining method based on a multi-stage network, comprising the following steps:

[0008] Get the rain-removing image.

[0009] Based on the channel attention mechanism, the first image deraining network and the second image deraining network are constructed.

[0010] Based on the channel attention mechanism and the spatial attention mechanism, a third image de-raining network is constructed. The first image de-raining network, the second image de-raining network, and the third image de-raining network constitute a multi-stage network single-image de-raining network.

[0011] The image to be de-rained is input into the first image de-raining network, and the first image de-raining network outputs the first de-rained feature map.

[0012] The image to be de-rained and the first de-rained feature map are input into the second image de-raining network, and the second image de-raining network outputs the second de-rained feature map.

[0013] The image to be de-rained and the second de-rained feature map are input into the third image de-raining network, and the third image de-raining network outputs the de-rained image.

[0014] As a preferred solution, the first image de-raining network includes a convolutional layer, a channel attention module, an encoder-decoder network, and a supervised attention module. Among them:

[0015] After the image to be de-rained is input into the first image de-raining network, the image to be de-rained undergoes a convolutional operation through the convolutional layer and then is transmitted to the channel attention module for feature extraction. The channel attention module outputs the feature extraction result. The feature extraction result undergoes another convolutional operation through the convolutional layer and then is processed by the encoder-decoder network and the supervised attention module in sequence. The supervised attention module outputs the first de-rained feature map.

[0016] As a preferred solution, the second image de-raining network includes a convolutional layer, a channel attention module, an encoder-decoder network, and a supervised attention module. Among them:

[0017] After the image to be de-rained is input into the second image de-raining network, the image to be de-rained undergoes a convolutional operation through the convolutional layer and then is transmitted to the channel attention module for feature extraction. The channel attention module outputs the feature extraction result. The feature extraction result undergoes another convolutional operation through the convolutional layer and then is superimposed with the first de-rained feature map. The result of the superimposed operation is processed by the encoder-decoder network and the supervised attention module in sequence. The supervised attention module outputs the second de-rained feature map.

[0018] As a preferred solution, the third image de-raining network includes a convolutional layer, a channel attention module, a spatial attention module, and a supervised attention module. Among them:

[0019] The image to be de-rained is input into the third image de-raining network, and the spatial attention module extracts the spatial information of the image to be de-rained to obtain the spatial information of the image to be de-rained.

[0020] Meanwhile, the image to be de-rained is transmitted to the channel attention module for feature extraction after performing a convolution operation through a convolutional layer. The channel attention module outputs the feature extraction result. The feature extraction result is then subjected to a convolution operation through a convolutional layer and superimposed with the second de-rained feature map. The result of the superimposition operation is processed by the original resolution module and a convolutional layer in sequence, and then superimposed with the spatial information of the image to be de-rained to obtain the de-rained image.

[0021] As a preferred solution, the encoder-decoder network includes an encoder and a decoder; the encoder includes m downsampling modules connected in sequence; the decoder includes m upsampling modules connected in sequence; each downsampling module of the encoder is skip-connected to the upsampling module of the corresponding level of the decoder.

[0022] As a preferred solution, the channel attention module includes a first input channel and a second input channel.

[0023] After the image to be de-rained performs a convolution operation, it is respectively input into the first input channel and the second input channel. The first input channel and the second input channel respectively output feature data, and the feature data is superimposed with the image to be de-rained after performing a convolution operation to obtain the feature extraction result.

[0024] The first input channel includes a first convolutional layer, a first RELU activation layer, a second convolutional layer, an average pooling layer, and a Sigmod activation layer connected in sequence.

[0025] The second input channel includes a third convolutional layer, a second RELU activation layer, a fourth convolutional layer, a fifth convolutional layer, a third RELU activation layer, a sixth convolutional layer, and a Sigmod activation layer connected in sequence.

[0026] As a preferred solution, before inputting the image to be de-rained into the first image de-raining network, the image to be de-rained is sliced once, and the image to be de-rained is divided into 4 small pictures of equal size; before inputting the image to be de-rained into the second image de-raining network, the image to be de-rained is sliced once, and the image to be de-rained is divided into 2 small pictures of equal size.

[0027] As a preferred solution, the single-image de-raining method further includes training the multi-stage network single-image de-raining network, specifically including the following steps:

[0028] Step A: Set the parameters of the multi-stage network single-image de-raining network.

[0029] Step B: Input the image to be de-rained into the multi-stage network single-image de-raining network, and the multi-stage network single-image de-raining network outputs the de-rained image.

[0030] Step C: Calculate the loss function according to the de-rained image.

[0031] Step D: Update the parameters of the multi-stage network single-image de-raining network through backpropagation according to the calculated loss function.

[0032] Step E: Repeat loop steps B - D to continuously update the parameters of the multi-stage network single-image de-raining network until the de-raining accuracy rate of the multi-stage network single-image de-raining network converges.

[0033] As an optimal solution, the de-raining method further includes evaluating the multi-stage network single-image de-raining network with the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) as the evaluation criteria.

[0034] In a second aspect, the present invention also proposes a single-image de-raining system based on a multi-stage network, which is applied to the single-image de-raining method based on a multi-stage network described in any of the above solutions, and includes:

[0035] A data acquisition module for acquiring the image to be de-rained.

[0036] The multi-stage network single-image de-raining network includes a first image de-raining network, a second image de-raining network, and a third image de-raining network. The first image de-raining network and the second image de-raining network are both provided with channel attention modules. The third image de-raining network is provided with a channel attention module and a spatial attention module.

[0037] Input the image to be de-rained into the first image de-raining network, and the first image de-raining network outputs a first de-rained feature map. Input the image to be de-rained and the first de-rained feature map into the second image de-raining network, and the second image de-raining network outputs a second de-rained feature map. Input the image to be de-rained and the second de-rained feature map into the third image de-raining network, and the third image de-raining network outputs a de-rained image.

[0038] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0039] (1) By constructing a three-stage network of a first image de-raining network, a second image de-raining network, and a third image de-raining network to perform de-raining processing on the image to be de-rained, using the de-rained feature map of the previous stage network to guide the learning of the next stage network, so as to make full use of the useful information of the previous stage network to achieve multi-stage de-raining and improve the de-raining accuracy rate.

[0040] (2) By using the channel attention mechanism and the spatial attention mechanism to focus on the features of the image to be de-rained, improving the feature extraction ability of the de-raining network, thereby improving the generalization ability of the de-raining network and the ability to protect and restore the details of the image background. Description of the Drawings

[0041] Figure 1 It is a flowchart of a single-image rain removal method based on a multi-stage network.

[0042] Figure 2 It is the schematic diagram of the multi-stage network single-image rain removal network in Embodiment 2.

[0043] Figure 3 It is the schematic diagram of the channel attention in Embodiment 2.

[0044] Figure 4 It is the schematic diagram of the encoding-decoding network in Embodiment 2.

[0045] Figure 5 It is the schematic diagram of the supervised attention module in Embodiment 2.

[0046] Figure 6 It is the schematic diagram of the spatial attention module in Embodiment 2.

[0047] Figure 7 It is the schematic diagram of the original resolution module in Embodiment 2.

[0048] Figure 8 It is a flowchart for training and evaluating the multi-stage network single-image rain removal network in Embodiment 3.

[0049] Figure 9 It is the architecture diagram of the single-image rain removal system based on the multi-stage network. Detailed implementation manners

[0050] The attached drawings are only for illustrative purposes and cannot be construed as limitations on this patent;

[0051] The technical solutions of the present invention will be further described below with reference to the attached drawings and embodiments.

[0052] Embodiment 1

[0053] This embodiment proposes a single-image rain removal method based on a multi-stage network. Refer to Figure 1 , Figure 1 It is a flowchart of a single-image rain removal method based on a multi-stage network, including the following steps:

[0054] Obtain the image to be de-rained. Based on the channel attention mechanism, construct the first image rain removal network and the second image rain removal network; based on the channel attention mechanism and the spatial attention mechanism, construct the third image rain removal network; the first image rain removal network, the second image rain removal network, and the third image rain removal network constitute the multi-stage network single-image rain removal network.

[0055] Input the image to be de-rained into the first image rain removal network, and the first image rain removal network outputs the first de-rained feature map.

[0056] Input the image to be de-rained and the first de-rained feature map into the second image de-raining network, and the second image de-raining network outputs the second de-rained feature map.

[0057] Input the image to be de-rained and the second de-rained feature map into the third image de-raining network, and the third image de-raining network outputs the de-rained image.

[0058] In the specific implementation process, first input the image to be de-rained into the first image de-raining network to perform feature learning in the first stage on the rain stripe features of the image to be de-rained, and obtain the first de-rained feature map containing a large amount of image feature information. Then transmit the image to be de-rained and the first de-rained feature map obtained from the first stage of learning to the second image de-raining network. The second image de-raining network combines the first de-rained feature map to perform feature learning on the image to be de-rained, and obtains the second de-rained feature map containing a large amount of image feature information. Finally, transmit the image to be de-rained and the second de-rained feature map obtained from the second stage of learning to the third image de-raining network. The third image de-raining network combines the second de-rained feature map to perform feature learning on the image to be de-rained, and obtains the de-rained image with rain stripes removed.

[0059] In this embodiment, the first image de-raining network, the second image de-raining network, and the third image de-raining network are all provided with channel attention modules, and perform feature learning and extraction on the image to be de-rained through the channel attention mechanism, improving the utilization rate of image features.

[0060] In this embodiment, the third image de-raining network is provided with a spatial attention module, which extracts the spatial information of the image to be de-rained and suppresses the redundant information of the image to be de-rained.

[0061] The present invention performs de-raining processing on the image to be de-rained by constructing a multi-stage network single-image de-raining network, and uses the de-rained feature spectrogram of the previous stage network to guide the learning of the next stage network, so as to make full use of the useful information of the previous stage network to achieve multi-stage de-raining and improve the accuracy of de-raining. At the same time, through the channel attention mechanism and the spatial attention mechanism, the image de-raining network focuses on the features of the image to be de-rained during the training process, improving the feature extraction ability of the network, thereby improving the generalization ability of the network and the protection and restoration ability of the image background details.

[0062] Embodiment 2

[0063] This embodiment makes improvements on the single-image de-raining method based on a multi-stage network proposed in Embodiment 1.

[0064] As Figure 2 shown, Figure 2This is the schematic diagram of the multi-stage network single-image de-raining network in this embodiment. The multi-stage network single-image de-raining network consists of a first image de-raining network, a second image de-raining network, and a third image de-raining network.

[0065] In this embodiment, the size of the image to be de-rained is fixed at 256*256. The multi-stage network single-image de-raining network can restore images of a unified size, which can reduce the debugging difficulty of the network model.

[0066] In the specific implementation process, first, the image to be de-rained is sliced once, and the image to be de-rained is divided into 4 small pictures of equal size and input into the first image de-raining network. The first image de-raining network includes a convolutional layer, a channel attention module, an encoding-decoding network, and a supervised attention module. After the image to be de-rained is input into the first image de-raining network, the image to be de-rained undergoes a convolutional operation in the convolutional layer and then is transmitted to the channel attention module (CAM, Channel Attention Module) for feature extraction. The channel attention module outputs the feature extraction result. The feature extraction result undergoes another convolutional operation in the convolutional layer and then is processed by the encoding-decoding network and the supervised attention module in sequence. The supervised attention module outputs the first de-rained feature map and the predicted image of the first stage.

[0067] The image to be de-rained is sliced once, and the image to be de-rained is divided into 2 small pictures of equal size and input into the second image de-raining network. The second image de-raining network includes a convolutional layer, a channel attention module, an encoding-decoding network, and a supervised attention module. After the image to be de-rained is input into the second image de-raining network, the image to be de-rained undergoes a convolutional operation in the convolutional layer and then is transmitted to the channel attention module for feature extraction. The channel attention module outputs the feature extraction result. The feature extraction result undergoes another convolutional operation in the convolutional layer and then is superimposed and calculated with the first de-rained feature map. The result of the superimposed operation is processed by the encoding-decoding network and the supervised attention module (SAM, Supervised Attention Module) in sequence. The supervised attention module outputs the second de-rained feature map and the predicted image of the second stage.

[0068] In this embodiment, by slicing the image to be de-rained, the multi-stage network single-image de-raining network can obtain more detailed information. After the image to be de-rained is sliced, the multi-stage network single-image de-raining network does not process the entire picture each time, but performs operations on each slice separately. The first image de-raining network and the second image de-raining network can obtain local rain feature information through slicing. The local rain feature information is more accurate than processing the entire picture. In the multi-stage network single-image de-raining network, each stage of the image de-raining network can infer the global features of the entire picture with the help of local features after obtaining the local information of the previous stage of the image de-raining network. Through such multi-scale feature information extraction, the rain features can be extracted more accurately.

[0069] In this embodiment, an improved channel attention is used to extract features from the image to be de-rained. As Figure 3 shown, Figure 3 is the schematic diagram of the channel attention in this embodiment. The channel attention includes a first input channel and a second input channel. After the image to be de-rained performs a convolution operation, it is respectively input into the first input channel and the second input channel. The first input channel and the second input channel respectively output feature data, and the feature data is superimposed and calculated with the image to be de-rained after performing a convolution operation to obtain a feature extraction result. The first input channel includes a first convolutional layer, a first RELU activation layer, a second convolutional layer, an average pooling layer, and a Sigmod activation layer connected in sequence. The second input channel includes a third convolutional layer, a second RELU activation layer, a fourth convolutional layer, a fifth convolutional layer, a third RELU activation layer, a sixth convolutional layer, and a Sigmod activation layer connected in sequence. In the improved channel attention, a parallel structure is added to the input unit. Instead of linearly extracting features from the input image data, the features are extracted through a parallel structure, which improves the utilization rate of the image features and thus improves the accuracy of the multi-stage network single-image de-raining network.

[0070] In this embodiment, the encoder-decoder network (u-net) includes an encoder and a decoder. The encoder includes 3 downsampling modules connected in sequence. The decoder includes 3 upsampling modules connected in sequence. Each downsampling module of the encoder is skip-connected to the corresponding-level upsampling module of the decoder. Through the encoder-decoder network, upsampling and downsampling operations are performed on the image to obtain the deep semantic information of the image. The encoder-decoder network first encodes the input image, that is, performs a downsampling operation, and then transmits the result of the encoder to the decoder for upsampling operation to restore the complete image. As Figure 4 shown, Figure 4This is the schematic diagram of the encoding and decoding network in this embodiment. Skip connections are made between the encoder and the decoder, so that the shallow layers of the image can be retained. Among them, the encoding and decoding network in the first image deraining network only accepts the image data that has undergone feature extraction and is transmitted by the channel attention module; the encoding and decoding network in the second image deraining network, in addition to accepting the image data that has undergone feature extraction and is transmitted by the channel attention module, also accepts the feature information in the first deraining feature map and then performs encoding and decoding.

[0071] In this embodiment, the supervised attention module (SAM, Supervised Attention Module) receives the operation results of the encoding and decoding network and the original image to be derained, as Figure 5 shown. Figure 5 This is the schematic diagram of the supervised attention module in this embodiment. The supervised attention module performs supervised calculation on the original image to be derained to help suppress the features with less information in the current stage. The supervised attention module will output two results. One is the predicted image of the network in the current stage, and the other is the deraining feature map containing a large amount of information. This deraining feature map will be transmitted to the network in the next stage for further operations.

[0072] Input the image to be derained into the third image deraining network. The third image deraining network includes a convolutional layer, a channel attention module, a spatial attention module, and a supervised attention module. The spatial attention module (SAB, Spatial Attention Block) extracts the spatial information of the image to be derained to obtain the spatial information of the image to be derained. At the same time, the image to be derained is transmitted to the channel attention module for feature extraction after performing a convolutional operation through the convolutional layer. The channel attention module outputs the feature extraction result. The feature extraction result is superimposed and calculated with the second deraining feature map after a convolutional operation. The result of the superimposed operation is processed by the original resolution module (Original Resolution Block) and a convolutional layer in sequence, and then superimposed and calculated with the spatial information of the image to be derained to obtain the derained image.

[0073] In this embodiment, as Figure 6 shown. Figure 6 This is the schematic diagram of the spatial attention module in this embodiment. The spatial attention module is only used in the third image deraining network. The spatial attention module extracts the spatial information of the image to be derained as much as possible through a series of convolutions, and uses this spatial information to help the restoration of the image, so that while removing the rain, the relevant information of the original background details of the image can be retained as much as possible.

[0074] In this embodiment, the ORB algorithm is used to operate on the de-rained feature map information extracted in three stages by the original resolution module to restore the image, so as to retain the original details of the image and prevent some background details of the image from being removed together when removing rain streaks. As Figure 7 shown, Figure 7 is the schematic diagram of the original resolution module in this embodiment. The original resolution module consists of two CAB modules and one convolutional layer.

[0075] Embodiment 3

[0076] This embodiment makes improvements on the single-image de-raining method based on a multi-stage network proposed in Embodiment 2.

[0077] In this embodiment, the test set and the training set are respectively constructed by using the image to be de-rained, and the multi-stage network single-image de-raining network is trained and evaluated, as Figure 8 shown, Figure 8 is the flowchart for training and evaluating the multi-stage network single-image de-raining network in this embodiment.

[0078] In this embodiment, the data in the training set is input into the multi-stage network single-image de-raining network to train the multi-stage network single-image de-raining network, which specifically includes the following steps:

[0079] Step A: Set the parameters of the multi-stage network single-image de-raining network. The parameters of the multi-stage network single-image de-raining network include the image input format, learning rate, data batch size, number of training times, etc.

[0080] Step B: Input the image to be de-rained into the multi-stage network single-image de-raining network, and the multi-stage network single-image de-raining network outputs the de-rained image.

[0081] Step C: Calculate the loss function according to the de-rained image.

[0082] Step D: Update the parameters of the multi-stage network single-image de-raining network by backpropagation according to the calculated loss function.

[0083] Step E: Repeat steps B - D in a loop, continuously update the parameters of the multi-stage network single-image de-raining network until the de-raining accuracy of the multi-stage network single-image de-raining network converges.

[0084] In this embodiment, taking the peak signal-to-noise ratio PSNR (Peak Signal to Noise Ratio) and the structural similarity SSIM (Structural Similarity) as the evaluation criteria, the data in the test set is input into the multi-stage network single-image de-raining network to evaluate the multi-stage network single-image de-raining network.

[0085] The calculation formula for Peak Signal-to-Noise Ratio (PSNR) is as follows:

[0086]

[0087]

[0088] Among them, MSE represents the mean square error between the image to be de-rained and the de-rained image, col represents the number of columns of the input image matrix, raw represents the number of rows of the input image matrix, X represents the input image to be de-rained, Y represents the output de-rained image, i represents the abscissa of the image, j represents the ordinate of the image, and n is a constant that can be set.

[0089] Peak Signal-to-Noise Ratio (PSNR) is an objective standard for evaluating images. The larger the PSNR value, the less distortion the image represents.

[0090] The calculation formula for Structural Similarity (SSIM) is as follows:

[0091]

[0092] Among them, μ X represents the average value of image X, μ Y represents the average value of image Y, σ XY σ XY represents the covariance of image X and image Y, represents the variance of image X, represents the variance of image Y, c 1 and c 2 are arbitrary constants respectively.

[0093] Structural Similarity (SSIM) is an index for measuring the similarity between two images. The value range of structural similarity is from -1 to 1. When image X and image Y are exactly the same, the value of SSIM is equal to 1.

[0094] Evaluate the multi-stage network single-image de-raining network according to the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), and further optimize the multi-stage network single-image de-raining network according to the evaluation results.

[0095] Example 4

[0096] This example proposes a single-image de-raining system based on a multi-stage network, as Figure 9 shown, Figure 9It is an architecture diagram of a single-image de-raining system based on a multi-stage network, including a data acquisition module and a multi-stage network single-image de-raining network. The multi-stage network single-image de-raining network includes a first image de-raining network, a second image de-raining network, and a third image de-raining network; both the first image de-raining network and the second image de-raining network are provided with channel attention modules; the third image de-raining network is provided with a channel attention module and a spatial attention module.

[0097] In the specific implementation process, the data acquisition module acquires the image to be de-rained. The image to be de-rained is input into the first image de-raining network. After a convolution operation, the image to be de-rained is transmitted to the channel attention module for feature extraction. The channel attention module outputs the feature extraction result. After a convolution operation, the feature extraction result passes through an encoding-decoding network and a supervised attention module for processing. The supervised attention module outputs the first de-rained feature map and the predicted image of the first stage.

[0098] The image to be de-rained is input into the second image de-raining network. After a convolution operation, the image to be de-rained is transmitted to the channel attention module for feature extraction. The channel attention module outputs the feature extraction result. After a convolution operation, the feature extraction result is superimposed with the first de-rained feature map. The superimposed operation result passes through an encoding-decoding network and a supervised attention module for processing. The supervised attention module outputs the second de-rained feature map and the predicted image of the second stage.

[0099] The image to be de-rained is input into the third image de-raining network. The spatial attention module extracts the spatial information of the image to be de-rained to obtain the spatial information of the image to be de-rained. At the same time, after a convolution operation, the image to be de-rained is transmitted to the channel attention module for feature extraction. The channel attention module outputs the feature extraction result. After a convolution operation, the feature extraction result is superimposed with the second de-rained feature map. The superimposed operation result passes through the original resolution module and a convolution operation, and then is superimposed with the spatial information of the image to be de-rained to obtain the de-rained image.

[0100] The terms describing the positional relationship in the drawings are only for illustrative purposes and should not be construed as a limitation of this patent;

[0101] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or variations can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A single-image de-raining method based on a multi-stage network, characterized in that, it includes the following steps: Obtain the image to be de-rained; Based on the channel attention mechanism, construct the first image de-raining network and the second image de-raining network; Based on the channel attention mechanism and the spatial attention mechanism, construct the third image de-raining network; the first image de-raining network, the second image de-raining network and the third image de-raining network form a multi-stage network single-image de-raining network; Divide the image to be de-rained into 4 equal-sized small pictures and input them into the first image de-raining network, and the first image de-raining network outputs the first de-rained feature map; After dividing the image to be de-rained into 2 equal-sized small pictures, input them together with the first de-rained feature map into the second image de-raining network, and the second image de-raining network outputs the second de-rained feature map; Input the image to be de-rained and the second de-rained feature map into the third image de-raining network, and the third image de-raining network outputs the de-rained image; The third image de-raining network includes a convolutional layer, a channel attention module, a spatial attention module and a supervised attention module; among them: Input the image to be de-rained into the third image de-raining network, and the spatial attention module extracts the spatial information of the image to be de-rained to obtain the spatial information of the image to be de-rained; At the same time, the image to be de-rained is transmitted to the channel attention module for feature extraction after performing a convolutional operation through the convolutional layer. The channel attention module outputs the feature extraction result. The feature extraction result is then subjected to a convolutional operation through the convolutional layer and then superimposed with the second de-rained feature map. The result of the superimposed operation is processed by the original resolution module and a convolutional layer in sequence, and then superimposed with the spatial information of the image to be de-rained to obtain the de-rained image.

2. The single-image de-raining method based on a multi-stage network according to claim 1, characterized in that, The first image de-raining network includes a convolutional layer, a channel attention module, an encoder-decoder network and a supervised attention module; among them: After the image to be de-rained is input into the first image de-raining network, the image to be de-rained is transmitted to the channel attention module for feature extraction after performing a convolutional operation through the convolutional layer. The channel attention module outputs the feature extraction result. The feature extraction result is then subjected to a convolutional operation through the convolutional layer and then processed by the encoder-decoder network and the supervised attention module in sequence. The supervised attention module outputs the first de-rained feature map.

3. The single-image de-raining method based on a multi-stage network according to claim 1, characterized in that, The second image de-raining network includes a convolutional layer, a channel attention module, an encoder-decoder network and a supervised attention module; among them: After the image to be de-rained is input into the second image de-raining network, the image to be de-rained is transmitted to the channel attention module for feature extraction after performing a convolution operation through a convolutional layer. The channel attention module outputs the feature extraction result. The feature extraction result is then transmitted to a convolutional layer for another convolution operation and then superimposed with the first de-rained feature map. The result of the superimposed operation is processed by an encoder-decoder network and a supervised attention module in sequence, and the supervised attention module outputs the second de-rained feature map.

4. The single-image de-raining method based on a multi-stage network according to claim 2 or 3, wherein, the encoder-decoder network includes an encoder and a decoder; the encoder includes m downsampling modules connected in sequence; the decoder includes m upsampling modules connected in sequence; each downsampling module of the encoder is skip-connected to the corresponding-level upsampling module of the decoder.

5. The single-image de-raining method based on a multi-stage network according to any one of claim 4, wherein, the channel attention module includes a first input channel and a second input channel; after the image to be de-rained performs a convolution operation, it is respectively input into the first input channel and the second input channel. The first input channel and the second input channel respectively output feature data, and the feature data is superimposed with the image to be de-rained after performing a convolution operation to obtain the feature extraction result; the first input channel includes a first convolutional layer, a first RELU activation layer, a second convolutional layer, an average pooling layer, and a Sigmod activation layer connected in sequence; the second input channel includes a third convolutional layer, a second RELU activation layer, a fourth convolutional layer, a fifth convolutional layer, a third RELU activation layer, a sixth convolutional layer, and a Sigmod activation layer connected in sequence.

6. The single-image de-raining method based on a multi-stage network according to claim 1, wherein, the single-image de-raining method further includes training the multi-stage network single-image de-raining network, specifically including the following steps: Step A: Set the parameters of the multi-stage network single-image de-raining network; Step B: Input the image to be de-rained into the multi-stage network single-image de-raining network, and the multi-stage network single-image de-raining network outputs the de-rained image; Step C: Calculate the loss function according to the de-rained image; Step D: Update the parameters of the multi-stage network single-image de-raining network by backpropagation according to the calculated loss function; Step E: Repeat steps B - D in a loop to update the parameters of the multi-stage network single-image de-raining network until the de-raining accuracy of the multi-stage network single-image de-raining network converges.

7. The single-image de-raining method based on a multi-stage network according to claim 1, wherein, the de-raining method further includes evaluating the multi-stage network single-image de-raining network using peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) as evaluation criteria.

8. A single-image de-raining system based on a multi-stage network, applied to the single-image de-raining method based on a multi-stage network according to any one of claims 1 to 7, wherein, it includes: a data acquisition module for acquiring the image to be de-rained; A multi-stage network single-image rain removal network, including a first image rain removal network, a second image rain removal network, and a third image rain removal network; both the first image rain removal network and the second image rain removal network are provided with channel attention modules; the third image rain removal network is provided with a channel attention module and a spatial attention module; Input the rain-containing image to be removed into the first image rain removal network, and the first image rain removal network outputs a first rain-removed feature map; input the rain-containing image to be removed and the first rain-removed feature map into the second image rain removal network, and the second image rain removal network outputs a second rain-removed feature map; input the rain-containing image to be removed and the second rain-removed feature map into the third image rain removal network, and the third image rain removal network outputs a rain-removed image.

Images