Single image rain removal method, system, medium, device and terminal

Abstract

The present invention belongs to the field of image processing technology and discloses a single-image rain removal method, system, medium, device, and terminal. The method uses an improved weighted guided image filter to extract high-frequency information from raindrop images, and then uses a deep convolutional neural network to learn this high-frequency information to obtain raindrop information. A feature extraction network is used to transfer the input image and raindrop image from the image domain to the feature domain, adaptively learning useful rain removal features. A multi-scale enhancement strategy is designed, utilizing a dual-scale network with a multi-scale attention mechanism to restore high-quality images from latent features. The method uses iWGIF to extract high-frequency information from the image, and then uses a deep convolutional neural network to learn this high-frequency information to obtain raindrop information, thereby avoiding interference from other factors and improving network training efficiency. Deep learning is used to convert the input image and raindrop image from the image domain to the feature domain, adaptively utilizing features beneficial to image restoration, improving network training efficiency and rain removal performance.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a single image rain removal method, system, medium, device and terminal. Background Art

[0002] Currently, as the most common adverse weather condition, the impact of rain can reduce the visual quality of images and seriously affect the performance of outdoor vision systems. When it rains, raindrops not only produce a blurring effect in the image, but also cause haze due to light scattering, low scene visibility, and background scene occlusion. The contrast and color of the target in the image will be attenuated to varying degrees, resulting in unclear expression of background information (i.e., the target image), which makes some video or image systems unable to work properly. Therefore, it is necessary to eliminate the impact of rainfall on image scenes. In fact, image rain removal has always been an important part of image restoration and computer vision research, and is mainly used in video surveillance, autonomous driving and other fields. Currently, rain removal algorithms are mainly divided into video stream-based and image-based rain removal algorithms.

[0003] Video-based rain removal algorithms exploit the temporal correlation between adjacent frames, making it relatively easy to remove rain from videos. Garg and Nayar proposed an appearance model for rain streaks based on the photometric characteristics and temporal dynamics of rain videos. Zhang et al. assumed that the intensity changes of the R, G, and B channels caused by rainfall are roughly the same, and studied video derailment using the temporal and color characteristics of pixels in rainy videos. α-blending is used to remove rain streaks. α-blending is similar to weighted frame averaging, which is an effective method to reduce noise in static scene videos captured by fixed cameras. This algorithm is suitable for rainy static background videos, but may produce erroneous results for specific foreground colors. Chen and Zhou proposed a high-dynamic scene video rain removal algorithm based on dynamic scene motion segmentation. Spatial and temporal correlations are exploited to restore rain-free pixels.

[0004] Single image rain removal is more challenging than video-based rain removal due to the lack of temporal information and has attracted extensive attention. These methods can be divided into two categories: model-driven methods and data-driven methods.

[0005] Several priors have been used to remove rain from a single image. Assume that rain streaks R are sparse and have similar directions. A rain image O is decomposed into two layers: a rain-free background scene B and a rain streak layer R. Zhu et al. first detected the dominant rain regions and then used the detected regions as a guide image to help separate the rain streaks R from the background layer B. Liu et al. decomposed the rain image into low-frequency and high-frequency regions using a low-pass filter, capturing rain-free image details from the high-frequency region. The rain-free image details were superimposed on the low-frequency region and a dark channel prior algorithm was used to further mitigate the blur caused by rain streaks. However, the rain streaks in the resulting rain-removed image were too obvious. Li et al. proposed a rain removal method based on a Gaussian mixture model, which achieved good results in many cases. However, rain streaks still remained in the rain-removed image, making it difficult for the Gaussian mixture model to accurately estimate them. Due to air resistance and wind, the shape and direction of rain are random. Model-driven algorithms struggle to capture these regularities, and thus rain removal may not be ideal.

[0006] Data-driven single-image rain removal algorithms are very popular. Fu et al. learned the mapping relationship between the rain detail layer and the clean detail layer and added the predicted detail layer to the low-pass filter base layer to remove rain streaks. Yang et al. proposed a deep recurrent dilation network to jointly detect and remove rain streaks. Zhang et al. considered rain density and proposed a multi-task CNN to jointly estimate rain density and remove rain. Later, they further improved their work and proposed a conditional generative adversarial network for removing rain streaks in specific situations. Li et al. introduced a single-image rain removal algorithm based on multi-stage deep learning. According to the properties of the rain streak layer at each stage, α values ​​are assigned to different rain streak layers, and a recurrent neural network is combined to retain useful information from the previous stage, which is beneficial for residual prediction in the later stage. Data-driven methods rely on massive amounts of data and outperform model-driven algorithms in performance. However, due to the randomness of rainfall, even data-driven methods have difficulty capturing regularity.

[0007] Multi-scale methods are applied to single image deraining. Jiang et al. constructed a multi-scale pyramid structure and introduced an attention mechanism to guide the fine fusion of relevant information at different scales, which not only promotes cooperative representation but also promotes end-to-end training. Fu et al. introduced the mature Gaussian-Laplacian image pyramid decomposition technology into neural networks and proposed a lightweight pyramid network (LPNet) for single image deraining. Although multi-scale methods have shown strong effectiveness in these tasks, the correlation between different scales is often ignored. Different scales have different contextual information. How to effectively utilize this information remains to be explored.

[0008] Through the above analysis, the problems and defects of the existing technology are as follows:

[0009] (1) The rain marks on the derained images obtained by existing technologies are too obvious; there are still residual rain lines in the derained images; the model-driven algorithm is difficult to capture rules, and the rain removal effect is not ideal.

[0010] (2) Due to the randomness of rainfall, data-driven methods are difficult to capture regularity; when applying multi-scale technology to deraining a single image, the correlation between different scales is often ignored.

[0011] (3) Rain streak distribution and shape are diverse. Existing rain removal algorithms will cause problems such as image background blur and poor generalization performance when removing rain. Summary of the Invention

[0012] In response to the problems existing in the prior art, the present invention provides a single image rain removal method, system, medium, device and terminal, and more particularly relates to a single image rain removal method, system, medium, device and terminal based on knowledge guidance.

[0013] The present invention is implemented as follows: a single image deraining method includes: using an improved weighted guided image filter to extract high-frequency information of raindrop images, and using a deep convolutional neural network to learn the high-frequency information to obtain raindrop information; using a feature extraction network to transfer the input image and raindrop image from the image domain to the feature domain, and adaptively learning useful deraining features; designing a multi-scale enhancement strategy, and finally using a dual-scale network with a multi-scale attention mechanism to restore a high-quality image from the latent features.

[0014] Furthermore, the single image deraining method includes the following steps:

[0015] Step 1: Decompose the raindrop image using an improved weighted guided filter;

[0016] Step 2: Design a single image raindrop removal model and obtain rain removal features through feature extraction network;

[0017] Step 3: Design a feature-based deep convolutional neural network to achieve single image rain removal.

[0018] Furthermore, the step 1 of decomposing the image using the improved weighted guided filter includes:

[0019] When I is the image to be processed, G is the guide image; let Ω ζ (p) is a square window with radius ζ centered at pixel p; when I(p) is the window Ω ζ The linear transformation of the guidance image G(p) in (p′) is:

[0020]

[0021] Where a p′and b p′ is the window Ω ζ The two constants in (p′); a p′ and b p′ The optimal value of is obtained by minimizing the cost function E(a p′ ,b p′ ) is obtained, then:

[0022]

[0023] Where λ is the regularization parameter used to penalize a p′ .

[0024] Edge-aware weights Defined by the local variance of a 3×3 window of all pixels, then:

[0025]

[0026] Where M is the total number of pixels in image I and ε is a small constant.

[0027] By solving the optimization problem, we get a p′ and b p′ The optimal value of is:

[0028]

[0029] b p′ =μ I,ζ (p′)-a p′ μ G,ζ (p′),;

[0030] In the formula, the operation is the element-wise product of two matrices, then:

[0031]

[0032] Where μ G,ζ (p′), μ I,ζ (p′) and G, I and In the window Ω ζ The average value in (p′).

[0033] Furthermore, the final value of I(p) is calculated using the optimal method:

[0034]

[0035] Where W p′ is the weight factor, then:

[0036]

[0037] Where η is a small positive constant, |Ω ζ (p′)| is the set Ω ζ The cardinality of (p′).

[0038] The optimal solution is calculated as:

[0039]

[0040] and They are derived from the following formulas:

[0041]

[0042]

[0043] Where, for:

[0044]

[0045] but:

[0046]

[0047] iWGIF is used to decompose the raindrop image I into a base layer and a detail layer. The guide image G is the same as the raindrop image I. The detail layer contains high-frequency information, and most of the raindrops are in the detail layer.

[0048] Furthermore, the single image raindrop removal framework design in step 2 includes:

[0049] The rainfall image model is expressed as:

[0050] I(p)=B(p)+S(p);

[0051] Where I is the rainfall image with rain streaks, B is the background layer, S is the rain streak layer, and p is the pixel.

[0052] The rain image and raindrops are converted from the image domain to the feature domain through the feature extraction network to obtain the deraining features. The deraining enhancement branch is then used to perform image enhancement and restoration. Then:

[0053] F(I(p))=F(B(p))+F(S(p));

[0054] Where F(·) represents the feature extraction function obtained by the feature extraction network.

[0055] Furthermore, the feature-based deep convolutional neural network design in step 3 includes:

[0056] The CNN used in the single image raindrop removal framework is built on top of DeNoiseNet. The recursive residual group (RRG) in DeNoiseNet contains dual attention blocks (DAB), each of which performs spatial attention and channel attention operations simultaneously. The feature extraction network contains two RRGs and a 3*3 convolution, and the raindrop extraction network contains two 3*3 convolutions and two RRGs. The loss function is used to extract the raindrops from N pairs of images {(I,B T )} Train CNN, where I is the rainfall image, B T is a rain-free image. Using the L1 loss function to regularize the network, the loss function is:

[0057]

[0058] Another object of the present invention is to provide a single image rain removal system using the single image rain removal method, the single image rain removal system comprising:

[0059] The information extraction module is used to extract high-frequency information from raindrop images using an improved weighted guided image filter and to learn the high-frequency information using a deep convolutional neural network to obtain raindrop information.

[0060] The rain removal feature extraction module is used to build a feature extraction network, use deep learning to transfer the input image and raindrop image from the image domain to the feature domain, and adaptively learn useful rain removal features;

[0061] The image deraining module is used to design a multi-scale enhancement strategy and utilize a dual-scale network with a multi-scale attention mechanism based on deraining features to restore high-quality images from latent features.

[0062] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor executes the steps of the single image deraining method.

[0063] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the single image rain removal method.

[0064] Another object of the present invention is to provide an information data processing terminal, which is used to implement the single image rain removal system.

[0065] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0066] First, in view of the technical problems existing in the above-mentioned prior art and the difficulty of solving these problems, we closely combine the technical solutions to be protected by the present invention and the results and data during the research and development process, and conduct a detailed and in-depth analysis of how the technical solutions of the present invention solve the technical problems and some creative technical effects brought about by solving the problems. The specific description is as follows:

[0067] The present invention provides a knowledge-guided single-image rain removal method that, based on prior knowledge, can facilitate more targeted data-driven learning. Since both rain and noise are random, they both exist in high-frequency components. The high-frequency portion of the present invention can be obtained using an edge-preserving filter, and raindrop features can be learned directly from high-frequency information rather than from the input image to avoid interference from other information. The present invention then converts the input image and raindrop image from the image domain to the feature domain and adaptively extracts useful features to remove rain. Finally, the present invention utilizes a dual-scale network with a multi-scale attention mechanism to restore high-quality images from latent features, but due to model inaccuracies, errors are inevitable. The network, primarily composed of a multi-scale recursive residual group (MSRRG), can adaptively utilize useful features for rain removal based on contextual information at different scales. The present invention's prior method is highly helpful for single-image rain removal, and the algorithm has certain practical value for autonomous navigation under rainy conditions.

[0068] The knowledge in this paper guides the deep learning single image rain removal algorithm, fully utilizing the advantages of data-driven and model-driven approaches to design a reasonable and effective combination scheme to achieve the purpose of complementary advantages. The specific innovations are as follows:

[0069] 1) Raindrops, like noise, exist in high-frequency information. This approach first uses iWGIF to extract high-frequency information from images, then uses a deep convolutional neural network to learn raindrop information from this high-frequency information, thus avoiding interference from other factors and improving network training efficiency.

[0070] 2) To further improve the performance of rain removal, deep learning is used to transform the input image and raindrop image from the image domain to the feature domain, allowing them to adaptively utilize features that are beneficial to image restoration, improving network training efficiency and rain removal performance;

[0071] 3) Due to modeling inaccuracies and the output blurring defects of deep convolutional neural networks, a multi-scale enhancement strategy was designed. The overall framework adopts two scales, and the network weights of each scale share parameters to avoid computational complexity. A multi-scale attention module is used in the network structure. The contributions of features at different scales to image restoration are inconsistent, and different weights are adaptively assigned for learning, which improves the efficiency and performance of rain removal.

[0072] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by the present invention are described in detail as follows:

[0073] The knowledge-guided single-image rain removal method proposed in this paper first uses an improved weighted guided image filter (iWGIF) to extract high-frequency information and learn raindrop information to prevent other information from interfering with the input image. The input image and raindrops are then adaptively transferred from the image domain to the feature domain, learning useful features to achieve high-quality image denoising. Finally, a network with a multi-scale attention mechanism is used to restore a high-quality image from the latent features. This knowledge-guided single-image rain removal method utilizes a combined model-driven and data-driven strategy to reduce data dependence and enhance algorithm robustness.

[0074] Third, as auxiliary evidence for the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0075] The technical solution of the present invention solves the technical problems that people have been eager to solve but have never been able to solve successfully:

[0076] 1) Combining model-driven and data-driven approaches for raindrop detection improves the algorithm's robustness while reducing its reliance on data. 2) By converting the input from the image domain to the feature domain and then utilizing a deraining model for constrained learning, the data-driven approach adaptively leverages features that contribute to image restoration, enhancing training efficiency. 3) A multi-level and multi-scale attention mechanism is employed to enhance latent features. This multi-layered strategy facilitates preserving image detail and avoids excessively blurry output. Since features of different scales contribute inconsistently to image restoration, adaptively allocating more attention to features that contribute to image restoration further improves data-driven training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0078] Figure 1 This is a flow chart of a single image rain removal method provided by an embodiment of the present invention;

[0079] Figure 2 This is a schematic diagram of a single image rain removal method provided by an embodiment of the present invention;

[0080] Figure 3 2. It is a schematic diagram of the recursive residual dual attention module provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0082] In view of the problems existing in the prior art, the present invention provides a single image rain removal method, system, medium, device and terminal. The present invention is described in detail below with reference to the accompanying drawings.

[0083] like Figure 1 As shown, the single image rain removal method provided by the embodiment of the present invention includes the following steps:

[0084] S101, using an improved weighted guided image filter to extract high-frequency information from raindrop images, and using a deep convolutional neural network to learn the high-frequency information to obtain raindrop information;

[0085] S102, construct a feature extraction network, use deep learning to transfer the input image and raindrop image from the image domain to the feature domain, and adaptively learn useful deraining features;

[0086] S103, designs a multi-scale enhancement strategy and utilizes a dual-scale network with a multi-scale attention mechanism based on deraining features to recover high-quality images from latent features.

[0087] As a preferred embodiment, Figure 2 As shown, the single image deraining method combining data-driven and model-driven methods provided by the embodiment of the present invention specifically includes the following steps:

[0088] Step 1: Decompose the image using the improved weighted guided filter. Assume I is an image to be processed and G is a guided image. Let Ω ζ (p) is a square window with radius ζ centered at pixel p. Assume I(p) is the window Ω ζ Linear transformation of the guidance image G(p) in (p′):

[0089]

[0090] Among them, a p′ and b p′ is the window Ω ζ The two constants in (p′).

[0091] It should be noted that the guidance image G and the image to be processed I can be the same. p′ and b p′ The optimal value of is obtained by minimizing the cost function E(a p′ ,bp′ ) is obtained, which is defined as:

[0092]

[0093] Among them, λ is a regularization parameter that can be used to penalize a large a p′ .

[0094] Similar to WGIF, edge-aware weights It is defined by using the local variance of a 3×3 window of all pixels as follows:

[0095]

[0096] where M is the total number of pixels in image I and ε is a small constant.

[0097] By solving the optimization problem (2), we can get a p′ and b p′ The optimal value of is:

[0098]

[0099] b p′ =μ I,ζ (p′)-a p′ μ G,ζ (p′) (5)

[0100] Among them, the operation is the element-wise product of two matrices.

[0101]

[0102] Among them, μ G,ζ (p′), μ I,ζ (p′) and G, I and In the window Ω ζ The average value in (p′).

[0103] The final value of I(p) calculated using the optimal method is:

[0104]

[0105] Among them, W p′ is a weighting factor given as:

[0106]

[0107] η is a small positive constant, |Ω ζ (p′)| is the set Ω ζ The cardinality of (p′).

[0108] The optimal solution is calculated as:

[0109]

[0110] a p and b p They are derived from the following formulas:

[0111]

[0112]

[0113] in, for:

[0114]

[0115] It is easy to deduce:

[0116]

[0117] Using iWGIF, the raindrop image I is decomposed into a base layer and a detail layer. The guide image G is identical to the raindrop image I. The detail layer primarily contains high-frequency information, and most raindrops are located in the detail layer. Using the detail layer to learn raindrop features is more direct, avoids interference from other information, and improves the network's learning efficiency.

[0118] Step 2: Single image raindrop removal framework design. The widely used rainfall image model is expressed as:

[0119] I(p)=B(p)+S(p) (14)

[0120] Where I is a rain image with rain streaks, B is the background layer, S is the rain streak layer, and p is a pixel. Single-image rain removal aims to recover image B from the rain image I. However, removing raindrops from a single image is an inappropriate problem. Especially when the structure and orientation of objects are similar to raindrops, even with the strong learning ability of DCNNs, it is difficult to remove raindrops while preserving image details.

[0121] This method combines data-driven and model-driven methods to propose a new single image raindrop removal algorithm. Figure 2As shown in the figure, based on the model I(p) = B(p) + S(p) and the prior knowledge that raindrops exist in high-frequency information, directly using rain images to learn raindrop features is not ideal due to the shortcomings of DCNN in learning high-frequency information. Therefore, an improved weighted guided filter (iWGIF) is used to obtain high-frequency components, which mainly include noise, rain streaks, etc. A rain streak extraction network is proposed to obtain rain streaks. Due to the limitations of prior knowledge, the obtained high-frequency information cannot include all raindrops, and the learned raindrop features may contain inaccurate information and need further enhancement. Therefore, converting rain images and raindrops from the image domain to the feature domain can reduce the interference of noise and other useless information, fully utilize the effective information, and improve network training efficiency.

[0122] Through the feature extraction network, the rainfall image and raindrops are converted from the image domain to the feature domain to obtain the rain removal features:

[0123] F(I(p))=F(B(p))+F(S(p)) (15)

[0124] Here, F(·) represents the feature extraction function obtained by the feature extraction network. Finally, the rain removal enhancement branch (DERB) is used for image enhancement and restoration. The combination of prior knowledge and deep learning enhances the robustness and physical interpretability of the algorithm.

[0125] Step 3: Design a feature-based deep convolutional neural network. Due to the limited modeling capabilities of prior knowledge, high-frequency information cannot cover all raindrops. Furthermore, DCNNs are not good at acquiring high-frequency information, and the learned raindrop features contain inaccurate information. Therefore, it is necessary to enhance the derained image. This method uses a feature extraction network to convert rainfall images and raindrops from the image domain to the feature domain. It adaptively learns useful features, achieves high-quality image deraining, and improves network efficiency. Figure 2 and Figure 3 The algorithm flow is summarized. In order to effectively utilize useful features, the CNN used in the framework is built on top of DeNoiseNet, which achieves excellent performance in denoising. The recursive residual group (RRG) in DeNoiseNet is as follows Figure 3 As shown. RRG contains dual attention blocks (DAB), each DAB performs spatial attention and channel attention operations simultaneously. The feature extraction network contains two RRGs and a 3*3 convolution, which can suppress features with little effect and only allow features containing more information to pass. The raindrop extraction network contains two 3*3 convolutions and two RRGs. Under the action of the attention mechanism, it can pay more attention to the learning of raindrop features. The loss function is T )} plays an important role in the process of training CNN. Here, I is a rainfall image, BT is a rain-free image. In order to better regularize the network, the L1 loss function is applied. The loss function is:

[0126]

[0127] The single image rain removal system provided by an embodiment of the present invention includes:

[0128] The information extraction module is used to extract high-frequency information from raindrop images using an improved weighted guided image filter and to learn the high-frequency information using a deep convolutional neural network to obtain raindrop information.

[0129] The rain removal feature extraction module is used to build a feature extraction network, use deep learning to transfer the input image and raindrop image from the image domain to the feature domain, and adaptively learn useful rain removal features;

[0130] The image deraining module is used to design a multi-scale enhancement strategy and utilize a dual-scale network with a multi-scale attention mechanism based on deraining features to restore high-quality images from latent features.

[0131] In order to prove the creativity and technical value of the technical solution of the present invention, this section provides application examples of the claimed technical solution on specific products or related technologies.

[0132] Application Example 1

[0133] Based on the image deraining method based on a combination of data-driven and model-driven methods provided in the above embodiments, an application embodiment of the present invention provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0134] Application Example 2

[0135] Based on the image deraining method based on a combination of data-driven and model-driven methods provided in the above embodiments, an application embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the steps in the above method embodiments.

[0136] Application Example 3

[0137] Based on the image deraining method based on a combination of data-driven and model-driven methods provided in the above embodiments, an application embodiment of the present invention provides an information data processing terminal. When the information data processing terminal is implemented on an electronic device, it provides a user input interface to implement the steps in the above method embodiments. The information data processing terminal is not limited to mobile phones, computers, and switches.

[0138] Application Example 4

[0139] Based on the image deraining method based on a combination of data-driven and model-driven methods provided in the above embodiments, an application embodiment of the present invention provides a server, which is used to provide a user input interface to implement the steps in the above method embodiments when executed on an electronic device.

[0140] Application Example 5

[0141] Based on the image deraining method based on a combination of data-driven and model-driven methods provided in the above embodiments, an application embodiment of the present invention provides a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps in the above method embodiments when executing the computer program product.

[0142] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art will appreciate that the above-mentioned devices and methods can be implemented using computer-executable instructions and / or contained in processor control code, for example, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, or can be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0143] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A single image rain removal method, characterized in that: The following steps are involved: Step 1: Input processing: obtain the rainy input image I and perform preliminary processing through the improved weighted guided filter iWGIF; Step 2: Rain streak perception branch RSAB processing: the image processed by iWGIF is sequentially passed through convolution 3×3, rain streak extraction network, convolution 3×3, and feature extraction network; Step 3: The input image I is directly fed into another feature extraction network, which shares weights with the feature extraction network in the rain streak sensing branch. Step 4: Add the outputs of the two feature extraction networks above, perform channel concatenation on the added features, and obtain the fused features; Step 5: Multi-scale rain removal enhancement branch MSDERB processing: Main branch: The fused features are convolved 3×3, processed by multiple MSRRG modules in sequence, and then convolved 3×3 to output the derained image B; Sub-branch: The fusion feature is first down-sampled, then passes through convolution 3×3, four MSRRG modules, and convolution 3×3. The four MSRRG modules share the MSRRG weights of the main branch. The sub-branch output is compared with the derained image B to calculate the loss. At the same time, the derained image B is compared with the rain-free image B. T Calculate losses and jointly drive network optimization; The structure of the rain streak extraction network is: a 3×3 convolutional layer, a recursive residual module RRG, a recursive residual module RRG, and a 3×3 convolutional layer are connected in series in sequence; The structure of the feature extraction network is: a 3×3 convolutional layer, a multi-scale recursive residual group MSRRG, and a multi-scale recursive residual group MSRRG are connected in series in sequence; The feature processing of the multi-scale recursive residual group MSRRG includes: The input features are processed in three parallel channels, with different features extracted through 3×3 convolutional layers, 5×5 convolutional layers, and 7×7 convolutional layers respectively. After each convolutional output, a dual attention module DAB is connected. The three DAB output channels are spliced, the features are fused through a 3×3 convolutional layer, and then added point by point to the input of the MSRRG to achieve a residual connection. The feature processing of the dual attention module DAB includes: The input features are sequentially transformed through a 3×3 convolutional layer, a ReLU activation function layer, and a 3×3 convolutional layer; Then perform parallel calculations on the attention branches: Spatial attention branch: For the above transformed features, global average pooling and global maximum pooling are performed in parallel. After splicing the output channels of the two, they are passed through a 3×3 convolution layer and a sigmoid activation function layer to generate spatial attention weights, which are multiplied point by point with the initial transformed features. Channel attention branch: Perform global average pooling on the transformed features, pass through a 1×1 convolution layer, a ReLU activation function layer, a 1×1 convolution layer, and a Sigmoid activation function layer to generate channel attention weights, which are multiplied point by point with the initial transformed features. Feature fusion and residual connection: The output channels of the spatial attention and channel attention branches are concatenated, compressed by a 1×1 convolutional layer, and then added point by point to the output of the initial feature transformation to achieve residual connection.

2. The single image rain removal method according to claim 1, wherein: The decomposition of the image using the improved weighted guided filter in step 1 includes: When I is the image to be processed, G is the guide image; let Ω ζ (p) is a square window with radius ζ centered at pixel p; when I(p) is the window Ω ζ The linear transformation of the guidance image G(p) in (p′) is: Where a p′ and b p′ is the window Ω ζ The two constants in (p′); a p′ and b p′ The optimal value of is obtained by minimizing the cost function E(a p′ ,b p′ ) is obtained, then: Where λ is the regularization parameter used to penalize a p′ ; Edge-aware weights Defined by the local variance of a 3×3 window of all pixels, then: Where M is the total number of pixels in image I, and ε is a small constant; By solving the optimization problem, we get a p′ and b p′ The optimal value of is: b p′ =μ I,ζ (p′)-a p′ m G,ζ (p′),; Where the operation ⊙ is the element-wise product of two matrices, then: cov I,G,ζ (p′)=μ G⊙I,ζ (p′)-μ G,ζ (p′)μ I,ζ (p′),; Where μ G,ζ (p′), μ I,ζ (p′) and μ G⊙I,ζ (p′) are G, I and G⊙I in the window Ω ζ The average value in (p′).

3. The single image rain removal method according to claim 2, wherein: The final value of I(p) calculated using the optimal method is: Where W p′ is the weight factor, then: Where η is a small positive constant, |Ω ζ (p′)| is the set Ω ζ the cardinality of (p′); The optimal solution is calculated as: and They are derived from the following formulas: Where, for: but: iWGIF is used to decompose the raindrop image I into a base layer and a detail layer. The guide image G is the same as the raindrop image I. The detail layer contains high-frequency information, and most of the raindrops are in the detail layer.

4. The single image rain removal method according to claim 1, wherein: The rainfall image model is expressed as: I(p)=B(p)+S(p); Where I is the rainfall image with rain streaks, B is the background layer, S is the rain streak layer, and p is the pixel; The rain image and raindrops are converted from the image domain to the feature domain through the feature extraction network to obtain the deraining features. The deraining enhancement branch is then used to perform image enhancement and restoration. Then: F(I(p))=F(B(p))+F(S(p)); Where F(·) represents the feature extraction function obtained by the feature extraction network.

5. The single image rain removal method according to claim 1, wherein: It also includes using the L1 loss function to regularize the network, so the loss function is:

6. A single image rain removal system using the single image rain removal method according to any one of claims 1 to 5, characterized in that: The single image deraining system includes: The information extraction module is used to extract high-frequency information from raindrop images using an improved weighted guided image filter and to learn the high-frequency information using a deep convolutional neural network to obtain raindrop information. The rain removal feature extraction module is used to build a feature extraction network, use deep learning to transfer the input image and raindrop image from the image domain to the feature domain, and adaptively learn useful rain removal features; The image deraining module is used to design a multi-scale enhancement strategy and utilize a dual-scale network with a multi-scale attention mechanism based on deraining features to restore high-quality images from latent features.

7. A computer device, characterized in that: The computer device includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor performs the steps of the single image deraining method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the single image rain removal method according to any one of claims 1 to 5.

9. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the single image rain removal system as claimed in claim 6.

Images