Image rain removal method based on wavelet transform and digital filtering detail guidance

Abstract

The application relates to the fields of deep learning and image processing, and discloses an image rain removal method based on wavelet transform and digital filter detail guidance, which comprises applying a trained image rain removal model to process an input rainy image, and the method comprises the following steps: acquiring the rainy image, inputting the rainy image into the image rain removal model for processing, and then outputting a rain-removed image. By introducing image prior knowledge, the application can better adapt to different image rain removal tasks, effectively suppresses image noise in combination with digital filtering, thereby improving the quality and precision of recovery, and improves the accuracy and robustness of image rain removal by fusing feature information of different scales.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of deep learning and image processing, and particularly relates to an image rain removal method based on wavelet transform and digital filtering detail guidance. BACKGROUND

[0002] The basic goal of image rain removal is to restore a clear image as close to the real scene as possible from a low-quality image affected by raindrops, rain fog, etc. before performing downstream tasks, because the deterioration of image quality (such as raindrops, haze, noise, etc.) not only affects the visual effect of the image, but also has a negative impact on the accuracy of image processing and the accuracy of downstream tasks, leading to false recognition and judgment. SUMMARY

[0003] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art. The present application provides an image rain removal method based on wavelet transform and digital filtering detail guidance. By introducing image prior knowledge, the method can better adapt to different image rain removal tasks, and effectively suppress image noise by combining digital filtering, thereby improving the quality and accuracy of the recovered image. By fusing feature information of different scales, the accuracy and robustness of image rain removal are improved.

[0004] To achieve the above-mentioned purpose, the first aspect of the present application provides an image rain removal method based on wavelet transform and digital filtering detail guidance, which comprises applying a trained image rain removal model to process an input rainy image. The method comprises: obtaining a rainy image, inputting the rainy image into the image rain removal model for processing, and then outputting a rain-removed image.

[0005] The image rain removal model processes the input rainy image, comprising the following steps:

[0006] S1, learnable wavelet decomposition:

[0007] The input rainy image is decomposed into multiple levels of low-frequency subbands and high-frequency subbands by a learnable wavelet;

[0008] S2, detail modulation:

[0009] The rainy image and the multiple levels of low-frequency subbands are subjected to level-by-level band-pass filtering to obtain multiple levels of detail information. The multiple levels of detail information are fused, and the background information and rain layer information in the rainy image are subjected to detail modulation by the fused detail information to obtain background features and rain layer features.

[0010] S3, dual-branch processing and dynamic interactive perception decoupling:

[0011] The background feature is processed through a background branch capable of effectively preserving low-frequency structure information of the rainy image, and the rain layer feature is processed through a rain layer branch capable of obtaining rain streak distribution through global modeling;

[0012] The background branch at least includes: processing the background feature through a depth separable convolution, then processing the background feature through a multi-scale grouping convolution, performing SE channel attention enhancement on the background feature enhanced by the multi-scale grouping convolution, performing weighted residual fusion processing on the background feature after SE channel attention enhancement and the original background feature, and finally outputting the background feature processed by the background branch;

[0013] The outputs of the background branch and the rain layer branch are processed through a dynamic interactive perception mechanism, the dynamic interactive perception mechanism is a global attention obtained by integrating global information through a global gating weight, a local attention is generated through spatial modulation, and the global attention and the local attention are fused through an attention weight to obtain decoupled background features and rain layer features;

[0014] S4, image reconstruction is performed on the decoupled background features and rain layer features to obtain a rain-removed image.

[0015] Optionally, in S2, the rainy image is sequentially band-pass filtered with the multi-level low-frequency subbands to obtain multi-level detail information, the multi-level detail information is fused, and the background information and the rain layer information in the rainy image are modulated by the fused detail information to obtain background features and rain layer features, including:

[0016] Sequentially subtracting low-frequency subbands to obtain a medium-frequency detail feature BP;

[0017] Generating a detail attention map for the medium-frequency detail feature BP through convolution , using a Sigmoid activation function to convert to generate rain layer detail weight d, and background detail weight is 1-d;

[0018] The background feature is obtained by performing element-wise multiplication between the initial background feature generated by the last low-frequency subband through convolution and the background detail weight 1-d;

[0019] The rain layer feature is obtained by performing convolution on the initial rain layer feature generated by the high-frequency subband through feature fusion and the rain layer detail weight d.

[0020] Optionally, the depth separable convolution processing in S3 includes:

[0021] Performing a depth convolution operation, the depth convolution operation will independently perform convolution operation on each input channel without cross-channel information fusion to extract local structure information;

[0022] Through the point convolution operation, information fusion is performed on each channel using a 1*1 convolution kernel.

[0023] Optionally, the multi-scale grouped convolution processing in S3 includes:

[0024] The background feature after the depth separable convolution processing is divided into multiple groups, and each group is processed by a convolution kernel.

[0025] The convolution results of different groups are spliced along the channel dimension to obtain a merged feature map, that is, the background feature after the multi-scale grouped convolution processing.

[0026] Optionally, the SE channel attention enhancement of the background feature after the multi-scale grouped convolution enhancement in S3 includes:

[0027] Global information of each channel is obtained through global average pooling;

[0028] An attention weight of each channel is generated through a two-layer fully connected network; and

[0029] The background feature after the multi-scale grouped convolution enhancement is weighted processed using the generated attention weight of each channel, and finally the background feature after the SE channel attention enhancement is obtained.

[0030] Optionally, the SE channel attention enhanced background feature and the original background feature are connected along the channel through a weighted residual fusion processing in S3, and finally the background feature processed by the background branch is output, including:

[0031] The SE channel attention enhanced background feature and the original background feature are connected along the channel;

[0032] A layer of depth separable convolution is used to further extract the feature connected along the channel;

[0033] An SE attention enhanced fusion output; and

[0034] A learnable residual weight is introduced A weighted residual connection is performed, and finally the background feature processed by the background branch is output.

[0035] Optionally, the dynamic interaction perception mechanism in S3 is to synthesize global information through a global gating weight, that is, global attention, and to generate local attention through spatial modulation, including:

[0036] Global attention:

[0037] The background enhancement feature and the rain layer enhancement feature are connected along the channel dimension to form a joint feature, denoted as:

[0038] ;

[0039] wherein, is the background feature processed by the background branch, is the rain layer feature processed by the rain layer branch, denotes the joint feature;

[0040] The global context information is integrated by adaptive average pooling to generate a single-point global description, denoted as:

[0041] ;

[0042] wherein, denotes the single-point global description formed, avoiding interference of spatial positions, denotes global average pooling;

[0043] A gating network is introduced, using a multi-layer convolution sequence, adding an activation function layer by layer, and applying a soft-maximum activation to generate global gating weights, denoted as:

[0044] ;

[0045] wherein, is a learnable parameter, is a channel compression ratio, is a Sigmoid function, denotes a Relu activation function, denotes the generated background gating weight, denotes the generated rain layer gating weight, denotes g B and g R comprise a list;

[0046] Local spatial attention:

[0047] Spatial attention is applied to the joint feature using a multi-layer convolution to generate a pixel-level spatial weight mask, denoted as:

[0048] ;

[0049] wherein, is a Sigmoid function, is a convolution weight, denotes the shape of the pixel-level spatial weight mask , batch, channel number, width and height, respectively.

[0050] Optionally, in S3, the global attention and the local attention are fused through attention weight to obtain the decoupled background feature and the rain layer feature, including:

[0051] A learnable parameter is introduced to control the relationship between the global gating weight and the local spatial attention weight, and is expressed as:

[0052] ;

[0053] wherein, the background layer weight after attention weight fusion is represented by B, the rain layer weight after attention weight fusion is represented by R, the pixel-level spatial weight mask is represented by M, the learnable parameter is represented by w,

[0054] The normalized weight is applied to the features output by the background branch and the rain layer branch to obtain the decoupled background feature and the rain layer feature, and is expressed as:

[0055] ;

[0056] ;

[0057] wherein, the decoupled background feature is represented by B, the output of the background branch before decoupling is represented by B, the normalized background weight is represented by B, the decoupled rain layer feature is represented by R, the output of the rain layer branch before decoupling is represented by R, the normalized rain layer weight is represented by R, and epsilon is a constant to ensure numerical stability and avoid division by zero and gradient explosion, the element-wise multiplication is represented by.

[0058] After the above technical solution is adopted, the present application has the following beneficial effects compared with the prior art:

[0059] In the present application, the learnable wavelet transform and the multi-modal data fusion technology are introduced to effectively solve the separation problem of complex background and rain layer in image rain removal. Compared with the traditional single modal or simple fusion method, the present application can more accurately separate the background image and the rain layer feature, significantly improve the accuracy and detail restoration ability of image rain removal, especially in the face of high noise, blur or dynamic change scene, the weight of each modal can be adaptively adjusted to ensure the stability and accuracy of the recovery operation;

[0060] This application combines a dual-branch processing method for background and rain layer decoupling with a detail modulation mechanism to further optimize performance in complex image environments. By processing background and rain layer information separately through a dual-branch structure, interference from background and rain layer features is effectively avoided, making the restoration process more accurate and efficient in complex environments. Compared with traditional methods, this application provides higher accuracy when processing complex image background and rain layer mixtures, especially demonstrating superior performance in detail restoration.

[0061] This application employs a guided details mechanism and a multi-scale separation strategy, and optimizes memory usage through depthwise separable convolution, enabling the image deraining process to reduce unnecessary computational overhead while maintaining high accuracy. Compared with traditional deep networks, this application optimizes the feature extraction and fusion process, avoids redundant overcomputation, and improves inference efficiency. It is particularly suitable for deployment on edge devices, reducing hardware costs and energy consumption, and enabling the image deraining system to run efficiently on resource-constrained platforms.

[0062] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. Attached Figure Description

[0063] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application. The illustrative embodiments and descriptions of the application are used to explain the application, but do not constitute an undue limitation of the application. Obviously, the drawings described below are merely some embodiments, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0064] In the attached diagram:

[0065] Figure 1 This is a flowchart illustrating the image deraining method based on wavelet transform and digital filtering details in this specific embodiment.

[0066] Figure 2 This is a system schematic diagram of the image deraining method based on wavelet transform and digital filtering details guided in this specific embodiment;

[0067] Figure 3 This is a flowchart illustrating the S1 learnable wavelet transform process in the image deraining method based on wavelet transform and digital filtering details in this specific embodiment.

[0068] Figure 4 This is a flowchart illustrating the S2 detail modulation process in the image deraining method guided by wavelet transform and digital filtering in this specific embodiment.

[0069] Figure 5A flowchart of the S3 dual-branch processing and dynamic interactive perception decoupling in the image rain removal method based on wavelet transform and digital filtering detail guidance in the embodiment;

[0070] Figure 6 A flowchart of the training of the image rain removal model in the image rain removal method based on wavelet transform and digital filtering detail guidance in the embodiment;

[0071] Figure 7 A feature heat map of the image rain removal method based on wavelet transform and digital filtering detail guidance in the embodiment. DETAILED DESCRIPTION

[0072] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments with reference to the drawings in the embodiments of the present application. The following embodiments are used to explain the present application but not to limit the scope of the present application.

[0073] Most existing image rain removal methods rely on image digital filtering and prior information-based algorithms. The filtering method usually processes based on certain local parameters (such as brightness, shape, direction, etc.) of the image, which are usually strongly affected by noise. In processing, the traditional method generates sub-images for specific directions, frequencies, and shapes, but this method is easy to lose the detailed features of the image, and there may be large information loss and limitations in the image information processing process. Secondly, the prior-based method relies on prior knowledge and model structure designed in advance, which makes it difficult to adapt to the changing image features in complex real-world scenarios, resulting in unsatisfactory recovery results.

[0074] Existing deep learning-based image rain removal methods, such as convolutional neural networks (CNN) and generative adversarial networks (GAN), do not fully utilize the image prior information with a single deep learning model. Since deep learning networks mainly focus on extracting information from image data, they lack guidance on frequency domain features in images, which limits the recovery effect to the learning ability of the network itself and the quality of the training data.

[0075] See Figure 1 and Figure 2 Based on the technical defects of existing methods in image rain removal, the present application provides an image rain removal method based on wavelet transform and digital filtering detail guidance, which includes applying a trained image rain removal model to process the input rain image. The method includes: obtaining a rain image, inputting the rain image into the image rain removal model for processing, and then outputting a rain-removed image.

[0076] The image rain removal model processes the input rainy image, including the following steps:

[0077] S1, learnable wavelet decomposition:

[0078] The input rainy image is decomposed into multi-level low-frequency subbands and high-frequency subbands through the learnable wavelet;

[0079] S2, detail modulation:

[0080] The rainy image and the multi-level low-frequency subbands are subjected to step-by-step band-pass filtering to obtain multi-level detail information; the multi-level detail information is fused, and the background information and the rain layer information in the rainy image are modulated by the fused detail information to obtain background features and rain layer features;

[0081] S3, dual-branch processing and dynamic interactive perception decoupling:

[0082] The background features are processed through the background branch, which can effectively preserve the low-frequency structure information of the rainy image, and the rain layer features are processed through the rain layer branch, which obtains the rain streak distribution through global modeling;

[0083] The background branch at least includes: processing the background features through depth separable convolution, then processing through multi-scale grouping convolution, enhancing the background features after multi-scale grouping convolution, and enhancing the background features after SE channel attention, and finally outputting the background features processed by the background branch through weighted residual fusion processing;

[0084] The outputs of the background branch and the rain layer branch are processed through the dynamic interactive perception mechanism, which is a global gating weight that integrates global information, i.e. global attention, generates local attention through spatial modulation, and fuses global attention and local attention through attention weight to obtain decoupled background features and rain layer features;

[0085] S4, image reconstruction is performed on the decoupled background features and rain layer features to obtain the final rain-removed image.

[0086] The present application combines digital filtering and image prior information, uses the powerful feature learning ability of deep learning network, and integrates the advantages of traditional image processing methods. By introducing image prior knowledge, it can better adapt to different image rain removal tasks, and combine digital filtering to effectively suppress image noise, thereby improving the quality and precision of the recovery, and through the fusion of feature information of different scales, the accuracy and robustness of image rain removal are improved.

[0087] It should be noted that the execution subject of the image rain removal method based on wavelet transform and digital filtering detail guidance in this embodiment is an image rain removal device based on wavelet transform and digital filtering detail guidance. The device can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Illustratively, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, or the like, and the non-mobile electronic device can be a server, a personal computer, or the like, which is not limited in the present application. The image rain removal method based on wavelet transform and digital filtering detail guidance in this embodiment is described below with the execution subject being a server as an example.

[0088] In addition, the terms "first" and "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0089] Please refer to Figure 3 In order to introduce the prior information of the image and improve the separation performance of the background rain layer of the image rain removal model, the input image is hierarchically processed by a learnable wavelet. In addition, a static wavelet transform (SWT) is used to decompose the image into a low-frequency subband and a high-frequency subband. The downsampling operation in the traditional wavelet transform is avoided, and the resolution is kept consistent by padding convolution, thereby avoiding information loss and facilitating subsequent feature fusion. The filter of the learnable wavelet transforms the traditional Haar wavelet filter into a learnable filter, and the initial state of the filter is obtained from the low-pass filter and the high-pass filter of the Haar wavelet. The parameters of the filter can be dynamically adjusted during the training process to adapt to the spectral distribution of a specific task:

[0090] ;

[0091] wherein LL subband represents the low-frequency part of wavelet decomposition, HH, HL, and HH represent the high-frequency part of wavelet decomposition, represents a depth separable convolution, is a learnable wavelet kernel, and the initialization satisfies the Haar basis orthogonality, , , , respectively represent the parameterization of the filter set of the traditional two-dimensional Haar wavelet into a trainable depth separable convolution kernel, represents the original input image, is the number of wavelet decomposition layers, = 1, means wavelet transform is performed on the input image;

[0092] Learnable wavelet tree: Since the single-level wavelet transform is insufficient to mine the spectral information, in order to ensure sufficient extraction of image information, a hierarchical frequency band separation strategy is adopted, and a learnable wavelet tree is used to generate multi-level subbands to extract low-frequency background (LL subband) and high-frequency details (LH, HL, HH subband) step by step. Multi-level decomposition allows the model to capture multi-scale information from coarse to fine granularity;

[0093] In order to balance the amount of calculation and performance, a two-level wavelet tree is adopted:

[0094] Level : ;

[0095] Level : ;

[0096] Wherein, x is the original input image, the first level decomposes the original input image, represents a continuous wavelet transform using a learnable wavelet filter, 、 、 、 respectively represent the first level subband, 、 、 、 respectively represent the second level subband. Unlike traditional down-sampling wavelet transform, continuous wavelet transform maintains consistent subband resolution through padding convolution, thereby avoiding information loss and facilitating subsequent feature fusion.

[0097] Please refer to Figure 4 , S2 is used to perform band-pass filtering on the rainy image and the multi-level low-frequency subband to obtain multi-level detail information, and the multi-level detail information is fused to modulate the background information and the rain layer information in the rainy image, thereby obtaining background features and rain layer features, including:

[0098] Step-by-step subtraction of low-frequency subbands to obtain intermediate frequency detail features BP;

[0099] The intermediate frequency detail features BP are used to generate a detail attention map using a Sigmoid activation function to convert to generate rain layer detail weights d, and the background detail weight is 1-d;

[0100] The background features are obtained by performing element-wise multiplication on the initial background features generated by convolution of the last level low-frequency subband and the background detail weight 1-d;

[0101] The rain layer feature is obtained by element-wise multiplication of the initial rain layer feature generated by convolution of the high-frequency subband after feature fusion and the rain layer detail weight d.

[0102] Specifically, since the learnable wavelet transform lacks the downsampling operation of the traditional wavelet transform, it will contain redundant frequency band decomposition information. A detail modulation mechanism is designed to dynamically fuse background and detail features through band-pass filtering and gating mechanism. First, a multi-level band-pass component is calculated, that is, the low-frequency subband is subtracted step by step to obtain the intermediate-frequency detail feature, denoted as:

[0103] ;

[0104] wherein BP represents the obtained intermediate-frequency detail feature, is the wavelet decomposition level. When is 1, the first-level low-frequency subband LL and the rainy image are band-pass filtered; is 2, the second-level low-frequency subband and the first-level low-frequency subband are band-pass filtered.

[0105] Specifically, a detail attention map is generated by convolution, and a Sigmoid activation function is used to convert the generated detail weight, denoted as:

[0106] ;

[0107] ;

[0108] wherein, represents the detail attention map, represents convolution, is a Sigmoid activation function. The detail feature obtained by the step-by-step band-pass filtering represents the missing high-frequency detail, so the generated weight d is used as the rain layer detail weight, and 1-d is used as the background detail weight.

[0109] Specifically, the background feature is obtained by element-wise multiplication of the initial background feature generated by convolution of the last-level low-frequency subband and the background detail weight 1-d; the rain layer feature is obtained by element-wise multiplication of the initial rain layer feature generated by convolution of the high-frequency subband after feature fusion and the rain layer detail weight d, respectively denoted as:

[0110] ;

[0111] ;

[0112] wherein, represents the background feature, represents the rain layer feature, LH and HL represent the LH subband and the HL subband, represents channel concatenation, This represents element-wise multiplication. This indicates the first-level HL sub-band and the second-level HL sub-band. This indicates the first-level LH ​​subband and the second-level LH ​​subband.

[0113] After obtaining the embedding information of the two branches, the background branch and the rain layer branch, since the background and the rain layer need to retain different image information, the background branch can effectively retain the low-frequency structural information of the image, while the purpose of the rain layer branch is to obtain the distribution of rain patterns through global modeling. In this embodiment, a dual-branch processing network for image restoration with decoupled background and rain layers is designed. The network is an encoder-decoder UET architecture, and the encoder and decoder are assembled by a feature reconstructor and a dynamic interaction module.

[0114] Please see Figure 5 The depthwise separable convolution processing described in S3 includes:

[0115] Perform depthwise convolution operations, which are performed independently on each input channel without cross-channel information fusion, to extract local structural information;

[0116] By using point convolution operations, information is fused for each channel using a 1×1 convolution kernel.

[0117] Specifically, performing a depthwise convolution operation is represented as:

[0118] ;

[0119] in, This represents the features after a depthwise convolution operation. Represents depthwise convolution. Indicate background features;

[0120] Then, through point convolution, it can be represented as:

[0121] ;

[0122] in, Represents the features after depthwise convolution. Features obtained by point convolution Represents point convolution;

[0123] Information is fused across each channel using a 1×1 convolutional kernel, ensuring information transfer across channels. Depthwise separable convolution refines the input feature map through depthwise and pointwise convolution operations while maintaining low computational cost.

[0124] Please continue reading Figure 5 The multi-scale grouped convolution processing described in S3 includes:

[0125] The background feature processed by the depth separable convolution is divided into multiple groups, and each group is processed by a convolution kernel;

[0126] The convolution results of different groups are spliced along the channel dimension to obtain a merged feature map, that is, the background feature processed by the multi-scale group convolution.

[0127] Specifically, the feature map after feature extraction by the depth separable convolution is processed by the multi-scale group convolution, and more sufficient information is extracted by different convolution kernels. First, the input feature is divided into multiple groups, and the input feature channel number is C, which is divided into 3 groups, and each group is processed by a convolution kernel, which is represented as:

[0128]

[0129] wherein, G represents the group number, G represents the group convolution, a different scale of convolution kernel is applied to each group of features , the convolution kernel is 3x3, 5x5, 7x7, and the result of each group convolution is obtained .

[0130] Then, the convolution results of different groups are spliced along the channel dimension to obtain a merged feature map, which is represented as:

[0131]

[0132] wherein, G represents the convolution result of different groups, the feature map obtained after splicing along the channel dimension, G represents the convolution result of the first, second, …, G group, and channel splicing.

[0133] Please continue to refer to Figure 5 , the background feature enhanced by the multi-scale group convolution in S3 is enhanced by SE channel attention, including:

[0134] Global information of each channel is obtained by global average pooling;

[0135] The attention weight of each channel is generated by a two-layer fully connected network; and

[0136] The background feature enhanced by the multi-scale group convolution is weighted processed by using the attention weight of each channel, and finally the background feature after SE channel attention enhancement is obtained. ​​

[0137] Specifically, after obtaining the multi-scale information of the image, the SE attention mechanism is used to highlight the important background features in the image. The global information of each channel is obtained by global average pooling,

[0138] ;

[0139] wherein, represents the global information of each channel obtained after global average pooling, represents global average pooling, represents the convolution results of different groups the feature map obtained after concatenation along the channel dimension;

[0140] Then, a two-layer fully connected network is used to generate the attention weight of each channel, denoted as:

[0141] ;

[0142] wherein, represents the weight of each channel, represents a Sigmoid activation function, , represents the learnable weight matrix of the fully connected layer, represents an activation function;

[0143] Finally, the input features are weighted using the attention weight of each channel to obtain the features of the grouped convolution and channel enhancement, denoted as:

[0144] ;

[0145] wherein, represents the convolution results of different groups the feature map obtained after concatenation along the channel dimension is subjected to channel attention enhancement.

[0146] Please continue to refer to Figure 5 , in S3, the background features enhanced by the SE channel attention are connected with the original background features along the channel, and the final output of the background features processed by the background branch includes:

[0147] the background features enhanced by the SE channel attention are connected with the original background features along the channel;

[0148] a layer of depth separable convolution is used to further extract the features connected along the channel;

[0149] output by SE attention enhancement fusion; and

[0150] Introducing learnable residual weights The weighted residual connection is performed, and the background feature processed by the background branch is finally output.

[0151] Specifically, to achieve adaptive fusion of background features, first, the background features enhanced by SE channel attention are connected with the original background features along the channel to prevent gradient explosion, denoted as:

[0152] ;

[0153] Wherein, denotes the spliced feature, denotes channel splicing, denotes the original background feature;

[0154] Then, a layer of depth separable convolution is used to further extract the fused features, denoted as:

[0155] ;

[0156] Wherein, denotes the feature enhanced by depth separable convolution, denotes depth separable convolution;

[0157] SE attention is applied to enhance the fusion output, denoted as:

[0158] ;

[0159] Wherein, denotes that the feature after depth separable convolution is enhanced by se attention, denotes SE attention;

[0160] Finally, learnable residual weights are introduced to perform weighted residual connection to obtain the output background feature , denoted as:

[0161] ;

[0162] Wherein, denotes the learnable residual weight, is a learnable parameter that helps to control the balance between the fused feature and the residual feature, which can adaptively adjust the fusion ratio of the background feature and the rain layer feature, avoiding overfitting or unnecessary interference.

[0163] Please continue to refer to Figure 5 The dynamic interactive perception mechanism in S3 is to integrate global information through global gating weights, that is, global attention, and generate local attention through spatial modulation, including:

[0164] Global attention:

[0165] The background enhancement features and rain layer enhancement features are concatenated along the channel dimension to form a joint feature, represented as follows:

[0166] ;

[0167] in, Background features after background branch processing The rain layer features after rain layer branching processing. Indicates joint features;

[0168] By integrating global context information through adaptive average pooling, a single-point global description is generated, represented as:

[0169] ;

[0170] in, This indicates the generated single-point global description, avoiding interference from spatial location; GAP indicates global average pooling.

[0171] A gated network is introduced, using a multi-layer convolutional sequence, adding activation functions layer by layer, and applying soft maximization activation to generate global gate weights, represented as:

[0172] ;

[0173] in, For learnable parameters, This refers to the channel compression ratio. For the Sigmoid function, express Activation function This represents the generated rain layer gating weights. This represents the generated background gating weights. Represents a list of components;

[0174] Local spatial attention:

[0175] Spatial attention is applied to the joint features, and multiple convolutions are used to generate a pixel-level spatial weight mask, represented as follows:

[0176] ;

[0177] in, For the Sigmoid function, For convolution weights, Represents pixel-level spatial weight mask The shape, These represent batch, number of channels, width, and height, respectively.

[0178] Please continue to see Figure 5 In the rain layer branch, the first input feature map is processed by down-sampling. By using the convolutional neural network to down-sample the rainy image, the feature map size of the image is reduced, which provides more compact feature representation for subsequent network processing, and then the down-sampled feature is processed by Restormer, which is represented as:

[0179] ;

[0180] Wherein, is the down-sampled feature map, is the feature map processed by the Restormer module;

[0181] After Restormer processing, in order to restore the rain layer details, an up-sampling operation is adopted, using transposed convolution to up-sample the feature map to restore it to the original size. The up-sampling operation can be represented by the formula:

[0182] ;

[0183] Wherein, is the feature map processed by the Restormer, is the feature map up-sampled by transposed convolution, represents the up-sampling operation, represents the transposed convolution.

[0184] In order to further optimize the generation of rain layer feature map, first, the rain layer feature processed by Restormer is fused with the original rain layer feature, and then weighted by the gating mechanism to ensure that the rain layer details are reasonably restored, which is represented as:

[0185] ;

[0186] Wherein, is the fused rain layer feature map, which is the output of the rain layer branch, represents feature fusion, represents the original background feature.

[0187] Please continue to see Figure 5 In S3, the global attention and local attention are fused by attention weight to obtain decoupled background feature and rain layer feature, including:

[0188] The learnable parameter is introduced to control the relationship between the global gating weight and the local spatial attention weight, which is represented as:

[0189] ;

[0190] wherein, denotes the background weight after attention weight fusion, denotes the rain layer weight after attention weight fusion, denotes the element-wise spatial weight mask, denotes the learnable parameters;

[0191] apply the normalized weight to the features output by the background branch and the rain layer branch to obtain the decoupled background features and rain layer features, denoted as:

[0192]

[0193]

[0194] wherein, denotes the decoupled background features, denotes the output of the background branch of the pre-decoupling feature reconstructor, denotes the normalized background weight, denotes the decoupled rain layer features, denotes the output of the rain layer branch of the pre-decoupling feature reconstructor, denotes the normalized rain layer weight, and ε is a very small constant to ensure numerical stability, avoid division by zero and gradient explosion, denotes element-wise multiplication.

[0195] In a preferred embodiment, the training of the image rain removal model is achieved through the following steps: data collection and preprocessing, model initialization, feature extraction and decoupling, dynamic interactive fusion, multi-scale reconstruction, loss function design, optimization strategy, training iteration cycle, test evaluation and parameter optimization. The whole process aims to make the model learn the decoupled representation of background and rain layer from low-quality images (such as rainy images) through supervised learning, and generate high-quality restored images (such as rain-removed images).

[0196] Please refer to Figure 2 and Figure 6 , the training process of the image rain removal model is briefly described below. For the processing steps of the image rain removal model on the input rainy image, please refer to the steps as previously described:

[0197] ​​Data collection: Pairwise samples are obtained from the public Rain dataset. The training set uses Rain13K, and the corresponding five test datasets are used for testing: Test100, Test1200, Test2800, Rain100L, and Rain100H. Each set of samples consists of two parts: rainy images, which contain rain streaks, rain fog, water reflection, and other degradation factors, providing appearance and texture information under real interference; and clear images (rain-free images), which are strictly aligned with rainy images at the pixel level and serve as reference images for supervised learning and objective metric evaluation. The data covers multiple weather, lighting, texture, and scale scenarios.

[0198] Specifically, the Rain13k dataset is used to ensure that the dataset covers different types of raindrop characteristics, scene lighting, and background complexity. Various data augmentation techniques such as rotation, cropping, translation, flipping, and random noise are used to increase the diversity of the training set, prevent overfitting, and improve the generalization ability of the model.

[0199] This embodiment uses two main loss functions to optimize the rain removal model: SSIM loss, which calculates the structural similarity between the output image and the target rain-removed image, and reduces image blurring during the rain removal process. Char loss is used to measure the image's ability to recover details. This loss focuses on the recovery of small-scale structures, especially the preservation of details during the rain removal process.

[0200] This embodiment uses the Adam optimizer to train the image rain removal model and adopts a dynamic learning rate strategy to improve the convergence speed of the model. The gradient is calculated through backpropagation, and the model's weights are updated to minimize the SSIM loss and Char loss. By optimizing the SSIM loss and Char loss, the image rain removal model can simultaneously recover the structural information and details of the image. This process adjusts the parameters in the image rain removal model to better remove rain from rainy images and obtain rain-free images.

[0201] It should be noted that during data preprocessing, the input color image (i.e., the rainy image) is first normalized to scale the pixel values to the [0, 1] interval. Then, data diversity is enhanced through random flipping, rotation, and cropping to increase the robustness of the model. For paired samples (low-quality images and clear target images), reflection padding is used to ensure consistent size and adjust to a uniform resolution. Noise interference data is initially removed by Gaussian filtering to ensure consistency in spatial and channel dimensions, facilitating subsequent feature extraction and fusion. Through this preprocessing method, the most recognizable features can be accurately extracted from the original image, significantly improving the accuracy of the recovery.

[0202] In a preferred embodiment, the average SSIM, PSNR and running time of the restoration results of the method proposed in this embodiment and the following comparative methods on multiple test data sets R100L, R100H, Test100, Test1200, Test2800, etc. are compared. The comparative algorithms include: DIDMDN algorithm, RESCAN method, LPNet algorithm, unsupervised multi-residual learning deblurring method proposed in UMRL, PreNet algorithm, MSPFN network, PCNet network, MPRNet network, ELFormer method, Uformer network, Restormer network, MFDnet network and the method proposed in this embodiment. The comparison results of each algorithm are shown in Table 1:

[0203] Table 1: Comparison of objective indicators of restoration results of several algorithms

[0204]

[0205] As can be seen from Table 1, the method proposed in this embodiment has achieved the optimal or near-optimal PSNR and SSIM values on most test data sets, with an average PSNR of 33.37 and an average SSIM of 0.932, which has obvious performance advantages compared with other methods, fully verifying the effectiveness of the method proposed in this embodiment.

[0206] See Figure 7 , each row shows an example of a feature heat map, from left to right: input rain map input, original background feature Braw, original rain layer feature Rraw, guided background feature Bfea, guided rain layer feature Rfea, detail guide map Dfea, and target map Target. Braw is obtained by splicing low-frequency subbands LL and their multi-scale information, and still has obvious rain layer noise pollution; Rraw is obtained by cross-scale splicing of high-frequency subbands (LH / HL / HH), which can highlight high-frequency components such as stripes / edges, but is mixed with some background details. We learn the detail guide map Rfea (high response at noise, low response at background) from Rraw, and respectively modulate the two branches: on the one hand, suppress the rain streaks in Braw, original rain layer feature Rraw, guided background feature Bfea, guided rain layer feature Rfea, original rain layer feature Braw to obtain guided background feature Bfea (cleaner background, more complete structure); on the other hand, enhance the noise response of the original rain layer feature Rraw to obtain a more "pure" rain layer representation (corresponding to Rfea / Dfea in the figure). As can be seen, the guided Bfea / Rfea are closer to their target layers (background / rain layer) than the original background feature Braw and the original rain layer feature Rraw, respectively.

[0207] The above merely preferred embodiments of the present application, and not any form of the present application, although the present application has been disclosed as above, however, not in order to limit the present application, any skilled in the art of the technical personnel in the present application within the scope of the present application, when can use the above-mentioned technical content of the prompt to make some more changes or modification of equivalent embodiments of equivalent changes, the implementation of the above-mentioned embodiments can be further combined or replaced, but as long as it does not deviate from the technical scheme of the present application, according to the technical essence of the present application, the above-mentioned embodiments of any simple modification, equivalent change and modification, still belongs to the scope of the present application.

Claims

1. An image deraining method based on wavelet transform and digital filtering detail guidance, characterized in that, The method includes applying a trained image deraining model to process an input rainy image, the method comprising: acquiring a rainy image, inputting the rainy image into the image deraining model for processing, and then outputting a derained image; The image deraining model processes the input rainy image, including the following steps: S1. Learnable wavelet decomposition: The input rainy image is decomposed into multiple levels of low-frequency subbands and high-frequency subbands using learnable wavelets; S2, Detail Modulation: The rainy image is subjected to stepwise bandpass filtering with the multiple levels of the low-frequency sub-bands to obtain multi-level detail information; the multi-level detail information is fused and the background information and rain layer information in the rainy image are modulated with detail information after fusion to obtain background features and rain layer features; S3, decoupling of dual-branch processing and dynamic interaction awareness: The background features are processed through a background branch, which can effectively preserve the low-frequency structural information of the rain image. The rain layer features are processed through a rain layer branch, which obtains the rain pattern distribution through global modeling. The background branch includes at least the following steps: processing the background features through depthwise separable convolution, then processing them through multi-scale grouped convolution, enhancing the background features enhanced by the multi-scale grouped convolution with SE channel attention, and fusing the background features enhanced by SE channel attention with the original background features through weighted residual fusion, and finally outputting the background features processed by the background branch. The outputs of the background branch and the rain layer branch are processed by a dynamic interactive perception mechanism. The dynamic interactive perception mechanism integrates global information through global gating weights, i.e., global attention, generates local attention through spatial modulation, and fuses global attention and local attention through attention weights to obtain decoupled background features and rain layer features. S4. Perform image reconstruction on the decoupled background features and rain layer features to finally obtain the rain-removed image.

2. The method according to claim 1, characterized in that, In S2, the rainy image and the multiple levels of low-frequency sub-bands are subjected to step-by-step bandpass filtering to obtain multi-level detail information. This multi-level detail information is then fused, and the fused detail information is used to modulate the background and rain layer information in the rainy image to obtain background features and rain layer features, including: The low-frequency subbands are subtracted stepwise to obtain the mid-frequency detail features (BP). The mid-frequency detail features are used to generate a detail attention map through convolution. The Sigmoid activation function is used to convert the rain layer detail weights d to generate the background detail weights 1-d. The background features are obtained by generating initial background features through convolution of the last-level low-frequency subband, and then multiplying them element-wise with the background detail weights 1-d. The rain layer features are obtained by combining the initial rain layer features generated by feature fusion of high-frequency subbands through convolution with the rain layer detail weights d.

3. The method according to claim 1, characterized in that, The depthwise separable convolution processing described in S3 includes: Perform depthwise convolution operations, which are performed independently on each input channel without cross-channel information fusion, to extract local structural information; By using point convolution operations, information is fused for each channel using a 1×1 convolution kernel.

4. The method according to claim 1, characterized in that, The multi-scale grouped convolution processing described in S3 includes: The background features after the depth-separable convolution are divided into multiple groups, and each group is processed by a convolution kernel. The convolution results of different groups are concatenated along the channel dimension to obtain the merged feature map, which is the background feature after the multi-scale grouped convolution processing.

5. The method according to claim 1, characterized in that, S3 performs SE channel attention enhancement on the background features enhanced by the multi-scale grouped convolution, including: Global information for each channel is obtained through global average pooling; Attention weights for each channel are generated using a two-layer fully connected network; and, The background features enhanced by the multi-scale grouped convolution are weighted using the attention weights generated for each channel, and the background features enhanced by SE channel attention are finally obtained.

6. The method according to claim 5, characterized in that, In S3, the background features enhanced with attention in the SE channel are fused with the original background features using weighted residual fusion. The final output background features after the background branching process include: The background feature enhanced with attention in the SE channel is connected to the original background feature along the channel; Use a depthwise separable convolution layer to further extract features connected along the channels; Enhanced fusion output through SE attention; and, Introducing learnable residual weights A weighted residual connection is performed, and the final output is the background feature processed by the background branch.

7. The method according to claim 1, characterized in that, The dynamic interactive perception mechanism described in S3 integrates global information through global gating weights, i.e., global attention, and generates local attention through spatial modulation, including: Global attention: The background enhancement features and rain layer enhancement features are concatenated along the channel dimension to form a joint feature, represented as follows: ； in, Background features after background branch processing The rain layer features after rain layer branching processing. Indicates joint features; By integrating global context information through adaptive average pooling, a single-point global description is generated, represented as: ； in, This represents a single-point global description, avoiding interference from spatial location. Indicates global average pooling; A gated network is introduced, using a multi-layer convolutional sequence, adding activation functions layer by layer, and applying soft maximization activation to generate global gate weights, represented as: ； in, For learnable parameters, This refers to the channel compression ratio. For the Sigmoid function, This represents the ReLU activation function. This represents the generated background gating weights. This represents the generated rain layer gating weights. G represents B With g R A list of components; Local spatial attention: Spatial attention is applied to the joint features, and multiple convolutions are used to generate a pixel-level spatial weight mask, represented as follows: ； in, For the Sigmoid function, For convolution weights, Represents pixel-level spatial weight mask The shape, These are batch size, number of channels, width, and height.

8. The method according to claim 7, characterized in that, In S3, global and local attention are fused through attention weights to obtain decoupled background and rain layer features, including: Introducing learnable parameters The relationship between the global gating weights and the local spatial attention weights is expressed as: ； in, This represents the background layer weights after attention weight fusion. This represents the rain layer weights after attention weight fusion. Represents a pixel-level spatial weight mask. Represents the learnable parameters. By multiplying the features output by the background branch and the rain layer branch by the normalized weights, the decoupled background features and rain layer features are obtained, as follows: ； ； in, This represents the background features after decoupling. This represents the output of the background branch before decoupling. Represents the normalized background weights. This represents the characteristics of the rain layer after decoupling. This represents the output of the rain layer branch before decoupling. This represents the normalized rain layer weights, where ε is a constant used to ensure numerical stability and avoid division by zero and gradient explosion. This indicates element-wise multiplication.

Images