Image rain removal method based on phase perception and dynamic optimization

Abstract

The invention discloses an image rain removal method based on phase perception and dynamic optimization, and the method comprises the steps: constructing a phase perception dynamic rain removal network, converting a rain-containing image into a rain-containing image of a large-resolution flow, a medium-resolution flow and a small-resolution flow, and carrying out the parallel processing of the rain-containing image; channel attention operation is carried out on space, frequency and biological visual features to generate multi-modal fusion features, the multi-modal fusion features are combined with rain-containing image input features of various resolutions to generate decoding features, details are reconstructed through INR after fusion, preliminary rain removal features are output, and raindrop complexity is evaluated to dynamically optimize and output rain removal images; when the phase perception dynamic rain removal network is trained, network parameters are optimized through a composite loss function; and inputting a to-be-processed rain-containing image into the trained phase perception dynamic rain removal network for multi-scale feature processing and dynamic optimization refinement, and outputting a clear image after rain removal. According to the invention, multi-modal feature fusion, dynamic rain condition adaptation and high-precision detail recovery are realized.

Description

A Phase-Aware and Dynamically Optimized Image Deraining Method Technical Field

[0001] This invention relates to computer vision and image processing technology, specifically to an image deraining method based on phase sensing and dynamic optimization. Background Technology

[0002] Image deraining is a key preprocessing task in the field of image processing. Its core objective is to separate the rain layer from the background in degraded images affected by raindrops, restoring the image's details and structural information. In rainy conditions, raindrops reduce image contrast and blur edges, severely impacting the reliability of computer vision systems. For example, if the camera of an autonomous vehicle is obscured by raindrops, it may misidentify traffic lights or pedestrians; intelligent monitoring systems may miss or misjudge targets due to raindrop interference. Therefore, efficient image deraining technology has significant practical implications. Existing image deraining methods can be divided into two categories: traditional signal processing methods and deep learning methods.

[0003] Traditional signal processing methods rely on manually designed image priors or physical models to construct optimization problems. Typical techniques include: sparse coding methods, which assume raindrops are sparsely distributed in the image and transform the rain removal problem into solving for sparse coefficients by constructing raindrop dictionaries and background dictionaries; low-rank matrix recovery methods, which model rain-containing images as "low-rank background matrix + sparse rain layer matrix" and separate the two using convex optimization algorithms (such as RPCA); and physical model methods, which construct physical models based on the optical scattering characteristics of raindrops and achieve rain removal by estimating parameters such as raindrop density and orientation. While these methods are effective under simple rain conditions (such as regular sparse rain lines), they have limitations: rigid assumptions about rain patterns prevent them from adapting to complex real-world rain conditions; and the iterative optimization process has high computational complexity, making it difficult to meet the needs of real-time applications.

[0004] With the development of Convolutional Neural Networks (CNNs) and Transformer technologies, deep learning methods have become mainstream. CNN-based methods extract raindrop features through multi-scale convolutions and achieve end-to-end rain removal using an encoder-decoder architecture. Transformer-based methods capture long-range feature dependencies through self-attention mechanisms, improving global rain pattern modeling capabilities. GAN-based methods utilize generative adversarial networks to generate realistic rain-removed images and improve visual consistency through adversarial training.

[0005] The existing technologies have the following main drawbacks: (1) First, the existing methods have single feature representation and weak ability to distinguish between rain and background. Most methods rely on image intensity (amplitude) information, while the intensity distribution of raindrops and background textures often overlaps (such as dark raindrops on a dark background), resulting in incomplete removal of raindrops or loss of background details, especially in complex rain and fog scenes; (2) Second, most existing methods lack multimodal fusion, have poor generalization ability, and do not fully integrate key dimension features such as phase and frequency. Phase information is robust to changes in illumination and can effectively represent image structure, but existing methods often ignore this dimension; (3) Third, existing methods have weak dynamic adaptability, and it is difficult to balance efficiency and effect. Using fixed network structure and parameters to process all rain conditions leads to computational redundancy in low-complexity rain conditions and insufficient feature capture in high-complexity rain conditions; (4) Fourth, existing methods have low detail recovery accuracy and limited visual quality. Traditional discrete image representation (pixel grid) relies on the local receptive field of the convolution kernel, making it difficult to reconstruct high-frequency details such as edges and textures. After rain removal, the image is prone to excessive smoothing, affecting the accuracy of subsequent visual tasks. Summary of the Invention

[0006] The technical problem to be solved by this invention is to provide an image deraining method based on phase sensing and dynamic optimization, which addresses the above-mentioned problems in the prior art. The core of the method is the phase sensing dynamic deraining network CPDDNet. This network achieves multimodal feature fusion, dynamic rain condition adaptation and high-precision detail restoration through the collaborative work of its various modules.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: an image deraining method based on phase sensing and dynamic optimization, comprising the following steps: S1) Constructing a phase-aware dynamic deraining network CPDDNet, wherein the phase-aware dynamic deraining network CPDDNet converts rain-containing images into rain-containing images of high-resolution stream, medium-resolution stream, and low-resolution stream respectively, and uses a phase-aware rain pattern pre-analysis module CPPRPM to extract phase features from the rain-containing image of the medium-resolution stream to generate rain pattern sensing features. The multimodal bio-inspired feature fusion module MBIFM is used based on rain pattern perception features. Channel attention operations are performed on spatial, frequency, and biological visual features to generate multimodal fusion features. It uses a hierarchical lightweight dilated convolutional core module HLDCCM, corresponding to high-resolution, medium-resolution, and low-resolution streams, combined with multimodal fusion features. The corresponding rain-inducing image input features are used to generate corresponding decoded features, and the decoded features are fused across scales. The details of the fused decoded features are reconstructed using INR, and the preliminary rain-removing features are output. The intelligent dynamic optimization and refinement module IDORM was used to evaluate the initial rain removal characteristics. Reduce raindrop complexity and dynamically optimize the output rain-removed image. S2) The phase-aware dynamic deraining network CPDDNet is trained using a rain-clear image dataset. The network parameters are optimized using a composite loss function, which is a weighted sum of Charbonnier loss, edge loss, frequency domain loss, L1 loss, and perceptual loss. S3) The rain-laden image to be processed is input into the trained phase-aware dynamic deraining network CPDDNet for multi-scale feature processing and dynamic optimization and refinement, and then the clear image after deraining is output.

[0008] Furthermore, the phase-aware rain pattern pre-analysis module CPPRPM includes a complex convolutional network CCN, a phase-sensitive pooling layer PSP, and a residual thinning submodule. When using the phase-aware rain pattern pre-analysis module CPPRPM to extract phase features from a rain-containing image of a medium-resolution stream, the following steps are included: S101) The complex convolutional network CCN uses complex convolution kernels. Rain-bearing images of medium-resolution streams Perform complex convolution to output real part features. Features of the imaginary part ,satisfy:

[0009] in, , Let be the real part weight matrix. The imaginary part weight matrix, The imaginary unit, This is a complex convolution operation, specifically a parallel computation of rain-containing images. With the real part weight matrix Convolution and rain-containing images With the imaginary part weight matrix The convolution; S102) The phase-sensitive pooling layer PSP with window size Calculate pixels within the window phase angle , For row coordinates, Using column coordinates, filter the pixels with the largest phase angles and retain their real features. Features of the imaginary part The value of the output pooled real part feature Features of the imaginary part of pooling ,satisfy:

[0010] in, For phase-sensitive pooling operation; S103) The residual refinement submodule refines the real part features of the pooling. Features of the imaginary part of pooling Residual processing is performed separately, and then the data is stitched together along the channel dimension to generate rain-pattern sensing features. ,satisfy: ,

[0011]

[0012] in, For residual block operations, This is for channel splicing operations.

[0013] Furthermore, the multimodal bio-inspired feature fusion module MBIFM is used based on rain pattern perception features. When performing channel attention operations on spatial, frequency, and bio-visual features, the following steps are included: S201) Extracting rain pattern perception features through concatenated residual blocks in the spatial branch of the multimodal bio-inspired feature fusion module MBIFM. Local details ,satisfy:

[0014] in, For spatial feature extraction operations; S202) In the frequency branch of the multimodal bio-inspired feature fusion module MBIFM, rain pattern sensing features are extracted by simulating discrete wavelet transform. Frequency domain characteristics ,satisfy:

[0015] in, For frequency feature extraction operations; S203) In the biological vision branch of the multimodal bio-inspired feature fusion module MBIFM, Gabor-like convolutional kernels are used to extract rain pattern perception features. directional features ,satisfy:

[0016] in, For biological visual feature extraction operations; S204) for local detail features Frequency domain characteristics With directional features The generation is obtained by splicing along the channel dimension. Attention weights are then generated sequentially through global average pooling, 1×1 convolution, ReLU activation, 1×1 convolution, and Sigmoid activation. Finally, attention weights are used to apply attention to local detail features. Frequency domain characteristics With directional features Weighted summation yields multimodal fusion features ,satisfy:

[0017]

[0018] in, For weight generation operations, These are the weights for spatial, frequency, and biological vision branches, respectively. This is element-wise multiplication.

[0019] Furthermore, a layered lightweight dilated convolutional core module HLDCCM, corresponding one-to-one with high-resolution, medium-resolution, and low-resolution streams, is used in conjunction with multimodal fusion features. When generating corresponding decoded features from the input features of the rainy image, the process includes the following steps: S301) Projecting the rainy image using a 1×1 convolution to obtain projected features, and then fusing the multimodal features. A resolution adjustment operation is performed to match the resolution of the rain-containing image, resulting in multimodal fusion features with the matched resolution. Then, the projected features are fused with the multimodal features of the matching resolution. S302) The initial encoded features are added together to obtain the initial encoded features; the initial encoded features are input into the encoding path of the hierarchical lightweight dilated convolutional core module HLDCCM, the encoding path includes concatenated encoding blocks and the encoding blocks are hierarchical lightweight dilated Transformer blocks, and the output features of each encoding block are downsampled to reduce the resolution, satisfying:

[0020] in, To enable layered, lightweight, and hollow Transformer block operations, For downsampling operation; S303) Encode the final features of the path The input features are first normalized by layering the input features, then global feature dependencies are captured through multi-head self-attention, and residual connections are used to preserve the original information, resulting in features after multi-head self-attention and residual connections. Then, for the features The feature is normalized again, and a nonlinear transformation is achieved through a feedforward network. Finally, the output feature of the bottleneck layer is output through residual connections. ,satisfy:

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] in, For layer normalization operation, For multi-head self-attention operation, For a feedforward network; S304) the output features of the bottleneck layer The input is the decoding path of the hierarchical lightweight dilated convolutional core module HLDCCM. The decoding path includes concatenated decoding blocks, and each decoding block is a hierarchical lightweight dilated Transformer block. The first decoding block simultaneously receives the skip connection features of the corresponding encoding block and the output features of the bottleneck layer. Subsequent decoding blocks simultaneously receive the skip connection features of the corresponding encoding block and the output features of the previous decoding block. Each decoding block concatenates all simultaneously received features along the channel dimension before performing a layered lightweight diffusing Transformer block operation. Furthermore, the output features of each decoding block are upsampled to restore resolution, satisfying the following conditions:

[0027] in, For the upsampling operation, To enable layered, lightweight, and hollow Transformer block operations, This is a channel-dimensional splicing operation.

[0028] Furthermore, when fusing the decoded features across scales, the decoded features output by the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the low-resolution stream are downsampled and input into the first decoding block of the decoding path of the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the medium-resolution stream. Similarly, the decoded features output by the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the medium-resolution stream are downsampled and input into the first decoding block of the decoding path of the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the high-resolution stream, so that the decoded features output by the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the high-resolution stream are the fused decoded features.

[0029] Furthermore, when performing hierarchical lightweight dilated Transformer block operations, the following steps are included: processing the input features... After layer normalization, three 3×3 convolutional layers with different dilation rates are input in parallel. The effective output is dynamically selected through attention weights, satisfying the following:

[0030] In the formula: , For layer normalization operation; The void ratio has values ​​of 1, 2, and 3, corresponding to small, medium, and large receptive fields, respectively. For dilated convolution operations, the padding is... To capture the characteristics of raindrops at different scales; void ratio Attention weights; For adaptive dilated convolution output, and Dimensionality consistent; input features With adaptive dilated convolution output After being summed via residual connections, the input layer is normalized, and then global feature dependencies are captured through multi-head self-attention. These features are then fed into a feedforward network via residual connections, ultimately outputting block features that satisfy the following conditions:

[0031]

[0032] In the formula: This is a multi-head self-attention operation; It is a feedforward network; For self-attention output features, the dimension and Consistent; The final output feature of the block, with dimensions and Consistent.

[0033] Furthermore, when reconstructing details from the fused decoded features using INR, the following steps are included: S401) Generating normalized coordinates for the target resolution. (S402) Map the normalized coordinates to a high-dimensional space to obtain the corresponding position encoding vector. ,satisfy:

[0034] In the formula, S403) The position encoding vector is the number of encoding layers. The decoded features of the high-resolution stream hierarchical lightweight dilated convolutional core module HLDCCM are input into the multilayer perceptron to obtain a detailed image reconstructed by INR, satisfying the following:

[0035] In the formula, The decoding features of the hierarchical lightweight dilated convolutional core module HLDCCM are sampled to coordinates via bilinear interpolation. The corresponding eigenvalues; It is a multilayer perceptron; The Sigmoid activation function (S404) is used to overlay the decoded features of the hierarchical lightweight dilated convolutional core module HLDCCM of high-resolution streaming with the detailed image reconstructed by INR to obtain preliminary rain removal features. .

[0036] Furthermore, the intelligent dynamic optimization and refinement module IDORM was used to evaluate the initial rain removal characteristics. Reduce raindrop complexity and dynamically optimize the output rain-removed image. At that time, the process includes the following steps: S501) Initial rain removal characteristics Inputting a convolutional neural network (CNN), and sequentially passing it through 3×3 convolution, ReLU activation, max pooling, 3×3 convolution, ReLU activation, global average pooling, a linear layer, and sigmoid activation, yields the raindrop complexity. S502) If raindrop complexity If the value exceeds the upper threshold, the initial rain removal feature will be applied. The input consists of a residual attention refinement network (RARNet) cascaded with two layered lightweight dilated Transformer blocks and one channel attention layer to remove residual raindrops, and the output is the optimized features. If raindrop complexity If the value is less than the upper threshold and greater than the lower threshold, the initial rain removal feature will be applied. The input consists of a lightweight residual correction network (LRCNet) composed of two concatenated residual blocks to remove residual raindrops, and the output is the optimized features. If raindrop complexity If the value is less than the lower threshold, the initial rain removal feature will be directly applied. Output as optimized features S503) Optimized features The final derained image is obtained by mapping the image to RGB channels using a 1×1 convolution. .

[0037] Furthermore, preliminary rain characteristics will be determined. When removing residual raindrops from the input of a residual attention refinement network (RARNet) consisting of two hierarchical lightweight dilated Transformer blocks cascaded with one channel attention layer, the following steps are included: Each hierarchical lightweight dilated Transformer block acquires the input features and performs hierarchical lightweight dilated Transformer block operations, where the input features of the first hierarchical lightweight dilated Transformer block are the initial rain-removing features. Furthermore, the input features of the second layered lightweight perforated Transformer block are the output features of the first layered lightweight perforated Transformer block; the channel attention layer acquires the output features of the second layered lightweight perforated Transformer block. , respectively for features Perform global average pooling and global max pooling to obtain the channel statistics vector of the global average pooling output. Channel statistics vector of global max pooling output Then channel statistics vector With channel statistical vector The concatenated features are sequentially passed through a fully connected layer, ReLU activation, another fully connected layer, and a Sigmoid activation layer to obtain channel attention weights. These channel attention weights are then... With features Perform channel-by-channel and element-by-element multiplication to obtain the weighted features, which are then used as the optimized features.

[0038] Furthermore, the composite loss function satisfies:

[0039] in: For Charbonnier loss, Here are the weights of the Charbonnier loss, which satisfies the following:

[0040] In the formula: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale Deraining output image; For scale Clear label images; For scale The total number of pixels; For smoothing parameters; Representing scale Deraining output image The i-th pixel and scale Clear label images The i-th pixel The square of the L2 norm of the difference; For edge loss, Let be the weighting coefficients of the edge loss, and let the edge loss satisfy:

[0041] In the formula: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale Deraining output image; For scale Clear label images; These are gradient operators in the x and y directions, respectively; It is an L1 norm; For frequency domain loss, Here are the weighting coefficients for the frequency domain loss, which satisfy the following conditions:

[0042] In the formula: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale Deraining output image; For scale Clear label images; It is an L1 norm; It is a two-dimensional fast Fourier transform; For L1 loss, Let be the weights of the L1 loss function. The L1 loss function satisfies:

[0043] In the formula: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale The total number of pixels; and They are scales Deraining output image The i-th pixel and scale Clear label images The i-th pixel; In order to perceive loss, Let these be the weighting coefficients for the perceptual loss. The perceptual loss satisfies:

[0044] In the formula: Output the derained image at the maximum resolution scale; Clear label images at maximum resolution scale; It is an L1 norm; This is the feature extraction function for the pre-trained VGG19 network.

[0045] Compared with the prior art, the advantages of the present invention are as follows: (1) The present invention designs a phase-aware rain pattern pre-analysis module CPCRPM, which extracts the unique features of raindrops in the phase domain, breaks through the bottleneck of intensity domain representation, and achieves accurate differentiation between rain and background texture; (2) The present invention constructs a multimodal bio-inspired feature fusion module MBIFM, which integrates spatial details, frequency domain raindrop noise, and biological visual direction features, and forms a comprehensive rain pattern representation through adaptive attention weighting, thereby improving the generalization ability of the model; (3) The present invention designs an intelligent dynamic optimization and refinement module IDORM, which dynamically selects refinement strategies (attention refinement for high complexity, lightweight correction for medium complexity, and direct output for low complexity) through rain condition complexity assessment, balancing the rain removal effect and computational efficiency; (4) The present invention integrates implicit neural representation INR in the hierarchical lightweight dilated convolution core module HLDCCM, and reconstructs high-frequency details through continuous image modeling, ensuring the structural integrity and visual fidelity of the rain removal image. Attached Figure Description

[0046] Figure 1 is a flowchart of the method according to an embodiment of the present invention.

[0047] Figure 2 is a schematic diagram of the overall architecture of the phase-aware dynamic rain removal network CPDDNet in an embodiment of the present invention.

[0048] Figure 3 is a schematic diagram of the rain removal effect of the method according to an embodiment of the present invention. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0050] In the field of image deraining, existing deep learning models still face several technical bottlenecks, resulting in insufficient deraining effects and practicality under complex rain conditions. Specific technical problems that need to be addressed include: 1. Difficulty in distinguishing rain from background texture: Existing methods rely on image intensity (amplitude) information to distinguish raindrops from the background. However, the intensity distribution of raindrops and background textures often overlaps (e.g., dark raindrops on a dark background, bright raindrops on a light background), leading to incomplete raindrop removal or loss of background details; 2. Inefficient multimodal feature fusion: Single-modal features (e.g., spatial domain features) cannot comprehensively represent the "sparse rain lines - dense rain fog - 1. Complex rain patterns such as "non-uniform raindrops" lack integration of key dimensions such as phase and frequency, resulting in poor model generalization ability; 2. Poor adaptability to rain conditions: Using a fixed network structure and parameters to process all rain conditions, it cannot adapt to the dynamic changes of rain conditions - there is computational redundancy in low-complexity rain conditions, and incomplete rain removal due to insufficient feature capture in high-complexity rain conditions; 3. Low precision in detail restoration: Traditional discrete image representation (pixel grid) is difficult to accurately reconstruct high-frequency details such as edges and textures. After rain removal, the image is prone to excessive smoothing, which affects the accuracy of subsequent visual tasks such as object detection and segmentation.

[0051] To address the aforementioned problems, this invention proposes an image deraining method based on phase awareness and dynamic optimization, as shown in Figure 1, comprising the following steps: S1) Constructing a phase-aware dynamic deraining network CPDDNet, which converts rain-containing images into rain-containing images in high-resolution, medium-resolution, and low-resolution streams, respectively. The phase-aware rain pattern pre-analysis module CPPRPM is used to extract phase features from the rain-containing image in the medium-resolution stream, generating rain pattern-aware features. The multimodal bio-inspired feature fusion module MBIFM is used based on rain pattern perception features. Channel attention operations are performed on spatial, frequency, and biological visual features to generate multimodal fusion features. It uses a hierarchical lightweight dilated convolutional core module HLDCCM, corresponding to high-resolution, medium-resolution, and low-resolution streams, combined with multimodal fusion features. The corresponding rain-inducing image input features are used to generate corresponding decoded features, and the decoded features are fused across scales. The details of the fused decoded features are reconstructed using INR, and the preliminary rain-removing features are output. The intelligent dynamic optimization and refinement module IDORM was used to evaluate the initial rain removal characteristics. Reduce raindrop complexity and dynamically optimize the output rain-removed image. S2) The phase-aware dynamic deraining network CPDDNet is trained using a rain-clear image dataset. The network parameters are optimized using a composite loss function, which is a weighted sum of Charbonnier loss, edge loss, frequency domain loss, L1 loss, and perceptual loss. S3) The rain-laden image to be processed is input into the trained phase-aware dynamic deraining network CPDDNet for multi-scale feature processing and dynamic optimization and refinement, and then the clear image after deraining is output.

[0052] In step S1 of this embodiment, the phase-aware dynamic rain removal network CPDDNet is constructed using an end-to-end multi-scale Transformer-Encoder-Decoder architecture, with the input being a rain-containing image. (in Image height, in pixels; (3 represents the image width in pixels; 3 represents the RGB three channels), the output is a clear image with rain removed. .

[0053] The Phase-Aware Dynamic Rain Deraining Network (CPDDNet) comprises three parallel resolution processing streams, corresponding to different detail modeling requirements. The architecture is shown in Figure 2. The high-resolution X stream processes the original-size image (…). ), focusing on the restoration of high-frequency details such as edges and textures; medium-resolution M-stream: processing downsampled 1 / 2 images ( ), responsible for global rain pattern modeling and phase feature extraction; small-resolution S-flow: processes downsampled 1 / 4 images ( This allows for the extraction of low-frequency background features, reducing computational complexity.

[0054] The Phase-Aware Dynamic Rain Deraining Network CPDDNet comprises a Phase-Aware Rain Pattern Pre-analysis module CPCRPM, a Multimodal Bio-Inspired Feature Fusion module MBIFM, a Hierarchical Lightweight Dilated Convolutional Core module HLDCCM, and an Intelligent Dynamic Optimization and Refinement module IDORM. These modules are connected in a concatenated manner according to the process "CPRPM→MBIFM→HLDCCM→INR→IDORM" to achieve end-to-end rain deraining. The process is as follows: 1. The Phase-Aware Rain Pattern Pre-analysis module CPCRPM extracts phase features from the rain-bearing image of the medium-resolution flow (M-flow) to generate rain pattern-aware features. 2. The Multimodal Bioinspired Feature Fusion Module (MBIFM) for rain pattern-based feature perception. Channel attention operations are performed on spatial, frequency, and biological visual features to generate multimodal fusion features. 3. The hierarchical lightweight dilated convolutional core module HLDCCM is combined with multimodal fusion features. 4. Generate corresponding decoding features from the input features of the corresponding rain-containing image and fuse the decoding features across scales to capture multi-scale raindrop decoding features; 5. Reconstruct details from the fused decoding features (X-stream decoding features) using INR to output preliminary deraining features. 5. Evaluation of the Intelligent Dynamic Optimization Refinement Module IDORM Reduce raindrop complexity and dynamically optimize the output rain-removed image. .

[0055] As shown in Figure 2, in this embodiment, the Phase-Aware Rain Pattern Pre-Analysis Module (CPPRPM) includes a Complex Convolutional Network (CCN), a Phase-Sensitive Pooling Layer (PSP), and a Residual Refinement Submodule. The CPPRPM module extracts the unique features of raindrops in the phase domain through a three-stage process of "complex convolution - phase-sensitive pooling - residual refinement," solving the problem of rain being confused with the background in the intensity domain. When using the Phase-Aware Rain Pattern Pre-Analysis Module (CPPRPM) to extract phase features from a medium-resolution stream image containing rain, the following steps are included: S101) The Complex Convolutional Network (CCN) uses complex convolution kernels... Rain-bearing images of medium-resolution streams Perform complex convolution to output real part features. Features of the imaginary part Traditional real-valued convolution only captures intensity features, while CCN, through complex convolution kernels, extracts both the real and imaginary parts of the features simultaneously, modeling the modulation effect of raindrops on the phase of the light field. Let a medium-resolution rain-containing image be... (Depend on The result was obtained after bilinear downsampling (downsampling factor of 2), and the complex convolution kernel was... (in Let be the real part weight matrix, with dimension . ; The imaginary part weight matrix has dimensions of . Consistent; The imaginary unit; (where the number of feature channels is set to 32 in this embodiment), then the output of CCN is: (1) In the formula: For complex convolution operations, it is defined as "real part convolution ( and Convolution + Imaginary convolution and Parallel computation of "convolution", with a kernel size of 3×3, a stride of 1, and padding of 1 (to ensure that the feature map size is consistent with the input). The feature map is the real part, with dimension . Record information related to the intensity of the features; For the imaginary part feature map, the dimension is... Consistent, recording phase-related information of features.

[0056] S102) Phase-sensitive pooling layer PSP with window size Calculate pixels within the window phase angle , For row coordinates, Using column coordinates, filter the pixels with the largest phase angles and retain their real features. Features of the imaginary part The value of the output pooled real part feature Features of the imaginary part of pooling Raindrops appear as local phase abrupt changes in an image. PSP (Pooling Filtering) focuses on the raindrop region's features by selecting pixels with the largest phase angle within a local window, thus suppressing background interference. Let the pooling window size be... (This invention uses a 2×2 pixel configuration), any pixel within the window The phase angle is (in, It is a two-parameter arctangent function; These are pixel row coordinates, ranging from... ; Pixel column coordinates, range Then the output of the PSP is: (2) Where: For phase-sensitive pooling, the logic is: "Calculate the phase angle of each pixel within the window → Filter the pixel with the largest phase angle → Keep that pixel." and "Value", window step size 2 (to ensure resolution is reduced to 1 / 4); This is the real feature map after pooling, with dimension 1. ; This is the imaginary feature map after pooling, with dimensions equal to... Consistent.

[0057] S103) Residual refinement submodule for pooling real part features Features of the imaginary part of pooling Residual processing is performed separately, and then the data is stitched together along the channel dimension to generate rain-pattern sensing features. To suppress noise in the phase characteristics, and The residual blocks (containing two 3×3 convolutional layers, a batch normalization layer, and a ReLU activation function) are refined separately and then concatenated along the channel dimension to generate rain-shaped perceptual features. , (3) (4) Where: For residual block operations, the structure is "3×3 convolution → batch normalization → ReLU → 3×3 convolution → batch normalization → residual connection", ensuring that the feature dimensions remain unchanged; For channel-dimensional splicing operations, and Channel number merging; This is the refined real feature map, with dimensions and Consistent; This is the refined imaginary feature map, with dimensions and... Consistent; Rain pattern perception features, with dimensions of (2C=64), used for subsequent multimodal fusion.

[0058] As shown in Figure 2, in this embodiment, the Multimodal Bio-Inspired Feature Fusion Module (MBIFM) extracts spatial, frequency, and bio-visual modal features through three parallel branches, and then uses channel attention to achieve adaptive weighted fusion, solving the problem of insufficient single-modal representation. The MBIFM is used based on rain pattern perception features. When performing channel attention operations on spatial, frequency, and bio-visual features, the following steps are included: S201) Extracting rain pattern perception features through concatenated residual blocks in the spatial branch of the multimodal bio-inspired feature fusion module MBIFM. Local details To extract local details; spatial branching focuses on local detail features such as image edges and textures, through two concatenated residual blocks (structured the same as in step S103). Extract features and output spatial features. : (5) Where: For spatial feature extraction operations, the residual block has a convolution kernel size of 3×3, a stride of 1, and padding of 1 to ensure that the feature size remains unchanged; This is a spatial feature map with dimension 1. (2C=64), recording the local structural information of the image.

[0059] (S202) In the frequency branch of the multimodal bio-inspired feature fusion module MBIFM, rain pattern perception features are extracted by simulating discrete wavelet transform. Frequency domain characteristics To separate high-frequency rain noise, a frequency branch is used to separate high-frequency raindrop noise from low-frequency background features. This is achieved by simulating discrete wavelet transform using a "1×1 convolution + 3×3 convolution" to output frequency features. : (6) Where: For frequency feature extraction, 1×1 convolution is used for channel compression (dimensionality is maintained at 2C=64), and 3×3 convolution is used to capture frequency domain differences (distinguishing between high-frequency raindrop noise and low-frequency background). For frequency feature maps, the dimensions are... Consistent, recording the frequency domain information of the image.

[0060] (S203) In the bio-vision branch of the Multimodal Bioinspired Feature Fusion Module (MBIFM), Gabor-like convolutional kernels are used to extract rain pattern perception features. directional features To extract the direction of raindrops; the biological vision branch simulates the sensitivity of the human visual system to direction, extracting the directional features of raindrops through Gabor-like convolution kernels (including four directions: horizontal, vertical, 45°, and 135°), and outputting biological visual features. : (7) Where: For biological visual feature extraction operations, the wavelength of the Gabor convolution kernel is set to 4, and the directional spacing is 45° to ensure coverage of common raindrop directions; For biological visual feature maps, dimensions and Consistent, recording the directional distribution information of raindrops.

[0061] S204) Adaptive channel attention fusion first integrates local detail features Frequency domain characteristics With directional features spliced ​​along the channel dimension (dimension) (6C=192); then, through the channel attention module, weights are generated sequentially through global average pooling, 1×1 convolution, ReLU activation, 1×1 convolution, and Sigmoid activation to address multimodal features, i.e., local detail features. Frequency domain characteristics With directional features Adaptive weighting is performed to obtain multimodal fusion features. : (8) (9) Where: The attention weight generation operation follows this process: "Global average pooling (compressing the feature map to 1×1×6C) → 1×1 convolution (reducing the number of channels to 6C / 4=48) → ReLU activation → 1×1 convolution (restoring the number of channels to 6C=192) → Sigmoid activation", outputting the weights. ; The attention weights for the spatial branches are determined by... The first 2C=64 channels are split to obtain the value range [0,1]. The larger the value, the more important the contribution of spatial features to rain removal. The attention weights for the frequency branch are given by... The middle 2C=64 channels are split to obtain the value range [0,1]; The attention weights for the biological vision branch are determined by... The last 2C=64 channels are split to obtain the value range [0,1]; This is an element-wise multiplication operation that enables adaptive adjustment of weights to features; For multimodal fusion features, the dimension is (2C=64), integrating multi-source information for subsequent core rain removal.

[0062] As shown in Figure 2, in this embodiment, the layered lightweight dilated convolution core module HLDCCM is the core rain removal unit of CPDDNet. Through the structure of "multi-scale encoding - bottleneck layer (Transformer) - multi-scale decoding", it realizes multi-scale raindrop capture and high-precision detail restoration.

[0063] The Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) features a U-shaped architecture, designing independent encoder-decoder paths for the three resolution streams. It transmits detailed information through cross-scale skip connections (as shown in Figure 2, where skip connections between encoder and decoder blocks are achieved by concatenating channel dimensions). The HLDCCM module, corresponding one-to-one with the high-resolution, medium-resolution, and low-resolution streams, is combined with multimodal fusion features. When generating corresponding decoded features from the input features of the rainy image, the process includes the following steps: S301) Projecting the rainy image using a 1×1 convolution to obtain projected features, and then fusing the multimodal features. A resolution adjustment operation is performed to match the resolution of the rain-containing image, resulting in multimodal fusion features with the matched resolution. Then, the projected features are fused with the multimodal features of the matching resolution. The summation yields the initial encoded features, including: feature projection of the input images at each scale, mapping the RGB channels to a high-dimensional feature space; for large scales (X-stream), the input is the original rain-containing image. , through 1×1 convolution projection For mesoscale (M-flow), the input is a downsampled image. , through 1×1 convolution projection For small-scale (S-flow), the input is a double-downsampled image. Projected via 1×1 convolution .

[0064] The projection features are then fused with multimodal features matching the resolution. Add them together to obtain the initial encoded features: (10) Where: To match the maximum scale resolution (After bilinear upsampling); To match the mesoscale resolution (After bilinear upsampling); To match small-scale resolution ; For adaptive resolution adjustment, when fusing small-scale and large-scale features, the small-scale feature map is bilinearly upsampled, and the output of the MBIFM is... When distributed to the X stream, it is upsampled by ×4 (matching H×W); when distributed to the M stream, it is upsampled by ×2 (matching H / 2×W / 2); when distributed to the S stream, no sampling is required (matching H / 4×W / 4). These are the initial encoded features at large, medium, and small scales, respectively, with dimensions consistent with the corresponding projected features.

[0065] S302) The initial encoded features are input into the encoding path of the hierarchical lightweight dilated convolutional core module HLDCCM. The encoding path includes concatenated encoding blocks, and each encoding block is a hierarchical lightweight dilated Transformer block. The output features of each encoding block are downsampled to reduce the resolution, satisfying the following: (11) Among them, To enable layered, lightweight, and hollow Transformer block operations, This is a downsampling operation; specifically, as shown in Figure 2, the initial encoded features are input into three concatenated hierarchical lightweight dilated Transformer blocks (encoding blocks). After each block, the resolution is reduced through downsampling (Pixel Unshuffle, stride 2), ultimately yielding the bottleneck layer output features. The maximum scale is... (After 3 downsampling iterations, the resolution was reduced to 1 / 8); the mesoscale was... (After 3 downsampling iterations, the resolution is reduced to 1 / 16); Small scale: (After 3 downsampling steps, the resolution was reduced to 1 / 32).

[0066] S303) Encode the final features of the path The input features are first normalized by layering the input features, then global feature dependencies are captured through multi-head self-attention, and residual connections are used to preserve the original information, resulting in features after multi-head self-attention and residual connections. Then, for the features The feature is normalized again, and a nonlinear transformation is achieved through a feedforward network. Finally, the output feature of the bottleneck layer is output through residual connections. The bottleneck layer of the hierarchical lightweight dilated convolutional core module HLDCCM is a subsequent module in the encoding path. Its function is to integrate the global features after downsampling of all encoding blocks, providing latent features with long-distance dependencies for the decoding path (e.g., the input of the X-stream bottleneck layer is the features after downsampling of encoding block 3, and the output is...). Its core is the lightweight Transformer module, and the specific process is as follows: Let the input features of the bottleneck layer (taking X flow as an example) be... ,in, The number of input feature channels (the number of channels after downsampling of coded block 3, such as...) , (Base channel number) The input feature resolution (the resolution after downsampling of the X stream coded block 3, such as...) ); The final compressed features of the encoded path contain global raindrop distribution and background structure information.

[0067] The bottleneck layer achieves global feature integration through "layer normalization + MSA + FFN". First, the input features are normalized by layer, then global feature dependencies are captured through multi-head self-attention, and the original information is preserved by combining residual connections. (12) (13) (14) Where: For layer normalization, the channel dimension is normalized using the following formula: ( The characteristic mean, For characteristic variance, For learnable parameters, To prevent smoothing terms with a denominator of zero); For multi-head self-attention operations, features are mapped to query (Q), key (K), and value (V), and then concatenated after parallel computation by multiple attention heads, as shown in the formula: (15) Where: The number of attention heads (set to 4 in this invention); For the first The output of each attention head is given by the formula: ; , , For the first The query, key, and value matrix of the size. For learnable projective weights; For the dimension of a single attention head (e.g.) ), Used to prevent excessive emphasis on attention; The attention output projection weights are used to map the concatenated multi-head features back to the input. aisle; The features obtained after MSA and residual connection retain global dependencies and original encoded feature information.

[0068] right The layer is normalized again, and the feature nonlinear transformation is achieved through a feedforward network. Finally, the output feature of the bottleneck layer is output through residual connections. (16) (17) (18) Where: For the feedforward network, a lightweight structure of "1×1 convolution + ReLU + 1×1 convolution" is adopted, and the formula is as follows: (19) Where: This represents the number of intermediate channels (expansion factor 2.66). For a 1×1 convolution operation, the number of channels of the input X is mapped to... ; The final output features of the bottleneck layer (corresponding to the technical solution) Taking X stream as an example, This provides global feature guidance for the decoding path.

[0069] S304) The output characteristics of the bottleneck layer The input is the decoding path of the hierarchical lightweight dilated convolutional core module HLDCCM. The decoding path includes concatenated decoding blocks, and each decoding block is a hierarchical lightweight dilated Transformer block. The first decoding block simultaneously receives the skip connection features of the corresponding encoding block and the output features of the bottleneck layer. Subsequent decoding blocks simultaneously receive the skip connection features of the corresponding encoding block and the output features of the previous decoding block. Each decoding block concatenates all simultaneously received features along the channel dimension before performing a layered lightweight diffusing Transformer block operation. Furthermore, the output features of each decoding block are upsampled to restore resolution, satisfying the following conditions: (20) Among them, For the upsampling operation, To enable layered, lightweight, and hollow Transformer block operations, This involves channel-level concatenation. Specifically, as shown in Figure 2, the bottleneck layer output features are input into three concatenated decoding blocks. After each block, the resolution is restored through upsampling (Pixel Shuffle, stride 2). Simultaneously, these features are added along the corresponding channel elements to the features of the corresponding layer at the encoding end (upsampled / downsampled to match the resolution) to obtain the decoded features. The maximum scale is... (Original resolution restored after 3 upsampling cycles); mesoscale is (After 3 upsampling iterations, mesoscale resolution is restored); small scale is... (After 3 upsampling operations, small-scale resolution is restored).

[0070] As shown in Figure 2, when fusing decoded features across scales, the decoded features output by the HLDCCM (Hyperlayer Lightweight Dilated Convolutional Core Module) of the low-resolution stream are downsampled and input into the first decoding block (S stream → M stream downsampled × 1 / 4) of the decoding path of the HLDCCM of the medium-resolution stream. Similarly, the decoded features output by the HLDCCM of the medium-resolution stream are downsampled and input into the first decoding block (M stream → X stream downsampled × 1 / 4) of the decoding path of the HLDCCM of the high-resolution stream, resulting in the decoded features output by the HLDCCM of the high-resolution stream being the fused decoded features.

[0071] For the hierarchical lightweight dilated Transformer block used as both the encoding and decoding blocks in this embodiment, this block integrates adaptive dilated convolution (capturing local multi-scale raindrops) and self-attention (capturing global dependencies). The specific process for operating the hierarchical lightweight dilated Transformer block is as follows: First, adaptive dilated convolution is applied to the input features... After layer normalization, three 3×3 convolutional layers with different dilation rates are input in parallel, and the effective output is dynamically selected through attention weights. (21) Where: , For layer normalization, the channel dimension is normalized to stabilize training; The void ratio has values ​​of 1, 2, and 3, corresponding to small, medium, and large receptive fields, respectively. For dilated convolution operations, the padding is... (Ensuring the feature size remains constant) Capture raindrop features at different scales; void ratio The attention weights are generated by "global average pooling + 1×1 convolution + Softmax", satisfying... The contribution of different receptive fields is dynamically adjusted. For adaptive dilated convolution output, and With consistent dimensions, they are added together through residual joins ( ).

[0072] Then, for the self-attention and feedforward network, the input features are... With adaptive dilated convolution output After being summed via residual connections, the input layer is normalized, and then global feature dependencies are captured through multi-head self-attention (MSA). These features are then fed into a feedforward network (FFN) via residual connections, ultimately outputting block features. (twenty two) (23) Where: For multi-head self-attention operations, the number of heads is set to 4, and the attention dimension is... (2C=64), the formula is ( These are query, key, and value matrices, respectively. (obtained by linear transformation) It is a feedforward network with a structure of "1×1 convolution (2C channels → 2C×2.66) → ReLU → 1×1 convolution (2C×2.66 → 2C)" to achieve non-linear transformation of features; For self-attention output features, the dimension and Consistent; The final output feature of the block, with dimensions and Consistent.

[0073] As shown in Figure 2, in this embodiment, when performing Implicit Neural Representation (INR) detail reconstruction on the decoded features output by the hierarchical lightweight dilated convolution core module HLDCCM of the X stream after cross-scale fusion, in order to improve the accuracy of high-frequency detail recovery, INR models the image as a continuous function and outputs pixel RGB values ​​through coordinate encoding and MLP. The process is as follows: S401) Coordinate generation: Generate normalized coordinates for the target resolution. Specifically, regarding target resolution (such as maximum scale) Generate normalized coordinates The coordinate range is independent of the image size to ensure generalization; row coordinates. Linear mapping from -1 (top of image) to 1 (bottom of image); column coordinates A linear mapping from -1 (left side of the image) to 1 (right side of the image).

[0074] S402) Position Encoding: Mapping normalized coordinates to a high-dimensional space to obtain the corresponding position encoding vector. Specifically, mapping coordinates to a higher-dimensional space enhances the ability to model high-frequency details. (24) Where: The number of encoding layers is set to 4 in this invention to ensure coverage of high-frequency details; The location encoding vector has a dimension of This maps low-dimensional coordinates to high dimensions, enhancing the MLP's ability to model high-frequency features.

[0075] S403) MLP Mapping: Transforming the position encoding vector The decoded features of the high-resolution streaming layered lightweight dilated convolutional core module HLDCCM are input into the multilayer perceptron (MLP) to obtain a detailed image reconstructed from INR. Specifically, the encoded coordinates and the decoded features of HLDCCM are input into the MLP, and the pixel RGB values ​​are output. (25) Where: Decoding features for HLDCCM (such as maximum scale) ), sampled to coordinates via bilinear interpolation The corresponding eigenvalues; It is a multilayer perceptron with the structure of "linear layer (16+2C→64)→LayerNorm→ReLU→linear layer (64→64)→LayerNorm→ReLU→linear layer (64→3)"; The Sigmoid activation function normalizes the output to [0,1], corresponding to the RGB pixel value range; Detailed image reconstructed from INR, with dimensions of The resolution is consistent with the original image.

[0076] S404) overlays the decoding features of the hierarchical lightweight dilated convolutional core module HLDCCM of high-resolution streaming with the detailed image reconstructed by INR to obtain preliminary rain removal features. The output is (dimension) ).

[0077] As shown in Figure 2, in this embodiment, the Intelligent Dynamic Optimization Refinement Module (IDORM) dynamically selects a refinement strategy based on raindrop complexity to address the problem of poor adaptability to fixed structures. The IDORM module is used to evaluate the initial rain removal features. Reduce raindrop complexity and dynamically optimize the output rain-removed image. At that time, the process includes the following steps: S501) Initial rain removal characteristics Inputting a convolutional neural network (CNN), and sequentially passing it through 3×3 convolution, ReLU activation, max pooling, 3×3 convolution, ReLU activation, global average pooling, a linear layer, and sigmoid activation, yields the raindrop complexity. Specifically, a lightweight CNN is used to evaluate preliminary rain removal features. raindrop complexity Quantifying the complexity of rainfall conditions: (26) Where: For the complexity evaluation operation, the structure is "3×3 convolution (3→16) → ReLU → max pooling (stride 2) → 3×3 convolution (16→32) → ReLU → global average pooling → linear layer (32→1) → Sigmoid". The raindrop complexity is represented by a value in the range [0,1]. A larger value indicates that the raindrops are more dense and have more complex shapes (such as rainstorms or rain and fog).

[0078] S502) Dynamic refinement strategy based on upper limit of threshold With threshold lower limit (set up , Choose a refinement strategy: (1) High complexity refinement: If the raindrop complexity is high... Greater than the upper limit of the threshold ( (This will indicate the initial characteristics of rain removal) The input consists of a residual attention refinement network (RARNet) cascaded with two layered lightweight dilated Transformer blocks and one channel attention layer to remove residual raindrops, and the output is the optimized features. Specifically, when raindrops are dense or have complex shapes, a Residual Attention Refinement Network (RARNet) is used to focus on dense raindrop regions through Transformer blocks and attention mechanisms. (27) Where: For RARNet operations, two hierarchical lightweight dilated Transformer blocks are concatenated with one channel attention layer to ensure complete removal of residual raindrops. Each hierarchical lightweight dilated Transformer block acquires input features and performs hierarchical lightweight dilated Transformer block operations, where the input features of the first hierarchical lightweight dilated Transformer block are the preliminary rain removal features. Furthermore, the input features of the second layered lightweight perforated Transformer block are the output features of the first layered lightweight perforated Transformer block.

[0079] The channel attention layer is a feature weighting mechanism based on "channel importance assessment". The input of the channel attention layer is the output feature of the second hierarchical lightweight dilated Transformer block. (dimension) ,in Based on the number of channels, (where the feature map resolution is used), the output is the weighted feature map. The process consists of three steps: "global information aggregation → weight generation → feature weighting". The specific formula is as follows: By using two methods, global average pooling (GAP) and global max pooling (GMP), the spatial features of each channel are compressed into a single scalar, comprehensively capturing the global statistical information of the channels (GAP reflects the average response of the channel, GMP reflects the extreme response of the channel, and the combination of the two improves robustness). (28) (29) Where: This is a global average pooling operation that averages the values ​​of all spatial pixels in each channel. This is a global max pooling operation that takes the maximum value of all spatial pixels for each channel. The channel statistics vector output by GAP, dimension ; The channel statistics vector output by GMP, with dimensions and Consistent; Features All values ​​in the channel dimension (“:” represents all channels), spatial location The pixel value.

[0080] Will and After concatenation, channel attention weights are generated using a shared fully connected layer and sigmoid activation to dynamically evaluate channel importance. (30) (31) (32) (33) (34) Where: For channel-dimensional splicing operations, and splicing into dimensions ; , It is a fully connected layer (linear transformation), where for The weight matrix (compressing the 2C channel to the C channel). The bias is C-dimensional; for The weight matrix (keeping the number of channels constant). The bias is C-dimensional; As a nonlinear activation function, nonlinearity is introduced to fit the complex dependencies between channels; The Sigmoid activation function maps the weight values ​​to the range [0,1], which facilitates subsequent feature weighting. For the final generated channel attention weights, the dimension is... Each value corresponds to the importance of a channel (the closer the value is to 1, the greater the channel's contribution to rain removal).

[0081] Channel attention weights Input features Perform channel-wise element-wise multiplication to dynamically adjust the feature channels, and output the weighted features as the optimized features: (35) Where: For element-wise multiplication (Hadamard product), Each channel weight is broadcast to the entire spatial dimension, and Multiply all pixels of the corresponding channel; The output features of the channel attention layer, with dimensions and Consistent, key channels have been strengthened and redundant channels have been suppressed through weight adjustments.

[0082] (2) Medium complexity correction: If the raindrop complexity Less than the upper threshold and greater than the lower threshold ( (This will indicate the initial characteristics of rain removal) The input consists of a lightweight residual correction network (LRCNet) composed of two concatenated residual blocks to remove residual raindrops, and the output is the optimized features. Specifically, when the raindrop density is moderate, a lightweight residual correction network (LRCNet) is used to achieve efficient refinement through a small number of residual blocks: (36) Where: For LRCNet operations, two residual blocks are concatenated, which reduces computational complexity while ensuring rain removal.

[0083] (3) Low-complexity output: If the complexity of raindrops is low... Less than the lower threshold ( ), directly indicating the initial rain-removal characteristics Output as optimized features Specifically, when raindrops are sparse or nearly nonexistent, output directly. Avoid over-calculation: (37)S503) Final Derained Image Output: Output the optimized features The final derained image is obtained by mapping the image to RGB channels using a 1×1 convolution. , dimension The resolution is consistent with the input rainy image.

[0084] In step S2 of this embodiment, to balance the thoroughness of rain removal and the fidelity of details, a composite loss function is used to optimize CPDDNet, and the total loss is a weighted sum of five components: (38) Among them: For Charbonnier loss, These are the weighting coefficients for the Charbonnier loss, which suppresses outliers (such as residual raindrops) and improves the robustness of the loss function. (39) Where: For resolution scale identification (x=large, m=medium, s=small); For scale Deraining output image; For scale A clear label image (obtained by downsampling the original clear image); Representing scale Deraining output image The i-th pixel and scale Clear label images The i-th pixel The square of the L2 norm of the difference; For scale The total number of pixels (e.g.) ); As the smoothing parameter, set to To avoid zero within the square root; The weighting coefficient is set to 0.6, which dominates pixel-level fidelity.

[0085] For edge loss, These are the weighting coefficients for the edge loss, which preserves image edge details and avoids over-smoothing. (40) Where: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale Deraining output image; For scale Clear label images; Gradient operators in the x and y directions (implemented based on 3×3 Laplacian filtering) are used to extract edge features; L1 norm is used to quantify marginal differences; The weighting factor is set to 0.08 to enhance edge detail protection.

[0086] For frequency domain loss, These are the weighting coefficients for the frequency domain loss, which suppresses frequency domain raindrop noise and maintains background spectrum consistency. (41) Where: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale Deraining output image; For scale Clear label images; It is an L1 norm; This is a two-dimensional Fast Fourier Transform, which transforms the image from the spatial domain to the frequency domain; The weighting coefficient is set to 0.05 to suppress high-frequency raindrop noise.

[0087] For L1 loss, The weights for the L1 loss function provide the underlying pixel-level constraints, stabilizing the training process. (42) Where: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale The total number of pixels; and They are scales Deraining output image The i-th pixel and scale Clear label images The i-th pixel; The weighting coefficient is set to 0.15 to assist in pixel-level fidelity.

[0088] In order to perceive loss, The weights for the perceptual loss are used to improve the visual consistency of the derained image. Features are extracted based on a pre-trained VGG19 network. (43) Where: It is an L1 norm; The feature extraction function for the VGG19 network uses the output features of the 5th convolutional block of the network. Output the derained image at the largest scale; For the clearest label image at the largest scale; The weighting factor is set to 0.2 to improve visual quality.

[0089] The experimental setup for this embodiment is as follows: In weight initialization, convolutional and linear layers are initialized using a Kaiming normal distribution, and the γ of the batch normalization layer is initialized to 1 and the β to 0; in the INR module, the linear layers of the MLP are initialized using a Xavier uniform distribution. The Adam optimizer is selected for model training, with an initial learning rate of... , , Weight decay The training epochs were set to 1000, the batch size was set to 1, the total loss was calculated according to formula (36), and the parameters were updated by backpropagation. The experimental results are shown in Figure 3. It can be seen that the method in this embodiment can achieve a good rain removal effect.

[0090] In summary, the image deraining method based on phase perception and dynamic optimization proposed in this invention innovatively combines phase perception, multimodal fusion, dynamic optimization and continuous detail reconstruction to form an efficient image deraining scheme. The key points are as follows: (1) Phase-aware rain pattern pre-analysis module (CPPRPM): extracts phase features through complex convolution and focuses raindrop regions by combining phase-sensitive pooling to solve the problem of confusion between intensity domain rain and background. The feature extraction method is innovative; (2) Multimodal bio-inspired feature fusion module (MBIFM): integrates spatial, frequency and bio-visual three-modal features and achieves multi-source information synergy through adaptive channel attention weighting. The fusion mechanism is innovative; (3) Layered lightweight hollow Transformer block (LHDC Transformer) Block: Integrating adaptive dilated convolution and self-attention, dynamically adjusting the receptive field to match the raindrop scale, taking into account both local details and global dependencies, the module structure is innovative; (4) INR-based detail reconstruction mechanism: combining implicit neural representation with hierarchical convolution, restoring high-frequency details through continuous image modeling, the detail reconstruction method is innovative; (5) Intelligent dynamic optimization and refinement module (IDORM): dynamically selecting refinement strategy based on raindrop complexity evaluation, balancing rain removal effect and computational efficiency, the dynamic optimization mechanism is innovative; (6) Composite loss function: integrating Charbonnier loss, edge loss, frequency domain loss, L1 loss and perceptual loss, optimizing rain removal performance in multiple dimensions, the loss design is innovative; (7) CPDDNet overall architecture: connecting the above modules in the process of "pre-analysis-fusion-core rain removal-dynamic refinement" to form an end-to-end multi-scale rain removal network, the architecture design is innovative.

[0091] Compared with the best existing technologies (Restormer and phase-aware image restoration methods), the advantages of the present invention are as follows: (1) Stronger ability to distinguish between rain and background: Existing technologies (such as Restormer) rely on intensity features and cannot distinguish the intensity overlap area between rain and background texture; the present invention extracts phase features through CPCRPM, and the phase information is robust to illumination and can characterize the essential structure of the image, which can improve the raindrop removal rate in dense rain and fog scenes.

[0092] (2) More comprehensive multimodal fusion: Existing phase-aware methods only use phase for preprocessing and do not combine it with other modalities; This invention integrates spatial, frequency and biological visual features through MBIFM, and the adaptive weighting mechanism improves the model's generalization ability for different rain types (sparse rain, dense rain, rain fog).

[0093] (3) Better adaptability to rain conditions: Existing technologies use a fixed structure, which results in computational redundancy in low-complexity rain conditions and incomplete rain removal in high-complexity rain conditions; This invention uses an IDORM dynamic adjustment strategy to improve the inference speed in low-complexity scenarios and reduce residual raindrops in high-complexity scenarios.

[0094] (4) Higher precision in detail recovery: Existing technologies rely on discrete convolution, which easily leads to the loss of high-frequency details such as edges and textures; This invention achieves continuous detail reconstruction through INR, which can improve the edge integrity of rain-removed images.

[0095] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0096] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An image deraining method based on phase sensing and dynamic optimization, characterized in that, Includes the following steps: S1) Construct a phase-aware dynamic rain removal network CPDDNet. The CPDDNet converts rain-containing images into rain-containing images of high-resolution, medium-resolution, and low-resolution streams, respectively. The phase-aware rain pattern pre-analysis module CPCRPM extracts phase features from the rain-containing image of the medium-resolution stream to generate rain pattern sensing features. The multimodal bio-inspired feature fusion module MBIFM is used based on rain pattern perception features. Channel attention operations are performed on spatial, frequency, and biological visual features to generate multimodal fusion features. It uses a hierarchical lightweight dilated convolutional core module HLDCCM, corresponding to high-resolution, medium-resolution, and low-resolution streams, combined with multimodal fusion features. The corresponding rain-inducing image input features are used to generate corresponding decoded features, and the decoded features are fused across scales. The details of the fused decoded features are reconstructed using INR, and the preliminary rain-removing features are output. The intelligent dynamic optimization and refinement module IDORM was used to evaluate the initial rain removal characteristics. Reduce raindrop complexity and dynamically optimize the output rain-removed image. S2) The phase-aware dynamic deraining network CPDDNet is trained using a rain-clear image dataset. The network parameters are optimized using a composite loss function, which is a weighted sum of Charbonnier loss, edge loss, frequency domain loss, L1 loss, and perceptual loss. S3) The rain-laden image to be processed is input into the trained phase-aware dynamic deraining network CPDDNet for multi-scale feature processing and dynamic optimization and refinement, and then the clear image after deraining is output.

2. The image deraining method based on phase sensing and dynamic optimization according to claim 1, characterized in that, The phase-aware rain pattern pre-analysis module CPPRPM includes a complex convolutional network CCN, a phase-sensitive pooling layer PSP, and a residual thinning submodule. When using the phase-aware rain pattern pre-analysis module CPPRPM to extract phase features from a rain-containing image of a medium-resolution stream, the following steps are included: S101) The complex convolutional network CCN uses complex convolution kernels. Rain-bearing images of medium-resolution streams Perform complex convolution to output real part features. Features of the imaginary part ,satisfy: in, ， Let be the real part weight matrix. The imaginary part weight matrix, The imaginary unit, This is a complex convolution operation, specifically a parallel computation of rain-containing images. With the real part weight matrix Convolution and rain-containing images With the imaginary part weight matrix The convolution; S102) The phase-sensitive pooling layer PSP with window size Calculate pixels within the window phase angle ， For row coordinates, Using column coordinates, filter the pixels with the largest phase angles and retain their real features. Features of the imaginary part The value of the output pooled real part feature Features of the imaginary part of pooling ,satisfy: in, For phase-sensitive pooling operation; S103) The residual refinement submodule refines the real part features of the pooling. Features of the imaginary part of pooling Residual processing is performed separately, and then the data is stitched together along the channel dimension to generate rain-pattern sensing features. ,satisfy: ， in, For residual block operations, This is for channel splicing operations.

3. The image deraining method based on phase sensing and dynamic optimization according to claim 1, characterized in that, The multimodal bio-inspired feature fusion module MBIFM was used based on rain pattern perception features. When performing channel attention operations on spatial, frequency, and bio-visual features, the following steps are included: S201) Extracting rain pattern perception features through concatenated residual blocks in the spatial branch of the multimodal bio-inspired feature fusion module MBIFM. Local details ,satisfy: in, For spatial feature extraction operations; S202) In the frequency branch of the multimodal bio-inspired feature fusion module MBIFM, rain pattern sensing features are extracted by simulating discrete wavelet transform. Frequency domain characteristics ,satisfy: in, For frequency feature extraction operations; S203) In the biological vision branch of the multimodal bio-inspired feature fusion module MBIFM, Gabor-like convolutional kernels are used to extract rain pattern perception features. directional features ,satisfy: in, For biological visual feature extraction operations; S204) for local detail features Frequency domain characteristics With directional features The generation is obtained by splicing along the channel dimension. Attention weights are then generated sequentially through global average pooling, 1×1 convolution, ReLU activation, 1×1 convolution, and Sigmoid activation. Finally, attention weights are used to apply attention to local detail features. Frequency domain characteristics With directional features Weighted summation yields multimodal fusion features ,satisfy: in, For weight generation operations, These are the weights for spatial, frequency, and biological vision branches, respectively. This is element-wise multiplication.

4. The image deraining method based on phase sensing and dynamic optimization according to claim 1, characterized in that, The system utilizes a hierarchical lightweight dilated convolutional core module (HLDCCM) corresponding to high-resolution, medium-resolution, and low-resolution streams, combined with multimodal fusion features. When generating corresponding decoded features from the input features of the rainy image, the process includes the following steps: S301) Projecting the rainy image using a 1×1 convolution to obtain projected features, and then fusing the multimodal features. A resolution adjustment operation is performed to match the resolution of the rain-containing image, resulting in multimodal fusion features with the matched resolution. Then, the projected features are fused with the multimodal features of the matching resolution. S302) The initial encoded features are added together to obtain the initial encoded features; the initial encoded features are input into the encoding path of the hierarchical lightweight dilated convolutional core module HLDCCM, the encoding path includes concatenated encoding blocks and the encoding blocks are hierarchical lightweight dilated Transformer blocks, and the output features of each encoding block are downsampled to reduce the resolution, satisfying: in, To enable layered, lightweight, and hollow Transformer block operations, For downsampling operation; S303) Encode the final features of the path The input features are first normalized by layering the input features, then global feature dependencies are captured through multi-head self-attention, and residual connections are used to preserve the original information, resulting in features after multi-head self-attention and residual connections. Then, for the features The feature is normalized again, and a nonlinear transformation is achieved through a feedforward network. Finally, the output feature of the bottleneck layer is output through residual connections. ,satisfy: in, For layer normalization operation, For bullish self-attention operations, For a feedforward network; S304) the output features of the bottleneck layer The input is the decoding path of the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM). The decoding path includes concatenated decoding blocks, and each decoding block is a hierarchical lightweight dilated Transformer block. The first decoding block simultaneously receives the skip connection features of the corresponding encoding block and the output features of the bottleneck layer. Subsequent decoding blocks simultaneously receive the skip connection features of the corresponding encoding block and the output features of the previous decoding block. Each decoding block concatenates all simultaneously received features along the channel dimension before performing a layered lightweight diffusing Transformer block operation. Furthermore, the output features of each decoding block are upsampled to restore resolution, satisfying the following conditions: in, For the upsampling operation, To enable layered, lightweight, and hollow Transformer block operations, This is a channel-dimensional splicing operation.

5. The image deraining method based on phase sensing and dynamic optimization according to claim 4, characterized in that, When fusing decoded features across scales, the decoded features output by the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the low-resolution stream are downsampled and input into the first decoding block of the decoding path of the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the medium-resolution stream. Similarly, the decoded features output by the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the medium-resolution stream are downsampled and input into the first decoding block of the decoding path of the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the high-resolution stream, so that the decoded features output by the Hierarchical Lightweight Dilated Convolutional Core Module (HLDCCM) of the high-resolution stream are the fused decoded features.

6. The image deraining method based on phase sensing and dynamic optimization according to claim 4, characterized in that, When performing layered lightweight hollow Transformer block operations Includes the following steps: input features After layer normalization, three 3×3 convolutional layers with different dilation rates are input in parallel. The effective output is dynamically selected through attention weights, satisfying the following: In the formula: ， For layer normalization operation; The void ratio has values ​​of 1, 2, and 3, corresponding to small, medium, and large receptive fields, respectively. For dilated convolution operations, the padding is... To capture the characteristics of raindrops at different scales; void ratio Attention weights; For adaptive dilated convolution output, and Dimensionality consistent; input features With adaptive dilated convolution output After being summed via residual connections, the input layer is normalized, and then global feature dependencies are captured through multi-head self-attention. These features are then fed into a feedforward network via residual connections, ultimately outputting block features that satisfy the following conditions: In the formula: This is a bullish self-attention operation; It is a feedforward network; For self-attention output features, the dimension and Consistent; The final output feature of the block, with dimensions and Consistent.

7. The image deraining method based on phase sensing and dynamic optimization according to claim 1, characterized in that, When reconstructing details from the fused decoded features using INR, the following steps are included: S401) Generating normalized coordinates for the target resolution. (S402) Map the normalized coordinates to a high-dimensional space to obtain the corresponding position encoding vector. ,satisfy: In the formula, S403) The position encoding vector is the number of encoding layers. The decoded features of the high-resolution stream hierarchical lightweight dilated convolutional core module HLDCCM are input into the multilayer perceptron to obtain a detailed image reconstructed by INR, satisfying the following: In the formula, The decoding features of the hierarchical lightweight dilated convolutional core module HLDCCM are sampled to coordinates via bilinear interpolation. The corresponding eigenvalues; It is a multilayer perceptron; The Sigmoid activation function (S404) is used to overlay the decoded features of the hierarchical lightweight dilated convolutional core module HLDCCM of high-resolution streaming with the detailed image reconstructed by INR to obtain preliminary rain removal features. 。 8. The image deraining method based on phase sensing and dynamic optimization according to claim 1, characterized in that, The initial rain removal characteristics were assessed using the Intelligent Dynamic Optimization and Refinement Module (IDORM). Reduce raindrop complexity and dynamically optimize the output rain-removed image. At that time, the process includes the following steps: S501) Initial rain removal characteristics Inputting a convolutional neural network (CNN), and sequentially passing it through 3×3 convolution, ReLU activation, max pooling, 3×3 convolution, ReLU activation, global average pooling, a linear layer, and sigmoid activation, yields the raindrop complexity. S502) If raindrop complexity If the value exceeds the upper threshold, the initial rain removal feature will be applied. The input consists of a residual attention refinement network (RARNet) cascaded with two layered lightweight dilated Transformer blocks and one channel attention layer to remove residual raindrops, and the output is the optimized features. If raindrop complexity If the value is less than the upper threshold and greater than the lower threshold, the initial rain removal feature will be applied. The input consists of a lightweight residual correction network (LRCNet) composed of two concatenated residual blocks to remove residual raindrops, and the output is the optimized features. If raindrop complexity If the value is less than the lower threshold, the initial rain removal feature will be directly applied. Output as optimized features S503) Optimized features The final derained image is obtained by mapping the image to RGB channels using a 1×1 convolution. 。 9. The image deraining method based on phase sensing and dynamic optimization according to claim 8, characterized in that, Initial signs of rain removal When removing residual raindrops from the input of a residual attention refinement network (RARNet) consisting of two hierarchical lightweight dilated Transformer blocks cascaded with one channel attention layer, the following steps are included: Each hierarchical lightweight dilated Transformer block acquires the input features and performs hierarchical lightweight dilated Transformer block operations, where the input features of the first hierarchical lightweight dilated Transformer block are the initial rain-removing features. Furthermore, the input features of the second layered lightweight perforated Transformer block are the output features of the first layered lightweight perforated Transformer block; the channel attention layer acquires the output features of the second layered lightweight perforated Transformer block. , respectively for features Perform global average pooling and global max pooling to obtain the channel statistics vector of the global average pooling output. Channel statistics vector of the output of global max pooling Then, channel statistical vectors With channel statistical vector The concatenated features are sequentially passed through a fully connected layer, ReLU activation, another fully connected layer, and a Sigmoid activation layer to obtain channel attention weights. These channel attention weights are then... With features Perform channel-by-channel and element-by-element multiplication to obtain the weighted features, which are then used as the optimized features.

10. The image deraining method based on phase sensing and dynamic optimization according to claim 1, characterized in that, The composite loss function satisfies: in: For Charbonnier loss, Here are the weights of the Charbonnier loss, which satisfies the following: In the formula: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale Deraining output image; For scale Clear label images; For scale The total number of pixels; For smoothing parameters; Representing scale Deraining output image The i-th pixel and scale Clear label images The i-th pixel The square of the L2 norm of the difference; For edge loss, Let be the weight coefficients of the edge loss, and let the edge loss satisfy: In the formula: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale Deraining output image; For scale Clear label images; These are gradient operators in the x and y directions, respectively; It is an L1 norm; For frequency domain loss, Here are the weighting coefficients for the frequency domain loss, which satisfy the following: In the formula: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale Deraining output image; For scale Clear label images; It is an L1 norm; It is a two-dimensional fast Fourier transform; For L1 loss, Let be the weights of the L1 loss function. The L1 loss function satisfies: In the formula: For resolution scale identification, x represents high resolution, m represents medium resolution, and s represents low resolution; For scale The total number of pixels; and They are scales Deraining output image The i-th pixel and scale Clear label images The i-th pixel; In order to perceive loss, Let these be the weighting coefficients for the perceptual loss. The perceptual loss satisfies: In the formula: Output the derained image at the maximum resolution scale; Clear label images at maximum resolution scale; It is an L1 norm; This is the feature extraction function for the pre-trained VGG19 network.

Images