A single image rain removal method and system based on direction perception

By combining a direction-aware single-image rain removal method with convolutional feature embedding, shallow feature enhancement, local enhancement Transformer multi-scale coding, and frequency-spatial domain complementary refinement, the robustness and detail fidelity of existing technologies in complex rainy scenes are solved, achieving efficient rain line suppression and background restoration.

CN122473005APending Publication Date: 2026-07-28EAST CHINA JIAOTONG UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
EAST CHINA JIAOTONG UNIVERSITY
Filing Date
2026-06-29
Publication Date
2026-07-28

AI Technical Summary

Technical Problem

Existing image deraining techniques suffer from poor generalization robustness and difficulty in completely reconstructing background texture and structural information when dealing with complex, variable, and varying degrees of degradation in real dynamic rainy scenes. They are particularly ineffective when there is large-area, long-range rain occlusion. Furthermore, existing methods fail to fully explore the unique distribution patterns of rain degradation in the frequency domain.

Method used

A direction-aware single-image rain removal method is adopted. Through convolutional feature embedding, shallow feature enhancement, local enhancement Transformer multi-scale coding, direction-aware residual compensation, and multi-scale spatial-frequency complementary refinement of frequency domain features and spatial domain features, a residual prediction map is generated and added pixel by pixel to the rain-degraded image to achieve the rain removal effect.

Benefits of technology

It effectively suppresses rain streak noise diffusion, enhances rain line direction modeling and maintains phase structure information, and preserves background texture and edge details while removing multi-scale rain lines, thereby improving rain removal integrity, restoration clarity and generalization ability in complex rainy scenes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122473005A_ABST
    Figure CN122473005A_ABST
Patent Text Reader

Abstract

The application discloses a single-image rain removal method and system based on direction perception. The method obtains a rain-degraded image, after convolution feature embedding and shallow feature enhancement, multi-scale coding is performed by using a local enhancement Transformer structure; direction perception residual compensation features are generated at the shallow coding level and residual injection is performed, frequency domain features and spatial domain features are generated at the deep coding level and gate fusion injection is performed; then, residual prediction images are obtained through bottleneck modeling, hierarchical decoding and refining output; the residual prediction images are added to the rain-degraded image pixel by pixel to obtain a rain-removed image. Through sparse attention and local enhancement of LSTB, direction residual compensation of DARC and air-frequency complementary refining of FSCR, the application can simultaneously inhibit rain streak noise diffusion, enhance rain line direction modeling and maintain phase structure information, remove multi-scale rain lines while preserving background texture and edge details, and improve rain removal integrity, recovery clarity and generalization ability in complex rainy weather scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method and system for deraining single images based on orientation awareness. Background Technology

[0002] Image deraining is a classic and highly challenging ill-posed problem in computer vision. Its fundamental goal is to recover clear background content from degraded observations obscured by rain streaks or fog, thus providing reliable, high-quality input for subsequent advanced visual understanding tasks such as object detection and image segmentation. With the deepening of research in this field, deraining techniques have undergone significant paradigm shifts. Early traditional algorithms relied heavily on manually designed image priors, such as Gaussian mixture models or sparse coding, as optimization constraints. Subsequently, data-driven convolutional neural networks (CNNs) significantly improved the performance ceiling of deraining models by introducing multi-scale feature fusion and various attention mechanisms. In recent years, the Transformer architecture, with its superior non-local information modeling and long-range dependency capture capabilities, has gradually replaced traditional CNNs, becoming the core backbone architecture for image restoration tasks.

[0003] However, existing technologies still exhibit significant limitations when dealing with complex, dynamic rainy scenes with varying degrees of degradation. Early physical models based on prior knowledge heavily rely on researchers' empirical observations, making it difficult to accurately depict the inherent statistical regularities of complex natural scenes. They often demonstrate poor generalization robustness when faced with real-world scenarios where rain scale and density fluctuate dramatically. CNN models, widely used to overcome this limitation, are constrained by the inherent local receptive field of convolutional kernels and their translation invariance assumptions. This structural locality fundamentally hinders the network's effective modeling of the global structure and long-range dependencies of images, resulting in difficulties in thoroughly reconstructing the underlying background texture and structural information when dealing with large-area, long-range rain occlusion.

[0004] As the current mainstream paradigm, the Transformer architecture also faces multiple insurmountable bottlenecks in practical applications. At the feature extraction level, standard dense self-attention not only causes computational costs to increase quadratically with image resolution, but more importantly, its indiscriminate global interaction easily spreads redundant noise from rain areas to clear areas, causing "secondary pollution" during feature aggregation. Simultaneously, due to the inherent lack of local inductive bias characteristic of convolution, Transformer tends to over-smooth image reconstruction when processing high-frequency regions requiring fine-grained perception, resulting in irreversible loss of high-frequency details. At the feature representation level, most existing rain removal models are still limited to single spatial domain feature mapping, failing to fully explore the unique distribution patterns of rain degradation in the frequency domain. That is, the periodic stripe structure of rain streaks corresponds to peak amplitudes in specific directions and frequency bands in the frequency domain. This prior knowledge could provide effective guidance for frequency domain rain streak suppression; however, existing methods do not fully utilize this, failing to implement structural suppression of rain streaks from the frequency domain dimension, thus hindering further improvements in rain removal performance. Summary of the Invention

[0005] The purpose of this invention is to overcome the limitations of existing image deraining techniques by providing a direction-aware single-image deraining method and system.

[0006] To achieve the above objectives, the present invention provides a direction-aware single-image rain removal method, comprising: Obtain a rain-degraded image, and perform convolutional feature embedding and shallow feature enhancement on the rain-degraded image to obtain shallow enhanced features; The shallow enhancement features are encoded at multiple scales using a local enhancement Transformer structure to obtain the backbone encoded features. At the shallow coding level, orientation-aware residual compensation features are generated based on the rain-degraded image. The orientation-aware residual compensation features are obtained by gating and fusing the orientation features extracted by at least two different directional convolutional paths, and are injected into the backbone coding features corresponding to the shallow coding level in a residual manner. At the deep coding level, frequency domain features and spatial domain features are generated based on the rain-degraded image. Fourier transform is performed on the deep features obtained from the rain-degraded image to obtain amplitude and phase spectra. The amplitude spectrum is modulated while preserving the phase spectrum, and then inverse Fourier transform is performed to obtain the frequency domain features. Multi-scale spatial convolution is performed on the deep features to obtain the spatial domain features. After gating and fusing the frequency domain features and spatial domain features, they are injected into the corresponding backbone coding features at the deep coding level in a residual manner. Bottleneck modeling and hierarchical decoding are performed on the injected backbone coding features to obtain the decoded features; A residual prediction map is generated based on the decoding features. The residual prediction map is then added pixel by pixel to the rain-degraded image to obtain the rain-free image.

[0007] As a further improvement to the above technical solution, the generation process of the shallow enhancement features includes: performing a 3×3 overlapping convolution projection on the rain-affected image to map the rain-affected image from the original RGB three-channel image to a high-dimensional feature space to obtain high-dimensional initial features; inputting the high-dimensional initial features into a shallow enhancement sub-network based on a multi-operator adaptive search mechanism for preprocessing, the shallow enhancement sub-network including alternating stacked operation perception layers and grouping operator layers; performing global average pooling on the input features through the operation perception layer, and mapping to generate an operator selection weight vector through a fully connected layer; and the grouping operator layer performing a weighted summation of the responses of multiple candidate feature extraction operators according to the operator selection weight vector to obtain the shallow enhancement features.

[0008] As a further improvement to the above technical solution, the locally enhanced Transformer structure includes an attention sublayer and a feedforward sublayer; The attention sublayer performs Top-K sparse attention processing on the input features, generates multiple sets of sparse attention masks according to various increasing Top-K ratios, sets the low phase key value response to negative infinity and then performs sparse Softmax normalization, and then fuses the multiple sparse attention outputs with learnable scalar coefficients for weighted fusion. The attention sublayer is also connected in parallel to a local convolution enhancement path composed of a series of depthwise convolutions and pointwise convolutions, and the output of the local convolution enhancement path is superimposed on the multi-path sparse attention output. The attention sub-layer and the feedforward sub-layer perform global average pooling on the current feature map, generate channel-wise gated weights through two layers of 1×1 convolution and Sigmoid activation, and perform channel-level modulation on the output of the corresponding sub-layer in the form of soft multiplication; The feedforward sublayer includes a hybrid-scale feedforward network, which performs a feedforward transformation on the features output by the attention sublayer to obtain the output features of the locally enhanced Transformer structure.

[0009] As a further improvement to the above technical solution, the generation process of the direction-aware residual compensation feature includes: projecting the rain-affected image through a 3×3 convolution to a feature dimension that matches the backbone coding feature corresponding to the shallow coding level, to obtain the direction input feature; inputting the direction input feature into a vertical convolution path and a horizontal convolution path, respectively, wherein the vertical convolution path includes a 3×1 stripe depth convolution and the horizontal convolution path includes a 1×3 stripe depth convolution; performing point-by-point projection on the vertical direction feature output by the vertical convolution path and the horizontal direction feature output by the horizontal convolution path; concatenating the point-by-point projected vertical and horizontal features along the channel dimension, and generating direction gating weights based on the concatenation result; and performing weighted fusion on the point-by-point projected vertical and horizontal features based on the direction gating weights and their complementary weights to obtain the direction-aware residual compensation feature.

[0010] As a further improvement to the above technical solution, the 3×1 strip depth convolution is a vertically dilated depth convolution with an inflation rate of 2, and the 1×3 strip depth convolution is a horizontally dilated depth convolution with an inflation rate of 2; the orientation gating weights are obtained by sequentially performing global average pooling, 1×1 convolution, and sigmoid activation on the concatenated results of the vertical and horizontal features after point-by-point projection; the point-by-point projection layers corresponding to the vertical and horizontal features are both initialized with zero initialization.

[0011] As a further improvement to the above technical solution, the generation process of the frequency domain features and spatial domain features includes: The rain-degraded image is subjected to 3×3 convolution to extract features, and the extracted features are layer normalized to obtain frequency domain-spatial domain input features; the frequency domain-spatial domain input features are then input into the frequency domain processing branch and the multi-scale spatial domain branch, respectively. In the frequency domain processing branch, a two-dimensional fast Fourier transform is performed on the frequency domain-spatial domain input features to obtain complex frequency domain features, and the complex frequency domain features are decoupled into amplitude spectrum and phase spectrum; the amplitude spectrum is subjected to learnable modulation while keeping the phase spectrum unchanged; the modulated amplitude spectrum and the unchanged phase spectrum are recombined and then subjected to a two-dimensional inverse fast Fourier transform to obtain the frequency domain features. In the multi-scale spatial branch, the spatial features are extracted through spatial convolution paths with at least two different receptive fields.

[0012] As a further improvement to the above technical solution, the multi-scale spatial branch includes a first spatial convolution path and a second spatial convolution path; the first spatial convolution path performs 5×5 depth convolution and pointwise convolution on the frequency domain-spatial input features sequentially to obtain the first spatial feature; the second spatial convolution path performs 3×3 depth convolution with an inflation rate of 3 on the frequency domain-spatial input features to obtain the second spatial feature; the first spatial feature and the second spatial feature are concatenated along the channel dimension and fused by zero-initialization 1×1 convolution to obtain the spatial complementary feature; the frequency domain feature and the spatial complementary feature are respectively processed by independent two-layer 1×1 convolution channel attention to generate corresponding gating weights, and the frequency domain feature and the spatial complementary feature are weighted and summed based on the gating weights to obtain the frequency domain-spatial complementary feature; The convolutional projection layer that performs learnable modulation on the amplitude spectrum is initialized using a zero-initialization method.

[0013] As a further improvement to the above technical solution, the multi-scale coding adopts a four-level hierarchical downsampling structure, with the resolution of each level decreasing sequentially and the number of channels increasing exponentially. The frequency-spatial complementary feature obtained by gating the fusion of the frequency domain feature and the spatial complementary feature is injected into the backbone coding feature corresponding to the deep coding level. The bottleneck modeling is achieved through bottleneck processing consisting of a stack of locally enhanced Transformer structures; The hierarchical decoding adopts a three-level upsampling structure corresponding to the multi-scale encoding. Each level of upsampling achieves 2x spatial resolution recovery through sub-pixel convolution. The upsampled decoding features are then concatenated with the backbone encoding features of the corresponding encoding level in the channel dimension via skip connections. After dimensionality reduction and alignment through 1×1 convolution, the features are then input into the corresponding local enhancement Transformer structure for decoding. The decoded features, after undergoing multi-operator adaptive processing corresponding to shallow feature enhancement, are used to generate the residual prediction map by a 3×3 convolutional output layer.

[0014] The present invention also provides a direction-aware single-image deraining system, comprising: The image acquisition module is used to acquire rain-degraded images; The feature processing module is used to perform convolutional feature embedding and shallow feature enhancement on the rain-degraded image to obtain shallow enhanced features; The backbone coding module is used to encode the shallow enhancement features at multiple scales using a local enhancement Transformer structure to obtain the backbone coding features. The residual compensation module is used to generate direction-aware residual compensation features based on the rain-degraded image at the shallow coding level, and inject them into the corresponding backbone coding features at the shallow coding level in a residual manner; the direction-aware residual compensation features are obtained by gating and fusing the directional features extracted from at least two directional convolutional paths in different directions respectively. The spatial-frequency feature processing module is used to generate frequency domain features and spatial domain features based on the rain-affected image at the deep coding level, and to inject the frequency domain features and spatial domain features into the corresponding backbone coding features at the deep coding level in a residual manner after gated fusion of the two features. The frequency domain features are obtained by performing Fourier transform on the deep features obtained based on the rain-affected image to obtain the amplitude spectrum and phase spectrum, modulating the amplitude spectrum while preserving the phase spectrum, and then performing inverse Fourier transform. The spatial domain features are obtained by performing multi-scale spatial convolution on the deep features. The decoding output module is used to perform bottleneck modeling and hierarchical decoding on the injected backbone coding features to obtain decoding features, generate a residual prediction map based on the decoding features, and add the residual prediction map to the rain-affected image pixel by pixel to obtain the rain-free image.

[0015] Compared with existing technologies, the single-image deraining method and system provided by this invention have the following advantages: By employing sparse attention and local enhancement of LSTB, directional residual compensation of DARC, and spatial-frequency complementary refinement of FSCR, this invention can simultaneously suppress rain streak noise diffusion, enhance rain line direction modeling, and maintain phase structure information. While removing multi-scale rain lines, it preserves background texture and edge details, improving rain removal integrity, restoration clarity, and generalization ability in complex rainy scenes. Attached Figure Description

[0016] Figure 1 This is a flowchart of the single-image rain removal method provided in the embodiments of the present invention; Figure 2 This is a functional block diagram of the single-image deraining system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the overall DFNet network structure and the LSTB, DARC, and FSCR module structures provided in this embodiment of the invention. Figure 4 The images provided in this embodiment of the invention are visual comparisons of the effects of rain on a real rainy day. Detailed Implementation

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described below are only for illustration and explanation of the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0018] This embodiment provides a direction-aware single-image deraining method and system. The method takes a rain-degraded image as the processing object and generates a residual prediction map by means of convolutional feature embedding, shallow feature enhancement, local enhancement Transformer multi-scale coding, direction-aware residual compensation, frequency domain-spatial domain complementary refinement, bottleneck modeling and hierarchical decoding. The residual prediction map is then added pixel by pixel to the rain-degraded image to obtain the derained image.

[0019] like Figure 1 As shown, the method includes processing steps S1 to S6; as Figure 2 As shown, the system corresponding to this method includes an image acquisition module, a feature processing module, a backbone coding module, a residual compensation module, a spatial frequency feature processing module, and a decoding output module. The following is combined with... Figures 1 to 4 The collaborative relationships between each step and module are explained.

[0020] Specifically, such as Figure 1 As shown, the method includes: S1. Acquiring a rain-degraded image and performing convolutional feature embedding and shallow feature enhancement to form shallow enhanced features; S2. Encoding the shallow enhanced features at multiple scales using a local enhancement Transformer structure to form backbone coding features; S3. Generating residual compensation features in the shallow coding level and injecting them into the corresponding backbone coding features in a residual manner; S4. Generating frequency domain features and spatial domain features at the deep coding level, gating and fusing them, and then injecting them into the deep backbone coding features in a residual manner; S5. Performing bottleneck modeling and hierarchical decoding on the backbone coding features that have been injected a priori to obtain decoded features; S6. Generating a residual prediction map based on the decoded features and adding it pixel-by-pixel to the rain-degraded image to output a rain-removed image.

[0021] Combination Figure 3 The overall implementation flow of DFNet shown is as follows: the rain-degraded image first undergoes shallow feature embedding and shallow enhancement to obtain the backbone input; then it enters a four-level encoder, in which the first coding level has a parallel DARC directional compensation branch and the third coding level has a parallel FSCR spatial frequency complementarity branch; after the bottleneck layer completes the low-resolution global modeling, the decoder restores the resolution step by step through pixel shuffling upsampling and skip connections, and outputs the residual prediction map at the end. Figure 3 The LSTB, DARC, and FSCR modules correspond to the key processing steps in S2, S3, and S4, respectively, while the decoding and output paths correspond to S5 and S6.

[0022] Example 1 S1. Image Input and Shallow Feature Extraction: like Figure 1 As shown in S1, firstly, a rain-degraded image is acquired, and then... Figure 2The image acquisition module sends the image to the feature processing module. The feature processing module performs a 3×3 overlapping convolution projection on the rain-degraded image, mapping the original RGB three-channel image to a high-dimensional feature space to obtain high-dimensional initial features. Subsequently, the high-dimensional initial features are input into a shallow enhancement sub-network based on a multi-operator adaptive search mechanism for preprocessing to obtain shallow enhancement features.

[0023] The shallow enhancement subnetwork comprises alternating stacked operation-aware layers and grouping operator layers: the operation-aware layers perform global average pooling on the input features and generate an operator selection weight vector through a fully connected layer; the grouping operator layers perform a weighted summation of the responses of multiple candidate feature extraction operators based on the operator selection weight vector. Thus, the shallow stage can adaptively select the feature extraction path based on rain pattern density, rain pattern distribution, and background texture complexity.

[0024] In one specific implementation, 3×3 overlapping convolutional projection preserves continuous spatial context through overlapping sampling between adjacent receptive regions, reducing the risk of feature breakage at block boundaries compared to non-overlapping block embedding. Each dimension in the operator selection weight vector corresponds to the relative contribution of a candidate feature extraction operator. During forward inference, the grouped operator layer fuses each candidate response according to this weight, thereby forming an implicit operator search result for the current rain pattern morphology.

[0025] DFNet consists of three parts: shallow feature embedding, a hierarchical encoder-decoder backbone, and two parallel degradation refinement branches, using LSTB as the basic building block. Given a rain-degraded image... DFNet first extracts shallow features through 3×3 overlapping block embedding. The data is then processed by a shallow semantic enhancement sub-network based on a multi-operator adaptive search mechanism. The backbone network adopts a symmetrical U-shaped structure, receiving corresponding encoder features through skip connections and progressively restoring spatial resolution through pixel-shuffle-based upsampling. After the shallowest decoding layer, the shallow enhancement sub-network is applied again for final refinement. Finally, the network outputs a residual map through a 3×3 convolution. The rain removal results were obtained using a global residual learning approach. In equation (1), I represents the input rainy day image, and R represents the residual image predicted by the network. This represents the output derained image.

[0026] The aforementioned shallow enhancement features serve as input for subsequent multi-scale coding. To leverage the two types of degradation priors—the anisotropy of rain lines in spatial direction and the periodic energy accumulation in the frequency domain—DFNet sets up two parallel degradation refinement branches connected to the backbone at key encoder levels: DARC is deployed at the first level, focusing on shallow directional texture; FSCR is deployed at the third level, focusing on deep spectral structure; and feature processing at the second encoder level is completed by the backbone LSTB to ensure a smooth transition between semantic levels.

[0027] S2. A local enhancement Transformer structure is used to encode the shallow enhancement features at multiple scales to obtain the backbone encoded features: like Figure 1 S2 and Figure 3 As shown in the backbone network, the backbone coding module uses a Local Enhancement Transformer structure to perform four-level hierarchical downsampling encoding on the shallow enhancement features. The resolution of each level decreases sequentially while the number of channels expands exponentially, thus forming backbone coding features from shallow texture to deep structure at different semantic levels. The feature transformation within each coding level is completed by the Local Enhancement Transformer block (LSTB).

[0028] Combination Figure 3 The encoding backbone in the code allows for 2x spatial resolution compression and channel expansion between encoding levels via downsampling modules, gradually transforming shallow texture information into higher-level semantic representations. At the decoding end, symmetrical upsampling restores the spatial scale. This four-level encoding / three-level decoding configuration... Figure 1 S2 and S5 are connected.

[0029] LSTB consists of an attention sublayer and a feedforward sublayer. The attention sublayer performs Top-K sparse attention processing, generating multiple sets of sparse attention masks according to various increasing Top-K ratios. After setting the low-phase key value responses to negative infinity, sparse Softmax normalization is performed, and the multiple sparse attention outputs are fused with learnable scalar coefficients. At the same time, the attention sublayer is connected in parallel to a local convolution enhancement path composed of depthwise convolution and pointwise convolution to supplement the local inductive bias.

[0030] LSTB adopts the classic Transformer's two-stage structure of attention and feedforward. First, a lightweight adaptive gating network... Generate two channel gating vectors at once and These are used to modulate the outputs of the attention branch and the feedforward branch, respectively. In equation (2), Indicates the current LSTB layer index. Indicates the first Layer output features, This represents a lightweight adaptive gating network. and These represent the channel gating vectors for the attention branch and the feedforward branch, respectively.

[0031] To enhance the perception of local details, this embodiment adds a lightweight local convolutional branch in parallel to the output of TKSA, and its output is processed by a learnable scalar weight. The modulated result is added to the TKSA output; the fused result is then passed through a gated vector. Perform content-adaptive modulation at the channel level and obtain the attention segment output through residual connections: In equation (3), This represents the output feature of the attention segment in LSTB. For layer normalization, This indicates Top-K sparse attention. Indicates a local convolution branch. Learnable scalar weights for the output of local convolutional branches. To multiply by channel, This represents the gating strength coefficient. Specifically, TKSA generates a sparse attention map through a multi-proportional Top-K sparsity strategy, ensuring that each query token interacts only with the most relevant key-value pairs, thereby suppressing interference from irrelevant features. The feedforward branch employs a hybrid-scale feedforward network (MSFN), applying multi-scale local spatial transformations to the features through two parallel deep convolutional branches (3×3 and 5×5); its output is then passed through a gating vector. After modulation, the final LSTB output is obtained through residual connection: In equation (4), Indicates the first The output characteristics of each LSTB block This represents a hybrid-scale feedforward network. represents the gating strength coefficient of the feedforward segment. LSTB uses sparse global attention for long-range dependency modeling, local convolution and mixed-scale feedforward for fine-grained texture modeling, and dual-path adaptive gating to achieve dynamic balance driven by content.

[0032] In specific implementation, the lightweight adaptive gating network can first perform global average pooling on the current feature map, and then generate channel-wise gating weights through two layers of 1×1 convolution and Sigmoid activation; the two gating vectors are applied to the attention output and feedforward output in a soft multiplication manner, respectively, so that LSTB can adaptively enhance the channel response related to rain removal under different input contents.

[0033] Through the above processing, the backbone coding features output by the backbone coding module at each level contain both long-range dependency information obtained from sparse global attention and fine-grained texture information obtained from local convolution and hybrid scale feedforward networks, providing a feature basis that matches each coding level for subsequent injection of directional priors and spatial frequency priors.

[0034] S3. At the shallow coding level, direction-aware residual compensation features are generated based on the rain-degraded image, and injected into the corresponding backbone coding features at the shallow coding level in a residual manner: like Figure 1 S3 and Figure 3 As shown in the DARC module, at the shallow coding level, the residual compensation module generates direction-aware residual compensation features based on the rain-affected image. These features are obtained by gating and fusing the directional features extracted from the vertical and horizontal convolutional paths, and are injected as residuals into the corresponding backbone coding features at the shallow coding level.

[0035] DARC receives the features of a rain-degraded image, which is then projected onto the C channel via a 3×3 convolution layer. After layer normalization, the following is obtained Then, sensing is performed along both the vertical and horizontal directions: In equation (5), X represents the input feature of DARC. This indicates its characteristics after layer normalization. and These represent vertical and horizontal features, respectively. , These represent the 1×1 point-by-point projections in the corresponding directions. This represents a depthwise convolution with a 3×1 kernel and a vertical dilation rate of 2. This represents a depthwise convolution with a 1×3 kernel and a horizontal dilation rate of 2. This dilated asymmetric strip convolution significantly expands the effective receptive field along the strip direction, better matching the physical characteristics of the long-range linear structure of rain lines; on the other hand, it strictly suppresses the mixing of cross-directional context, avoiding the directional ambiguity caused by isotropic convolution. and This involves two 1×1 pointwise convolutions. To achieve adaptive fusion of features from both directions, DARC generates directional gating vectors using lightweight bottleneck gating units: In equation (6), g represents the direction gating vector. This represents the Sigmoid activation function. The mapping weights of the direction-gated units are represented by GAP, which indicates global average pooling. Indicates channel-dimensional splicing. This represents a mapping operation.

[0036] Using gate vectors Its complementary quantity DARC dynamically balances the contributions of features in both directions using a complementary weighting method, and obtains the module output through residual connections. In equation (7), This represents the direction-aware residual compensation feature output by the DARC module, where 1-g represents the complementary weights of the direction gating. This indicates element-wise multiplication by channel. Under this complementary gating mechanism, DARC can adaptively emphasize a dominant direction based on the input content, achieving fine-grained scheduling of direction priors at the channel level. This embodiment uses two 1×1 projection layers. and All weights adopt a zero-initialization strategy, so that DARC is equivalent to a strict identity mapping in the early stage of training, and the directional prior features are stably connected to the backbone as the overall network optimization process progresses.

[0037] The directional gating weights can be obtained by concatenating the vertical and horizontal features after point-by-point projection along the channel dimension, and then sequentially performing global average pooling, 1×1 convolution, and Sigmoid activation. The directional gating weights and their complementary weights are then applied to the two directional features to form a directional adaptive weighted fusion result.

[0038] During the injection process, the direction-aware residual compensation features output by DARC maintain a match with the backbone coding features corresponding to the shallow coding level in terms of spatial size and channel dimension. The residuals are added together and incorporated into the backbone feature stream. Since the two pointwise projection layers are initialized with zero initialization, DARC does not disrupt the backbone coding distribution in the early stages of training. As the network is optimized, its ability to perceive the direction of vertical, horizontal, and tilted rain line textures is gradually enhanced.

[0039] S4. Deep coding level FSCR space-frequency complementarity refinement: like Figure 1 S4 and Figure 3 As shown in the FSCR module, at the deep coding level, the spatial frequency feature processing module generates frequency domain features and spatial domain features based on the rain-degraded image, and then injects the two into the corresponding backbone coding features of the deep coding level in a residual manner after gated fusion.

[0040] The FSCR receives rain-degraded images, downsamples them through three convolutional layers, and projects them onto a deep feature X with a spatial size of H / 4×W / 4 in the 4C channel. After layer normalization, X is obtained and then enters two branches in parallel.

[0041] exist Figure 3In the FSCR branch shown, the rain-degraded image is first processed by 3×3 convolution to extract frequency-spatial input features and then normalized. In order to match the backbone features of the third coding level, the spatial size and channel dimension can be aligned by continuous convolution or downsampling. Then, the images are sent to the frequency domain processing branch and the multi-scale spatial domain branch respectively.

[0042] Frequency domain branch. Using the two-dimensional real-number fast Fourier transform. Will Mapped to the complex frequency domain, and the complex representation decoupled to amplitude |F| and phase ∠F: In equation (8), the Fourier operator This represents the two-dimensional real-number Fast Fourier Transform, where F represents the complex frequency domain characteristics, |F| is the amplitude spectrum, and ∠F is the phase spectrum. It is a function that takes a complex phase.

[0043] Unlike frequency domain filtering based on a fixed mask, this embodiment directly introduces a 1×1 convolutional projection layer in the amplitude domain. The amplitude is adaptively modulated while the phase remains unchanged to maintain spatial structure consistency, followed by inverse real Fourier transform. Mapping the modulated complex representation back to the spatial domain yields the refined features in the frequency domain. : In equation (9), and These are the amplitude spectrum and phase spectrum obtained by Fourier transforming the input features, respectively. express The amplitude spectrum obtained after learnable modulation. This represents a 1×1 convolutional projection layer for frequency domain amplitude modulation. This represents the two-dimensional inverse real fast Fourier transform. It is an exponential function, where j is the imaginary unit. This represents the refined features in the frequency domain.

[0044] The amplitude-phase decoupling modulation strategy has a clear physical meaning: rain streak degradation is mainly manifested as the energy concentration of high-frequency components in a specific direction, while the spatial structure information of the image content is more encoded by the phase. By performing learnable modulation only on the amplitude and preserving the original phase, the energy of the frequency components related to rain streaks can be specifically weakened, and spatial structure misalignment can be avoided after the inverse transform, reducing the risk of ringing and edge distortion caused by pure frequency domain filtering.

[0045] Spatial Domain Branch. To compensate for potential loss of local detail due to frequency domain modulation, the spatial domain branch consists of two parallel complementary sub-paths: the large kernel branch uses 5×5 depthwise convolutions to capture dense local textures, while the hole branch uses 3×3 depthwise convolutions with a dilation rate of 3 to expand the receptive field and model larger-scale structural contexts. Both sub-paths are mapped to 1×1 convolutional channels and activated by GELU, then concatenated along the channel dimension and fused through a 1×1 fusion layer to obtain complementary spatial features. In equations (10) to (12), Indicates local features of the large core airspace. Indicates the contextual features of the void airspace. Indicates spatial complementarity; , These represent the 1×1 channel mappings of the two spatial sub-paths, and These represent a 5×5 depthwise convolution and a 3×3 depthwise convolution with an inflation rate of 3, respectively. This represents the GELU activation function. For 1×1 fused convolution, This indicates channel-level splicing. Gated fusion and residual injection are performed. To achieve synergy between frequency and spatial domain features, FSCR introduces dual independent channel attention gating, namely... and Generate their respective channel weights and The module output is obtained through residual connection: In equations (13) and (14), and These represent two-layer 1×1 convolutional channel attention units for the frequency domain branch and the spatial domain branch, respectively. and These represent the corresponding channel gating weights. This indicates the frequency-spatial complementary characteristics of the FSCR module output.

[0046] Amplitude modulation layer in frequency domain branch Fusion layer in the airspace branch All weights adopt a zero-initialization strategy to ensure that FSCR is equivalent to an identity mapping in the early stage of training, and frequency domain priors and spatial domain multi-scale structural features are stably connected to the backbone as the overall network optimization process progresses.

[0047] Among them, the gating weights of frequency domain features and spatial complementary features can be generated by independent two-layer 1×1 convolutional channel attention units, so that the frequency domain amplitude modulation results and multi-scale spatial structure results can be adaptively allocated fusion contributions according to the input content.

[0048] The frequency-spatial complementary features obtained through the above frequency domain branching and multi-scale spatial domain branching are injected into the backbone coding features corresponding to the third coding level. This deep injection, combined with the shallow directional residual compensation in S3, enables the backbone coding features to simultaneously carry directional rain line texture information, frequency domain amplitude modulation information, and multi-scale spatial structure information.

[0049] S5. Bottleneck Modeling and Hierarchical Decoding: like Figure 1 As shown in S5, bottleneck modeling and hierarchical decoding are performed on the backbone encoded features after DARC and FSCR injection to obtain the decoded features. Bottleneck modeling is achieved through bottleneck processing consisting of multiple LSTB stacks, which covers the global spatial range with low computational cost at a lower spatial resolution, capturing long-distance semantic associations between large areas of rain-covered regions and distant clear background structures.

[0050] In one implementation, after three consecutive downsampling steps, the spatial resolution of the bottleneck input feature can be compressed to 1 / 8 of the shallow input feature, and the number of channels is correspondingly expanded to a higher dimension, such as 8C; stacking LSTB on this low-resolution feature can cover a larger spatial range with lower computational cost.

[0051] Hierarchical decoding employs a three-level upsampling structure corresponding to the four-level multi-scale coding. Each upsampling level achieves a 2x spatial resolution recovery through sub-pixel convolution. The upsampled decoded features are then concatenated with the corresponding coding level's backbone coding features via skip connections along the channel dimension, followed by dimensionality reduction alignment using a 1×1 convolution before being input into the corresponding LSTB decoder module for decoding. Through this structure, shallow local texture information and deep global semantic information are synchronously fused during the progressive resolution recovery process.

[0052] Each subpixel upsampling level can specifically include: first, adjusting or expanding the number of channels using 3×3 convolution; then, rearranging the channel dimensions into spatial dimensions through pixel shuffling to achieve 2x resolution restoration; the upsampled decoded features are concatenated with the backbone coding features of the corresponding coding level, and then aligned using 1×1 convolution for dimensionality reduction, ensuring... Figure 3 Skip connections in the code will not cause channel dimension mismatch.

[0053] S6. Refining Post-Processing and Global Residual Output: like Figure 1As shown in S6, the decoding output module generates a residual prediction map based on the decoding features, and adds the residual prediction map to the rain-degraded image pixel by pixel to obtain the de-rained image. Specifically, the output feature map of the final stage of the decoder is again fed into the multi-operator adaptive processing process corresponding to the shallow feature enhancement, to perform secondary repair and enhancement on the fine rain artifacts and edge blurs that may still remain after the main network processing.

[0054] After refinement, a 3×3 convolutional output layer maps the high-dimensional feature map into a 3-channel residual prediction map of the same size as the input image. Finally, this residual prediction map is added pixel-by-pixel to the original rainy input image via global residual connections to obtain the rain-restored result. Global residual learning transforms the network optimization objective from directly regressing the absolute pixel values ​​of the clear image to predicting the residual components between the rainy image and the clear background image, which helps reduce the difficulty of end-to-end optimization and improve the quantization accuracy of the restored result.

[0055] Combining S1 to S6, this embodiment forms an end-to-end processing closed loop: convolutional feature embedding and shallow enhancement—LSTB multi-scale coding—DARC shallow directional residual compensation—FSCR deep spatial-frequency complementary refinement—bottleneck modeling and hierarchical decoding—refined residual output. This processing closed loop is strictly constructed around the data flow between the rain-degraded image, backbone coding features, direction-aware residual compensation features, frequency domain features, spatial domain features, decoding features, and residual prediction map. Each branch is connected to the backbone network in a residual manner to avoid disrupting the backbone feature distribution.

[0056] Furthermore, the end-to-end processing closed loop of this embodiment has the capability of automated real-time image restoration. After inputting a rain-degraded image, there is no need to manually select the rain streak region or manually configure the filtering parameters. The restoration result can be obtained through a single network forward propagation. This is suitable for scenarios that require continuous processing of rainy day images, such as autonomous driving perception, intelligent traffic monitoring, and drone inspection.

[0057] From the perspective of directional degradation modeling, DARC explicitly decouples linear rain patterns in different directions through vertically and horizontally dilated depth convolutions, and then uses channel gating to fuse the two directional features, making the shallow representation more sensitive to vertical, horizontal and oblique rain lines while preserving the background edges, thus reducing the probability of missed detection and undersuppression of shallow rain patterns.

[0058] From the perspective of frequency domain and spatial domain collaboration, FSCR uses amplitude modulation and phase preservation in frequency domain processing to suppress the periodic energy of rain streaks. At the same time, it uses 5×5 large kernel depth convolution and 3×3 depth convolution with an expansion rate of 3 to supplement local texture and larger-scale structural context. After the two types of features are fused through independent channel gating, the risk of edge ringing and detail blurring can be reduced while reducing residual rain streaks.

[0059] From the perspective of backbone feature modeling, LSTB uses Top-K sparse attention to reduce noise propagation between irrelevant regions, enhances the path with local convolution and supplements the convolution induction bias with a hybrid scale feedforward network, and adjusts the output intensity of the attention sublayer and the feedforward sublayer through channel gating, thereby taking into account both global rain pattern distribution modeling and high-frequency background detail restoration.

[0060] Therefore, in engineering implementation, the proposed method can perform multi-dimensional modeling around the real degradation characteristics such as variable rain line direction, large scale differences, and obvious frequency domain energy accumulation. This not only improves the integrity of rain removal but also maintains the fidelity of the background structure, providing clear input for subsequent tasks such as detection, segmentation, and visual navigation.

[0061] Example 2 like Figure 2 As shown, this embodiment also provides a direction-aware single-image rain removal system. An image acquisition module is used to acquire rain-degraded images; a feature processing module is used to perform convolutional feature embedding and shallow feature enhancement on the rain-degraded images to obtain shallow enhanced features; a backbone coding module is used to perform multi-scale coding on the shallow enhanced features using a local enhancement Transformer structure to obtain backbone coded features.

[0062] The residual compensation module generates direction-aware residual compensation features based on the rain-affected image at the shallow coding level, and injects them into the corresponding backbone coding features at the shallow coding level in a residual manner. The direction-aware residual compensation features are obtained by gating and fusing directional features extracted from at least two directed convolutional paths in different directions. The spatial-frequency feature processing module generates frequency domain features and spatial domain features based on the rain-affected image at the deep coding level, and injects them into the corresponding backbone coding features at the deep coding level in a residual manner after gating and fusing the frequency domain features and spatial domain features.

[0063] The decoding output module is used to perform bottleneck modeling and hierarchical decoding on the injected backbone coding features to obtain decoded features; it is also used to generate a residual prediction map based on the decoded features, and add the residual prediction map to the rain-affected image pixel by pixel to obtain a rain-free image. Thus, Figure 2 The various functional modules and Figure 1 The methods and steps are interconnected, enabling the restoration of rainy images within the same end-to-end network.

[0064] In system implementation, the above modules can be deployed as functional branches within the same neural network in the processor or image processing acceleration unit: the image acquisition module corresponds to the input stage of S1, the feature processing module corresponds to the convolutional embedding and shallow enhancement of S1, the backbone encoding module corresponds to S2, the residual compensation module and the spatial frequency feature processing module correspond to S3 and S4 respectively, and the decoding output module corresponds to S5 and S6. The feature tensors passed between modules are aligned in size and channels before injection to ensure the effective execution of residual addition and skip connections.

[0065] Example 3 During the training phase, this embodiment performs end-to-end optimization of DFNet within the PyTorch deep learning framework and employs a peak signal-to-noise ratio (PSNR)-guided loss function to constrain network parameters. During the inference phase, the network completes rain image restoration through a single forward propagation, outputting a rain-free image with the same size as the input image. To illustrate the effectiveness of this embodiment, the following references Tables 1 and 2. Figure 4 Please provide an explanation.

[0066] During training, paired samples consisting of rainy images and corresponding clear images can be used to optimize the network parameters end-to-end. The loss function can directly constrain the pixel error or PSNR-related target between the rain-removed image and the clear image output by the network. During inference, the parameters obtained during training are fixed, and a single forward calculation is performed on a rainy image to output the rain-removed image.

[0067] Table 1 compares the quantitative performance of the present invention with existing mainstream rain removal methods on four public benchmark datasets (Rain200L, Rain200H, DID-Data, and DDN-Data). The evaluation metrics are peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). Bold values ​​in the table represent the best results on each dataset, and underlined values ​​represent the second-best results.

[0068] Table 1. Comparison of quantitative performance between the present invention and existing methods (PSNR / SSIM) As shown in Table 1, the method proposed in this invention achieves the best performance on the Rain200H, DID-Data and DDN-Data datasets, and the second best performance on the Rain200L dataset. Its overall performance is better than the existing mainstream methods, including DRSformer, Restormer and IDT, which verifies the effectiveness of the collaborative design of the innovative modules proposed in this invention.

[0069] Table 2 compares the computational complexity of this invention with that of three representative methods, SPD-Net, Restormer, and DRSformer, in terms of model parameter count (params, in M), floating-point operation count (FLOPs, in G), and single-image inference latency (Latency, in s).

[0070] Table 2 Comparison of computational complexity between the present invention and the comparative method As shown in Table 2, the method proposed in this invention has 37.45M parameters, 116.01G FLOPs, and an inference latency of 4.6499s. In terms of parameter scale and computational overhead, it is similar to the DRSformer baseline method, indicating that while achieving performance improvement, the additional computational cost introduced by this invention is controlled within a reasonable range, and it has the feasibility of practical deployment and application.

[0071] like Figure 4 As shown, the present invention, DRSformer, and SPD-Net were visually compared in a real rainy scene to demonstrate their rain removal effects. The four columns in the figure, from left to right, represent the rainy image, the restoration result of the DRSformer method, the restoration result of the SPD-Net method, and the restoration result of the present invention's method (corresponding to the column labeled "The Method of the Present Invention" in the figure). Figure 4 As can be seen, in the first set of distant forest scenes, both DRSformer and SPD-Net exhibit varying degrees of rain line residue and blurred background details. The method of this invention, however, more thoroughly eliminates dense vertical rain lines while preserving the outline layers of distant trees. In the second set of close-up vegetation scenes, DRSformer shows slight oversmoothing in leaf texture areas, and SPD-Net does not completely remove some slanted rain lines. The method of this invention performs better in preserving leaf edges and branch details. In the third set of dense vegetation texture scenes, the three methods perform similarly overall, but the method of this invention is slightly better than the comparison methods in restoring high-frequency textures of small leaves. In the fourth set of building scenes, DRSformer exhibits slight blurring in the roof edge area, and SPD-Net is slightly insufficient in preserving building wall textures. The method of this invention can more clearly restore building outlines and wall brick texture details. In summary, the method proposed in this invention outperforms DRSformer and SPD-Net in both the completeness of rain line removal and high-fidelity restoration of background structure, further verifying the effectiveness and superiority of the technical solution of this invention.

Claims

1. A direction-aware single-image rain removal method, characterized in that, include: Obtain a rain-degraded image, and perform convolutional feature embedding and shallow feature enhancement on the rain-degraded image to obtain shallow enhanced features; The shallow enhancement features are encoded at multiple scales using a local enhancement Transformer structure to obtain the backbone encoded features. At the shallow coding level, orientation-aware residual compensation features are generated based on the rain-degraded image. The orientation-aware residual compensation features are obtained by gating and fusing the orientation features extracted by at least two different directional convolutional paths, and are injected into the backbone coding features corresponding to the shallow coding level in a residual manner. At the deep coding level, frequency domain features and spatial domain features are generated based on the rain-degraded image; Fourier transform is performed on the deep features obtained based on the rain-degraded image to obtain the amplitude spectrum and phase spectrum. The amplitude spectrum is modulated while the phase spectrum is preserved, and then the frequency domain features are obtained by inverse Fourier transform. The deep features are obtained by performing multi-scale spatial convolution on the deep features; the frequency domain features and spatial features are then gated and fused and injected into the backbone coding features corresponding to the deep coding level in a residual manner. Bottleneck modeling and hierarchical decoding are performed on the injected backbone coding features to obtain the decoded features; A residual prediction map is generated based on the decoding features. The residual prediction map is then added pixel by pixel to the rain-degraded image to obtain the rain-free image.

2. The direction-aware single-image rain removal method according to claim 1, characterized in that, The shallow enhancement feature generation process includes: performing a 3×3 overlapping convolution projection on the rain-affected image to map the original RGB three-channel image to a high-dimensional feature space, obtaining high-dimensional initial features; inputting the high-dimensional initial features into a shallow enhancement sub-network based on a multi-operator adaptive search mechanism for preprocessing, the shallow enhancement sub-network including alternating stacked operation-aware layers and grouping operator layers; performing global average pooling on the input features through the operation-aware layer, and mapping to generate an operator selection weight vector through a fully connected layer; and the grouping operator layer performing a weighted summation of the responses of multiple candidate feature extraction operators according to the operator selection weight vector to obtain the shallow enhancement features.

3. The direction-aware single-image rain removal method according to claim 1, characterized in that, The locally enhanced Transformer structure includes an attention sublayer and a feedforward sublayer; The attention sublayer performs Top-K sparse attention processing on the input features, generates multiple sets of sparse attention masks according to various increasing Top-K ratios, sets the low phase key value response to negative infinity and then performs sparse Softmax normalization, and then fuses the multiple sparse attention outputs with learnable scalar coefficients for weighted fusion. The attention sublayer is also connected in parallel to a local convolution enhancement path composed of a series of depthwise convolutions and pointwise convolutions, and the output of the local convolution enhancement path is superimposed on the multi-path sparse attention output. The attention sub-layer and the feedforward sub-layer perform global average pooling on the current feature map, generate channel-wise gated weights through two layers of 1×1 convolution and Sigmoid activation, and perform channel-level modulation on the output of the corresponding sub-layer in the form of soft multiplication; The feedforward sublayer includes a hybrid-scale feedforward network, which performs a feedforward transformation on the features output by the attention sublayer to obtain the output features of the locally enhanced Transformer structure.

4. The direction-aware single-image rain removal method according to claim 1, characterized in that, The generation process of the direction-aware residual compensation feature includes: projecting the rain-affected image through a 3×3 convolution to a feature dimension that matches the backbone coding feature corresponding to the shallow coding level, to obtain the direction input feature; inputting the direction input feature into a vertical convolution path and a horizontal convolution path, respectively, wherein the vertical convolution path includes a 3×1 stripe depth convolution and the horizontal convolution path includes a 1×3 stripe depth convolution; performing point-by-point projection on the vertical direction feature output by the vertical convolution path and the horizontal direction feature output by the horizontal convolution path; concatenating the point-by-point projected vertical and horizontal features along the channel dimension, and generating direction gating weights based on the concatenation result; and performing weighted fusion on the point-by-point projected vertical and horizontal features based on the direction gating weights and their complementary weights to obtain the direction-aware residual compensation feature.

5. The direction-aware single-image rain removal method according to claim 4, characterized in that, The 3×1 strip depthwise convolution is a vertically dilated depthwise convolution with an inflation rate of 2, and the 1×3 strip depthwise convolution is a horizontally dilated depthwise convolution with an inflation rate of 2. The orientation gating weights are obtained by sequentially performing global average pooling, 1×1 convolution, and sigmoid activation on the concatenated results of the vertical and horizontal features after point-by-point projection. The point-by-point projection layers corresponding to the vertical and horizontal features are initialized with zero initialization.

6. The direction-aware single-image rain removal method according to claim 1, characterized in that, The generation process of the frequency domain features and spatial domain features includes: The rain-degraded image is subjected to 3×3 convolution to extract features, and the extracted features are layer normalized to obtain frequency domain-spatial domain input features; the frequency domain-spatial domain input features are then input into the frequency domain processing branch and the multi-scale spatial domain branch, respectively. In the frequency domain processing branch, a two-dimensional fast Fourier transform is performed on the frequency domain-spatial domain input features to obtain complex frequency domain features, and the complex frequency domain features are decoupled into amplitude spectrum and phase spectrum; the amplitude spectrum is subjected to learnable modulation while keeping the phase spectrum unchanged; the modulated amplitude spectrum and the unchanged phase spectrum are recombined and then subjected to a two-dimensional inverse fast Fourier transform to obtain the frequency domain features. In the multi-scale spatial branch, the spatial features are extracted through spatial convolution paths with at least two different receptive fields.

7. The direction-aware single-image rain removal method according to claim 6, characterized in that, The multi-scale spatial branch includes a first spatial convolution path and a second spatial convolution path; the first spatial convolution path performs a 5×5 depthwise convolution and a pointwise convolution on the frequency-spatial input features to obtain a first spatial feature; the second spatial convolution path performs a 3×3 depthwise convolution with an inflation rate of 3 on the frequency-spatial input features to obtain a second spatial feature; the first and second spatial features are concatenated along the channel dimension and fused by a zero-initialized 1×1 convolution to obtain a spatial complementary feature; The frequency domain features and the spatial complementary features are processed by independent two-layer 1×1 convolutional channel attention to generate corresponding gating weights. The frequency domain features and the spatial complementary features are then weighted and summed based on the gating weights to obtain the frequency domain-spatial complementary features. The convolutional projection layer that performs learnable modulation on the amplitude spectrum is initialized using a zero-initialization method.

8. The direction-aware single-image rain removal method according to claim 7, characterized in that, The multi-scale coding adopts a four-level hierarchical downsampling structure, with the resolution decreasing sequentially at each level and the number of channels increasing exponentially. The frequency-spatial complementary feature obtained by gating the fusion of the frequency domain feature and the spatial complementary feature is injected into the backbone coding feature corresponding to the deep coding level. The bottleneck modeling is achieved through bottleneck processing consisting of a stack of locally enhanced Transformer structures; The hierarchical decoding adopts a three-level upsampling structure corresponding to the multi-scale encoding. Each level of upsampling achieves 2x spatial resolution recovery through sub-pixel convolution. The upsampled decoding features are then concatenated with the backbone encoding features of the corresponding encoding level in the channel dimension via skip connections. After dimensionality reduction and alignment through 1×1 convolution, the features are then input into the corresponding local enhancement Transformer structure for decoding. The decoded features, after undergoing multi-operator adaptive processing corresponding to shallow feature enhancement, are used to generate the residual prediction map by a 3×3 convolutional output layer.

9. A direction-aware single-image rain removal system, characterized in that, include: The image acquisition module is used to acquire rain-degraded images; The feature processing module is used to perform convolutional feature embedding and shallow feature enhancement on the rain-degraded image to obtain shallow enhanced features; The backbone coding module is used to encode the shallow enhancement features at multiple scales using a local enhancement Transformer structure to obtain the backbone coding features. The residual compensation module is used to generate direction-aware residual compensation features based on the rain-degraded image at the shallow coding level, and inject them into the corresponding backbone coding features at the shallow coding level in a residual manner; the direction-aware residual compensation features are obtained by gating and fusing the directional features extracted from at least two directional convolutional paths in different directions respectively. The spatial-frequency feature processing module is used to generate frequency domain features and spatial domain features based on the rain-affected image at the deep coding level, and to inject the frequency domain features and spatial domain features into the corresponding backbone coding features at the deep coding level in a residual manner after gated fusion of the two features. The frequency domain features are obtained by performing Fourier transform on the deep features obtained based on the rain-affected image to obtain the amplitude spectrum and phase spectrum, modulating the amplitude spectrum while preserving the phase spectrum, and then performing inverse Fourier transform. The spatial domain features are obtained by performing multi-scale spatial convolution on the deep features. The decoding output module is used to perform bottleneck modeling and hierarchical decoding on the injected backbone coding features to obtain decoding features, generate a residual prediction map based on the decoding features, and add the residual prediction map to the rain-affected image pixel by pixel to obtain the rain-free image.