Single image rain removal method and system based on bright feature selection Transform

Abstract

The invention provides a single image rain removal method and system based on bright feature selection Transform, and relates to the technical field of computer vision, and the method comprises the steps: carrying out the feature extraction of a rain-containing image to be recovered, and obtaining a depth feature map; performing n times of first iterative processing and at least two times of image restoration processing by taking the depth feature map as an input parameter to obtain a first restored feature map; the first iteration processing comprises image restoration processing and down-sampling; the image restoration processing comprises bright anchor point feature screening and local and global information fusion; performing n times of second iteration processing and image restoration processing by taking the first restored feature map as an input parameter to obtain a second restored feature map; the second iteration processing comprises image restoration processing and up-sampling; and carrying out convolution processing on the second recovery feature map and adding the second recovery feature map with the rain-containing image to be recovered to obtain a recovered image. According to the method, a high-quality rain-free image restoration effect is realized.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method and system for deraining a single image based on a brightness feature selection Transformer. Background Technology

[0002] With the widespread application of outdoor vision systems, ensuring image quality under adverse weather conditions such as rain and fog has become a key technical challenge. Rain streaks in images exhibit complex characteristics such as non-uniform distribution, strong randomness in direction, and high coupling with background details, posing significant difficulties for image deraining tasks.

[0003] Early image deraining methods primarily relied on traditional filters and hand-designed prior knowledge, which often proved limited in handling complex rain pattern patterns. In recent years, deep learning methods based on convolutional neural networks have significantly improved deraining performance, automatically extracting rain pattern features through end-to-end learning. However, the local receptive field of standard convolutional operations limits their ability to model long-range rain pattern dependencies in images, easily leading to problems such as rain pattern retention or overly smoothed backgrounds. Summary of the Invention

[0004] In view of this, this application proposes a single-image deraining method and system based on a brightness feature selection Transformer.

[0005] Firstly, this application provides a single-image rain removal method based on a brightness feature selection Transformer, including: Convolutional layers are used to extract features from the rain-affected image to be restored, resulting in a depth feature map; The depth feature map is used as input parameters, and n first iterations are performed. The features after the nth first iteration are then subjected to at least two image restoration processes to obtain a first restored feature map. The first iteration process includes: performing image restoration processing on the input parameters and downsampling the output features after image restoration processing. The image restoration processing includes: selecting bright anchor point features from the input parameters and fusing spatial and frequency domain information from local and global perspectives on the selected bright anchor point features to obtain fused features. The first restored feature map is used as the input parameter, and the second iteration is performed n times. The features after the nth second iteration are then subjected to image restoration processing to obtain the second restored feature map. The second iteration processing includes: performing image restoration processing on the input parameter and upsampling the output features after image restoration processing. A convolutional layer is used to process the second restored feature map to generate a residual feature map, and the residual feature map is added to the rain-containing image to be restored to obtain the repaired image.

[0006] In one embodiment, the step of filtering bright anchor point features from the input parameters and fusing spatial and frequency domain information from local and global perspectives to obtain fused features includes: Based on a bright feature selection self-attention mechanism, bright anchor point features containing rain line information are filtered out from the depth feature map; By using cross-domain attention, rain line features similar to the bright anchor point features are selected from the depth feature map. After feature restoration processing of the rain line features, residual connection is performed with the depth feature map to obtain a preliminary restored feature map. A dual-branch feedforward network is used to extract global features that retain global key frequency components and local features that retain local key frequency components from the preliminary recovered feature map. The global features and local features are then fused and residually connected to the preliminary recovered feature map to generate the fused features.

[0007] In one embodiment, the bright feature selection self-attention mechanism for filtering bright anchor point features containing rain line information from the depth feature map includes: The depth feature map is subjected to layer normalization, and the number of channels in the normalized depth feature map is increased by 1x1 convolution to obtain an enhanced feature map. Non-overlapping local max pooling operations are performed on the enhanced feature map to filter out the feature points with the highest brightness in each local region, thereby obtaining candidate bright features containing the main rain line information. The candidate bright features are linearly projected into query, key, and value matrices, and then feature filtering is performed using self-attention calculation to obtain the bright anchor point features.

[0008] In one embodiment, selecting rain line features similar to the bright anchor point features from the depth feature map through cross-domain attention includes: The depth feature map is normalized, and then a 1x1 convolutional layer is used to project the normalized depth feature map to generate a query matrix. The bright anchor point features are input into a 1x1 convolutional layer, and the feature map output by the 1x1 convolutional layer is segmented to generate a key matrix for comparison and a value matrix containing rain line information. Calculate the multi-head cross-attention between the query matrix and the key matrix to obtain an attention graph; Perform matrix multiplication on the attention map and the value matrix to obtain the rain line feature.

[0009] In one embodiment, the step of performing feature restoration processing on the rain line features and then performing residual connection with the depth feature map to obtain a preliminary restored feature map includes: The rain line features are channel-adjusted by 1x1 convolution, and the channel-adjusted rain line features are restored to their original spatial dimensions to obtain the restored rain line features. The restored rain line features are residually concatenated with the depth feature map to obtain a preliminary restored feature map.

[0010] In one embodiment, the dual-branch feedforward network includes a global branch and a local branch; the step of extracting global features retaining global key frequency components and local features retaining local key frequency components from the initially recovered feature map using the dual-branch feedforward network includes: The preliminary recovered feature map is subjected to layer normalization to obtain a layer normalized recovered feature map, and the layer normalized recovered feature map is input into the dual-branch feedforward network; In the global branch, the channel dimension of the restored feature map is expanded by point convolution processing and normalized by layer. The expanded features are then activated by an activation function. Discriminative frequency domain operations are performed on the activated features to obtain the global features. The discriminative frequency domain operations include: transforming the input features to the frequency domain and multiplying them with a learnable quantization matrix to adaptively preserve key frequency information, and then restoring the product features to the spatial domain through inverse transformation. In the local branch, spatial features are extracted from a portion of the input channels of the normalized feature map of the layer using partial convolution. The extracted features are then transformed using point convolution and activation functions. The transformed features are processed using depthwise separable convolution. Discriminative frequency domain operations are performed on the processed features to obtain the local features.

[0011] In one embodiment, the step of transforming the input features to the frequency domain and multiplying them with a learnable quantization matrix to adaptively preserve key frequency information, and then restoring the product features to the spatial domain through an inverse transformation, includes: The input features are divided into multiple non-overlapping feature blocks in the spatial dimension; Perform a Fast Fourier Transform on each of the feature blocks to transform the image features of each feature block from the spatial domain to the frequency domain, thereby obtaining the frequency domain feature map corresponding to each feature block; Each of the frequency domain feature maps is element-wise multiplied with a learnable quantization matrix to adaptively amplify the weights of key frequency components and suppress invalid frequency components related to rain stripe degradation, resulting in multiple weighted frequency domain feature maps. Perform inverse fast Fourier transform on each of the weighted frequency domain feature maps to obtain multiple filtered spatial feature blocks; Based on the arrangement order of the feature blocks, the spatial feature blocks are restored and recombined, and the spatial resolution of the restored and recombined feature map is restored to the original size.

[0012] Secondly, this application also provides a single-image deraining system based on brightness feature selection Transformer, including: a feature extraction module, an encoder module, a bottleneck module, a decoder module and an image reconstruction module; The feature extraction module is used to extract features from the rain-containing image to be restored using a convolutional layer to obtain a depth feature map; The encoder module includes n cascaded first iteration units and one image complex atom unit. The first iteration unit includes cascaded image complex atom units and a downsampling layer. The image complex atom unit is used to perform bright anchor point feature filtering on the input parameters, and to fuse spatial and frequency domain information from local and global perspectives on the filtered bright anchor point features to obtain fused features. The bottleneck module includes at least one image complex atom unit. The input of the bottleneck module is connected to the output of the encoder module, and the output of the bottleneck module is connected to the input of the decoder module. The decoder module includes n cascaded second iteration units and one image complex atom unit, wherein the second iteration unit includes cascaded image complex atom units and an upsampling layer; The image reconstruction module is used to process the features output by the decoder module using a convolutional layer to generate a residual feature map, and then add the residual feature map to the rain-containing image to be restored to obtain the repaired image.

[0013] In one embodiment, the encoder module is used to save the current output as a skip connection feature after the first iteration processing module of the i-th level completes feature extraction. The skip connection feature is used for the fusion of input features in the subsequent second iteration processing module corresponding to the (n-i+1)-th level. The second iteration processing module of the (n-i+1)-th level is used to fuse the input features in the second iteration processing module of the (n-i+1)-th level with the corresponding skip connection feature before performing image restoration processing.

[0014] Thirdly, this application also provides a method for rain removal from a single image, including: Acquire rain-soaked images to be recovered; The rain-laden image to be restored is input into the single-image deraining system based on brightness feature selection Transformer as described in the second aspect, and the image output by the single-image deraining system based on brightness feature selection Transformer is determined as the rain-free restored image corresponding to the rain-laden image to be restored.

[0015] The single-image rain removal method based on brightness feature selection Transformer proposed in this application has the following advantages over related techniques: 1. The single-image rain removal method of this application extracts features from the rain-containing image to be restored through convolutional layers to obtain a deep feature map, which provides a comprehensive initial feature foundation for rain pattern removal and image restoration. Then, n first-iteration processing is performed. In each iteration, rain pattern-related features are accurately locked through bright anchor point feature screening. Then, spatial and frequency domain information is fused from local and global perspectives to generate fused features, realizing accurate identification and preliminary processing of rain pattern features. Subsequent downsampling operations gradually refine deep semantic features, enhance the perception of complex rain pattern patterns, and at least two additional image restoration processes are performed after the nth first iteration to further optimize the rain pattern separation effect, resulting in a first restored feature map. Then, the first restored feature map is used as input for n second-iteration processing. Similarly, the image restoration processing of bright anchor point screening and spatial-frequency domain fusion is used to deepen rain pattern removal. Combined with upsampling operations, the spatial details of the image are gradually restored, realizing the synergistic connection between deep semantic features and spatial details, resulting in a second restored feature map. Finally, the second restored feature map is processed through convolutional layers to generate a residual feature map, which is added to the rain-containing image to be restored to complete the final restoration. Through multiple iterations and the combination of upsampling and downsampling, the integrity of the image structure and the richness of details are ensured, ultimately achieving high-quality rain streak removal and image restoration.

[0016] 2. By extracting features from the rain-containing image to be restored, a depth feature map is obtained, laying a comprehensive feature foundation for rain stripe separation and image restoration. Then, a bright feature selection self-attention mechanism is used to accurately filter out bright anchor point features containing rain line information, making full use of the distribution characteristics of rain stripes in the brightness space to achieve preliminary accurate localization of rain stripe features. Subsequently, a cross-domain attention mechanism is used to match rain line features similar to bright anchor point features from the depth feature map. After feature restoration, it is connected to the residual of the depth feature map, locking the rain stripe features while preserving the original image details to the greatest extent, resulting in a preliminary restored feature map. Finally, a dual-branch feedforward network is used to extract global features that retain global key frequency components and local features that retain local key frequency components from the preliminary restored feature map. By fusing global and local frequency domain features, the spatial domain and frequency domain are optimized synergistically. Then, it is connected to the residual of the preliminary restored feature map, ultimately achieving accurate and efficient removal of rain stripes from a single rain-containing image. This significantly improves the thoroughness of rain removal while perfectly preserving the image details, thus achieving a high-quality rain-free image restoration effect. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a single-image rain removal method based on brightness feature selection Transformer in one embodiment of this application; Figure 2 This is a flowchart illustrating a single-image rain removal method based on brightness feature selection of Transformer in another embodiment of this application. Figure 3 This is a flowchart illustrating a single-image rain removal method based on brightness feature selection Transformer in another embodiment of this application. Figure 4 This is a flowchart illustrating the filtering process for bright anchor point features and rain line features in one embodiment of this application. Figure 5 This is a schematic diagram of the architecture of a dual-branch feedforward network in one embodiment of this application; Figure 6 This is a schematic diagram of the processing flow of frequency domain discrimination operation in one embodiment of this application; Figure 7 This is a schematic diagram of the structure of a single-image rain removal system based on brightness feature selection Transformer in one embodiment of this application; Figure 8 This is a schematic diagram of the structure of a single-image deraining system based on brightness feature selection Transformer in another embodiment of this application; Figure 9 This is a flowchart illustrating a single-image rain removal method according to an embodiment of this application. Detailed Implementation

[0019] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] In some embodiments, this application provides a single-image rain removal method based on a brightness feature selection Transformer, such as... Figure 1 As shown, the single-image rain removal method includes the following steps S101 to S104.

[0021] S101: Convolutional layers are used to extract features from the rain-affected image to be restored, resulting in a depth feature map.

[0022] In this process, the rain-laden image to be restored can be input into an initial feature embedding layer to extract shallow features and map them to a deep feature map. The corresponding formula can be as follows:

[0023] in, , represents the original rain-laden image to be restored, 3 is the number of channels, and H and W are the height and width; C represents the number of channels. This represents a single 3x3 convolution operation.

[0024] S102: Using the depth feature map as input parameters, perform n iterations of the first process, and perform at least two image restoration processes on the features after the nth iteration to obtain the first restored feature map. The first iteration process includes: performing image restoration processing on the input parameters, and downsampling the output features after image restoration processing; the image restoration processing includes: selecting bright anchor point features from the input parameters, and fusing spatial and frequency domain information from local and global perspectives on the selected bright anchor point features to obtain fused features.

[0025] It is understandable that image restoration processing is responsible for decomposing, selecting, and enhancing input features, while downsampling (e.g., using strided convolution or pixel merging) can halve the feature map size while multiplying the number of channels, thereby gradually expanding the model's receptive field and extracting deeper features with more semantic information. Therefore, performing n iterations of the first processing on the deep feature map can achieve accurate recognition and preliminary processing of rain pattern features, and, in conjunction with downsampling operations, gradually refine deep semantic features, enhancing the ability to perceive complex rain pattern patterns. Performing at least two image restoration processes on the features after the nth first iteration can further optimize the rain pattern separation effect, ultimately obtaining the first restored feature map.

[0026] S103: Using the first restored feature map as input parameters, perform n second iterations, and perform image restoration processing on the features after the nth second iteration to obtain the second restored feature map. The second iteration processing includes: performing image restoration processing on the input parameters and upsampling the output features after image restoration processing.

[0027] It is understandable that, using the first restored feature map obtained after n iterations of the first process and additional image restoration optimization as the core input, this feature map already has the foundation for initially stripping rain patterns and refining deep semantic features. Based on this, n second iterations are initiated. Each iteration first executes a standardized image restoration process, which can accurately lock the relevant features of residual rain patterns through bright anchor point feature filtering. Then, it fuses spatial domain detail information and frequency domain frequency features from both local and global perspectives to generate a fused feature that combines rain pattern recognition accuracy with feature completeness. Subsequently, upsampling operations are used to gradually improve the spatial resolution of the feature map, accurately aligning and coordinating the previously refined deep semantic features with the gradually restored spatial detail information, avoiding the disconnect between semantic features and detail information. After n such iterations and optimizations, a complete image restoration process is performed again on the final output feature map to further deepen the stripping effect of residual rain patterns, enhance the restoration of spatial details and the overall consistency of the feature map, ultimately resulting in a second restored feature map with significantly reduced rain pattern interference, rich and complete spatial details, and a high degree of coordination between deep semantics and surface details.

[0028] S104: The second restored feature map is processed by a convolutional layer to generate a residual feature map, and the residual feature map is added to the rain-containing image to be restored to obtain the repaired image.

[0029] It is understandable that a convolutional layer is used to refine the second restored feature map, which has already completed the collaborative connection between deep semantic features and spatial details. This convolutional layer, through adaptive adjustment of convolutional kernel parameters, accurately captures residual error information such as trace rain streaks, local brightness deviations, and missing details and textures that may still exist in the second restored feature map. These differential features to be corrected are extracted and transformed into residual feature maps with dimension matching and semantic correspondence. This not only preserves the core details of the image that need to be supplemented, but also specifically targets the rain streak interference traces that have not been completely eliminated. Subsequently, the residual feature map is fused with the original rain-containing image to be restored at the pixel level. With the advantage of residual learning, the positive restoration information contained in the residual feature map compensates and corrects the original structure of the rain-containing image, while offsetting the negative interference caused by residual rain streaks. Finally, a restored image with completely removed rain streaks, clear and natural image texture, accurate color restoration, and a coherent and harmonious overall visual effect is obtained.

[0030] The single-image rain removal method in this embodiment extracts features from the rain-containing image to be restored using convolutional layers to obtain a deep feature map, thus building a comprehensive initial feature foundation for rain pattern removal and image restoration. Then, it performs n first-iteration processing steps. In each iteration, it first accurately identifies rain pattern-related features through bright anchor point feature filtering, and then generates fused features by fusing spatial and frequency domain information from local and global perspectives, achieving accurate identification and preliminary processing of rain pattern features. Subsequent downsampling operations gradually refine deep semantic features, enhancing the perception of complex rain pattern patterns. After the nth first iteration, at least two additional image restoration processes are performed to further optimize the rain pattern separation effect, resulting in a first restored feature map. Next, using the first restored feature map as input, it performs n second-iteration processing steps. Similarly, it deepens rain pattern removal through bright anchor point filtering and spatial-frequency domain fusion image restoration processing, combined with upsampling operations to gradually restore image spatial details, achieving synergistic connection between deep semantic features and spatial details, resulting in a second restored feature map. Finally, it processes the second restored feature map through convolutional layers to generate a residual feature map, which is then added to the rain-containing image to be restored to complete the final restoration. Through multiple iterations and the combination of upsampling and downsampling, the integrity of the image structure and the richness of details are ensured, ultimately achieving high-quality rain streak removal and image restoration.

[0031] In some embodiments, such as Figure 2 and Figure 3 As shown, the input parameters are filtered for bright anchor point features, and the filtered bright anchor point features are fused with spatial and frequency domain information from local and global perspectives to obtain fused features, including the following steps S201 to S203.

[0032] S201: A bright anchor point feature containing rain line information is selected from the deep feature map based on a bright feature selection self-attention mechanism.

[0033] Understandably, based on the physical prior that rain streaks are typically brighter than their surroundings, the brightness feature selection self-attention mechanism first performs a full-dimensional feature scan of the input depth feature map, focusing on capturing the differences in brightness across feature locations. Simultaneously, through the association computation capability of self-attention, it establishes brightness correlations between different locations in the feature map. Subsequently, it quantitatively evaluates the matching degree between the brightness attributes of each feature point and the rain streak features. By dynamically allocating attention weights, it strengthens the weights of feature points whose brightness is significantly higher than the surrounding area and whose morphology is potentially correlated with the rain streak structure, while suppressing background bright interference features unrelated to the rain streak. Finally, it accurately selects feature points from the depth feature map that satisfy both the brightness prior conditions and contain the core information of the rain streak as bright anchor features, achieving preliminary purification and precise localization of the rain streak features.

[0034] S202: Select rain line features similar to bright anchor point features from the depth feature map through cross-domain attention, and perform feature restoration processing on the rain line features and perform residual connection with the depth feature map to obtain a preliminary restored feature map.

[0035] Understandably, using the selected bright anchor point features as the core reference benchmark, the cross-domain attention mechanism constructs a cross-domain feature association channel between the spatial and frequency domains. This comprehensively searches the depth feature map for rain line-related features that are highly similar to the bright anchor point features in terms of brightness attributes, structural morphology, and frequency domain distribution, accurately distinguishing rain line features from background image features and avoiding misidentification of non-rain line image details. Subsequently, the selected rain line features undergo feature restoration processing such as dimensional calibration and resolution matching to ensure consistency with the original depth feature map in feature dimension and spatial scale, meeting the adaptation requirements of residual connections. Finally, the restored rain line features are residually connected to the depth feature map. Through feature subtraction, redundant rain line information in the depth feature map is removed, while preserving key details such as texture and edges of the original image to the greatest extent possible. This results in a preliminary restored feature map with significantly reduced rain line interference and complete preservation of the core image information.

[0036] S203: Using a bi-branch feedforward network, global features that retain global key frequency components and local features that retain local key frequency components are extracted from the preliminary recovered feature map, respectively. The global and local features are then fused and residually connected to the preliminary recovered feature map to generate fused features.

[0037] In a dual-branch feedforward network, one branch focuses on the global frequency characteristics of the initial recovered feature map. By accurately extracting and preserving key frequency components of the overall image, it effectively avoids global frequency interference caused by residual rain streaks, while maintaining the overall brightness balance and structural integrity of the image. The other branch focuses on the frequency information of local details. By capturing key frequency components in local areas of the image, it accurately locates and removes fine rain streaks that were not completely removed in the initial recovered feature map. Subsequently, an adaptive feature fusion strategy is used to fuse the global and local key frequency features in a complementary manner. This ensures the rationality of the overall frequency distribution of the image and enhances the accurate restoration of local details. The fused comprehensive features are then residually connected to the initial recovered feature map to further offset the remaining rain streak interference while preserving the core details of the original image, such as texture, edges, and color, to the greatest extent possible. The final result is a fused image with completely removed rain streaks, clear and natural image quality, and rich and complete details.

[0038] In this embodiment, a depth feature map is obtained by extracting features from the rain-containing image to be restored, laying a comprehensive feature foundation for rain stripe separation and image restoration. Then, a bright feature selection self-attention mechanism is used to accurately filter out bright anchor point features containing rain line information, making full use of the distribution characteristics of rain stripes in the brightness space to achieve preliminary accurate localization of rain stripe features. Subsequently, a cross-domain attention mechanism is used to match rain line features similar to bright anchor point features from the depth feature map. After feature restoration, the features are connected to the residual of the depth feature map, locking the rain stripe features while preserving the original details of the image to the greatest extent, resulting in a preliminary restored feature map. Finally, a dual-branch feedforward network is used to extract global features that retain global key frequency components and local features that retain local key frequency components from the preliminary restored feature map. By fusing global and local frequency domain features, the spatial domain and frequency domain are optimized in a coordinated manner. The features are then connected to the residual of the preliminary restored feature map, ultimately achieving accurate and efficient removal of rain stripes from a single rain-containing image. This significantly improves the thoroughness of rain removal while perfectly preserving the details of the image, thus achieving a high-quality rain-free image restoration effect.

[0039] In some embodiments, step S201, selecting bright anchor point features containing rain line information from the depth feature map based on a bright feature selection self-attention mechanism, includes: performing layer normalization on the depth feature map and increasing the number of channels in the normalized depth feature map through 1x1 convolution to obtain an enhanced feature map; performing non-overlapping local max pooling on the enhanced feature map to select the feature points with the highest brightness in each local region to obtain candidate bright features containing the main rain line information; linearly projecting the candidate bright features into query, key, and value matrices respectively, and performing feature filtering through self-attention calculation to obtain bright anchor point features.

[0040] The process for filtering bright anchor point features and rain line features can be as follows: Figure 4 As shown, it can be understood that performing layer normalization on the deep feature map can stabilize the training process, and the corresponding formula is as follows:

[0041] in, For layer normalization operation; These are the features after normalization.

[0042] Then, 1x1 convolutions are used to increase the number of channels in the normalized features, enhancing their feature representation capabilities to obtain an enhanced feature map. Non-overlapping local max pooling is then performed on the enhanced feature map. This operation is the first key design step in achieving bright feature selection. This step utilizes max pooling to retain only the brightest feature points within each local window, thus computationally efficiently filtering out candidate bright features containing the main rain line information. The corresponding expression is as follows:

[0043] in, The representation layer normalization operation standardizes the training process by calculating the mean and variance of each feature channel, effectively stabilizing the training process and accelerating convergence. This represents a 1x1 convolution operation, used to enhance feature representation before pooling. This represents a local max pooling operation, used to filter the feature points with the highest brightness within each local region; This represents the local bright feature obtained after the above operations.

[0044] Then, the candidate bright features can be linearly projected into query, key, and value matrices, respectively. The corresponding formula is as follows:

[0045] in, Indicates feature segmentation operation; This represents the query, key, and value matrix for the first stage. Subsequently, based on another prior knowledge that rain line features in different regions are similar, self-attention computation is used to... Features that truly belong to the rain lines (those that are similar to each other) receive high response weights, while The weights of irrelevant background bright spot noise (which is dissimilar to each other) mixed in are suppressed, thus obtaining the bright anchor point features. The calculation formula is as follows:

[0046] in, is a learnable scaling parameter used to adjust the attention score; SoftMax() represents the standard SoftMax activation function used to normalize the attention score to a probability distribution between [0, 1]; Attn() represents the output of this self-attention operation, i.e., the bright anchor feature.

[0047] In some embodiments, step S202, selecting rain line features similar to bright anchor point features from the deep feature map through cross-domain attention, includes: normalizing the deep feature map and projecting the normalized deep feature map through a 1x1 convolutional layer to generate a query matrix; inputting the bright anchor point features into the 1x1 convolutional layer, segmenting the feature map output by the 1x1 convolutional layer to generate a key matrix for comparison and a value matrix containing rain line information; calculating the multi-head cross-attention between the query matrix and the key matrix to obtain an attention map; and performing matrix multiplication on the attention map and the value matrix to obtain the rain line features.

[0048] like Figure 4As shown, after normalizing the depth feature map, projecting it through an independent 1x1 convolutional layer can generate a query matrix:

[0049] in, This represents the query matrix for the second stage.

[0050] The bright anchor point features are then input into another 1x1 convolutional layer, and the output feature map is segmented to generate key matrices for comparison. and the value matrix containing information The corresponding formula is as follows:

[0051] in, This represents the key and value matrix of the second stage.

[0052] Then, calculate the multi-head cross-attention between the query matrix and the key matrix. First, calculate the attention score: [The query matrix is ​​then...] With key matrix transpose Perform matrix multiplication and divide by a learnable scaling parameter. The attention score is obtained, and the corresponding formula is as follows:

[0053] Next, the normalized score is calculated: the attention score is normalized using the SoftMax activation function to obtain the attention map, as shown in the following formula:

[0054] After obtaining the attention map (Map), the normalized attention map (Map) and the value matrix are compared... Perform matrix multiplication to obtain weighted output features. The rain line characteristic is represented by the following formula:

[0055] In some embodiments, in step S202, after the rain line features are processed for feature restoration, a residual connection is made with the depth feature map to obtain a preliminary restored feature map. This includes: adjusting the channels of the rain line features through 1x1 convolution, and restoring the channel-adjusted rain line features to their original spatial size to obtain restored rain line features; and making a residual connection between the restored rain line features and the depth feature map to obtain a preliminary restored feature map.

[0056] Among them, such as Figure 4 As shown, the rain line features Channel adjustment is performed using a 1x1 convolution, and the features after windowing are restored to their original spatial dimensions to obtain the processed features. This step is post-processing for the attention operation. A 1x1 convolution is used to fuse multi-head information, and the window restoration operation is a standard accompanying step of the window attention mechanism, used to merge the segmented features back into a complete feature map. The processed features are then residually concatenated with the initial depth feature map to obtain the preliminary recovered feature map Y, as shown in the following formula:

[0057] In some embodiments, such as Figure 5 As shown, the dual-branch feedforward network includes a global branch and a local branch. In step S203, the dual-branch feedforward network is used to extract global features (preserving global key frequency components) and local features (preserving local key frequency components) from the preliminary recovered feature map. This includes: performing layer normalization on the preliminary recovered feature map to obtain a layer-normalized recovered feature map, and inputting the layer-normalized recovered feature map into the dual-branch feedforward network; in the global branch, expanding the channel dimension of the layer-normalized recovered feature map through point convolution, activating the expanded features using an activation function, and performing discriminative frequency domain operations on the activated features to obtain global features; in the local branch, extracting spatial features from some input channels of the layer-normalized recovered feature map using partial convolution, transforming the extracted features through point convolution and an activation function, processing the transformed features using depthwise separable convolution, and performing discriminative frequency domain operations on the processed features to obtain local features.

[0058] The discriminative frequency domain operation includes: transforming the input features to the frequency domain and multiplying them with a learnable quantization matrix to adaptively preserve key frequency information, and then restoring the product features to the spatial domain through an inverse transformation.

[0059] Initial recovery of feature maps Layer normalization is performed, and the normalized features are then fed into a two-branch feedforward network consisting of global and local branches to obtain the normalized features. To stabilize the calculation process. The corresponding formula is as follows:

[0060] in, This represents the normalized features.

[0061] The global branch processes the input features. The purpose of this branch is to extract global contextual information from the features and preserve key global frequency components through frequency domain operations. In the global branch, the channel dimension is first expanded using a dotted convolution. Then, a non-linear transformation is performed using the GELU activation function. Finally, a discriminative frequency domain operation (FDO) is executed, which transforms the features to the frequency domain, multiplies them with a learnable quantization matrix, and then inversely transforms them back to the spatial domain to adaptively preserve key global frequency information. The calculation formulas for the above process are as follows:

[0062] in, This represents the GELU activation function; Indicates discriminative frequency domain operations; This indicates the output characteristics of the global branch.

[0063] The purpose of local branching is to efficiently extract local spatial features and preserve important local frequency components by combining frequency domain operations. First, partial convolution is employed. right The process involves processing. Unlike standard convolution, partial convolution performs a 3x3 spatial convolution on only a subset (e.g., a quarter) of the input feature map, while leaving the remaining channels unchanged. This significantly reduces computational redundancy and memory access while efficiently extracting spatially local features; the output of the partial convolution is then processed through point convolution. and GELU activation function Perform feature transformation to obtain intermediate features :

[0064] in, This indicates a partial convolution operation; This represents the GELU activation function; This represents the intermediate features of a local branch.

[0065] Next, depthwise separable convolution is used. right Further processing is performed to enhance the ability to extract local information across multiple ranges. The output of the depthwise separable convolution is processed again through GELU and point convolution, and then fed into discriminative frequency domain operations (FDO) to obtain the final output of the local branch. :

[0066] in, Indicates intermediate features of local branches; This represents a 3x3 depth separable convolution. This represents the output characteristics of a local branch.

[0067] Then the output features of the global branch Output characteristics of local branches By adding elements one by one, global contextual information and local detailed features are aggregated to obtain fused features. :

[0068] Finally, the fused features are compared with the initially recovered feature map. By combining residual connections, the final features are output. The formula for restoring the image without rain is as follows:

[0069] In some embodiments, such as Figure 6 As shown, the process involves transforming the input features to the frequency domain and multiplying them with a learnable quantization matrix to adaptively preserve key frequency information. The product features are then restored to the spatial domain via an inverse transform. This includes: dividing the input features into multiple non-overlapping feature blocks in the spatial dimension; performing a Fast Fourier Transform (FFT) on each feature block to transform the image features of each block from the spatial domain to the frequency domain, obtaining a frequency domain feature map corresponding to each feature block; performing element-wise multiplication of each frequency domain feature map with the learnable quantization matrix to adaptively amplify the weights of key frequency components and suppress invalid frequency components related to rain ripple degradation, obtaining multiple weighted frequency domain feature maps; performing an inverse FFT on each weighted frequency domain feature map to obtain multiple filtered spatial feature blocks; and restoring and recombining the spatial feature blocks based on their arrangement order, and restoring the spatial resolution of the restored and recombined feature maps to their original size.

[0070] It can be understood that the steps in this embodiment correspond to the aforementioned discriminative frequency domain operation. The discriminative frequency domain operation is embedded in the dual-branch discriminative frequency domain feedforward network, which is used to adaptively filter and retain frequency information that is beneficial to image restoration in the frequency domain, and suppress high-frequency noise related to degradation factors such as rain streaks.

[0071] In applications, input feature map segmentation can transform the feature map after layer normalization. The system is spatially divided into multiple non-overlapping feature blocks, each processed independently to enhance the discriminative power of local frequency features. Then, a Fast Fourier Transform is performed to analyze each block. The feature blocks are transformed from spatial domain features to frequency domain representation using Fast Fourier Transform (FFT), resulting in a complex-form frequency domain feature map. This feature map is then element-wise multiplied with a learnable quantization matrix to obtain multiple weighted frequency domain feature maps. During training, this matrix adaptively learns the importance weights of different frequency components, thereby enhancing key frequency components and suppressing irrelevant frequencies. Adaptive frequency filtering, through the aforementioned weighting operations, achieves adaptive filtering of the frequency domain features, preserving low- and mid-frequency components related to image structure and background information while weakening high-frequency components related to degradation factors such as rain streaks and noise.

[0072] Based on this, an inverse fast Fourier transform is performed on each weighted frequency domain feature map to convert the filtered frequency domain features back to the spatial domain, thus restoring them as... Spatial feature blocks. The feature blocks are reassembled, and all processed blocks are reassembled. The feature blocks are reassembled in their original order, restoring them to the same size as the original input feature map. The output feature map is used to adaptively filter and retain frequency information that is beneficial to image restoration, suppress high-frequency noise related to degradation factors such as rain streaks, and ensure that it is consistent with the original depth feature map in terms of feature dimension and spatial scale, thus meeting the adaptation requirements of residual connection.

[0073] In some embodiments, the single-image deraining method may further include: after the first iteration of the i-th level, saving the current output as a skip connection feature, which is used for subsequent fusion with the input features of the second iteration of the n-i+1-th level.

[0074] It is understandable that during the first iteration of the i-th level, after the bright anchor point feature selection, the fusion of spatial-frequency domain information from local and global perspectives, and the downsampling operation, the current output feature not only contains the core semantic information after the rain pattern is accurately separated at this level, but also retains the key spatial detail features at the corresponding resolution. Saving this feature as a skip connection feature can make up for the information loss of intermediate levels that may be caused by upsampling and downsampling during the iteration process. In the subsequent second iteration, the skip connection feature will be precisely matched to the (n-i+1)th level of the second iteration and fused with the input features of that level. Since the first iteration gradually refines the deep semantics along the downsampling direction (gradually deepening from level 1 to level n), and the second iteration gradually restores spatial details along the upsampling direction (gradually refining from level n to level 1), the skip connection feature of the first iteration at level i and the input features of the second iteration at level n-i+1 naturally adapt to each other in terms of resolution and feature semantic dimension. After fusion, the rain pattern processing results and original detail information of the first iteration at this level can be transmitted to the corresponding stage of the second iteration, providing a precise reference for the spatial detail restoration in the upsampling process. At the same time, it can avoid the feature blurring or detail loss problem caused by the second iteration relying solely on upsampling. This achieves precise cross-iteration connection between semantic features and spatial details at different levels, further enhancing the coherence of rain pattern removal and the integrity of image detail restoration.

[0075] In some embodiments, such as Figure 7 As shown, this application also provides a single-image deraining system based on brightness feature selection Transformer, including: feature extraction module 11, encoder module 12, bottleneck module 13, decoder module 14 and image reconstruction module 15.

[0076] The feature extraction module 11 is used to extract features from the rain-containing image to be restored using a convolutional layer to obtain a depth feature map.

[0077] The encoder module 12 includes n cascaded first iteration units and one image complex atom unit. The first iteration unit includes cascaded image complex atom units and a downsampling layer. The image complex atom unit is used to filter bright anchor point features of the input parameters and fuse spatial and frequency domain information from local and global perspectives to obtain fused features.

[0078] The bottleneck module 13 includes at least one image complex atom unit. The input of the bottleneck module 13 is connected to the output of the encoder module 12, and the output of the bottleneck module 13 is connected to the input of the decoder module 14.

[0079] The decoder module 14 includes n cascaded second iteration units and an image complex atom unit. The second iteration unit includes cascaded image complex atom units and an upsampling layer.

[0080] The image reconstruction module 15 is used to process the features output by the decoder module 14 using a convolutional layer to generate a residual feature map, and then add the residual feature map to the rain-containing image to be restored to obtain the repaired image.

[0081] It should be noted that the number of image complex atomic units in a single first iteration unit and a single second iteration unit can be multiple, and the number of image complex atomic units in the first iteration unit and the second iteration unit is the same. Multiple image complex atomic units in an iteration unit can be connected in series, and the downsampling layer can be located before or after multiple image complex atomic units.

[0082] In some embodiments, the encoder module 12 is used to save the current output as a skip connection feature after the first iterative processing module of the i-th level completes feature extraction. The skip connection feature is used for the fusion of input features in the second iterative processing module of the (n-i+1)-th level. The second iterative processing module of the (n-i+1)-th level is used to fuse the input features in the second iterative processing module of the (n-i+1)-th level with the corresponding skip connection feature before performing image restoration processing.

[0083] For example, such as Figure 8 As shown, a symmetrical two-stage encoder-decoder U-Net architecture is used for deep feature extraction. Each layer of the encoder contains N1 image complex atomic units, the bottleneck module contains N2 image complex atomic units, and each layer of the decoder contains the same number of image complex atomic units as the encoder. Feature downsampling and upsampling are achieved through convolution and pixel rearrangement operations, and skip connections are introduced to stabilize training.

[0084] In terms of experimental setup, this embodiment has been fully validated on synthetic datasets such as Rain200L / H, DDN-Data, and DID-Data, as well as real-world datasets such as SPA-Data and GT-RAIN. During the training phase, the input image is randomly cropped into 128×128 or 256×256 pixel blocks, and the Adam optimizer is used with an initial learning rate set to 4e. -4 or 2e -4 And gradually reduced to 1e through cosine annealing scheduling. -6 A hybrid loss function combining L1 loss and frequency domain loss is employed to guide model training. Experimental results demonstrate that the proposed method and system outperform state-of-the-art single-image rain removal methods in both synthetic datasets and real-world scenarios, achieving significant improvements in PSNR and SSIM metrics. Furthermore, a good balance is struck between model complexity and performance, validating the effectiveness and practicality of the proposed method and system.

[0085] Based on the above embodiments, in one example, the single-image rain removal system based on brightness feature selection Transformer can employ a carefully designed symmetrical U-shaped encoder-decoder architecture to achieve effective extraction and fusion of multi-scale features. Specifically, an initial shallow feature layer can first be extracted from the input rain image using a 3×3 convolutional layer. This feature layer contains both rain streak information and background content, serving as the input basis for subsequent deep processing.

[0086] In the encoder path, hierarchical processing of the feature map is achieved through two consecutive downsampling stages. Each downsampling stage employs a 3×3 convolution operation with a stride of 2, precisely halving the spatial size of the feature map while doubling the number of channels, thus compressing the spatial dimension while retaining more feature information. The first downsampling stage downsamples the feature map from its original size H×W to H / 2×W / 2, and the second downsampling stage further compresses it to H / 4×W / 4. After each downsampling stage, four image complex atomic units are configured, which are connected in a cascaded manner to deepen the feature extraction process layer by layer. In particular, each image complex atomic unit includes a brightness feature selection self-attention module and a dual-branch discriminative frequency domain feedforward network. The number of self-attention heads can be set according to the configuration scheme of [1,2,4,2,1], and the initial number of channels is fixed at 48 to ensure optimal feature processing capability at different resolutions.

[0087] A dedicated bottleneck module is placed between the encoder and decoder. This layer contains 16 consecutive image complex atomic units, performing deep feature transformation at the lowest spatial resolution (H / 4×W / 4). This dense block configuration allows the model to fully exploit the global characteristics and long-range dependencies of rain patterns within a compressed feature space, while enhancing the discriminative power of the features through multi-layer nonlinear transformations. The output features of the bottleneck module retain rich semantic information while also containing refined rain pattern feature representations.

[0088] The decoder path progressively restores the spatial resolution of the feature map through two upsampling stages. Each upsampling stage employs a pixel shuffle mechanism, using learnable upsampling convolutional kernels to precisely double the feature map size while halving the number of channels. The first upsampling stage restores the feature map from H / 4×W / 4 to H / 2×W / 2, and the second upsampling stage further restores it to the original size H×W. Each upsampling stage is followed by four image complex atomic units, progressively restoring the image's detailed information through feature reconstruction.

[0089] To ensure the effective transfer of multi-scale features, symmetrical skip connections are established between corresponding scales of the encoder and decoder. This symmetrical skip connection mechanism effectively alleviates the gradient vanishing problem while ensuring the full fusion of low-level visual features and high-level semantic features.

[0090] In the image reconstruction stage, the feature map output by the decoder is channel-adjusted using a 3×3 convolutional layer, converting it to have the same three channels as the input image, resulting in a residual image with the exact same size as the input image. This residual image mainly contains the rain streaks to be removed, as well as some high-frequency detail information. Finally, the residual image is added to the original input rain image using pixel-level addition operations. Leveraging the advantages of residual learning, the original content and texture details of the image are preserved to the maximum extent while removing rain streaks.

[0091] In the entire network structure, the number of image complex atomic units can be set according to a symmetrical configuration scheme of [4,4,16,4,4]. This configuration effectively controls computational complexity while ensuring model depth. The self-attention module in each image complex atomic unit adopts a multi-head attention mechanism, and the number of heads can be configured according to a scheme of [1,2,4,2,1], so that the model can maintain an appropriate receptive field and feature interaction capability at different resolutions. The initial number of channels is set to 48, which is dynamically adjusted through subsequent downsampling and upsampling operations to ensure that the model achieves the best balance between computational efficiency and feature representation capability. By organically combining physical priors (brightness characteristics of rain streaks) with the deep learning model, a dedicated bright feature selection mechanism is used to achieve accurate localization of rain streak features. Frequency domain processing is combined to enhance feature discrimination capability, and a symmetrical U-shaped architecture is used to ensure effective fusion of multi-scale features. Finally, superior rain removal performance is achieved on multiple standard test sets.

[0092] It should be noted that the single-image deraining system based on brightness feature selection Transformer provided in this application embodiment and the single-image deraining method based on brightness feature selection Transformer provided in this application embodiment are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned single-image deraining method based on brightness feature selection Transformer, and the repeated parts will not be described again.

[0093] In some embodiments, such as Figure 9 As shown, this application also provides a method for removing rain from a single image, including the following steps S901 and S902.

[0094] S901: Acquire the rain-containing image to be recovered.

[0095] S902: Input the rain-containing image to be restored into the single-image rain removal system based on brightness feature selection Transformer as described above, and determine the image output by the single-image rain removal system based on brightness feature selection Transformer as the rain-free restored image corresponding to the rain-containing image to be restored.

[0096] It should be noted that the beneficial effects of the single-image deraining method in this application embodiment compared to related technologies are the same as the beneficial effects of the single-image deraining system based on brightness feature selection Transformer in the above embodiment compared to related technologies, and the repeated parts will not be described again.

[0097] In some embodiments, an electronic device provided in this application includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the above-described single-image rain removal method based on brightness feature selection Transformer.

[0098] This application also provides a computer-readable medium storing a computer program that, when executed by a processor, implements the above-described single-image deraining method based on a brightness feature selection Transformer. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0099] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A single image rain removal method based on bright feature selection Transformer, characterized in that, The method comprises the steps of: performing feature extraction on the rain-containing image to be restored by using a convolutional layer to obtain a deep feature map; performing n times of first iteration processing on the deep feature map as an input parameter, and performing image restoration processing on the feature after the n times of first iteration processing at least twice to obtain a first restored feature map; the first iteration processing comprises: performing image restoration processing on the input parameter, and performing down-sampling on the output feature after the image restoration processing; the image restoration processing comprises: performing bright anchor point feature screening on the input parameter, and fusing spatial and frequency domain information from a local and global perspective on the screened bright anchor point feature to obtain a fusion feature; performing n times of second iteration processing on the first restored feature map as an input parameter, and performing image restoration processing on the feature after the n times of second iteration processing to obtain a second restored feature map; the second iteration processing comprises: performing image restoration processing on the input parameter, and performing up-sampling on the output feature after the image restoration processing; generating a residual feature map by performing processing on the second restored feature map by using a convolutional layer, and adding the residual feature map and the rain-containing image to be restored to obtain a repaired image.

2. The single image rain removal method based on bright feature selection Transformer of claim 1, wherein, the bright anchor point feature screening on the input parameter and the fusion feature obtained by fusing spatial and frequency domain information from a local and global perspective on the screened bright anchor point feature, comprises: screening the bright anchor point feature containing rain line information from the deep feature map based on a bright feature selection self-attention mechanism; selecting rain line features similar to the bright anchor point feature from the deep feature map through cross-domain attention, and performing residual connection on the rain line features after feature restoration processing and the deep feature map to obtain a preliminary restored feature map; extracting global features retaining global key frequency components and local features retaining local key frequency components from the preliminary restored feature map respectively by using a double-branch feedforward network, and performing residual connection on the global features and the local features after fusion and the preliminary restored feature map to generate the fusion feature.

3. The bright feature selection Transformer-based single image deraining method of claim 2, wherein, the screening of the bright anchor point feature containing rain line information from the deep feature map based on the bright feature selection self-attention mechanism, comprises: performing layer normalization processing on the deep feature map, and improving the channel number of the normalized deep feature map through 1x1 convolution to obtain an enhanced feature map; performing non-overlapping local maximum pooling operation on the enhanced feature map to screen the feature points with the highest brightness in each local region to obtain candidate bright features containing main rain line information; linearly projecting the candidate bright features into query, key and value matrices respectively, and performing feature filtering processing through self-attention calculation to obtain the bright anchor point feature.

4. The bright feature selection Transformer-based single image deraining method of claim 2, wherein, the selection of rain line features similar to the bright anchor point feature from the deep feature map through cross-domain attention, comprises: performing normalization processing on the deep feature map, and performing projection processing on the deep feature map after normalization processing through a 1x1 convolutional layer to generate a query matrix; inputting the bright anchor point feature into a 1x1 convolutional layer, and dividing the feature map output by the 1x1 convolutional layer to generate a key matrix used for comparison and a value matrix containing rain line information respectively; performing matrix multiplication operation on the attention graph and the value matrix to obtain the rain line feature. The rain line feature is connected in residual connection with the depth feature map after feature restoration processing to obtain a preliminary recovery feature map, including:

5. The bright feature selection Transformer-based single image deraining method of claim 2, wherein, The rain line feature is connected in residual connection with the depth feature map after feature restoration processing to obtain a preliminary recovery feature map, including: The rain line feature is connected in residual connection with the depth feature map after feature restoration processing to obtain a preliminary recovery feature map, including: The rain line feature is connected in residual connection with the depth feature map after feature restoration processing to obtain a preliminary recovery feature map, including:

6. The bright feature selection Transformer-based single image deraining method of claim 2, wherein, The double-branch feedforward network includes a global branch and a local branch; the global feature and the local feature are extracted from the preliminary recovery feature map by using the double-branch feedforward network, including: The layer normalization recovery feature map is input into the double-branch feedforward network; In the global branch, the channel dimension of the layer normalization recovery feature map is expanded by point convolution processing, and the expanded feature is activated by an activation function, and the activated feature is subjected to a discriminative frequency domain operation to obtain the global feature, wherein the discriminative frequency domain operation includes: multiplying the input feature converted to the frequency domain by a learnable quantization matrix to adaptively retain key frequency information, and restoring the product feature to the spatial domain by inverse transformation; In the local branch, spatial features are extracted from part of the input channels of the layer normalization recovery feature map by partial convolution, and the extracted features are transformed by point convolution and an activation function, and the transformed features are processed by depth separable convolution, and the processed features are subjected to a discriminative frequency domain operation to obtain the local feature.

7. The bright feature selection Transformer-based single image deraining method of claim 6, wherein, The input feature is converted to the frequency domain and multiplied by a learnable quantization matrix to adaptively retain key frequency information, and the product feature is restored to the spatial domain by inverse transformation, including: Each of the feature blocks is subjected to a fast Fourier transform to convert the image features of each of the feature blocks from the spatial domain to the frequency domain to obtain a frequency domain feature map corresponding to each of the feature blocks; Each of the frequency domain feature maps is multiplied by a learnable quantization matrix at the element level to adaptively amplify the weight of the key frequency component and suppress the invalid frequency component related to rain streak degradation to obtain a plurality of weighted frequency domain feature maps; Each of the weighted frequency domain feature maps is subjected to a fast inverse Fourier transform to obtain a plurality of filtered spatial feature blocks; Each of the spatial feature blocks is restored and reorganized based on the arrangement order of each of the feature blocks, and the spatial resolution of the restored and reorganized feature map is restored to the original size. including:

8. A single image rain removal system based on bright feature selection Transformer, characterized in that, a feature extraction module, an encoder module, a bottleneck module, a decoder module, and an image reconstruction module; The feature extraction module is configured to extract features from a rain-containing image to be restored by using a convolutional layer to obtain a depth feature map; ​ The encoder module comprises n cascaded first iteration units and an image restoration atomic unit, the first iteration unit comprises a cascaded image restoration atomic unit and a down-sampling layer; the image restoration atomic unit is used for bright anchor point feature screening on input parameters, and fusing spatial and frequency domain information from local and global perspectives on the screened bright anchor point features to obtain fused features; The bottleneck module comprises at least one image restoration atomic unit, the input end of the bottleneck module is connected with the output end of the encoder module, and the output end of the bottleneck module is connected with the input end of the decoder module; The decoder module comprises n cascaded second iteration units and an image restoration atomic unit, the second iteration unit comprises a cascaded image restoration atomic unit and an up-sampling layer; The image reconstruction module is used for processing the features output by the decoder module by adopting a convolution layer to generate a residual feature map, and adding the residual feature map and the rain-containing image to be restored to obtain a repaired image.

9. The bright feature selection Transformer-based single image deraining system of claim 8, wherein, The encoder module is used for saving the current output as a skip connection feature after the first iteration processing module at the i-th level completes feature extraction, and the skip connection feature is used for subsequent input feature fusion in the second iteration processing module at the n-i+1-th level; the second iteration processing module at the n-i+1-th level is used for fusing the input feature in the second iteration processing module at the n-i+1-th level with the corresponding skip connection feature before performing image restoration processing.

10. A single image deraining method, characterized in that, It comprises: An image containing rain to be restored is acquired; The image containing rain to be restored is input into the single-image rain removal system based on bright feature selection Transformer, and the image output by the single-image rain removal system based on bright feature selection Transformer is determined as the rain-free restored image corresponding to the image containing rain to be restored.

Images