Image rain removal method based on space channel coupling attention and related equipment

Abstract

The invention discloses an image rain removal method based on space channel coupling attention and related equipment, and aims to solve the technical problem of poor image rain removal effect caused by single feature extraction, insufficient detail recovery capability and poor universality in the prior art. According to the method, an image rain removal network comprising an encoder, a decoder and a detail reconstruction network is constructed, so that the rain removal effect is improved in multiple dimensions. Space channels in the encoder and the decoder are coupled with attention modules, so that the unicity of feature extraction can be avoided. And redundant information suppression processing is performed on the sample feature map, key features are highlighted, and the adaptability of the image rain removal network to different scenes and rain imprint is enhanced. And the detail reconstruction network further optimizes image details and recovers texture information, so that the problems of insufficient image detail recovery capability and poor universality of the existing image rain removal network are effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image deraining method based on spatial channel coupled attention and related equipment. Background Art

[0002] In applications such as autonomous driving and intelligent visual inspection, clear and complete target images are essential for ensuring efficient and accurate system operation. However, in real-world environments, images are often disrupted by adverse weather conditions such as rain, resulting in blurry and distorted images and missing critical information, which severely impacts the accuracy and stability of subsequent visual tasks. Therefore, research on methods for removing residual rain streaks from real-world scenes while preserving texture details and clear background information is crucial for enhancing the robustness and reliability of autonomous driving systems in harsh environmental conditions such as rain.

[0003] Traditional rain removal methods use algorithms such as dictionary learning, filters, sparse coding, and Gaussian mixture models. These algorithms often fail to adapt to the diversity of rain streaks in complex environments, suffer from poor generalization performance, and struggle to fully utilize the large amount of rich input data samples. Existing single-image rain removal algorithms, such as Vision Transformers (ViT), achieve good results, but are often not directly applicable to high-resolution images. They are computationally expensive and face bottlenecks in model construction, hindering their transfer to practical applications. Furthermore, existing rain removal algorithms, particularly those targeting real-world rain images, generally lack a targeted, detailed, and comprehensive algorithmic network for phased optimization and reconstruction.

[0004] Therefore, how to build a highly targeted image reconstruction optimization network for rain removal under limited computational cost and enhance the details and generalization performance of the restored image is the key to solving the current image rain removal task. Summary of the Invention

[0005] The purpose of the present invention is to provide an image deraining method and related equipment based on spatial channel coupled attention, so as to overcome the technical problems of poor image deraining effect caused by the existing technology due to single feature extraction, insufficient detail restoration ability and poor universality.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides an image deraining method based on spatial channel coupled attention, comprising: Construct an image deraining network, which includes an encoder, a decoder, and a detail reconstruction network. The encoder and decoder include a spatial channel coupling attention module. An image deraining network is used to obtain a sample image, and feature encoding is performed on the sample image to obtain a sample feature map. Performing redundant information suppression processing on the sample feature map to obtain a redundant information suppressed sample feature map; Performing feature decoding on the redundant information suppressed sample feature map to obtain a sample decoding feature map; The sample decoded feature map is reconstructed in detail to obtain the derained image.

[0007] The image deraining network is used to obtain a sample image, and the feature encoding of the sample image is performed to obtain a sample feature map, including: An image deraining network is used to obtain sample images. The encoder in the image deraining network processes the sample images, and the sample images are passed through the Transformer-like convolutional block in the encoder for feature extraction. The feature map processed by the Transformer-like convolution block is downsampled to complete feature encoding and obtain a sample feature map.

[0008] The Transformer-like convolutional block includes a spatial channel coupled attention module and a multi-stage parallel feedforward network.

[0009] Performing redundant information suppression processing on the sample feature map to obtain a redundant information suppressed sample feature map, including: In the image deraining network, the connection between the encoder and the decoder adopts convolutional gated connection. During the transmission of the sample feature map from the encoder to the decoder, the screening mechanism of convolutional gated connection is used to suppress redundant information and obtain the redundant information suppressed sample feature map.

[0010] Perform feature decoding on the redundant information suppressed sample feature map to obtain a sample decoding feature map, including: The decoder in the image deraining network is used to process the redundant information suppression sample feature map, and the redundant information suppression sample feature map is subjected to feature extraction by the Transformer-like convolution block in the decoder; The feature map processed by the Transformer-like convolution block is upsampled to complete feature decoding and obtain a sample feature decoding map.

[0011] Detailed reconstruction of the sample decoded feature map is performed to obtain a derained image, including: The detail reconstruction network in the image deraining network is used to reconstruct the details of the sample feature decoding map. The detail reconstruction network includes a long short-term memory module and a detail improvement reconstruction unit. The sample feature decoding map is passed through the long short-term memory module and the detail improvement reconstruction unit to achieve image detail optimization and obtain a derained image.

[0012] The detail improvement reconstruction unit consists of a multi-scale aggregation block and a spatial pixel attention module; the spatial pixel attention module consists of a spatial attention block and a pixel attention block.

[0013] In a second aspect, the present invention provides an image deraining system based on spatial channel coupled attention, comprising: A network construction module is used to construct an image deraining network. The image deraining network includes an encoder, a decoder, and a detail reconstruction network. The encoder and decoder include a spatial channel coupling attention module. A feature encoding module is used to obtain a sample image using an image deraining network, perform feature encoding on the sample image, and obtain a sample feature map; An information suppression module is used to perform redundant information suppression processing on the sample feature map to obtain a redundant information suppressed sample feature map; A feature decoding module is used to perform feature decoding on the redundant information suppression sample feature map to obtain a sample decoding feature map; The detail reconstruction module is used to reconstruct the details of the sample decoding feature map to obtain a derained image.

[0014] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the image deraining method based on spatial channel coupled attention as described above are implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the image deraining method based on spatial channel coupled attention as described above.

[0016] Compared with the prior art, the present invention has the following beneficial technical effects: In its first aspect, the present invention provides an image deraining method based on spatial channel coupled attention. By constructing an image deraining network comprising an encoder, a decoder, and a detail reconstruction network, the deraining effect is improved in multiple dimensions. The spatial channel coupled attention modules in the encoder and decoder can simultaneously focus on local contextual information and channel dimension information, achieving adaptive modeling of rain mark degradation features and avoiding the monotony of feature extraction. Redundant information suppression processing is performed on the sample feature map to remove irrelevant and redundant information, highlight key features, and enhance the adaptability of the image deraining network to different scenes and rain marks. The detail reconstruction network further optimizes image details and restores texture information, thereby effectively solving the problems of the existing image deraining network's shortcomings in image detail restoration capabilities and poor universality.

[0017] In the second aspect, the present invention provides an image deraining system based on spatial channel coupled attention, which effectively improves the image deraining effect through its unique architecture. The network construction module constructs an image deraining network containing an encoder, a decoder and a detail reconstruction network. The spatial channel coupled attention modules in the encoder and decoder can focus on local and global features at the same time to avoid single feature extraction. The feature encoding module performs feature encoding on the sample image, and the information suppression module removes irrelevant information and redundant information, highlights key features, and enhances adaptability. The feature decoding module performs feature decoding on the sample feature map after redundant information is suppressed, and the detail reconstruction module performs detail reconstruction on the sample decoded feature map, restores texture information, and makes up for the shortcomings of insufficient detail recovery and poor universality of the existing technology, thereby improving the deraining effect.

[0018] In a third aspect, the present invention provides a computer device that can efficiently implement the steps of the method of the present invention by executing a specific computer program through a processor. When performing data processing tasks, the computer device can accurately perform numerical calculations and logical judgments, avoiding errors caused by human factors; at the same time, since the computer program has a high degree of stability and reliability, the accuracy and consistency of the data processing results can be ensured.

[0019] In a fourth aspect, the present invention provides a computer-readable storage medium. By programming the steps of the method of the present invention into a computer program and storing it on a computer-readable storage medium, users can easily load these programs onto any compatible computer device and execute them without rewriting or converting the code, thereby greatly improving the convenience and flexibility of program execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of an image deraining method based on spatial channel coupled attention in an embodiment of the present invention.

[0021] Figure 2 Schematic diagram of an image deraining network architecture based on spatial channel coupled attention in an embodiment of the present invention.

[0022] Figure 3 Schematic diagram of the spatial channel coupled attention module in an embodiment of the present invention.

[0023] Figure 4 Schematic diagram of convolutional gating connection in an embodiment of the present invention.

[0024] Figure 5 Schematic diagram of pixel attention block in an embodiment of the present invention.

[0025] Figure 6 Schematic diagram of an image deraining network system based on spatial channel coupled attention in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In the field of image deraining, currently widely used methods include algorithms based on the Transformer architecture and deep convolutional neural network methods. Single-image deraining algorithms, represented by Vision Transformers (ViT), have achieved excellent results in removing rain streaks thanks to their powerful feature extraction capabilities. Deep convolutional neural network methods, by constructing complex convolutional layer structures, can effectively learn rain image features, achieving rain layer separation and background layer reconstruction. These existing technologies have played an important role in improving image clarity and preserving texture details.

[0027] However, existing technologies still have many problems and shortcomings. Existing image deraining technologies are single in feature extraction, making it difficult to fully capture the diverse characteristics of rain streaks; they are insufficient in detail recovery, which easily leads to loss of image details or the appearance of artifacts; in terms of adaptability, they have poor universality for different rain streak shapes and complex scenes. These shortcomings limit the deraining effect and image quality improvement. Existing Transformer-based deraining algorithms have too high computational costs when processing high-resolution images, and there are bottlenecks in model building; traditional deep convolutional neural network methods are not sufficient in feature extraction and information fusion, and have limited capabilities for locating and suppressing key rain streak areas, and are deficient in cross-stage long-term dependency modeling and pixel-level detail optimization. These problems result in insufficient robustness and generalization capabilities of existing technologies for deraining in complex real-world scenes, making it difficult to meet the needs of practical applications for high-quality rain-free images.

[0028] In summary, a new image deraining technology solution is urgently needed to address the shortcomings of existing technologies in feature extraction, detail restoration, and adaptability, in order to meet the new demands of the market, industry, and users for efficient and accurate image deraining technology. Therefore, how to overcome the problems of single feature extraction, weak detail restoration ability, and poor adaptability to different rain streak morphologies in existing image deraining technologies has become a key issue that needs to be urgently addressed by those skilled in the art. It is based on this background that the present invention is proposed. It aims to construct an image deraining method that can effectively improve the deraining effect, enhance the image detail restoration ability, and adapt to a variety of complex scenes by introducing key technologies such as spatial channel coupling attention module, multi-level parallel feedforward network, convolutional gated connection, and detail improvement and reconstruction unit, thereby providing clearer and more accurate image data support and promoting the development and application of related technologies.

[0029] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0030] Reference Figure 1 As shown in FIG, a specific embodiment of the image deraining method based on spatial channel coupled attention provided by the present invention includes: S1, build an image deraining network, which includes an encoder, a decoder, and a detail reconstruction network, where the encoder and decoder include a spatial channel coupling attention module; S2, using the image deraining network to obtain a sample image, perform feature encoding on the sample image, and obtain a sample feature map; S3, performing redundant information suppression processing on the sample feature map to obtain a redundant information suppressed sample feature map; S4, performing feature decoding on the redundant information suppressed sample feature map to obtain a sample decoding feature map; S5, reconstructing the details of the sample decoded feature map to obtain a derained image.

[0031] Specifically, the image deraining network constructed in S1 includes three components: encoder, decoder, and detail reconstruction network. The image deraining network architecture refers to Figure 2 shown.

[0032] The encoder network consists of a Refined Transformer-like Convolutional Block (RCB). After processing the input sample image, the RCB performs a downsampling operation on the processed sample image. Each RCB consists of a Spatial Channel Coupling Attention Block (SCCAB) and a Multi-Level Parallel Feed-Forward Network (MLPFFN). The SCCAB learns the local features of the input sample image while also considering channel dimensionality, achieving simultaneous attention to local contextual information and channel weights. The MLPFFN achieves complementary fusion of features at different levels, ensuring a rich and accurate information flow.

[0033] In this specific embodiment, the connection between the encoder and decoder uses a convolution gated connection (CGC). This convolution gated connection combines convolution operations with a gating mechanism to effectively filter and transmit information. It first performs a convolution operation on the input data and processes it using an activation function to enhance information exchange. The activation function result is then element-wise multiplied with another convolution result that has not been processed by the activation function, ultimately outputting a result that has suppressed redundant information. This convolution gated connection suppresses redundant information in the transmission of sample feature maps from the encoder to the decoder, ensuring that useful feature information flows into the decoder.

[0034] The decoder network consists of a Transformer-like convolution block and an upsampling operation. Similarly, the Transformer-like convolution block in the decoder is also composed of a spatial channel coupled attention module and a multi-stage parallel feedforward network. The decoder first extracts the sample feature map that has undergone redundant information suppression processing through the Transformer-like convolution block and performs feature extraction processing on it. Subsequently, the sample feature map that has undergone feature extraction processing will be upsampled.

[0035] The detail reconstruction network includes a long short-term memory (LSTM) module and a detail refinement reconstruction unit (DRRU). The LSTM module is used to implement long-term cross-stage propagation dependencies, while the DRRU consists of a multi-scale aggregation block (MSAB) and a spatial pixel-wise attention block (SPWAB) to further aggregate image contextual information and improve image details, thereby further enhancing the image deraining and restoration performance and adaptability of the image deraining network.

[0036] Specifically, S2 uses an image deraining network to obtain a sample image, performs feature encoding on the sample image, and obtains a sample feature map, including: An image deraining network is used to obtain sample images, which are then processed using the encoder in the image deraining network. The sample images are then passed through the Transformer-like convolutional block in the encoder for feature extraction. The feature map processed by the Transformer-like convolution block is downsampled to complete feature encoding and obtain a sample feature map.

[0037] The image deraining network first obtains a sample image, which is then input into the encoder in the image deraining network. The encoder in this specific embodiment includes a Transformer-like convolution block and an upsampling operation. In the encoder, the sample image passes through a Transformer-like convolution block, which includes a spatial channel coupling attention module and a multi-stage parallel feedforward network. The spatial channel coupling attention module can simultaneously focus on local context information and channel dimension information to achieve adaptive modeling of rain mark degradation features; the multi-stage parallel feedforward network uses depthwise convolutions of multiple mixed sizes to extract feature information at different scales. The sample feature map processed by the Transformer-like convolution block will then be processed by a downsampling operation. It should be noted that the downsampling operation includes but is not limited to pooling, convolution and other processing methods, the purpose of which is to reduce the spatial dimension of the sample feature map. After the downsampling operation, the feature encoding process is completed, and the sample feature map is obtained.

[0038] Among them, the specific implementation steps of the spatial channel coupling attention module refer to Figure 3 As shown in the figure, the Hadamard product between the spatial channel coupled attention and the value vector V is calculated and then implemented through residual convolution. The spatial channel coupled attention is the coupling weight between the spatial context information and the channel feature relationship. It is obtained by summing the dual large kernel convolutional attention and the channel attention, and then performing 1×1 convolution to reduce the dimension. The formulas for the value vector V, the spatial channel coupled attention, and the spatial channel coupled attention module are as follows:

[0039]

[0040]

[0041] Among them, SCCA is the space channel coupling attention (SCCA), SCCAB is the space channel coupling attention module, is the Hadamard product, DLKCA is the dual large kernel convolution attention, CA is the channel attention, and the input feature , that is, the input feature X is a three-dimensional real tensor with H rows, W columns and C channels, V is a value vector, yes Normalization layer, The convolution layer is 1×1. The dual large kernel convolution attention uses deep convolution with kernel size of 7×7 and 11×11 to fully learn the sample image features. Then, the deep convolution of the two large kernels is connected, and the dimension is doubled to achieve the fusion and representation of features of different scales. Then, the dimension is reduced by 1×1 convolution. The four-times GeLU activation function is used in the whole process to strengthen information interaction and improve the fitting effect of the spatial channel coupling attention module. The value vector V is a linear convolution layer of 1×1. For a given input , the formula of dual large kernel convolution attention is as follows:

[0042] In the formula is a 1×1 convolution, , They are depth convolutions with kernel sizes of 7×7 and 11×11 respectively. For the connection operation, is the activation function.

[0043] The multi-level parallel feedforward network adopts depth convolution of mixed sizes of 1×1, 3×3 and 5×5, as well as mixed operations and calculations of connection, addition and convolution multiplication to further enhance the expression and fusion of features. , the multi-level parallel feedforward network formula is expressed as follows:

[0044]

[0045] in, is a 1×1 convolution, yes Normalization layer, given input After layer normalization, it becomes the intermediate output. For connection operation. , , The depth convolutions are 1×1, 3×3, and 5×5 respectively.

[0046] Specifically, the S3 image deraining network performs redundant information suppression on the sample feature map to obtain a redundant information suppressed sample feature map, including: In the image deraining network, the connection between the encoder and the decoder adopts convolutional gated connection. The sample feature map is transmitted from the encoder to the decoder, and the redundant information is suppressed through the screening mechanism of convolutional gated connection to obtain the redundant information suppressed sample feature map.

[0047] In this specific embodiment, convolutional gated connection is used as the connection between the encoder and the decoder. This is because convolutional gated connection can not only realize the connection between the encoder and the decoder, but also filter the information in the sample feature map through the gating mechanism, suppress redundant information, and ensure the transmission of useful feature information. In this specific embodiment, the image processed by the Transformer-like convolutional block in each level of encoder, that is, the image after feature extraction, is processed by the Transformer-like convolutional block in each level of encoder. Figure 4 The upper and lower paths shown are multiplied and output separately. One path performs no operation; the other uses depthwise convolution to encode spatially adjacent pixel information, extracts features, and then performs normalization using a sigmoid activation function. The result of the multiplication of the two paths is added to the features output by the 3×3 depthwise convolution and then concatenated with the Transformer-like convolutional blocks (RCBs) in each decoder layer along the channel dimension. The formula for the convolutional gated connection is as follows:

[0048] Among them, CGC is a convolutional gated connection. is the Transformer-like convolutional block RCB in the encoder, is the Transformer-like convolutional block RCB in the decoder, is the Hadamard product, For the connection operation, It is a 3×3 depth convolution.

[0049] Specifically, S4 performs feature decoding on the redundant information suppressed sample feature map to obtain a sample decoding feature map, including: The decoder in the image deraining network is used to process the redundant information suppression sample feature map, and the redundant information suppression sample feature map is subjected to feature extraction by the Transformer-like convolution block in the decoder; The feature map processed by the Transformer-like convolution block is upsampled to complete feature decoding and obtain a sample feature decoding map.

[0050] The feature extraction process performed by the Transformer-like convolutional block is similar to that in the encoder, except that the input here is a redundant information suppression sample feature map. This redundant information suppression sample feature map is input into the decoder of the image deraining network. The main function of the decoder is to decode the encoded features and gradually restore the image structure and detail information. In the decoder, the redundant information suppression sample feature map passes through a Transformer-like convolutional block, which includes a spatial channel coupling attention module and a multi-stage parallel feedforward network. The spatial channel coupling attention module can simultaneously focus on local contextual information and channel dimension information, achieving adaptive modeling of rain streak degradation features; the multi-stage parallel feedforward network uses depthwise convolutions of multiple mixed sizes to extract feature information at different scales.

[0051] Subsequently, the redundant information suppressed sample feature map processed by the Transformer-like convolution block will be upsampled to increase the spatial dimension of the feature map, gradually restoring it to a size close to the original image while ensuring the integrity of the feature information, and finally completing the feature decoding.

[0052] Specifically, the S5 image deraining network reconstructs the details of the sample decoded feature map to obtain a derained image, including: The detail reconstruction network in the image deraining network is used to reconstruct the details of the sample feature decoding map. The detail reconstruction network includes a long short-term memory module and a detail improvement reconstruction unit. The sample feature decoding map is passed through the long short-term memory module and the detail improvement reconstruction unit to achieve image detail optimization and obtain a derained image.

[0053] In this specific implementation, the detail reconstruction network primarily comprises a long short-term memory (LSTM) module and a detail improvement and reconstruction unit. Its purpose is to further aggregate image contextual information and improve and optimize image details. The LSTM module's structure primarily consists of four components: a forget gate, an input gate, an input conditioning gate, and an output gate. The forget gate determines which information is forgotten, the input gate determines which new information is written to the memory unit, the input conditioning gate generates candidate information, and the output gate determines the latest output result and updates the current hidden state.

[0054] The detail improvement and reconstruction unit consists of a multi-scale aggregation block and a spatial pixel attention module. The multi-scale aggregation block first learns rain image features at multiple scales through parallel standard convolutional layers of 1×1, 3×3, and 5×5. This is then passed to 3×3 dilated convolutional layers with dilation factors of 1, 2, and 5 for further processing and learning of rain streak distribution features. The use of multi-factor dilated convolution enhances adaptability to different rain streaks and improves generalization. Next, adjacent branches are concatenated along the channel dimension to form two new branches, fusing features at different scales. These two new branches also require feature representation enhancement, but this process is performed using the Ghost module to extract features at different scales. After the outputs of these two new branches undergo dimensionality halving by the Ghost module, they are concatenated for fusion. The fused branches are then dimensionalized again by the Ghost module, resulting in a double dimensionality halving. This simple linear operation of the Ghost module enhances the richness of feature representation without significantly increasing computational burden, optimizing feature processing efficiency and effectiveness. Specifically, the formula is as follows:

[0055] in, , , They are 3×3 dilated convolutions with expansion factors of 1, 2, and 5, , , They are 1×1, 3×3, and 5×5 standard convolutions, Indicates a connection operation. is the input feature of the multi-scale aggregation block.

[0056] The spatial pixel attention module in the detail improvement reconstruction unit consists of the spatial attention block (SAB) and the pixel-wise attention block (PWAB) proposed in this paper.

[0057] The spatial attention block first processes the features of each region of the input features using average pooling and max pooling. After the two pooling operations, the two feature maps of size H × W × C are concatenated along the channel dimension. Next, a 7 × 7 convolutional layer and a sigmoid function are used to generate a normalized spatial attention weight map. Finally, the spatial attention weight map is multiplied by the input features to compress the channels of the weighted attention map of the input feature spatial information, aggregating pixels at the same horizontal coordinate in each channel into a single value.

[0058] Pixel attention block structure reference Figure 5As shown, the spatial pixel attention module can be expressed as:

[0059] Among them, SPWAB is the spatial pixel attention module, SAB is the spatial attention block, and PWAB is the pixel attention block.

[0060] In the pixel attention block, two 1×1 convolutions are used to achieve pixel-by-pixel attention to image features of size H×W×C, focusing on image texture details. The ReLU activation function is introduced to enhance the nonlinear expression of features. Finally, a normalized pixel weight map is obtained through the Sigmoid activation function. The normalized pixel weight map clarifies the weight value of the pixel information in the feature map that should be paid attention to, thereby performing pixel-by-pixel weighting processing on the input features, improving the image deraining network's ability to capture the image's semantic texture. The pixel attention block formula is expressed as:

[0061] in, is a 1×1 convolution, ReLU is the activation function, is the input feature.

[0062] When training the image deraining network provided in this embodiment, the clear image is degraded and randomly shuffled to form a training set and a test set. The training set is input into the image deraining network for processing until the output matches the image before degradation, thereby obtaining a trained image deraining network. Specifically, as shown below: Input the degraded and clear image pair to be restored , N is the sample size, is the degraded (rainy) image in the i-th training sample, is the clear (rain-free) image in the i-th training sample. In this specific implementation, 128×128 randomly cropped blocks are used to train the image deraining network. The dataset is randomly shuffled and divided into training and test sets.

[0063] The sample image is then input into the encoding part of the untrained image deraining network, and feature encoding is achieved through Transformer-like convolutional block processing and downsampling operations to obtain the training set feature map. The training set feature maps are then connected through convolutional gates in the encoder and decoder to suppress redundant information, ensuring that useful feature information flows into the decoder to assist in decoding. The feature maps of the training set with redundant information suppressed are input to the decoder part of the image deraining network, and feature decoding is completed through Transformer-like convolutional block processing and upsampling operations. The feature maps of the training set that have undergone feature decoding will be input into the detail reconstruction network, and will pass through the long short-term memory module and the detail improvement reconstruction unit in sequence to further aggregate the contextual information of the image and improve and optimize the image details, thereby enhancing the image deraining restoration effect and adaptability of the image deraining network model. When the deraining effect of the output training set feature map meets the standard, the training of the image deraining network is completed, and an image deraining network model based on spatial channel coupled attention is obtained.

[0064] Then the test set is input into the trained image deraining network model based on spatial channel coupled attention, and the image deraining network model based on spatial channel coupled attention is tested. Specifically, the trained image deraining network model based on spatial channel coupled attention is used to restore the image on the constructed test set.

[0065] The total loss function of the network training process consists of three sub-functions: Charbonnier penalty loss function, Edge loss function, and Mean Absolute Error (MAE). The formula is as follows:

[0066] Among them, L is the total loss function, is the Charbonnier penalty loss function, is the Edge loss function, is the mean absolute error loss function.

[0067] The three sub-function expressions are as follows:

[0068]

[0069]

[0070] in, is the Laplace filtering operation, is a constant term with a value of 0.001. is the restored image, is a clear (rain-free) image, is the sample size.

[0071] The Charbonnier penalty loss function maintains good convergence during training, while the Edge loss function further improves the realism and accuracy of high-frequency details. The mean absolute error loss function enhances robustness, reduces the impact of extreme values, and enables the image deraining network model based on spatial channel coupled attention to learn sparse weights.

[0072] The image deraining method based on spatial channel coupled attention proposed in the present invention effectively overcomes the problem of poor image deraining effect in the existing technology due to single feature extraction, insufficient detail recovery ability and poor universality by means of multi-dimensional feature extraction, enhanced detail recovery and improved model universality. In terms of feature extraction, the spatial channel coupled attention module can simultaneously focus on local context information and channel dimension information to achieve adaptive modeling of rain mark degradation features. The multi-level parallel feedforward network uses deep convolution of multiple mixed sizes to extract feature information of different scales, thereby avoiding the singleness of feature extraction. In terms of detail recovery, the detail improvement and reconstruction unit combines multi-scale aggregation blocks and spatial pixel attention to perform weighted attention on image features from the spatial dimension and pixel dimension respectively, realizing pixel-by-pixel weighted processing. The long short-term memory module uses long-term context information to guide detail reconstruction, which significantly improves the detail recovery effect. In addition, the convolutional gated connection between the encoder and decoder suppresses redundant information and ensures the transmission of useful feature information. The comprehensive loss function balances the overall error, edge details and robustness, so that the image deraining model can maintain stable deraining performance in different scenarios and conditions, effectively solving the problem of poor universality of existing technologies.

[0073] Reference Figure 6 As shown, a specific embodiment of the present invention further provides an image deraining system based on spatial channel coupled attention, comprising: A network construction module is used to construct an image deraining network. The image deraining network includes an encoder, a decoder, and a detail reconstruction network. The encoder and decoder include a spatial channel coupling attention module. A feature encoding module is used to obtain a sample image using an image deraining network, perform feature encoding on the sample image, and obtain a sample feature map; An information suppression module is used to perform redundant information suppression processing on the sample feature map to obtain a redundant information suppressed sample feature map; A feature decoding module is used to perform feature decoding on the redundant information suppression sample feature map to obtain a sample decoding feature map; The detail reconstruction module is used to reconstruct the details of the sample decoding feature map to obtain a derained image.

[0074] A specific embodiment of the present invention further provides a computer device. Specifically, the computer device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal and is suitable for implementing one or more instructions, specifically for loading and executing one or more instructions to implement corresponding method processes or corresponding functions. The processor described in the embodiment of the present invention can be used to construct an image deraining network, which includes an encoder, a decoder, and a detail reconstruction network, wherein the encoder and the decoder include a spatial channel coupled attention module. The image deraining network is used to obtain a sample image, perform feature encoding on the sample image, and obtain a sample feature map. The sample feature map is subjected to redundant information suppression processing to obtain a redundant information suppressed sample feature map. The redundant information suppressed sample feature map is subjected to feature decoding to obtain a sample decoded feature map. The sample decoded feature map is subjected to detail reconstruction to obtain a derained image.

[0075] A storage medium is also provided in a specific embodiment of the present invention. Specifically, it is a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the terminal device and, of course, extended storage media supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the relevant methods in the above embodiments; one or more instructions in the computer-readable storage medium are loaded by the processor and the following steps are executed: constructing an image deraining network, the image deraining network includes an encoder, a decoder and a detail reconstruction network, wherein the encoder and the decoder include a spatial channel coupling attention module; using the image deraining network to obtain a sample image, feature encoding the sample image, and obtaining a sample feature map; performing redundant information suppression processing on the sample feature map to obtain a redundant information suppressed sample feature map; performing feature decoding on the redundant information suppressed sample feature map to obtain a sample decoded feature map; and performing detail reconstruction on the sample decoded feature map to obtain a derained image.

[0076] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0078] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0080] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, from all points of view, the embodiments should be regarded as illustrative and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and range of equivalents of the claims are included in the present invention. Any reference signs in the claims should not be construed as limiting the claim to which they relate.

[0081] In addition, it should be understood that although this specification describes the embodiments, not every embodiment contains only one independent technical solution. This description is for clarity only. Those skilled in the art should consider the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art. The above content is only for the purpose of illustrating the technical concept of the present invention and cannot be used to limit the scope of protection of the present invention. Any changes made based on the technical solution in accordance with the technical concept proposed by the present invention fall within the scope of protection of the claims of the present invention.

Claims

1. An image deraining method based on spatial channel coupled attention, characterized in that: include: Construct an image deraining network, which includes an encoder, a decoder, and a detail reconstruction network. The encoder and decoder include a spatial channel coupling attention module. An image deraining network is used to obtain a sample image, and feature encoding is performed on the sample image to obtain a sample feature map. Performing redundant information suppression processing on the sample feature map to obtain a redundant information suppressed sample feature map; Performing feature decoding on the redundant information suppressed sample feature map to obtain a sample decoding feature map; The sample decoded feature map is reconstructed in detail to obtain the derained image.

2. The image deraining method based on spatial channel coupled attention according to claim 1, characterized in that: The method of using an image deraining network to obtain a sample image, performing feature encoding on the sample image, and obtaining a sample feature map includes: An image deraining network is used to obtain sample images, which are then processed using the encoder in the image deraining network. The sample images are then passed through the Transformer-like convolutional block in the encoder for feature extraction. The feature map processed by the Transformer-like convolution block is downsampled to complete feature encoding and obtain the sample feature map.

3. The image deraining method based on spatial channel coupled attention according to claim 2, characterized in that: The Transformer-like convolutional block includes a spatial channel coupled attention module and a multi-stage parallel feedforward network.

4. The image deraining method based on spatial channel coupled attention according to claim 1, characterized in that: The performing redundant information suppression processing on the sample feature map to obtain the redundant information suppressed sample feature map includes: In the image deraining network, the connection between the encoder and the decoder adopts convolutional gated connection. During the transmission of the sample feature map from the encoder to the decoder, the redundant information is suppressed through the screening mechanism of convolutional gated connection to obtain the redundant information suppressed sample feature map.

5. The image deraining method based on spatial channel coupled attention according to claim 1, characterized in that: The feature decoding of the redundant information suppressed sample feature map to obtain the sample decoding feature map includes: The decoder in the image deraining network is used to process the redundant information suppression sample feature map, and the redundant information suppression sample feature map is subjected to feature extraction by the Transformer-like convolution block in the decoder; The feature map processed by the Transformer-like convolution block is upsampled to complete feature decoding and obtain a sample feature decoding map.

6. The image deraining method based on spatial channel coupled attention according to claim 1, characterized in that: The detailed reconstruction of the sample decoding feature map to obtain the rain-free image includes: The detail reconstruction network in the image deraining network is used to reconstruct the details of the sample feature decoding map. The detail reconstruction network includes a long short-term memory module and a detail improvement reconstruction unit. The sample feature decoding map is passed through the long short-term memory module and the detail improvement reconstruction unit to achieve image detail optimization and obtain a derained image.

7. The image deraining method based on spatial channel coupled attention according to claim 6, characterized in that: The detail improvement reconstruction unit consists of a multi-scale aggregation block and a spatial pixel attention module; the spatial pixel attention module consists of a spatial attention block and a pixel attention block.

8. An image deraining system based on spatial channel coupled attention, characterized in that: include: A network construction module is used to construct an image deraining network. The image deraining network includes an encoder, a decoder, and a detail reconstruction network. The encoder and decoder include a spatial channel coupling attention module. A feature encoding module is used to obtain a sample image using an image deraining network, perform feature encoding on the sample image, and obtain a sample feature map; An information suppression module is used to perform redundant information suppression processing on the sample feature map to obtain a redundant information suppressed sample feature map; A feature decoding module is used to perform feature decoding on the redundant information suppression sample feature map to obtain a sample decoding feature map; The detail reconstruction module is used to reconstruct the details of the sample decoding feature map to obtain a derained image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the image deraining method based on spatial channel coupled attention are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the image deraining method based on spatial channel coupled attention are implemented.

Images