A pyramid-based method for SAR image speckle suppression

Abstract

The present invention provides a pyramid-based SAR image coherent speckle suppression method. First, a residual block formed by cascading four different residual modules is added to the noise estimation stage, which can make fuller use of the residual modules of different depths to extract feature information to promote the learning process; then, multi-scale feature information is sampled to obtain a five-level pyramid, which can make full use of the receptive fields of different scales to extract diversified feature information in different scale spaces; finally, the obtained multi-scale feature information is fused and denoised to obtain a denoised image, and the context information is helpful for retaining detail information of the denoised image. The present invention uses a residual block formed by cascading four different residual modules and a pyramid-based thinking to perform multi-scale feature extraction, fusion and denoising on SAR images, realizes denoising of real SAR images, and effectively retains the detail features of the denoised image, showing excellent denoising performance and having wide applicability.

Description

Technical Field

[0001] The present invention relates to image processing technology, and in particular to a pyramid-based SAR image speckle suppression method, which uses a residual block formed by cascading four different residual modules and a pyramid-based approach to perform multi-scale feature extraction, fusion and denoising on SAR images. Technical Background

[0002] Synthetic Aperture Radar (SAR) is an all-weather, all-day, high-resolution imaging sensor. Its unique imaging properties have led to its widespread use in military and civilian applications, including military mapping, ocean monitoring, resource exploration, and oil spill detection. However, the coherent imaging characteristics of SAR introduce a type of noise called "coherent speckle" during the imaging process. This noise severely degrades the readability of the image and poses a major challenge to downstream SAR imaging tasks such as target detection and classification. Therefore, effectively suppressing coherent speckle noise in SAR images is crucial for its subsequent applications. Although traditional methods have been widely used in coherent speckle suppression, it is worth noting that such methods are sensitive to the size of the search window and pixel blocks, and the denoising performance will degrade in the presence of severe noise. Therefore, the introduction of deep learning, especially convolutional neural networks (CNNs), has achieved remarkable success in coherent speckle suppression of SAR images due to its powerful nonlinear feature extraction capabilities.

[0003] The research on deep learning in SAR image speckle suppression mainly focuses on fixed receptive fields, which makes the extracted feature information lack diversity, especially the image's detail feature texture, edges and other subtle structural information. These detail features are crucial to the visual quality and perception of the image. To solve these problems, the residual module is fully utilized to effectively extract information at different network depths to promote the learning ability of the model. The extraction of contextual information features enriched by receptive fields of different scales and the research on multi-scale feature fusion and denoising have become the research focus of SAR image speckle suppression. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of suppressing coherent speckles in SAR images. An image pair consisting of an image containing simulated noise and an original image without noise in the same scene is provided. A method for extracting and fusing noise features through a deep learning network is used to learn and then denoise the noisy image. The method can be directly applied to real SAR images to obtain denoised SAR images.

[0005] In order to achieve the above-mentioned purpose, the present invention provides a pyramid-based SAR image coherent speckle suppression method, that is, using a residual block formed by cascading four different residual modules and a pyramid-based thinking to perform multi-scale feature extraction, fusion and denoising on the SAR image, which mainly includes five parts: the first part is to preprocess the data set; the second part is the noise estimation of the noise image; the third part is the multi-scale feature extraction of the noise image; the fourth part is the multi-scale denoising and feature fusion of the noise image; the fifth part is the network training and the test of the real SAR image, and finally the denoised SAR image is obtained.

[0006] The first part consists of two steps:

[0007] Step 1: Download the publicly available dataset of simulated SAR image speckle noise, select simulated image pairs (original images and simulated speckle noise images) with rich scenes, diverse details and appropriate speckle noise levels as training sets and validation sets, then crop the image pairs into image blocks of one-to-one corresponding sizes, expand the sample set, and form the final training set and validation set samples;

[0008] Step 2: Download the public SAR image dataset and select real SAR images with similar scenes and noise levels as the training set and validation set as test set samples for the network;

[0009] The second part consists of three steps:

[0010] Step 3: Input the training samples and verification samples in step 1 into the weight-sharing convolutional network, and preliminarily obtain the feature tensor of the noise image by transforming the number of channels;

[0011] Step 4, the tensor obtained in step 3 is subjected to a residual block cascaded with four different residual modules and a SimAM attention to extract discriminative features to obtain a feature tensor of estimated noise;

[0012] Step 5: For the attention feature tensor obtained in step 4, the final convolutional layer has 3 output channels, and the network output is represented as a 3-channel feature tensor;

[0013] The third part consists of three steps:

[0014] Step 6, performing a Concatenate operation on the feature tensor obtained in step 5 and the tensor of the input image to obtain a fused feature tensor;

[0015] Step 7: For the feature tensor obtained in step 6, a five-level pyramid is obtained by performing a maximum pooling downsampling operation, namely pool1:

[0016] The feature tensors are of five scales: 256*256, pool2: 128*128, pool3: 64*64, pool4: 32*32, and pool5: 16*16;

[0017] Step 8: The high-level features of pool5 obtained in step 7 are processed by an EVC block (mainly composed of two parallel connected blocks:

[0018] Lightweight MLP and learnable visual centers);

[0019] Part 4 consists of three steps:

[0020] Step 9: Perform denoising and feature fusion at five different scales on the feature tensors obtained in steps 7 and 8;

[0021] Step 10, performing a convolution operation on the feature tensor obtained in step 6 and the feature tensor obtained in step 9 by using the Concatenate operation to fuse the feature tensor;

[0022] Step 11, performing a convolution kernel selection module operation on the feature tensor obtained in step 10;

[0023] Part 5 consists of two steps:

[0024] Step 12, debug the network structure hyperparameters from step 3 to step 11, set the network model parameters, and obtain the final training model;

[0025] Step 13, input the test set in step 2 into the training model in step 12 to obtain a SAR image after speckle suppression.

[0026] The present invention provides a pyramid-based SAR image coherent speckle suppression method. First, a residual block formed by cascading four different residual modules is added to the noise estimation stage, which can make fuller use of the abstraction and extraction of feature information by residual modules of different depths to promote the learning process; then, multi-scale feature information is sampled to obtain a five-level pyramid, which can make full use of the receptive fields of different scales to extract diversified feature information in different scale spaces; finally, the obtained multi-scale feature information is fused and denoised to obtain a denoised image, and the context information is very helpful for retaining detail information of the denoised image. The present invention uses a residual block formed by cascading four different residual modules and a pyramid-based thinking to perform multi-scale feature extraction, fusion and denoising on SAR images, realizes effective denoising of real SAR images, and effectively retains the detail features of the denoised image, showing an excellent denoising effect, high operating efficiency, and good robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is the overall framework diagram of the network of the present invention;

[0028] Figure 2 A residual block module formed by cascading four different residual modules of the present invention;

[0029] Figure 3 The EVC module of the present invention;

[0030] Figure 4 are simulated image pairs in the training set and the validation set;

[0031] Figure 5 This is the result of using the present invention to process real SAR images in the test set. DETAILED DESCRIPTION

[0032] In order to better understand the present invention, the following is a more detailed description of the method of the present invention for extracting, fusing and denoising multi-scale features of SAR images using a residual block cascaded by four different residual modules and a pyramid-based thinking in combination with a specific embodiment. In the following description, the detailed description of the current prior art may dilute the subject content of the present invention, and these descriptions will be ignored here.

[0033] Figure 1 It is a specific network model diagram of the method of the present invention for extracting, fusing and denoising multi-scale features of SAR images by using a residual block cascaded by four different residual modules and a pyramid-based thinking. In this embodiment, the following steps are followed:

[0034] Step 1: Download the publicly available dataset of simulated SAR image speckle noise, select simulated image pairs (original images and simulated speckle noise images) with rich scenes, diverse details and appropriate speckle noise levels as training sets and validation sets, then crop the image pairs into image blocks of one-to-one corresponding sizes, expand the sample set, and form the final training set and validation set samples 101, as shown in Figure 4 As shown;

[0035] Step 2, downloading a public SAR image dataset, and selecting real SAR images with similar scenes and noise levels to the training set and the validation set as test set samples 102 of the network;

[0036] Step 3, input the training samples and verification samples in step 1 into 1*1 convolution (the convolution layer is normalized by BN and nonlinearized by ReLU activation function) to preliminarily obtain the feature tensor 103 of the noise image through channel number transformation;

[0037] Step 4, extracting discriminative features from the feature tensor 103 through a residual block cascaded with four different residual modules and a SimAM attention to obtain a feature tensor 104 of estimated noise, which is specifically implemented as follows:

[0038] Step 4-1, the residual block consists of Start_block, Middle_block1, Middle_block2 and End_block, which have 64 feature channels, such as Figure 2 As shown in the figure; the first layer of Start_block is a 1*1 convolutional layer (the convolutional layer is normalized by BN and nonlinearized by ReLU activation function), the middle layer is a 3*3 convolutional layer with a step size of 1 (the convolutional layer is normalized by BN and nonlinearized by ReLU activation function), and the last layer is a 1*1 convolutional layer (the convolutional layer is normalized by BN)201; Middle_block1 is first nonlinearized by ReLU activation function, and then passes through a 1*1 convolutional layer (the convolutional layer is normalized by BN and ReLU activation function The convolutional layer is then normalized by BN and nonlinearized by ReLU activation function, and finally a simple 1*1 convolutional layer 202. Middle_block2 adds a BN layer for normalization at the beginning compared to Middle_block1, and the other structures are the same 203. End_block has the same structure as Middle_block2, but adds a BN layer for normalization and a ReLU activation function layer for nonlinearization in the trunk path 204.

[0039] Step 4-2, SimAM attention, generates unified spatial and channel 3-D weights. In order to evaluate the importance of each pixel, an energy function is defined, as shown in formula (1):

[0040]

[0041] in, For simplicity, binary labels are used and regularization terms are added to obtain the final energy function, as shown in formula (2):

[0042]

[0043] Theoretically, each channel has M = H × W energy functions. The above formula has the following analytical solution, as shown in formula (3):

[0044]

[0045] in, Therefore, the minimum energy can be obtained by the following formula, as shown in formula (4):

[0046]

[0047] According to the definition of the attention mechanism, the features need to be enhanced, as shown in formula (5):

[0048]

[0049] That is, the attention feature tensor 104 is returned by expansion and fusion;

[0050] Step 5, the feature tensor 104 has 3 output channels through the last convolutional layer, and the network output is represented as a 3-channel feature tensor 105;

[0051] Step 6, performing a concatenation operation on the feature tensor 105 and the input image tensor 101 to obtain a fused 6-channel feature tensor 106;

[0052] Step 7, the feature tensor 106 is subjected to a maximum pooling downsampling operation, and the pooling kernel is set to 2×2 to obtain a five-level pyramid, and the 6-channel feature tensors of five scales of pool1: 256*256, pool2: 128*128, pool3: 64*64, pool4: 32*32, and pool5: 16*16 are 107, 108, 109, 110, and 111 respectively;

[0053] Step 8, the pool5 advanced feature 111 is operated through an EVC block, such as Figure 3 As shown (mainly composed of two parallel connected blocks: lightweight MLP and learnable visual center), the feature tensor 112 is obtained, which is specifically implemented as follows:

[0054] Step 8-1, the lightweight MLP 301 in the EVC block consists of two residual modules: a GSConv-based module and a channel-based MLP module, the input of the latter is the output of the former; GSConv-based module: the input 6-channel 16*16 features are first subjected to group normalization and a 1*1 convolution to obtain 24-channel 16*16 features, then passed through a 24-channel GSConv, and finally passed through 1*1 convolution, channel scaling operation and DropPath operation;; Channel-based MLP module: the features obtained by the previous module are first sent to group normalization, then passed through channel MLP, and then passed through channel scaling operation and DropPath operation;

[0055] Step 8-2, the learnable visual center 302 in the EVC block consists of two components: an intrinsic codebook and a set of scaling factors; the input features are first encoded through a set of convolutional layers consisting of 1*1 convolution, 3*3 convolution and 1*1 convolution, and then the encoded features are sent to a CBR block consisting of a 3*3 convolution with a BN layer and a ReLU activation function, and finally the encoded features are input into the codebook, and then the scaling factor S is used to map the pixels and the learnable visual code words to the corresponding position information, and then all the codeword information in the image is fused through a module composed of a BN layer, a ReLU layer and a mean layer, and then the obtained codebook is sent to a fully connected layer, a 1*1 convolution and a Sigmoid activation function to obtain the scaling factor coefficient, and then multiplied with the input feature channel to obtain the local feature, and finally the obtained local feature is added to the input feature channel;

[0056] Step 8-3, use the Concatenate operation to fuse the features obtained in step 8-1 and step 8-2, and then send them to 1*1 convolution to represent the network output as a 6-channel feature tensor;

[0057] Step 9, denoising and feature fusion of five different scales are performed on the feature tensors 107, 108, 109, 110, and 112, and the specific implementation is as follows:

[0058] Step 9-1, the feature tensor 112 is sent to the U-Net network for denoising to obtain a 3-channel 16*16 denoised feature, and then the denoised feature is upsampled to a 3-channel 32*32 feature tensor 113 using bilinear interpolation;

[0059] Step 9-2, the feature tensors 110, 112 and 113 are fused using the Concatenate operation, and then sent to a 1*1 convolution, and then the obtained feature tensor is sent to the U-Net network for denoising to obtain a 3-channel 32*32 denoised feature 114, and then the denoised feature is upsampled to a 3-channel 64*64 feature tensor 114 using bilinear interpolation;

[0060] Step 9-3, use the Concatenate operation to fuse the feature tensors 109, 112, and 114, then send them to a 1*1 convolution, and then send the obtained feature tensor to the U-Net network for denoising to obtain a 3-channel 64*64 denoised feature, and then use bilinear interpolation to upsample the denoised feature to a 3-channel 128*128 feature tensor 115;

[0061] Step 9-4, use the Concatenate operation to fuse the feature tensors 108, 112, and 115, then send them to a 1*1 convolution, and then send the obtained feature tensor to the U-Net network for denoising to obtain a 3-channel 128*128 denoised feature, and then use bilinear interpolation to upsample the denoised feature to a 3-channel 256*256 feature tensor 116;

[0062] Step 9-5, use the Concatenate operation to fuse the feature tensors 107 and 116, then send them to a 1*1 convolution, and then send the obtained feature tensor to the U-Net network for denoising to obtain a 3-channel 256*256 denoised feature 117;

[0063] Step 10, performing a convolution operation 118 on the feature tensor 106 and the feature tensor 117 by using the Concatenate operation to fuse the feature tensors;

[0064] Step 11, performing a convolution kernel selection module operation on the feature tensor 118, and finally mapping the feature tensor to a grayscale space 119;

[0065] Step 12, debug the network structure hyperparameters from step 3 to step 11, set the network model parameters, where the initial learning is set to 0.0001, Epochs is set to 150, the learning rate is reduced to 1 / 5 of the original after every 50 Epochs, Bachsize is set to 16, and the final training model is obtained;

[0066] Step 13, input the test set 102 into the training model in step 12 to obtain the SAR image 120 after the coherent speckle is suppressed, as shown in FIG. Figure 5 shown.

[0067] The present invention introduces the idea of ​​multi-scale receptive field of pyramid based on deep learning denoising network, and provides a pyramid-based SAR image coherent speckle suppression method. The method is more sufficient to extract discriminative noise features, adopts four different residual modules cascaded into residual blocks and introduces SimAM attention, obtains network information from different depth operations, and assigns different weights according to different noise features to obtain more accurate estimated noise feature tensors; then, the multi-scale feature information of the sampling is introduced into the EVC module in the high-level features to focus on the global long-range dependency (i.e., global information) of the high-level features, while retaining the local corner point area (i.e., local information) to strengthen the extraction of detail features; finally, the obtained multi-scale feature information is used for fusion and denoising, and the fusion of high-level features and low-level features is used to enrich the context information, which is very helpful for retaining detail information of the denoised image, and then the convolution kernel selection module operation is introduced, and multiple branches with different convolution kernel sizes are fused through linear combination, allowing different feature maps to be expressed by kernels of different sizes, and then mapped to the grayscale space to obtain the denoised image. The algorithm of the present invention is simple, operable, and has wide applicability.

[0068] Although the above describes an illustrative specific embodiment of the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiment. For ordinary technicians in this technical field, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A pyramid-based SAR image speckle suppression method, characterized in that: The residual block composed of four different residual modules and the pyramid-based thinking are used to extract, fuse and denoise multi-scale features of SAR images, including data set preprocessing, image noise estimation, multi-scale feature extraction, multi-scale denoising and feature fusion, network training and testing. The first part consists of two steps: Step 1: Download the publicly available simulated speckle noise dataset, select simulated image pairs with rich scenes, diverse details and appropriate speckle noise levels, which consist of original images and simulated speckle noise images, use the simulated image pairs as training sets and validation sets, then crop the image pairs into image blocks of one-to-one corresponding sizes, expand the sample set, and form the final training set and validation set samples; Step 2: Download the public SAR image dataset and select a real SAR image dataset with similar scenes and noise levels to the training set and validation set as the test set samples of the network; The second part consists of three steps: Step 3: Input the training samples and verification samples in step 1 into the weight-sharing convolutional network, and preliminarily obtain the feature tensor of the noise image by transforming the number of channels; Step 4: The tensor obtained in step 3 is subjected to a residual block cascaded with four different residual modules and a SimAM attention to extract discriminative features to obtain a feature tensor for estimating noise. The specific implementation is as follows: Step 4-1, the residual block consists of Start_block, Middle_block1, Middle_block2 and End_block, which have 64 feature channels; the first layer of Start_block is a 1*1 convolution layer normalized by BN and nonlinearized by ReLU activation function, the middle layer is a 3*3 convolution layer normalized by BN and nonlinearized by ReLU activation function with a step size of 1, and the last layer is a 1*1 convolution layer normalized by BN; Middle_block1 is first nonlinearized by ReLU activation function Linearization, then pass through a 1*1 convolution layer with BN normalization and ReLU activation function for nonlinearization, then pass through a 3*3 convolution layer with BN normalization and ReLU activation function for nonlinearization with a step size of 1, and finally pass through a simple 1*1 convolution layer; Middle_block2 adds a BN layer for normalization at the beginning compared to Middle_block1, and the other structures are the same; End_block has the same structure as Middle_block2, but adds a BN layer for normalization and a ReLU activation function layer for nonlinearization in the trunk path; Step 4-2, SimAM attention, generates 3-D weights with unified space and channels, and then returns the attention feature tensor through expansion and fusion; Step 5: For the attention feature tensor obtained in step 4, the final convolutional layer has 3 output channels, and the network output is represented as a 3-channel feature tensor; The third part consists of three steps: Step 6, performing a Concatenate operation on the feature tensor obtained in step 5 and the tensor of the input image to obtain a fused 6-channel feature tensor; Step 7, for the feature tensor obtained in step 6, the maximum pooling downsampling operation is performed, and the pooling kernel is set to 2×2 to obtain a five-level pyramid, and 6-channel feature tensors of five scales are obtained, namely pool1: 256*256, pool2: 128*128, pool3: 64*64, pool4: 32*32, and pool5: 16*16; Step 8: The pool5 high-level features obtained in step 7 are processed by an EVC block, which consists of two parallel connected blocks: a lightweight MLP and a learnable visual center. The specific implementation is as follows: Step 8-1, the lightweight MLP in the EVC block consists of two residual modules: a GSConv-based module and a channel-based MLP module, the input of the latter is the output of the former; GSConv-based module: the input 6-channel 16*16 features are first subjected to group normalization and a 1*1 convolution to obtain 24-channel 16*16 features, and then passed through a 24-channel GSConv, and then through 1*1 convolution, channel scaling operation and DropPath operation; Channel-based MLP module: the features obtained by the previous module are first sent to group normalization, then through channel MLP, and then through channel scaling operation and DropPath operation; Step 8-2, the learnable visual center in the EVC block consists of two components: an intrinsic codebook and a set of scaling factors; the input features are first encoded through a set of convolutional layers consisting of 1*1 convolution, 3*3 convolution and 1*1 convolution, and then the encoded features are sent to a CBR block consisting of a 3*3 convolution with a BN layer and a ReLU activation function, and finally the encoded features are input into the codebook, and then the scaling factor S is used to map the pixels and the learnable visual code words to the corresponding position information, and then all the codeword information in the image is fused through a module composed of a BN layer, a ReLU layer and a mean layer, and then the obtained codebook is sent to a fully connected layer, a 1*1 convolution and a Sigmoid activation function to obtain the scaling factor coefficient, and then multiplied with the input feature channel to obtain the local feature, and finally the obtained local feature is added to the input feature channel; Step 8-3, use the Concatenate operation to fuse the features obtained in step 8-1 and step 8-2, and then send them to 1*1 convolution to represent the network output as a 6-channel feature tensor; Part 4 consists of three steps: Step 9: Perform denoising and feature fusion at five different scales on the feature tensors obtained in steps 7 and 8. The specific implementation is as follows: Step 9-1, the 6-channel 16*16 feature tensor obtained in step 8 is sent to the U-Net network for denoising, and then the denoised features are upsampled to a 3-channel 32*32 feature tensor using bilinear interpolation; Step 9-2, the pool4 feature tensor obtained in step 7, the feature tensor obtained in step 8, and the feature tensor obtained in step 9-1 are fused using the Concatenate operation, and then sent to a 1*1 convolution, and then the obtained feature tensor is sent to the U-Net network for denoising, and then the denoised features are upsampled to a 3-channel 64*64 feature tensor using bilinear interpolation; Step 9-3, the pool3 feature tensor obtained in step 7, the feature tensor obtained in step 8, and the feature tensor obtained in step 9-2 are fused using the Concatenate operation, and then sent to a 1*1 convolution, and then the obtained feature tensor is sent to the U-Net network for denoising, and then the denoised features are upsampled to a 3-channel 128*128 feature tensor using bilinear interpolation; Step 9-4, the pool2 feature tensor obtained in step 7, the feature tensor obtained in step 8, and the feature tensor obtained in step 9-3 are fused using the Concatenate operation, and then sent to a 1*1 convolution, and then the obtained feature tensor is sent to the U-Net network for denoising, and then the denoised features are upsampled to a 3-channel 256*256 feature tensor using bilinear interpolation; Step 9-5, the pool1 feature tensor obtained in step 7 and the feature tensor obtained in step 9-4 are fused using the Concatenate operation, and then sent to a 1*1 convolution, and then the obtained feature tensor is sent to the U-Net network for denoising to obtain a 3-channel 256*256 denoised feature; Step 10, performing a convolution operation on the feature tensor obtained in step 6 and the feature tensor obtained in step 9 by using the Concatenate operation to fuse the feature tensor; Step 11, the feature tensor obtained in step 10 is operated by the convolution kernel selection module, and then mapped to the grayscale space; the fifth part includes two steps: Step 12, debug the network structure hyperparameters from step 3 to step 11, set the network model parameters, where the initial learning is set to 0.0001, Epochs is set to 150, the learning rate is reduced to 1 / 5 of the original after every 50 Epochs, Bach size is set to 16, and the final training model is obtained; Step 13, input the test set in step 2 into the training model in step 12 to obtain a SAR image after speckle suppression.

2. The pyramid-based SAR image speckle suppression method according to claim 1, characterized in that: In step 4, four different residual modules are cascaded into a residual block to more fully utilize the abstraction and extraction of information by residual modules of different depths to promote the learning process.

3. The pyramid-based SAR image speckle suppression method according to claim 1, characterized in that: In step 7, the downsampling operation of Maxpool is used to obtain a five-level pyramid of different scales, which can make full use of the receptive fields of different scales to extract diverse feature information in different scale spaces.

4. The pyramid-based SAR image speckle suppression method according to claim 1, characterized in that: In step 8, the EVC operation is used to focus on the global long-range dependencies of high-level features, that is, global information, while retaining the local corner areas, that is, local information, to strengthen the extraction of detail features.

5. The pyramid-based SAR image speckle suppression method according to claim 1, characterized in that: In step 9, the Concatenate operation is used to perform feature fusion between feature tensors of adjacent scales, and then the U-Net network is used to suppress coherent speckles. The obtained multi-scale feature information fusion is very helpful for retaining detail information in the denoised image.

Images