Synthetic aperture radar image denoising method and device and terminal equipment

Abstract

The invention is suitable for the technical field of synthetic aperture radars, and provides a synthetic aperture radar image denoising method and device, and terminal equipment. The method comprises the following steps: acquiring a synthetic aperture radar image to be processed; inputting a to-be-processed synthetic aperture radar image into the image denoising model, and outputting the denoised synthetic aperture radar image; the image denoising model comprises a plurality of encoders and a plurality of decoders, and each encoder comprises a PEB module and a DSC-Transform Block module; the PEB module performs weighting processing on the input image through a channel attention mechanism and a space attention mechanism; the DSC-Transform Block calculates the attention weight of the feature map output by the PEB module through a multi-head self-attention mechanism and performs weighted convergence, local feature extraction and nonlinear transformation are performed on the feature map after weighted convergence through MLP, and the processed features and the original features are added by using residual connection; the output of the encoder is transmitted to the next encoder and the corresponding decoder. The speckle noise can be effectively removed, and the texture information of the image can be reserved.

Description

Technical Field

[0001] The present application relates to the field of synthetic aperture radar technology, and in particular to a synthetic aperture radar image denoising method, device and terminal equipment. Background Art

[0002] Radar imaging is a landmark achievement in the field of modern detection science. It has brought about a qualitative change in the basic functions of radar and fundamentally expanded the ability of human beings to observe nature and the living environment. As an important remote sensing technology, SAR (Synthetic Aperture Radar) plays a vital role in fields such as environmental monitoring and resource management. Its unique observation capabilities make the images produced by SAR systems widely used in fields such as geographic information, disaster monitoring and national defense. However, due to the coherent characteristics of the SAR system, multiplicative coherent speckle noise is inevitably generated during the imaging process. The presence of this noise not only seriously reduces the visual quality of SAR images, but also has a negative impact on the accuracy of subsequent image processing and analysis. Therefore, denoising SAR images before interpretation is crucial to improve image quality and enhance the effect of subsequent interpretation processing. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a synthetic aperture radar image denoising method, apparatus and terminal device, which can effectively remove speckle noise while also better retaining the texture information of the image.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, an embodiment of the present application provides a synthetic aperture radar image denoising method, comprising:

[0006] Acquiring a synthetic aperture radar image to be processed;

[0007] Inputting the to-be-processed synthetic aperture radar image into an image denoising model, and outputting a denoised synthetic aperture radar image;

[0008] Among them, the image denoising model includes multiple encoders and multiple decoders connected in sequence, each encoder includes a PEB module and multiple DSC-Transformer Block modules connected in sequence; the PEB module performs weighted processing on the input image from the channel dimension and the spatial dimension respectively through the channel attention mechanism and the spatial attention mechanism; each DSC-Transformer Block module calculates the attention weights between the features of the feature map output by the PEB module through the multi-head self-attention mechanism and performs weighted aggregation on the features, performs local feature extraction and nonlinear transformation on the weighted aggregated feature map through MLP, and then uses residual connection to add the features processed by the multi-head self-attention mechanism and MLP to the original input features; the output of each encoder except the last encoder is passed to the next encoder and the corresponding decoder, and the output of the last encoder is passed to the first decoder.

[0009] Based on the first aspect, in some embodiments, each DSC-Transformer Block module includes a first normalization layer, a multi-head self-attention mechanism, a second normalization layer, and a second multi-layer perceptron network connected in sequence;

[0010] The first normalization layer is used to normalize the feature map output by the PEB module;

[0011] The multi-head self-attention mechanism is used to calculate the attention weights between the features in the feature map output by the first normalization layer and perform weighted aggregation on the features; wherein the feature map output by the multi-head self-attention mechanism is added to the feature map output by the PEB module and then input to the normalization layer for normalization processing;

[0012] The second multi-layer perceptron network is used to perform local feature extraction and nonlinear transformation on the feature map output by the normalization layer; wherein the feature map output by the second multi-layer perceptron network is added to the feature map output by the multi-head self-attention mechanism and then output to the next DSC-Transformer Block module.

[0013] Based on the first aspect, in some embodiments, the second multi-layer perceptron network includes a first fully connected layer, a depth-separable convolutional layer, a GELU activation function, a Dropout layer, and a second fully connected layer;

[0014] The first fully connected layer is used to convert the dimension of the feature map output by the normalization layer into a feature map of a preset hidden feature dimension;

[0015] The depthwise separable convolutional layer is used to convolve the feature map output by the first fully connected layer in depth and spatial dimensions;

[0016] The GELU activation function is used to perform nonlinear transformation on the feature map output by the depth-separable convolutional layer;

[0017] The Dropout layer is used to randomly set some feature values ​​in the feature maps output by the first fully connected layer, the depthwise separable convolutional layer and the GELU activation function to zero with a preset probability during the training process;

[0018] The second fully connected layer is used to process the feature representation processed by the GELU activation function and the Dropout layer, and the output feature map is added to the feature map output by the multi-head self-attention mechanism and then output to the next DSC-Transformer Block module.

[0019] Based on the first aspect, in some embodiments, the depthwise separable convolution layer performs depthwise convolution and pointwise convolution on the feature map output by the normalization layer in sequence; in the depthwise convolution, there is a corresponding convolution kernel for each channel of the input feature map, and each convolution kernel convolves the features of each channel to generate an output feature map of the corresponding channel; in the pointwise convolution, a 1×1 convolution is performed on the feature map output by the depthwise convolution, and the features of all channels are linearly combined at each position.

[0020] Based on the first aspect, in some embodiments, the PEB module includes a two-dimensional convolutional layer, a CBAM layer and a normalization layer connected in sequence;

[0021] The two-dimensional convolutional layer is used to divide the synthetic aperture radar image to be processed input into the PEB module into a plurality of overlapping feature blocks according to a preset size and stride;

[0022] The CBAM layer is used to perform attention weighting on the multiple overlapping feature blocks from the channel dimension and the spatial dimension respectively;

[0023] The normalization layer is used to normalize the weighted feature blocks.

[0024] Based on the first aspect, in some embodiments, the CBAM layer includes a channel attention unit and a spatial attention unit;

[0025] The channel attention unit is used to: perform a global pooling operation on each channel of the multiple overlapping feature blocks to obtain a global average pooling feature and a global maximum pooling feature of each channel; input the two pooled features into a first multi-layer perceptron network to extract features; add the features extracted by the global average pooling feature and the global maximum pooling feature by the first multi-layer perceptron network and perform sigmoid activation function processing to obtain a channel attention weight; use the channel attention weight to weight the feature map of each channel of the multiple overlapping feature blocks to obtain a weighted feature map;

[0026] The spatial attention unit is used to: perform maximum pooling and average pooling on the feature map weighted by the channel attention mechanism in the channel dimension, connect the pooled features through the concat function, extract features through the convolution layer to compress the channel dimension to 1, and finally obtain the spatial attention weight through the sigmoid activation function; use the spatial attention weight to weight the feature map weighted by the channel attention mechanism to obtain the final feature map.

[0027] Based on the first aspect, in some embodiments, each decoder includes an upsampling convolution layer and a residual module connected in sequence;

[0028] The upsampling convolution layer upsamples the input feature map by bilinear interpolation to double the size of the output feature map of the previous layer, and then performs convolution processing on the upsampled feature map to reduce the number of channels of the feature map. Finally, the size of the reconstructed feature map obtained by the convolution processing is the same as that of the input feature map, but the number of channels is reduced;

[0029] The residual module uses a residual connection at each scale to add the features output by the residual module to the original features input to obtain the residual features of each scale.

[0030] Based on the first aspect, in some embodiments, obtaining a synthetic aperture radar image to be processed includes:

[0031] Convert the original data of the three channels of the original image into a grayscale image to obtain a clean, spot-free image X;

[0032] The Nakagami-Rayleigh distribution is used to generate the speckle noise N;

[0033] Generate a synthetic aperture radar image to be processed through a speckle noise model Y=X*N of the synthetic aperture radar image, where Y is a synthetic aperture radar image with speckle noise;

[0034] Among them, the amplitude of speckle N obeys Nakagami distribution, and the probability density function of speckle N is n and L are both greater than 0, Γ(·) represents the Gamma function, and L represents the number of views. The number of views L is used to describe the number of times or the number of observation angles that the synthetic aperture radar system observes the target area.

[0035] In a second aspect, an embodiment of the present application provides a synthetic aperture radar image denoising device, comprising:

[0036] An image acquisition module, used for acquiring synthetic aperture radar images to be processed;

[0037] An image denoising module, used for inputting the synthetic aperture radar image to be processed into an image denoising model, and outputting a denoised synthetic aperture radar image;

[0038] Among them, the image denoising model includes multiple encoders and multiple decoders connected in sequence, each encoder includes a PEB module and multiple DSC-Transformer Block modules connected in sequence; the PEB module performs weighted processing on the input image from the channel dimension and the spatial dimension respectively through the channel attention mechanism and the spatial attention mechanism; each DSC-Transformer Block module calculates the attention weights between the features of the feature map output by the PEB module through the multi-head self-attention mechanism and performs weighted aggregation on the features, performs local feature extraction and nonlinear transformation on the weighted aggregated feature map through MLP, and then uses residual connection to add the features processed by the multi-head self-attention mechanism and MLP to the original input features; the output of each encoder except the last encoder is passed to the next encoder and the corresponding decoder, and the output of the last encoder is passed to the first decoder.

[0039] In a third aspect, an embodiment of the present application provides a terminal device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the synthetic aperture radar image denoising method as described in any one of the first aspects is implemented.

[0040] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0041] In the embodiment of the present application, a new patch embedding block PEB is designed to solve the problems of local information loss and insufficient global information fusion in traditional image feature extraction methods when processing images, and improve the perception ability of convolutional neural networks for multi-scale features; a new Transformer Block module is designed to solve the limitations of Transformer models in terms of computational efficiency and feature extraction capabilities when processing high-dimensional data, enhance the model's feature extraction and expression capabilities, thereby improving the model's ability to represent fine-grained features and reducing the computational complexity of the network. The present application can capture the inherent local and global dependencies in SAR images and effectively improve the noise removal effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] Figure 1 is a flow chart of a synthetic aperture radar image denoising method provided in an embodiment of the present application;

[0044] Figure 2 is a structural diagram of an image denoising model provided in an embodiment of the present application;

[0045] Figure 3 is a structural diagram of a PEB module provided in an embodiment of the present application;

[0046] Figure 4 is a structural diagram of a DSC-Transformer Block module provided in an embodiment of the present application;

[0047] Figure 5 It is the process of simulating SAR images provided by the embodiment of the present application;

[0048] Figure 6 This is a comparison diagram of the denoising effects of different methods on the oil tank image when L=1 provided in the embodiment of the present application;

[0049] Figure 7 This is a comparison diagram of the denoising effects of different methods on a harbor image when L=4 provided in an embodiment of the present application;

[0050] Figure 8 This is a comparison diagram of denoising effects of different methods on dense residential images when L=8 provided in an embodiment of the present application;

[0051] Fig. 9This is a comparison diagram of the denoising effects of different methods on an aircraft image when L=16 provided in an embodiment of the present application;

[0052] Fig.10 This is a comparison chart of real SAR image denoising results provided by the embodiments of the present application;

[0053] Fig.11 Schematic diagram of the structure of a synthetic aperture radar image denoising device provided in an embodiment of the present application;

[0054] Fig.12 It is a schematic diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The present application is described more clearly below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the effects of the present application, but are not intended to limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present application. These all fall within the scope of protection of the present application.

[0056] In order to make the purpose, technical solutions and advantages of the present application clearer, specific embodiments will be described below in conjunction with the accompanying drawings.

[0057] See also Figure 1 The synthetic aperture radar image denoising method provided in the embodiment of the present application may include the following steps:

[0058] Step 101: Acquire a synthetic aperture radar image to be processed.

[0059] In some embodiments, step 101 may include:

[0060] Convert the original data of the three channels of the original image into a grayscale image to obtain a clean, spot-free image X;

[0061] The Nakagami-Rayleigh distribution is used to generate the speckle noise N;

[0062] Generate a synthetic aperture radar image to be processed through a speckle noise model Y=X*N of the synthetic aperture radar image, where Y is a synthetic aperture radar image with speckle noise;

[0063] Among them, the amplitude of speckle N obeys Nakagami distribution, and the probability density function of speckle N is n and L are both greater than 0, Γ(·) represents the Gamma function, and L represents the number of views. The number of views L is used to describe the number of times or the number of observation angles that the synthetic aperture radar system observes the target area.

[0064] Step 102: input the to-be-processed synthetic aperture radar image into an image denoising model, and output a denoised synthetic aperture radar image.

[0065] Among them, the image denoising model includes multiple encoders and multiple decoders connected in sequence, each encoder includes a PEB module and multiple DSC-Transformer Block modules connected in sequence; the PEB module performs weighted processing on the input image from the channel dimension and the spatial dimension respectively through the channel attention mechanism and the spatial attention mechanism; each DSC-Transformer Block module calculates the attention weights between the features of the feature map output by the PEB module through the multi-head self-attention mechanism and performs weighted aggregation on the features, performs local feature extraction and nonlinear transformation on the weighted aggregated feature map through MLP, and then uses residual connection to add the features processed by the multi-head self-attention mechanism and MLP to the original input features; the output of each encoder except the last encoder is passed to the next encoder and the corresponding decoder, and the output of the last encoder is passed to the first decoder.

[0066] like Figure 2 As shown, five encoders (E1~E5) and five decoders (D1~D5) are used as examples for illustration. For ease of description, the image denoising model in the embodiment of the present application is referred to as HTC-Net below. HTC-Net can be applied to remote sensing imaging, military reconnaissance, geographic information and other fields. HTC-Net is a codec structure composed of a mixture of Transformer and CNN. The encoder and decoder use a symmetrical structure to implement the encoding and decoding process, and there is a residual connection between the encoder and the decoder, so that the output of the encoder (except the last stage) can be directly passed to the next encoder and decoder, thereby allowing more levels of feature representation to be retained and utilized. The noise image and the target image are used as the input of the encoder and the output of the decoder respectively. The output of the last encoder is only passed to the decoder. In order to gradually extract the features of the input image and let the model learn the representation of complex features, 5 codec stages were designed experimentally. Through this design model, the details and local features in the input image can be better utilized, thereby further improving the quality of image denoising and achieving better denoising performance.

[0067] Each encoder consists of a PEB (Patch Embedding Block) module and multiple DSC-TransformerBlock modules. For noisy input images, the Patch Embedding Block first extracts features. At each encoder stage, the feature map size gradually decreases and the number of channels gradually increases, so as to gradually extract higher-level features of the image. The encoded features are processed by the encoder and then passed to the decoder for further processing.

[0068] Each decoder consists of upsampling convolution and residual modules. The upsampling convolution contains a convolution operation and an upsampling operation to adjust the size of the input feature map to a higher resolution; and each residual module covers multiple convolution layers and normalization layers to extract higher-level feature representations. The decoder progressively upsamples and fuses the input features through a series of upsampling convolution layers and residual modules to obtain feature representations with higher resolution and richer semantic information. Finally, it is further processed through a convolutional projection layer to achieve the mapping from the feature domain to the image domain, and the features are mapped to the output clean image.

[0069] In some embodiments, the PEB module may include a two-dimensional convolutional layer (Conv), a CBAM (Convolutional Block Attention Module) layer and a normalization layer (Norm) connected in sequence, such as Figure 3 As shown in (a) in FIG. 1 . The two-dimensional convolutional layer is used to divide the synthetic aperture radar image to be processed input to the PEB module into multiple overlapping feature blocks according to a preset size and stride. The CBAM layer is used to perform attention weighting on the multiple overlapping feature blocks from the channel dimension and the spatial dimension respectively. The normalization layer is used to normalize the weighted feature blocks.

[0070] For the input noisy image, PEB first performs a two-dimensional convolution Conv process on it to divide the input image into a series of overlapping feature blocks according to a certain size and stride. There is partial overlap between these feature blocks to retain local continuity. The divided image blocks are projected into a low-dimensional embedding space to capture the feature information of the local area in the image, which helps the model better understand the local structure and details of the image. This projection operation maps each feature block to an embedding vector and applies appropriate padding to keep the size of the image block unchanged. The expressive power of the embedding vector is then enhanced through the CBAM attention mechanism. CBAM can effectively capture the spatial and channel correlations between image blocks and weight different parts of the embedding vector to improve the quality of feature representation. The generated embedding vector is then standardized to ensure that each embedding vector has similar scale and distribution characteristics between different image blocks, which helps to reduce the scale differences between different image blocks, thereby improving the training stability and convergence speed of the model.

[0071] This application uses CBAM to perform attention weighting on the features after convolution processing to enhance the perception and utilization of important features while suppressing the response to unimportant features. The specific structure of the CBAM module is as follows Figure 3 As shown in (b) in the figure. CBAM combines the channel attention mechanism and the spatial attention mechanism to weight the feature map from the channel dimension and the spatial dimension respectively. When processing each image block, CBAM improves the performance of the model by enhancing its feature representation ability. This process is due to the fact that CBAM can fully capture the feature associations in the channel and spatial dimensions when considering each image block. In addition, the design of CBAM allows it to be seamlessly embedded in multiple levels of convolutional neural networks, thereby enhancing the network's feature extraction and representation capabilities at different levels, further improving the performance of the model on various visual tasks.

[0072] See also Figure 3 In part (b), the CBAM layer may include a channel attention unit and a spatial attention unit.

[0073] The channel attention unit is used to: perform a global pooling operation on each channel of multiple overlapping feature blocks in an input feature map to obtain a global average pooling feature and a global maximum pooling feature of each channel; input the two pooled features into a first multi-layer perceptron network to extract features; use the first multi-layer perceptron network to add and process the features extracted by the global average pooling feature and the global maximum pooling feature, respectively, to obtain a channel attention weight; and use the channel attention weight to weight the feature map of each channel of the multiple overlapping feature blocks to obtain a weighted feature map.

[0074] The spatial attention unit is used to: perform maximum pooling and average pooling on the feature map weighted by the channel attention mechanism in the channel dimension, connect the pooled features through the concat function, extract features through the convolution layer to compress the channel dimension to 1, and finally obtain the spatial attention weight through the sigmoid activation function; use the spatial attention weight to weight the feature map weighted by the channel attention mechanism to obtain the final feature map.

[0075] The structure of the channel attention module is as follows Figure 3 As shown in (c) in the figure. In this model, CBAM performs attention weighting on the embedding vector of each feature block of the input feature map. First, the channel attention mechanism performs global pooling operations on each channel of the embedding vector, including MaxPool and AvgPool, to obtain the global average pooling and global maximum pooling features of each channel. Then the two pooled features are sent to a shared multi-layer perceptron Shared MLP network to extract features, and the pooled features of the MLP are added and the sigmoid activation function is used to obtain the final channel attention weight. Finally, the learned channel weight is applied to the original feature map, and the weighted feature map Mc is obtained by weighting the feature map of each channel. In the weighting process, the feature map of the channel with a larger weight will be strengthened, while the feature map of the channel with a smaller weight will be weakened. The calculation process of the channel attention mechanism is shown in formula (1):

[0076]

[0077] Among them, σ represents the activation function, MLP is a multi-layer perceptron, and W 0 and W 1 represents the weight matrix, and They represent average pooling features and maximum pooling features respectively.

[0078] Figure 3 (d) in Figure 1 shows the structure of the spatial attention mechanism. The spatial attention mechanism performs maximum pooling MaxPool and average pooling AvgPool on the channel dimension of the feature map Mc' calculated by the channel attention mechanism, concatenates the pooled features, extracts features through a convolutional layer Conv, compresses the channel dimension to 1, and obtains the spatial weight matrix through sigmoid activation. Finally, the normalized weights are multiplied element-by-element with the original feature map to obtain the final spatial attention feature map Ms. The entire calculation process is shown in formula (2):

[0079]

[0080] Among them, σ represents the activation function, f 7×7 Represents a 7×7 convolution operation.

[0081] By combining channel attention and spatial attention, CBAM can dynamically adjust the weights of feature maps while taking into account the correlation between channels and the importance of spatial positions, thereby improving the network's perception of important features.

[0082] In some embodiments, each DSC-Transformer Block module may include a first normalization layer, a multi-head self-attention mechanism, a second normalization layer, and a second multi-layer perceptron network connected in sequence, such as Figure 4 shown.

[0083] The first normalization layer is used to normalize the feature map output by the PEB module.

[0084] The multi-head self-attention mechanism is used to calculate the attention weights between the features in the feature map output by the first normalization layer and perform weighted aggregation on the features; wherein the feature map output by the multi-head self-attention mechanism is added to the feature map output by the PEB module and then input into the normalization layer for normalization processing.

[0085] The second multi-layer perceptron network is used to perform local feature extraction and nonlinear transformation on the feature map output by the normalization layer; wherein the feature map output by the second multi-layer perceptron network is added to the feature map output by the multi-head self-attention mechanism and then output to the next DSC-Transformer Block module.

[0086] like Figure 4 As shown on the right, the second multi-layer perceptron network may include a first fully connected layer, a depth-separable convolution layer, a GELU activation function, a Dropout layer, and a second fully connected layer. The first fully connected layer is used to convert the dimension of the feature map output by the normalization layer into a feature map of a preset hidden feature dimension. The depth-separable convolution layer is used to convolve the feature map output by the first fully connected layer in the depth and space dimensions. The GELU activation function is used to perform a nonlinear transformation on the feature map output by the depth-separable convolution layer. The Dropout layer is used to randomly set the feature values ​​of the feature maps output by the first fully connected layer, the depth-separable convolution layer, and the GELU activation function to zero with a preset probability during the training process. The second fully connected layer is used to process the feature representation processed by the GELU activation function and the Dropout layer, and the output feature map is added to the feature map output by the multi-head self-attention mechanism and output to the next DSC-Transformer Block module.

[0087] Exemplarily, the depthwise separable convolution layer performs depthwise convolution and pointwise convolution on the feature map output by the normalization layer in sequence; in the depthwise convolution, there is a corresponding convolution kernel for each channel of the input feature map, and each convolution kernel convolves the features of each channel to generate an output feature map of the corresponding channel; in the pointwise convolution, a 1×1 convolution is performed on the feature map output by the depthwise convolution, and the features of all channels are linearly combined at each position.

[0088] This application designs a new Transformer Block (DSC-Transformer Block) structure to address the limitations of the traditional Transformer model in terms of computational efficiency and feature extraction when processing high-dimensional data. This improvement effectively enhances the model's feature extraction and expression capabilities, thereby improving the model's ability to represent fine-grained features.

[0089] The traditional Transformer Block is usually composed of a multi-head self-attention mechanism and an MLP network, which is used to capture the global correlation and nonlinear transformation of features. However, this structure has certain limitations when processing local information and detail features. In order to solve this problem, this application transforms the traditional Transformer architecture, integrating DSC with the traditional Transformer architecture to form a new DSC-Transformer block structure to further enhance the feature extraction and representation capabilities of the model.

[0090] DSC-Transformer Block first calculates the attention weights between features through the Multi-Head Self-Attention mechanism and performs weighted aggregation on the features to capture the global dependencies between input features, so that the features at each position can obtain the contribution of global information and better understand the whole. After that, the features processed by the Multi-Head Self-Attention mechanism are sent to the MLP for local feature extraction and nonlinear transformation.

[0091] The features input to the MLP first pass through a fully connected layer to convert the dimensions of the input features from the initial dimensions to the specified hidden feature dimensions. The fully connected layer uses the learned weight matrix to perform a linear transformation on the input features and adds a bias term to generate a new feature representation. After that, the features are convolved in the depth and spatial dimensions through DSConv to extract and integrate feature information, thereby enhancing the expressiveness of the features.

[0092] DSConv decomposes the convolution operation into two independent steps, namely depth convolution and point-by-point convolution. Depth convolution performs a separate convolution operation on each channel of the input feature map. There is a corresponding convolution kernel for each channel of the input feature map. These convolution kernels convolve the features of each channel to generate the output feature map of the corresponding channel. This process only involves convolution operations in the spatial dimension and does not involve interactions between different channels. Point-by-point convolution performs a 1×1 convolution after depth convolution. It integrates the information between different channels by linearly combining the features of all channels at each position to obtain the final output feature map. It is then transformed nonlinearly through the GELU activation function. In order to prevent overfitting, this application introduces a Dropout layer, which is used to randomly set feature values ​​to zero with a certain probability during training to enhance the generalization ability of the model. After that, the features are passed through another fully connected layer to generate the final feature representation.

[0093] Through the above series of operations, MLP performs complex nonlinear transformation and feature extraction on the input features, thereby generating feature representations with higher expressiveness. Finally, DSC-Transformer Block uses residual connections to add the features processed by the multi-head self-attention mechanism and MLP to the original input features to retain the information of the original features. This residual connection not only improves the training efficiency of the model but also alleviates the problems of gradient disappearance and gradient explosion, making the model more stable.

[0094] When processing feature maps, standard convolution needs to perform convolution calculations on all input channels for each convolution kernel, so the number of parameters is large and the computational complexity is high. Assume that the size of the input feature map is D f ×D f , the convolution kernel size is K×K, the input channel and output channel are M and N respectively, and the size of the output feature map is D g ×D g , then the parameter quantity P after the standard convolution 1 and the amount of computation C 1 As shown in formula (3) and formula (4) respectively:

[0095] P 1 =K×K×M×N (3)

[0096] C 1 =K×K×M×N×D g ×D g (4)

[0097] The number of parameters P after depth-wise separable convolution 2 and the amount of computation C 2 As shown in formula (5) and formula (6):

[0098] P 2 =K×K×M + M×N (5)

[0099] C 2 =K×K×M×D g ×D g + M×N×D g ×D g (6)

[0100] The parameter ratio R for depthwise separable convolution and standard convolution 1 Ratio of the amount of calculation to R 2 , as shown in formula (7) and formula (8):

[0101]

[0102] From the above analysis, it can be seen that the number of parameters and the amount of calculation of DSConv are only a fraction of the standard convolution. The specific reduction depends on the convolution kernel size K and the output channel N. For example, when K is large and N is high, the reduction is more significant. At the same time, DSC can effectively extract spatial features and channel features, so that the network can better learn the local and global information of the input data, thereby improving the feature expression ability.

[0103] In some embodiments, each decoder includes an upsampling convolution layer and a residual module connected in sequence. The upsampling convolution layer upsamples the input feature map by bilinear interpolation to double the size of the output feature map of the previous layer, and then performs convolution processing on the upsampled feature map to reduce the number of channels of the feature map, and finally outputs the reconstructed feature map obtained by the convolution processing, the size of which is the same as the input feature map, but the number of channels is reduced. The residual module uses a residual connection at each scale to add the features output by the residual module to the original features of the input to obtain the residual features of each scale.

[0104] For the input noisy image I i ∈R H×W×C , H and W represent the height and width of the input image, respectively, and C represents the number of channels. The size of the input image is 256×256, and the number of channels is 1. First, the 5 patch embedding blocks in the encoder stage divide the input noisy image into multiple overlapping image patches and convert them into vector representations. The first patch embedding block receives the input image and decomposes it into overlapping image patches with a size of 7×7, a stride of 2, and uses a 3×3 padding method to ensure the integrity of the boundary information.

[0105] The features extracted by the patch embedding block are input into the DSC-Transformer Block module for encoding, where each encoder stage contains multiple DSC-Transformer Blocks. As each encoder stage proceeds, the resolution of the feature map output after encoding is gradually halved so that the DSC-Transformer Block can learn the coarse and fine details of the image. The DSC-Transformer Block first performs layer normalization on the input features to reduce the internal covariate shift problem in model training and enhance the stability of the network. Then, the multi-head self-attention mechanism network can calculate the attention weights between features in parallel on different heads to capture global information and positional relationships. Subsequently, the layers of the DSC-Transformer Block are randomly discarded through the random depth mechanism to enhance the generalization ability of the network. Finally, the multi-layer perceptron is used to perform linear transformation and nonlinear activation processing on the features processed by the multi-head self-attention mechanism to increase the representation diversity and adaptability of the features. The whole process enables the DSC-Transformer Block to better understand the relationship between input features, improve the representation ability of features and the performance of the network. These stages gradually extract features of the image from coarse to fine, at scales of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32. The output of each encoder stage is passed to the next encoder and decoder except for the last stage. The output of the final encoder stage is passed only to the decoder.

[0106] The output feature map size of the DSC-Transformer module in the first stage after encoding is 128×128, and the number of channels is 32. The output feature map size of the second stage is 64×64, and the number of channels is 64; the output feature map size of the third stage is 32×32, and the number of channels is 128; the output feature map size of the fourth stage is 16×16, and the number of channels is 320; the output feature map size of the last stage is 8×8, and the number of channels is 512.

[0107] After the encoder stage, the encoded feature representation is obtained, and then these encoded features are sent to the decoder. During the decoding process, the features will gradually recover to the original image space. The decoder contains a series of upsampling convolutional layers and residual blocks. The upsampling operation upsamples the input feature map by bilinear interpolation to double the size of the output feature map of the previous layer. The convolution operation is used to convolve the upsampled feature map to reduce the number of channels of the feature map. The final output is the reconstructed feature map obtained by the convolution operation. The size is the same as the input feature map, but the number of channels is reduced. The output feature map is then sent to the residual module to learn the residual information between feature maps. At each scale (except the last decoding stage), residual connections are used to add the features output by the residual block to the original features of the input to obtain the residual features of each scale to retain more original information. This process is completed by a series of deconvolutions and residual connections, so that the low-dimensional encoded features are reconstructed into features of the same size as the original image. The feature representation obtained after the decoding and reconstruction stages is processed by convolution and activation functions, and the denoised image is finally output.

[0108] The above synthetic aperture radar image denoising method designs a new patch embedding block PEB to solve the problems of local information loss and insufficient global information fusion in traditional image feature extraction methods when processing images, and improves the perception ability of convolutional neural networks for multi-scale features; designs a new Transformer Block module to solve the limitations of Transformer models in computational efficiency and feature extraction capabilities when processing high-dimensional data, enhances the model's feature extraction and expression capabilities, thereby improving the model's ability to represent fine-grained features and reducing the computational complexity of the network. This application can capture the inherent local and global dependencies in SAR images and effectively improve the noise removal effect.

[0109] The effects of the embodiments of the present application are described below through experiments.

[0110] The experimental hardware information is shown in Table 1:

[0111] Table 1 Experimental hardware information

[0112]

[0113]

[0114] The experimental steps are as follows:

[0115] (1) Data preprocessing:

[0116] For optical images, noise image simulation generally adds additive Gaussian white noise to the image to generate training images. SAR images are different. The SAR imaging mechanism inherently produces coherent speckle noise. Coherent speckle noise is a common phenomenon in SAR images. It originates from the coherence of radar waves. When radar waves are irradiated to the ground or the surface of an object, the reflected waves will produce random phase differences due to the variation of the roughness of the ground or the surface of the object within the wavelength scale. When these waves with different phases are superimposed, a seemingly random pattern of bright and dark spots will be formed on the radar image. This is speckle noise.

[0117] The speckle noise model of SAR images can be expressed as Y = X * N, where Y is the speckle noise image, X is the corresponding clean image or speckle-free image, and N is the corresponding speckle. The amplitude of speckle N follows the Nakagami distribution. In the case of single view (i.e., only one observation), the amplitude of speckle N follows the Rayleigh distribution. The probability density function of speckle N is n and L are both greater than 0, Γ(·) represents the Gamma function, and L represents the number of views, which is used to describe the number of times or the number of observation angles that the synthetic aperture radar system observes the target area. Each observation can obtain a SAR image, and the data of multiple observations can be combined or superimposed to improve the quality and resolution of the image. Therefore, increasing the number of views can effectively reduce the noise level in the image, thereby improving the overall image quality.

[0118] The simulation process of SAR image is as follows Figure 5 First, the three-channel raw data is converted into a grayscale image to obtain a clean image X. Next, the Nakagami-Rayleigh distribution is used to generate speckle noise N. Finally, the final simulated SAR image Y is obtained through the speckle noise model described by the speckle noise model.

[0119] Since SAR images lack real noise-free images, this application chooses to use simulated SAR images for network training. This experiment uses the BSDS500 (Berkeley Segmentation Dataset and Benchmark500, BSDS500) dataset provided by the University of Berkeley for network training, which contains 500 natural images. During the network training process, 400 images in the BSDS500 dataset are randomly selected as the training set, and the remaining 100 images are used as the validation set. This experiment obtains simulated SAR images through the multiplicative noise model described by formula (9). The number of views L is set to 1, 4, 8, and 16 respectively to generate four different levels of noise samples, which constitute training image pairs with the above-mentioned noise-free samples. In the testing phase, this experiment uses two public datasets, Kodak24 dataset and UC Merced Land-Use, to test the simulated SAR images, and uses real SAR images taken by Sentinel-1A for testing.

[0120] (2) Model comparison experiment;

[0121] In order to verify the coherent speckle suppression and edge preservation performance of the method of this application, five representative methods, including PPB, SAR-CNN, SAR-DCNN, MONet and SAR-Transformer, were used as comparison methods in the experiment, and two groups of experiments were carried out on simulated SAR images and real SAR images respectively.

[0122] (2.1) Evaluation indicators

[0123] When evaluating the performance of image denoising algorithms, a series of objective evaluation indicators are often used to quantify the quality of the denoising effect. This application uses Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) and Equivalent Number of Looks (ENL) as evaluation indicators of denoised image quality. In addition, it is also necessary to conduct qualitative analysis to observe the visual effects of the denoised image, including clarity, texture preservation, detail preservation, artifacts and residues, in order to comprehensively evaluate the performance of the denoising algorithm.

[0124] (2.1.1)PSNR

[0125] PSNR is mainly used to measure the similarity between the original image and the denoised image. This indicator evaluates the impact of noise in the image by calculating the maximum possible pixel value difference and mean square error between the two. Its unit is decibel (dB). The larger the PSNR value, the lower the image distortion and the higher the image quality.

[0126] PSNR is defined as: Where g represents the multiplication of two elements, MAX is the maximum possible value of the image pixel value, and MSE represents the mean square error between the original image I and the denoised image K. The smaller the MSE value, the closer the two images are.

[0127] (2.1.2)SSIM

[0128] SSIM is an image similarity metric that comprehensively evaluates brightness, contrast, and structural information. It is widely used to evaluate the effect of denoising algorithms on the preservation of image texture structure. The value range of this indicator is from 0 to 1. The closer the value is to 1, the smaller the difference between the denoised image and the original image. SSIM is defined as:

[0129] SSIM(X,Y)=l(X,Y)·c(X,Y)·s(X,Y)

[0130]

[0131] Among them, μ X and μ Y are the means of X and Y respectively, and and Denote the variance of X and Y, σ XY represents the covariance of X and Y, C 1 , C 2 and C 3 are all constants.

[0132] (2.1.3)ENL

[0133] When processing real SAR images, due to the lack of noise-free reference images, the above two indicators cannot be used to evaluate the denoising performance. Therefore, in addition to judging the denoising effect of real SAR images by visual effects, ENL is also used as a measurement standard. ENL is mainly used to evaluate the smoothness of uniform areas in denoised images. The larger the value, the higher the smoothness of the image in this area, and the more obvious the noise suppression effect. The calculation formula of ENL is: Among them, μ I represents the mean of the image, σ I Represents variance.

[0134] (2.2) Simulated SAR image experiment

[0135] (2.2.1) Qualitative analysis

[0136] Figure 6 to Figure 9The following figure shows the visual effect comparison of four different categories of images selected from the test set UC Merced Land-Use after denoising using different methods when L=1, L=4, L=8 and L=16. Figure 6 to Figure 9 (a) is the original image, (b) is the noisy image, and (c) to (h) are the denoising effect diagrams of PPB, SAR-CNN, SAR-DCNN, MONet, SAR-Transformer and HTC-Net of the present application respectively.

[0137] In order to make an intuitive and accurate comparison, the specific areas of the four types of images are enlarged and placed under each denoised image. It can be seen that the six denoising methods can remove noise to varying degrees. However, in terms of processing effect, although PPB shows good denoising performance, the processed image is not clear enough, and the image denoised by the algorithm will produce additional textures; in the case of severe noise, the image denoised by the SAR-CNN algorithm still has some residual noise; compared with SAR-CNN, the denoising performance of the MONet algorithm is slightly improved, but the denoised image is too smooth and loses a lot of detail information; after denoising by the SAR-DCNN method, the local area of ​​the image will be distorted to a certain extent; SAR-Transformer has a better denoising effect and retains some details of the image, but there are still some differences in the local detail map. Compared with the other five denoising methods, the HTC-Net method proposed in this application retains details better while maintaining denoising performance, the object outline is clear, the edge is also retained to a certain extent, and it is closer to the original image. In summary, for more complex noise situations, deep learning methods are superior to traditional denoising methods. SAR-CNN, SAR-DCNN, MONet and SAR-Transformer all showed good denoising performance, but in terms of detail preservation and texture reconstruction, the HTC-Net method proposed in this application performed best and achieved better performance.

[0138] (2.2.2) Quantitative analysis

[0139] Tables 2 and 3 show the average PSNR and SSIM of different algorithms at different view counts (L = 1, 4, 8, 16) on the Kodak24 dataset. It can be seen from the table that the HTC-Net proposed in this application has achieved good results at different view counts, especially when the view count is 1 (i.e., in the most complex noise scene), HTC-Net can still achieve high performance indicators. From the data in Tables 1 and 2, as the view count increases, the average PSNR of the noisy image increases from 13.18dB to 24.62dB, which reflects the effect of multi-view SAR imaging technology by integrating multi-view observation information to improve image quality. Although the PPB denoising algorithm shows obvious improvement with the increase in the view count, its effect is relatively weak in deep learning methods. Both SAR-CNN and MONet are better than PPB, and MONet is slightly better than SAR-CNN in some cases. SAR-DCNN and SAR-Transformer both show high PSNR and SSIM values ​​at all view counts, but the HTC-Net method proposed in this application performs best at all view counts. In the most complex noise case (L=1), the average PSNR of HTC-Net is improved by 11.1%, 5.0%, 1.6%, 0.9% and 1.1% compared with PPB, SAR-CNN, MONet, SAR-DCNN and SAR-Transformer, respectively. In addition, its average SSIM value is improved by 42.7%, 23.2%, 3.4%, 2.1% and 0.9% compared with other denoising algorithms, respectively, which shows that the images processed by HTC-Net have better performance in maintaining structural information. Comprehensive analysis shows that HTC-Net performs better than other methods in PSNR and SSIM at all view counts. This excellent performance is due to its hybrid architecture, which effectively combines the advantages of Transformer and CNN, thereby optimizing global and local feature extraction capabilities.

[0140] Table 2 Average PSNR of Kodak24 dataset using different denoising methods at different view counts

[0141]

[0142]

[0143] Table 3 Average SSIM of denoising on Kodak24 dataset using different methods under different view counts

[0144]

[0145] (2.3) Real SAR image comparison experiment

[0146] (2.3.1) Qualitative analysis

[0147] To further demonstrate the effectiveness of the HTC-Net algorithm proposed in this application, the experiment uses real SAR images taken by Sentinel-1A in IW scanning mode to verify the denoising performance. PPB, SAR-CNN, SAR-DCNN, MONet, SAR-Transformer and the HTC-Net method proposed in this application are used to denoise real SAR images. The processed images are shown in Figure 2. Fig.10 As shown, (a) is the original image, (b) to (g) are the effect images after denoising by PPB, SAR-CNN, SAR-DCNN, MONet, SAR-Transformer and the HTC-Net method proposed in this application, and the second row of images is the corresponding enlarged result of the blue area.

[0148] from Fig.10 From the results, it can be seen that the image denoised by the PPB method shows the characteristics of over-smoothing, resulting in texture distortion in some areas. Although SAR-CNN and MONet can restore more detailed textures, the detailed information of the reconstructed images is relatively chaotic and fails to effectively present the original image texture. Although SAR-DCNN can effectively remove noise, there are still some residual artifacts while retaining the image structure. Both SAR-Transformer and HTC-Net show good results after denoising, but HTC-Net performs better in terms of texture richness and detail clarity. Compared with the other five denoising algorithms, the method proposed in this application can clearly display the contour and detail information while achieving good denoising effects, retaining more edge details, so that the overall visual effect of the denoised image is better.

[0149] (2.3.2) Quantitative analysis

[0150] In order to quantitatively analyze the denoising effect of real SAR images, this experiment uses ENL as an evaluation index to quantitatively evaluate each denoising algorithm. The objective evaluation index ENL is usually used to evaluate the denoising effect of homogeneous areas. The homogeneous areas selected in this application are the red area and the green area in the image. The larger the ENL value, the better the denoising algorithm suppresses speckle noise in the smooth area. Table 4 gives the ENL in two rectangular homogeneous areas in the image, and the optimal value is underlined in the table.

[0151] Table 4 ENL values ​​of real SAR images

[0152] method Red Zone Green Area Original Image 11.9982 9.0122 PPB 488.6285 54.6642 SAR-CNN 377.6213 69.5273 MONet 392.4977 86.0493 SAR-DCNN 465.3756 99.0714 SAR-Transformer 420.1972 115.4409 HTC-Net 602.0012 163.7111

[0153] The ENL value of the original SAR image is relatively low, which represents the noise level existing in the real SAR image without any processing. In all cases, the ENL value of the PPB method is significantly higher than that of the original image, especially in the red area, where its ENL value increases significantly. This reflects that the PPB method has achieved certain effects in noise suppression, but its performance in the green area is relatively weak. In contrast, the ENL value of the SAR-CNN method is relatively low in the green area, but it reaches a higher value in the red area. This shows that the SAR-CNN method can suppress noise more effectively in some cases. The ENL values ​​of the MONet method are relatively high in both the red and green areas. This shows that the MONet method can better handle the noise in the real SAR image. The ENL values ​​of SAR-DCNN and SAR-Transformer in different areas are relatively high, indicating that they can effectively handle the noise in the real SAR image and achieve certain noise suppression effects. The ENL values ​​of the HTC-Net method are the highest in all cases, which shows that the HTC-Net method has significant advantages in dealing with noise in real SAR images. In summary, compared with the other five denoising algorithms, the HTC-Net algorithm proposed in this application shows better denoising ability both in terms of visual effects and quantitative results.

[0154] This application proposes a network HTC-Net based on Transformer and CNN, and applies it to the SAR image denoising task. HTC-Net captures global information through the Transformer model, and with the help of the local feature extraction capability of the CNN model, it achieves efficient denoising of SAR images. Through a series of denoising experiments on simulated SAR images and real SAR images, it is verified that the proposed algorithm has achieved a good balance between noise removal and texture preservation, and shows obvious effectiveness.

[0155] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0156] Corresponding to the synthetic aperture radar detection performance evaluation method described in the above embodiment, Fig.11 A structural block diagram of a synthetic aperture radar detection performance evaluation device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0157] See also Fig.11 The synthetic aperture radar image denoising device includes an image acquisition module 201 and an image denoising module 202.

[0158] The image acquisition module 201 is used to acquire the synthetic aperture radar image to be processed.

[0159] The image denoising module 202 is used to input the SAR image to be processed into an image denoising model, and output a denoised SAR image.

[0160] Among them, the image denoising model includes multiple encoders and multiple decoders connected in sequence, each encoder includes a PEB module and multiple DSC-Transformer Block modules connected in sequence; the PEB module performs weighted processing on the input image from the channel dimension and the spatial dimension respectively through the channel attention mechanism and the spatial attention mechanism; each DSC-Transformer Block module calculates the attention weights between the features of the feature map output by the PEB module through the multi-head self-attention mechanism and performs weighted aggregation on the features, performs local feature extraction and nonlinear transformation on the weighted aggregated feature map through MLP, and then uses residual connection to add the features processed by the multi-head self-attention mechanism and MLP to the original input features; the output of each encoder except the last encoder is passed to the next encoder and the corresponding decoder, and the output of the last encoder is passed to the first decoder.

[0161] Optionally, each DSC-Transformer Block module includes a first normalization layer, a multi-head self-attention mechanism, a second normalization layer, and a second multi-layer perceptron network connected in sequence;

[0162] The first normalization layer is used to normalize the feature map output by the PEB module;

[0163] The multi-head self-attention mechanism is used to calculate the attention weights between the features in the feature map output by the first normalization layer and perform weighted aggregation on the features; wherein the feature map output by the multi-head self-attention mechanism is added to the feature map output by the PEB module and then input to the normalization layer for normalization processing;

[0164] The second multi-layer perceptron network is used to perform local feature extraction and nonlinear transformation on the feature map output by the normalization layer; wherein the feature map output by the second multi-layer perceptron network is added to the feature map output by the multi-head self-attention mechanism and then output to the next DSC-Transformer Block module.

[0165] Optionally, the second multi-layer perceptron network includes a first fully connected layer, a depthwise separable convolutional layer, a GELU activation function, a Dropout layer, and a second fully connected layer;

[0166] The first fully connected layer is used to convert the dimension of the feature map output by the normalization layer into a feature map of a preset hidden feature dimension;

[0167] The depthwise separable convolutional layer is used to convolve the feature map output by the first fully connected layer in depth and spatial dimensions;

[0168] The GELU activation function is used to perform nonlinear transformation on the feature map output by the depth-separable convolutional layer;

[0169] The Dropout layer is used to randomly set some feature values ​​in the feature graph output by the first fully connected layer, the depthwise separable convolutional layer and the GELU activation function to zero with a preset probability during the training process;

[0170] The second fully connected layer is used to process the feature representation processed by the GELU activation function and the Dropout layer, and the output feature map is added to the feature map output by the multi-head self-attention mechanism and then output to the next DSC-Transformer Block module.

[0171] Optionally, the depthwise separable convolution layer performs depthwise convolution and pointwise convolution on the feature map output by the normalization layer in sequence; in the depthwise convolution, there is a corresponding convolution kernel for each channel of the input feature map, and each convolution kernel convolves the features of each channel to generate an output feature map of the corresponding channel; in the pointwise convolution, a 1×1 convolution is performed on the feature map output by the depthwise convolution, and the features of all channels are linearly combined at each position.

[0172] Optionally, the PEB module includes a two-dimensional convolutional layer, a CBAM layer and a normalization layer connected in sequence;

[0173] The two-dimensional convolutional layer is used to divide the synthetic aperture radar image to be processed input into the PEB module into a plurality of overlapping feature blocks according to a preset size and stride;

[0174] The CBAM layer is used to perform attention weighting on the multiple overlapping feature blocks from the channel dimension and the spatial dimension respectively;

[0175] The normalization layer is used to normalize the weighted feature blocks.

[0176] Optionally, the CBAM layer includes a channel attention unit and a spatial attention unit;

[0177] The channel attention unit is used to: perform a global pooling operation on each channel of the multiple overlapping feature blocks to obtain a global average pooling feature and a global maximum pooling feature of each channel; input the two pooled features into a first multi-layer perceptron network to extract features; add the features extracted by the global average pooling feature and the global maximum pooling feature by the first multi-layer perceptron network and perform sigmoid activation function processing to obtain a channel attention weight; use the channel attention weight to weight the feature map of each channel of the multiple overlapping feature blocks to obtain a weighted feature map;

[0178] The spatial attention unit is used to: perform maximum pooling and average pooling on the feature map weighted by the channel attention mechanism in the channel dimension, connect the pooled features through the concat function, extract features through the convolution layer to compress the channel dimension to 1, and finally obtain the spatial attention weight through the sigmoid activation function; use the spatial attention weight to weight the feature map weighted by the channel attention mechanism to obtain the final feature map.

[0179] Optionally, each decoder includes an upsampling convolutional layer and a residual module connected in sequence;

[0180] The upsampling convolution layer upsamples the input feature map by bilinear interpolation to double the size of the output feature map of the previous layer, and then performs convolution processing on the upsampled feature map to reduce the number of channels of the feature map. Finally, the size of the reconstructed feature map obtained by the convolution processing is the same as that of the input feature map, but the number of channels is reduced;

[0181] The residual module uses a residual connection at each scale to add the features output by the residual module to the original features input to obtain the residual features of each scale.

[0182] Optionally, the image acquisition module 201 is specifically used for:

[0183] Convert the original data of the three channels of the original image into a grayscale image to obtain a clean, spot-free image X;

[0184] The Nakagami-Rayleigh distribution is used to generate the speckle noise N;

[0185] Generate a synthetic aperture radar image to be processed through a speckle noise model Y=X*N of the synthetic aperture radar image, where Y is a synthetic aperture radar image with speckle noise;

[0186] Among them, the amplitude of speckle N obeys Nakagami distribution, and the probability density function of speckle N is n and L are both greater than 0, Γ(·) represents the Gamma function, and L represents the number of views. The number of views L is used to describe the number of times or the number of observation angles that the synthetic aperture radar system observes the target area.

[0187] Fig.12 Schematic diagram of a terminal device provided by an embodiment of the present application. Fig.12 As shown, the terminal device 300 of this embodiment includes a processor 310 and a memory 320, wherein the memory 320 stores a computer program that can be run on the processor 310, such as a synthetic aperture radar image denoising program. When the processor 310 executes the computer program, the steps in the above-mentioned synthetic aperture radar image denoising method embodiment are implemented, such as Figure 1 Alternatively, when the processor 310 executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Fig.11 Functions of modules 201 to 202 are shown.

[0188] The terminal device 300 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 310 and a memory 320. Those skilled in the art will appreciate that Fig.12 It is only an example of the terminal device 300 and does not constitute a limitation of the terminal device 300. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0189] The processor 320 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0190] The memory 320 may be an internal storage unit of the terminal device 300, such as a hard disk or memory of the terminal device 300. The memory 320 may also be an external storage device of the terminal device 300, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 300. Further, the memory 320 may also include both an internal storage unit of the terminal device 300 and an external storage device. The memory 320 is used to store the computer program and other programs and data required by the terminal device. The memory 320 may also be used to temporarily store data that has been output or is to be output.

[0191] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0192] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0193] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0195] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A synthetic aperture radar image denoising method, characterized in that: include: Acquiring a synthetic aperture radar image to be processed; Inputting the to-be-processed synthetic aperture radar image into an image denoising model, and outputting a denoised synthetic aperture radar image; Among them, the image denoising model includes multiple encoders and multiple decoders connected in sequence, each encoder includes a PEB module and multiple DSC-Transformer Block modules connected in sequence; the PEB module performs weighted processing on the input image from the channel dimension and the spatial dimension respectively through the channel attention mechanism and the spatial attention mechanism; each DSC-Transformer Block module calculates the attention weights between the features of the feature map output by the PEB module through the multi-head self-attention mechanism and performs weighted aggregation on the features, performs local feature extraction and nonlinear transformation on the weighted aggregated feature map through MLP, and then uses residual connection to add the features processed by the multi-head self-attention mechanism and MLP to the original input features; the output of each encoder except the last encoder is passed to the next encoder and the corresponding decoder, and the output of the last encoder is passed to the first decoder.

2. The synthetic aperture radar image denoising method according to claim 1, characterized in that: Each DSC-Transformer Block module includes a first normalization layer, a multi-head self-attention mechanism, a second normalization layer, and a second multi-layer perceptron network connected in sequence; The first normalization layer is used to normalize the feature map output by the PEB module; The multi-head self-attention mechanism is used to calculate the attention weights between the features in the feature map output by the first normalization layer and perform weighted aggregation on the features; wherein the feature map output by the multi-head self-attention mechanism is added to the feature map output by the PEB module and then input to the normalization layer for normalization processing; The second multi-layer perceptron network is used to perform local feature extraction and nonlinear transformation on the feature map output by the normalization layer; wherein the feature map output by the second multi-layer perceptron network is added to the feature map output by the multi-head self-attention mechanism and then output to the next DSC-Transformer Block module.

3. The synthetic aperture radar image denoising method according to claim 2, characterized in that: The second multi-layer perceptron network includes a first fully connected layer, a depth-separable convolutional layer, a GELU activation function, a Dropout layer, and a second fully connected layer; The first fully connected layer is used to convert the dimension of the feature map output by the normalization layer into a feature map of a preset hidden feature dimension; The depthwise separable convolutional layer is used to convolve the feature map output by the first fully connected layer in depth and spatial dimensions; The GELU activation function is used to perform nonlinear transformation on the feature map output by the depth-separable convolutional layer; The Dropout layer is used to randomly set some feature values ​​in the feature maps output by the first fully connected layer, the depthwise separable convolutional layer and the GELU activation function to zero with a preset probability during the training process; The second fully connected layer is used to process the feature representation processed by the GELU activation function and the Dropout layer, and the output feature map is added to the feature map output by the multi-head self-attention mechanism and then output to the next DSC-Transformer Block module.

4. The synthetic aperture radar image denoising method according to claim 3, characterized in that: The depthwise separable convolution layer performs depthwise convolution and pointwise convolution on the feature map output by the normalization layer in sequence; in the depthwise convolution, there is a corresponding convolution kernel for each channel of the input feature map, and each convolution kernel convolves the features of each channel to generate an output feature map of the corresponding channel; in the pointwise convolution, a 1×1 convolution is performed on the feature map output by the depthwise convolution, and the features of all channels are linearly combined at each position.

5. The synthetic aperture radar image denoising method according to claim 1, characterized in that: The PEB module includes a two-dimensional convolutional layer, a CBAM layer and a normalization layer connected in sequence; The two-dimensional convolutional layer is used to divide the synthetic aperture radar image to be processed input into the PEB module into a plurality of overlapping feature blocks according to a preset size and stride; The CBAM layer is used to perform attention weighting on the multiple overlapping feature blocks from the channel dimension and the spatial dimension respectively; The normalization layer is used to normalize the weighted feature blocks.

6. The synthetic aperture radar image denoising evaluation method according to claim 5, characterized in that: The CBAM layer includes a channel attention unit and a spatial attention unit; The channel attention unit is used to: perform a global pooling operation on each channel of the multiple overlapping feature blocks to obtain a global average pooling feature and a global maximum pooling feature of each channel; input the two pooled features into a first multi-layer perceptron network to extract features; add the features extracted by the global average pooling feature and the global maximum pooling feature by the first multi-layer perceptron network and perform sigmoid activation function processing to obtain a channel attention weight; use the channel attention weight to weight the feature map of each channel of the multiple overlapping feature blocks to obtain a weighted feature map; The spatial attention unit is used to: perform maximum pooling and average pooling on the feature map weighted by the channel attention mechanism in the channel dimension, connect the pooled features through the concat function, extract features through the convolution layer to compress the channel dimension to 1, and finally obtain the spatial attention weight through the sigmoid activation function; The spatial attention weight is used to weight the feature map weighted by the channel attention mechanism to obtain the final feature map.

7. The synthetic aperture radar image denoising method according to claim 1, characterized in that: Each decoder consists of upsampling convolutional layers and residual modules connected in sequence; The upsampling convolution layer upsamples the input feature map by bilinear interpolation to double the size of the output feature map of the previous layer, and then performs convolution processing on the upsampled feature map to reduce the number of channels of the feature map. Finally, the size of the reconstructed feature map obtained by the convolution processing is the same as that of the input feature map, but the number of channels is reduced; The residual module uses a residual connection at each scale to add the features output by the residual module to the original features input to obtain the residual features of each scale.

8. The synthetic aperture radar image denoising method according to claim 1, characterized in that: The step of obtaining a synthetic aperture radar image to be processed comprises: Convert the original data of the three channels of the original image into a grayscale image to obtain a clean, spot-free image X; The Nakagami-Rayleigh distribution is used to generate the speckle noise N; Generate a synthetic aperture radar image to be processed through a speckle noise model Y=X*N of the synthetic aperture radar image, where Y is a synthetic aperture radar image with speckle noise; Among them, the amplitude of speckle N obeys Nakagami distribution, and the probability density function of speckle N is n and L are both greater than 0, Γ(·) represents the Gamma function, and L represents the number of views. The number of views L is used to describe the number of times or the number of observation angles that the synthetic aperture radar system observes the target area.

9. A synthetic aperture radar image denoising device, characterized in that: include: An image acquisition module, used for acquiring synthetic aperture radar images to be processed; An image denoising module, used for inputting the synthetic aperture radar image to be processed into an image denoising model, and outputting a denoised synthetic aperture radar image; Among them, the image denoising model includes multiple encoders and multiple decoders connected in sequence, each encoder includes a PEB module and multiple DSC-Transformer Block modules connected in sequence; the PEB module performs weighted processing on the input image from the channel dimension and the spatial dimension respectively through the channel attention mechanism and the spatial attention mechanism; each DSC-Transformer Block module calculates the attention weights between the features of the feature map output by the PEB module through the multi-head self-attention mechanism and performs weighted aggregation on the features, performs local feature extraction and nonlinear transformation on the weighted aggregated feature map through MLP, and then uses residual connection to add the features processed by the multi-head self-attention mechanism and MLP to the original input features; the output of each encoder except the last encoder is passed to the next encoder and the corresponding decoder, and the output of the last encoder is passed to the first decoder.

10. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the synthetic aperture radar image denoising method as described in any one of claims 1 to 8 are implemented.

Images