Complex format SAR image speckle noise suppression method based on virtual-real mutual guidance network

Abstract

The application relates to the field of image noise suppression, and discloses a complex format SAR image speckle noise suppression method based on a virtual-real mutual guidance network, which comprises the following steps: acquiring an original complex format SAR image, separating the real part and the imaginary part of the original complex format SAR image, and obtaining a real part two-dimensional matrix and an imaginary part two-dimensional matrix; constructing a virtual-real mutual guidance speckle noise suppression network; the virtual-real mutual guidance speckle noise suppression network comprises a real part sub-network and an imaginary part sub-network; training the virtual-real mutual guidance speckle noise suppression network by using a self-supervised loss function, obtaining a trained network; inputting the real part two-dimensional matrix and the imaginary part two-dimensional matrix into the trained network, and completing complex format SAR image speckle noise suppression; and the application effectively suppresses the speckle noise while keeping the SAR data phase accuracy.

Description

Technical Field

[0001] This invention relates to the field of image noise suppression, and more particularly to a method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network. Background Technology

[0002] SAR imagery, as an active detection technology, can be used for imaging around the clock and in all weather conditions, and is widely applied in various industries. Raw SAR data is typically in single-look complex format, containing the radiometric characteristics and phase information of the corresponding detection area. However, due to the imaging characteristics of SAR images, they are subject to severe speckle noise, which seriously affects subsequent SAR image interpretation and detection. Therefore, speckle noise suppression is a fundamental aspect of SAR image research.

[0003] Typically, researchers reduce speckle noise in raw single-look complex SAR images through multi-look processing, but this reduces spatial and phase resolution. Then, intensity images are calculated from the multi-look processed complex data, and dynamic range adjustment techniques are used to refine the SAR intensity image to obtain a relatively clear one. Currently, most researchers focus on suppressing speckle noise in the intensity images obtained through this process to improve the display quality of SAR images. However, suppressing speckle noise in the intensity domain often results in information loss. This is due to both the loss of spatial and phase resolution caused by multi-look processing and the loss of bit width accuracy caused by compressing the raw complex data to 8-bit intensity data. Summary of the Invention

[0004] The purpose of this invention is to propose a method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network, thereby solving the technical problem of information loss caused by existing methods when suppressing noise in SAR images.

[0005] Specifically, this invention provides a method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network, comprising the following steps:

[0006] S1. Obtain the original complex format SAR image and separate its real and imaginary parts to obtain a two-dimensional matrix of the real part and a two-dimensional matrix of the imaginary part;

[0007] S2. Construct a virtual-real mutually guided speckle noise suppression network; the virtual-real mutually guided speckle noise suppression network includes a real part subnetwork and a virtual part subnetwork;

[0008] S3. The virtual-real mutual guidance speckle noise suppression network is trained using a self-supervised loss function to obtain the trained network.

[0009] S4. Input the real part of the two-dimensional matrix and the imaginary part of the two-dimensional matrix into the trained network to complete the speckle noise suppression of complex format SAR images.

[0010] A storage device stores instructions and data for implementing a method to suppress speckle noise in complex format SAR images based on a virtual-real mutual guidance network.

[0011] A speckle noise suppression device for complex format SAR images based on a virtual-real mutual guidance network includes: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement a speckle noise suppression method for complex format SAR images based on a virtual-real mutual guidance network.

[0012] The beneficial effects provided by this invention are as follows: It proposes a speckle noise suppression method for complex format SAR images based on a virtual-real mutual guidance network. The method directly uses a virtual-real mutual guidance speckle noise suppression network in the complex domain to suppress speckle noise in the real and imaginary parts of the original complex format SAR image. The proposed self-supervised loss function is used to distinguish the amplitude and phase components contained in the real and imaginary parts, effectively suppressing speckle noise while maintaining the phase accuracy of SAR data. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the process for the complex format SAR image speckle noise suppression method based on a virtual-real mutual guidance network according to the present invention;

[0014] Figure 2 This is a schematic diagram of a mutually guiding speckle noise suppression network structure;

[0015] Figure 3 This is a schematic diagram of the speckle noise suppression module.

[0016] Figure 4 This is a result illustration of a method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network;

[0017] Figure 5 This is a schematic diagram of the hardware device used in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0019] Before formally describing the present invention, a general description of the solution of the present invention will be given first to facilitate understanding.

[0020] Please refer to Figure 1The present invention provides a method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network, comprising the following steps:

[0021] S1. Obtain the original complex format SAR image and separate its real and imaginary parts to obtain a two-dimensional matrix of the real part and a two-dimensional matrix of the imaginary part;

[0022] It should be noted that, firstly, this invention separates the real and imaginary two-dimensional matrices of the original complex-format SAR image I, transforming it into two two-dimensional matrices composed of real and imaginary coefficients, respectively. The original complex-format SAR image I is represented as follows:

[0023] (1)

[0024] Where I is the input original complex format SAR image, A and B are the real and imaginary two-dimensional matrices of the original complex format SAR image I, respectively, and i is the imaginary unit. Here, the size of I, A, and B is H×W. In subsequent steps, the real two-dimensional matrix A and the imaginary two-dimensional matrix B are processed separately, and speckle noise is suppressed separately.

[0025] S2. Construct a virtual-real mutually guided speckle noise suppression network; the virtual-real mutually guided speckle noise suppression network includes a real part subnetwork and a virtual part subnetwork;

[0026] It should be noted that the real part sub-network mentioned in step S2 includes: a first real part noise suppression module, a second real part noise suppression module, and a third real part noise suppression module.

[0027] It should be noted that the virtual part sub-network mentioned in step S2 includes: a first virtual part noise suppression module, a second virtual part noise suppression module, and a third virtual part noise suppression module.

[0028] S3. The virtual-real mutual guidance speckle noise suppression network is trained using a self-supervised loss function to obtain the trained network.

[0029] It should be noted that the self-supervised loss function mentioned in step S3 is as follows:

[0030]

[0031] in, These are the weighting coefficients. It is an L1 norm. It is an L2 norm; These are intensity images before and after speckle noise suppression, respectively. These are phase images before and after speckle noise suppression, respectively.

[0032] S4. Input the real part of the two-dimensional matrix and the imaginary part of the two-dimensional matrix into the trained network to complete the speckle noise suppression of complex format SAR images.

[0033] It should be noted that step S4 is as follows:

[0034] S41. Perform convolution operations on the real part of the two-dimensional matrix A and the imaginary part of the two-dimensional matrix B respectively to extract the shallow feature vectors, thus obtaining the real part shallow feature vectors. ;

[0035] S42. The extracted shallow feature vectors... The data are input into the first real part noise suppression module and the first imaginary part noise suppression module, respectively, to obtain the real part deep feature vector. Imaginary part deep feature vector ;

[0036] S43. For the deep feature vector of the first real part First imaginary part deep feature vector Perform feature concatenation to obtain the first feature vector after feature concatenation. ;

[0037] S44. The concatenated first feature vector The inputs are respectively fed into the second real part noise suppression module and the second imaginary part noise suppression module to obtain the second real part deep feature vector. Second imaginary part deep feature vector ;

[0038] S45. For the deep feature vector of the second real part Second imaginary part deep feature vector Perform feature concatenation to obtain the second feature vector after feature concatenation. ;

[0039] S46. The concatenated second feature vector The inputs are respectively fed into the third real part noise suppression module and the third imaginary part noise suppression module to obtain the third real part deep feature vector. The third imaginary part deep feature vector ;

[0040] S47. The third real part deep feature vector The third imaginary part deep feature vector The inputs are fed into two convolutional layers respectively. After learning the residuals of the corresponding parts, the real part of the input two-dimensional matrix A and the imaginary part of the input two-dimensional matrix B are added to the output real part two-dimensional matrix after speckle noise suppression. Imaginary two-dimensional matrix ;

[0041] S48. The real part of the two-dimensional matrix after suppressing speckle noise. Imaginary two-dimensional matrix By combining these methods, a complex-format SAR image with speckle noise suppression is obtained.

[0042] It should be noted that the network structures of the first real noise suppression module, the first imaginary noise suppression module, the second real noise suppression module, the second imaginary noise suppression module, the third real noise suppression module, and the third imaginary noise suppression module are the same, only the number of input and output channels are different.

[0043] It should be noted that the network structure of the first real part noise suppression module, the first imaginary part noise suppression module, the second real part noise suppression module, the second imaginary part noise suppression module, the third real part noise suppression module, and the third imaginary part noise suppression module consists of three sets of alternately connected 3×3 convolutional layers, activation functions, and one self-attention weight unit.

[0044] As an example, the present invention further elaborates on the method.

[0045] The real part of the two-dimensional matrix A and the imaginary part of the two-dimensional matrix B are input into the speckle noise suppression network that mutually guides the real and imaginary parts. The network then performs the suppression work on the speckle noise suppression network it contains, as shown below:

[0046] (2)

[0047] in, These are the real and imaginary two-dimensional matrices output by the speckle noise suppression network, respectively. GDSNet is the speckle noise suppression network proposed in this invention, which uses a combination of real and imaginary parts for mutual guidance. A schematic diagram of the corresponding network structure is shown below. Figure 2 As shown, the invention will be described in detail in the remainder of this document.

[0048] In a virtual-real mutually guided coherent speckle noise suppression network, the present invention

[0049] First, convolution operations are performed on the real part of the two-dimensional matrix A and the imaginary part of the two-dimensional matrix B to extract shallow feature vectors, as shown below:

[0050] (3)

[0051] (4)

[0052] Here, These are the real shallow feature vectors and imaginary shallow feature vectors extracted by the corresponding convolution operations, respectively. Both are 3×3 convolution operations, with 1 input channel and 64 output channels.

[0053] Then, the present invention extracts the shallow feature vectors. The noise is input into the first real part noise suppression module and the first imaginary part noise suppression module respectively to perform speckle noise suppression, as shown below:

[0054] (5)

[0055] (6)

[0056] in, These are the first real part deep feature vector and the first imaginary part deep feature vector after the first real part noise suppression module and the first imaginary part noise suppression module have suppressed speckle noise, respectively. These are the first real part noise suppression module and the first imaginary part noise suppression module, respectively. Their network structures are identical, and their corresponding structural diagrams are shown below. Figure 3 As shown, it has 64 input and output channels. However, during training, different parameters are assigned to emphasize the real and imaginary speckle noise suppression process, which will be described in detail in the next step.

[0057] also, Figure 2 The first real noise suppression module, the first imaginary noise suppression module, the second real noise suppression module, the second imaginary noise suppression module, the third real noise suppression module, and the third imaginary noise suppression module shown in the figure all use the same network structure, only differing in the number of input and output channels, with the corresponding number of channels being 64, 64, 128, 128, 256, and 256, respectively.

[0058] Next, the present invention performs feature concatenation on the first real part deep feature vector and the first imaginary part deep feature vector after the first real part noise suppression module and the first imaginary part noise suppression module have suppressed speckle noise. This integrates the information of the real part and the imaginary part, providing guidance information for subsequent noise suppression modules. The concatenation process is as follows:

[0059] (7)

[0060] Here, `cat` refers to the `torch.cat()` function in PyTorch, which concatenates feature vectors along the channel dimension. This is the first feature vector after feature concatenation, with 128 channels.

[0061] Then, the present invention will concatenate the feature vectors The noise is input into the second real part noise suppression module and the second imaginary part noise suppression module, respectively, to selectively suppress noise in the real and imaginary eigenvectors, as shown below:

[0062] (8)

[0063] (9)

[0064] in, These are the second real part deep feature vector and the second imaginary part deep feature vector after the second real part noise suppression module and the second imaginary part noise suppression module have suppressed speckle noise, respectively. These are the second real part noise suppression module and the second imaginary part noise suppression module, respectively. Their network structures are identical, and the corresponding structural diagrams are shown below. Figure 3 As shown, it has 128 input and output channels.

[0065] Next, the present invention repeats the above steps to transform the feature vector. The concatenation is performed along the channel dimension, as shown below:

[0066] (10)

[0067] Here, `cat` refers to the `torch.cat()` function in PyTorch, which concatenates feature vectors along the channel dimension. The second feature vector is the result of feature concatenation, and it has 256 channels.

[0068] Then, the present invention will concatenate the feature vectors The inputs are respectively sent to the third real part noise suppression module and the third imaginary part noise suppression module, as shown below:

[0069] (11)

[0070] (12)

[0071] in, These are the real and imaginary deep feature vectors after suppressing speckle noise, respectively. These are the third real part noise suppression module and the third imaginary part noise suppression module, respectively. Their network structures are identical, and the corresponding structural diagrams are shown below. Figure 3 As shown, it has 256 input and output channels.

[0072] Then, the present invention inputs the third real part deep feature vector and the third imaginary part deep feature vector after suppressing speckle noise into two convolutional layers respectively. After learning the residuals of the corresponding parts, the input real and imaginary parts are added to them respectively to output the real part two-dimensional matrix and the imaginary part two-dimensional matrix after speckle noise suppression, as shown below:

[0073] (13)

[0074] (14)

[0075] in, These are the real and imaginary two-dimensional matrices of the speckle noise suppression network output, respectively. conv3 and conv4 are both 3×3 convolution operations. The input channel is 256 and the output channel is 1.

[0076] In the above process, this invention has provided a detailed description of the speckle noise suppression network with mutual guidance between real and virtual components. Next, this invention will provide a detailed description of the first real-part noise suppression module, the first imaginary-part noise suppression module, the second real-part noise suppression module, the second imaginary-part noise suppression module, the third real-part noise suppression module, and the third imaginary-part noise suppression module used therein. They all employ the same network structure, differing only in the number of input and output channels, which are 64, 64, 128, 128, 256, and 256 respectively. Therefore, this invention will only provide a detailed description of one module, explaining its network structure, and its corresponding schematic diagram is shown below. Figure 3 As shown. Here, the present invention will be described using the first real part noise suppression module as an example. First, the present invention will... The input is fed into the successive convolutional layers and activation functions, as shown below:

[0077] (15)

[0078] in, The extracted real part deep feature vector is 1. ReLU is the torch.nn.RELU function in PyTorch, conv5 is a 3×3 convolution operation, and both the input and output channels are 64.

[0079] Then, the present invention repeats the above operations to obtain the second fully extracted real part deep feature vector, as shown below:

[0080] (16)

[0081] in, To extract the real part deep feature vector, conv6 is a 3×3 convolution operation with 64 input and 64 output channels.

[0082] Next, this invention utilizes a vector self-attention mechanism to... Assign appropriate weights to generate the residual feature vector after self-attention weight processing, as shown below:

[0083] (17)

[0084] in, This is the residual feature vector after self-attention processing. sigmod is the torch.nn.Sigmoid function in PyTorch, conv7 is a 3×3 convolution operation, and both the input and output channels are 64.

[0085] Finally, the present invention uses the residual feature vector obtained above. Add the real part of the shallow feature vector as input to this module The real part deep feature vector of this module after speckle noise suppression is obtained as follows:

[0086] (18)

[0087] in, This is the real deep feature vector output by this module after speckle noise suppression.

[0088] (4) Establish a self-built supervisory loss function

[0089] Through the above steps, this invention inputs the real two-dimensional matrix A and the imaginary two-dimensional matrix B into a speckle noise suppression network that mutually guides real and imaginary components, thereby obtaining the real two-dimensional matrix after speckle noise suppression. Imaginary two-dimensional matrix Next, this invention utilizes the input and output of the speckle noise suppression network, which mutually guides the real and virtual data, to establish a self-supervised loss function, thereby training the model to obtain the fitted model parameters. First, this invention calculates the phase and intensity images before and after speckle noise suppression, as shown below:

[0090] (19)

[0091] (20)

[0092] (twenty one)

[0093] (twenty two)

[0094] in, These are intensity images before and after speckle noise suppression, respectively. These are phase images before and after speckle noise suppression, respectively.

[0095] Then, this invention uses the phase and intensity images before and after speckle noise suppression calculated above to establish a self-supervised loss function, as shown below:

[0096] (twenty three)

[0097] in, These are the weighting coefficients. It is an L1 norm. It is the L2 norm. In this paper, the invention refers to... The values ​​are 0.01 and 1.

[0098] Finally, this invention utilizes the established loss function to perform back gradient propagation on the mutually guided speckle noise suppression network to train the network. When the number of network iterations is greater than... When the network reaches a fit, the present invention considers it to have achieved this fit and outputs the parameters of the coherent speckle noise suppression network that guide each other at this point. Perform the model reasoning process.

[0099] Please refer to Figure 4 , Figure 4 This image shows the results of a method for suppressing speckle noise in complex-format SAR images based on a virtual-real mutual guidance network. Figure 4 In the image, (a) represents the intensity image calculated from a complex-format SAR image with speckle noise; Figure 4 In the diagram, (b) represents the intensity image calculated from the complex format SAR image after speckle noise suppression using this method.

[0100] Please see Figure 5 , Figure 5 This is a schematic diagram of the hardware device in operation according to an embodiment of the present invention. The hardware device specifically includes: a complex format SAR image speckle noise suppression device 401 based on a virtual-real mutual guidance network, a processor 402, and a storage device 403.

[0101] A speckle noise suppression device 401 for complex format SAR images based on a virtual-real mutual guidance network: The speckle noise suppression device 401 for complex format SAR images based on a virtual-real mutual guidance network implements the speckle noise suppression method for complex format SAR images based on a virtual-real mutual guidance network.

[0102] Processor 402: The processor 402 loads and executes the instructions and data in the storage device 403 to implement the method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network.

[0103] Storage device 403: The storage device 403 stores instructions and data; the storage device 403 is used to implement the method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network.

[0104] In summary, the beneficial effects of this invention are: it proposes a speckle noise suppression method for complex format SAR images based on a virtual-real mutual guidance network, which directly uses a virtual-real mutual guidance speckle noise suppression network in the complex domain to suppress speckle noise in the real and imaginary parts of the original complex format SAR image, and uses a proposed self-supervised loss function to distinguish the amplitude and phase components contained in the real and imaginary parts, thereby effectively suppressing speckle noise while maintaining the phase accuracy of SAR data.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network, characterized in that: The method includes the following steps: S1. Obtain the original complex format SAR image and separate its real and imaginary parts to obtain a two-dimensional matrix of the real part and a two-dimensional matrix of the imaginary part; S2. Construct a virtual-real mutually guided speckle noise suppression network; the virtual-real mutually guided speckle noise suppression network includes a real part subnetwork and a virtual part subnetwork; S3. The virtual-real mutual guidance speckle noise suppression network is trained using a self-supervised loss function to obtain the trained network. S4. Input the real part two-dimensional matrix and the imaginary part two-dimensional matrix into the trained network to complete the speckle noise suppression of complex SAR images; The real part sub-network mentioned in step S2 includes: a first real part noise suppression module, a second real part noise suppression module, and a third real part noise suppression module; The virtual part sub-network mentioned in step S2 includes: a first virtual part noise suppression module, a second virtual part noise suppression module, and a third virtual part noise suppression module; Step S4 is as follows: S41. Perform convolution operations on the real part of the two-dimensional matrix A and the imaginary part of the two-dimensional matrix B respectively to extract the shallow feature vectors, thus obtaining the real part shallow feature vectors. ; S42. The extracted shallow feature vectors... The data are input into the first real part noise suppression module and the first imaginary part noise suppression module, respectively, to obtain the real part deep feature vector. Imaginary part deep feature vector ; S43. For the deep feature vector of the first real part First imaginary part deep feature vector Perform feature concatenation to obtain the first feature vector after feature concatenation. ; S44. The concatenated first feature vector The inputs are respectively fed into the second real part noise suppression module and the second imaginary part noise suppression module to obtain the second real part deep feature vector. Second imaginary part deep feature vector ; S45. For the deep feature vector of the second real part Second imaginary part deep feature vector Perform feature concatenation to obtain the second feature vector after feature concatenation. ; S46. The concatenated second feature vector The inputs are respectively fed into the third real part noise suppression module and the third imaginary part noise suppression module to obtain the third real part deep feature vector. The third imaginary part deep feature vector ; S47. The third real part deep feature vector The third imaginary part deep feature vector The inputs are fed into two convolutional layers respectively. After learning the residuals of the corresponding parts, the real part of the input two-dimensional matrix A and the imaginary part of the input two-dimensional matrix B are added to the output real part two-dimensional matrix after speckle noise suppression. Imaginary two-dimensional matrix ; S48. The real part of the two-dimensional matrix after suppressing speckle noise. Imaginary two-dimensional matrix By combining these methods, a complex-format SAR image with speckle noise suppression is obtained.

2. The method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network as described in claim 1, characterized in that: The self-supervised loss function described in step S3 is as follows: in, These are the weighting coefficients. It is an L1 norm. It is an L2 norm; These are intensity images before and after speckle noise suppression, respectively. These are phase images before and after speckle noise suppression, respectively.

3. The method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network as described in claim 1, characterized in that: The network structures of the first real part noise suppression module, the first imaginary part noise suppression module, the second real part noise suppression module, the second imaginary part noise suppression module, the third real part noise suppression module, and the third imaginary part noise suppression module are the same, only the number of input and output channels are different.

4. The method for suppressing speckle noise in complex format SAR images based on a virtual-real mutual guidance network as described in claim 3, characterized in that: The network structure of the first real part noise suppression module, the first imaginary part noise suppression module, the second real part noise suppression module, the second imaginary part noise suppression module, the third real part noise suppression module, and the third imaginary part noise suppression module consists of three sets of alternately connected 3×3 convolutional layers, activation functions, and one self-attention weight unit.

5. A storage device, characterized in that: The storage device stores instructions and data for implementing the complex format SAR image speckle noise suppression method based on a virtual-real mutual guidance network as described in any one of claims 1 to 4.

6. A speckle noise suppression device for complex format SAR images based on a virtual-real mutual guidance network, characterized in that: include: A processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the complex format SAR image speckle noise suppression method based on a virtual-real mutual guidance network as described in any one of claims 1 to 4.

Images