SAR image speckle noise processing method based on deep learning

Abstract

The application discloses a SAR image coherent speckle noise processing method based on deep learning, wherein a real coherent speckle noise SAR image is first input into a coherent speckle noise coding network to extract a coherent speckle noise representation vector. Then, the coherent speckle noise representation vector and an input noise-free optical remote sensing image are input into a coherent speckle noise reconstruction network, the coherent speckle noise of the input real SAR image is transferred to the input noise-free optical remote sensing image, thereby obtaining a real coherent speckle noise degraded optical remote sensing image, and a data set of a coherent speckle noise-free SAR image and a coherent speckle noise SAR image is constructed. The problem that a real coherent speckle noise-free SAR image is difficult to obtain in a SAR image coherent speckle noise suppression method based on deep learning is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a SAR image speckle noise processing method based on deep learning. BACKGROUND

[0002] SAR images are remote sensing images generated by a synthetic aperture radar (SAR) system. As an active detection technology for earth observation, the synthetic aperture radar can be used without being affected by the night and adverse weather conditions, has the characteristics of all-weather and all-time imaging, and is successfully applied in many fields such as military, surveying and mapping, geological detection, climate warning, forest fire prevention, and rescue and disaster relief.

[0003] However, due to the limitation of the imaging principle of the synthetic aperture radar, the SAR image is seriously affected by the speckle noise, which greatly hinders the interpretation of the image and the subsequent application work. Therefore, a large number of researchers have carried out corresponding work on the suppression of SAR image speckle. At present, the mainstream method is to use the method based on deep learning for research, and the method based on deep learning mostly uses the technical means of supervised learning for training work, which has a significant effect on the suppression of SAR image speckle noise. However, the method of supervised learning usually needs to construct a training data pair of a SAR image without speckle damage and a SAR image damaged by speckle, and the SAR image without speckle damage cannot be obtained in principle because the speckle noise will inevitably be introduced in the imaging process of the SAR image. At present, the mainstream technical means mostly uses synthetic data to construct the training data pair, converts the clear visible light image after denoising into a gray image as the SAR image without speckle damage, and adds the synthetic speckle noise to the gray image as the SAR image damaged by speckle. However, the data set constructed in this way cannot effectively simulate the distribution state and the corresponding real speckle noise generation process of the real speckle noise, so as to cause the real speckle damaged SAR image to be unable to be effectively processed to effectively suppress the real speckle noise. SUMMARY

[0004] The purpose of the present application is to overcome the shortcomings of the prior art, provide a more reasonable speckle noise processing method, use the speckle noise encoding network to extract the speckle noise in the real speckle noise SAR image, and then use the speckle noise reconstruction network to transfer the extracted speckle noise to the optical remote sensing image to generate the SAR image damaged by speckle, so that the synthesized SAR image damaged by speckle is more in line with the characteristics of the real speckle noise.

[0005] By using the coherent speckle noise processing method, a highly real image is obtained, and the processed image is selected according to different geological characteristics to establish a training set, and a corresponding supervised learning model achieves a better coherent speckle noise suppression result.

[0006] To achieve the above technical purposes, the technical scheme adopted by the present application is:

[0007] A SAR image coherent speckle noise processing method based on deep learning, comprising the following steps,

[0008] 1) converting a SAR image with real coherent speckle noise into a high-dimensional deep feature vector,

[0009] 2) inputting the high-dimensional deep feature vector through multiple convolution layers and activation functions into an average pooling layer for feature simplification, and then again through multiple convolution layers and activation functions to obtain a coherent speckle noise representation vector,

[0010] 3) converting an optical remote sensing image into a high-dimensional shallow feature vector,

[0011] 4) adjusting the dimension of the coherent speckle noise representation vector and calculating the modulated weight,

[0012] 5) inputting the modulated weight and the high-dimensional shallow feature vector into a demodulation module to obtain a demodulated feature vector,

[0013] 6) inputting the demodulated feature vector into a convolution layer and an activation function to obtain an optical remote sensing feature vector containing coherent speckle noise,

[0014] 7) replacing the high-dimensional shallow feature vector with the optical remote sensing feature vector containing coherent speckle noise, and repeatedly executing steps 4) to 6) a predetermined number of times to output an optical remote sensing image feature vector with sufficient coherent speckle noise transmission,

[0015] 8) inputting the optical remote sensing image feature vector with sufficient coherent speckle noise transmission into a convolution layer to obtain an image with real coherent speckle noise.

[0016] As one of the preferred schemes, the network with UNet structure in step 1) is used to obtain the high-dimensional deep feature vector.

[0017] As one of the preferred schemes, in step 2), the high-dimensional deep feature vector is inputted into an average pooling layer for feature simplification after passing through 3 convolution layers and activation functions, and then again through 3 convolution layers and activation functions to obtain a coherent speckle noise representation vector.

[0018] As one of the preferred schemes, step 4) further comprises the step of optimizing and simplifying the shallow feature vector.

[0019] As one of the preferred solutions, the method for performing the dimension adjustment on the shallow feature vector in step 4) is ,

[0020]

[0021]

[0022] wherein, , are both 3x3 convolution operations, is an activation function in torch.nn, , are the optical remote sensing feature vectors after the successive dimension adjustment and optimization, respectively.

[0023] As one of the preferred solutions, the method for performing the dimension adjustment on the shallow feature vector in step 4) is ,

[0024]

[0025] wherein, is a learnable weight parameter, is a scale quantization parameter, is the weight of the modulated speckle noise representation vector E after the modulation by the modulation module.

[0026] As one of the preferred solutions, the modulated weight in step 4) and the dimension-adjusted feature vector after the dimension adjustment are input into the following demodulation module,

[0027]

[0028] wherein, is the summation operation on the 2nd, 3rd and 4th dimensions of the vector, respectively, is the demodulated noise-transferred feature vector output by the demodulation module.

[0029] As one of the preferred solutions, step 8) further includes an optimization processing step of performing a convolution layer and an activation function on the optical remote sensing image feature vector after the sufficient transmission of the speckle noise.

[0030] The advantages and beneficial effects of the present application are:

[0031] The application uses a coherent speckle noise coding network to extract a noise representation vector of a SAR image with real coherent speckle noise, so as to process an input optical remote sensing image by using the noise representation vector in a coherent speckle noise reconstruction network to obtain a synthetic image damaged by real coherent speckle noise, solving the problem that a real SAR image without coherent speckle noise is difficult to obtain in a SAR image coherent speckle noise suppression method based on deep learning. The method first inputs a SAR image with real coherent speckle noise into a coherent speckle noise coding network to extract a coherent speckle noise representation vector, which represents the intensity, spatial distribution state and corresponding potential degradation model of the coherent speckle noise of the input real SAR image. Then, the coherent speckle noise representation vector and the input noise-free optical remote sensing image are input into a coherent speckle noise reconstruction network, the coherent speckle noise of the input real SAR image is transmitted to the input noise-free optical remote sensing image, so as to obtain a real coherent speckle noise degraded optical remote sensing image, and the data set construction of the SAR image without coherent speckle noise and the SAR image with coherent speckle noise is completed. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 It is a flowchart of the application.

[0033] Figure 2 It is a schematic diagram of the coherent speckle noise coding network.

[0034] Figure 3 It is a schematic diagram of the coherent speckle noise reconstruction network.

[0035] Figure 4 It is a schematic diagram of the vector guidance generation module.

[0036] Figure 5 It is a real coherent speckle noise transmission result map.

[0037] For those skilled in the art, other related drawings can be obtained from the above drawings without creative labor. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the application scheme, the technical scheme of the application will be further described below in combination with the drawings and specific embodiments.

[0039] The SAR image coherent speckle noise processing method based on deep learning of the application comprises the following steps,

[0040] 1) converting a SAR image with real coherent speckle noise into a high-dimensional deep feature vector,

[0041] 2) input the high-dimensional deep feature vector into the average pooling layer after passing through multiple convolution layers and activation functions to simplify the features, and then pass through multiple convolution layers and activation functions again to obtain a speckle noise representation vector,

[0042] 3) convert the optical remote sensing image into a high-dimensional shallow feature vector, preferably using a noise-free optical remote sensing image,

[0043] 4) adjust the dimension of the speckle noise representation vector and calculate the modulated weight,

[0044] 5) input the modulated weight and high-dimensional shallow feature vector into a demodulation module to obtain a demodulated feature vector,

[0045] 6) input the demodulated feature vector into a convolution layer and an activation function to obtain an optical remote sensing feature vector containing speckle noise,

[0046] 7) replace the high-dimensional shallow feature vector with the optical remote sensing feature vector containing speckle noise, and repeat the iteration of steps 4) to 6) for a predetermined number of times to output an optical remote sensing image feature vector with sufficient speckle noise transmission,

[0047] 8) input the optical remote sensing image feature vector with sufficient speckle noise transmission into a convolution layer to obtain an image with real speckle noise.

[0048] The prior art simply considers the speckle noise as conforming to a gamma distribution, randomly generates a value of the gamma distribution with the same size as the image, and then multiplies it with the optical remote sensing grayscale image to obtain a synthesized SAR image damaged by speckle noise. The difference between the synthesis here and the synthesis of the present method is mainly that the synthesized speckle noise of the traditional method is only modeled as multiplicative noise through the gamma distribution, which cannot truly reflect the real degraded speckle noise; the present application uses a deep learning method to transfer the speckle noise on the real SAR image to the optical remote sensing image, which can be more consistent with the real speckle noise distribution.

[0049] The present application proposes a more reasonable method for constructing a speckle noise synthesis dataset, uses a speckle noise encoding network to extract the speckle noise in the real speckle noise SAR image, and then uses a speckle noise reconstruction network to transfer the extracted speckle noise to the optical remote sensing image to generate a SAR image damaged by speckle, so that the synthesized SAR image damaged by speckle is more consistent with the characteristics of real speckle noise, and the corresponding supervised learning model achieves better speckle noise suppression results.

[0050] The whole process will be described in two parts of coherent speckle noise extraction and reconstruction, the coherent speckle noise coding network is used to realize the coherent speckle noise extraction, and the coherent speckle reconstruction network is used to realize the coherent speckle noise reconstruction, which will be described in detail,

[0051] 1. Coherent speckle noise coding network

[0052] Firstly, the coherent speckle noise coding network is used to extract the coherent speckle noise representation vector of the SAR image with real coherent speckle, which is specifically as follows:

[0053] (1)

[0054] Wherein, is the extracted coherent speckle noise representation vector, which is specifically a one-dimensional vector with a size of 256. is the SAR image with real coherent speckle noise, is the coherent speckle noise coding network, and the detailed structure is shown in Figure 2 .

[0055] As one of the embodiments, the input SAR image with real coherent speckle noise is converted into a high-dimensional deep feature vector through a deep feature extraction module. The deep feature extraction module is a network with a classic UNet structure, and the corresponding structure information is: the network structure is 4 layers, the corresponding number of encoding blocks is 2, 2, 4 and 8 respectively, the corresponding number of decoding blocks is 8, 4, 2 and 2 respectively, and the number of modules in the middle layer is 16. Here, only the above introduction of the classic UNet network used in the present application is given, and no detailed description is given. Through the UNet structure deep feature extraction module, the size of is converted into a deep feature vector with a size of , which is specifically as follows:

[0056] (2)

[0057] Wherein, UNet is a deep feature extraction module, is the extracted deep feature vector, , W are the height and width of the input image respectively. The 1 in the input SAR image is the channel number of the input SAR image, which is 1, the in the deep feature vector is the vector dimension of the deep feature vector, ​is set to 64, the larger the dimension, the larger the model, and the higher the computational complexity. It can also be set to 32, but this will reduce the performance of the model, and setting it to 64 is a performance and computational load compromise.

[0058] Then, the extracted deep feature vector is input into the convolutional layer one and the activation function to obtain the second deep feature vector , as shown below:

[0059] (3)

[0060] wherein, is an activation function in torch.nn, and other appropriate activation functions can also be selected, and the same applies below, is a 3x3 convolution operation. After the convolutional layer one and the activation function, the size of the second deep feature vector becomes .

[0061] The second deep feature vector continues to be operated on and is input into the convolutional layer two and the activation function to obtain the third deep feature vector , as shown below:

[0062] (4)

[0063] Herein, is an activation function in torch.nn, is a 3x3 convolution operation. After the convolutional layer two and the activation function, the size of the third deep feature vector becomes .

[0064] The third deep feature vector continues to be operated on and is input into the convolutional layer three and the activation function to obtain the fourth deep feature vector , as shown below:

[0065] (5)

[0066] Herein, is an activation function in torch.nn, is a 3x3 convolution operation. After the convolutional layer three and the activation function, the size of the fourth deep feature vector becomes . After the above series of convolution operations, the fourth deep feature vector is input into the average pooling layer for feature reduction, as shown below:

[0067] (6)

[0068] wherein, is the fifth layer feature vector after feature reduction, is the pooling function in torch.nn. After the average pooling layer, the size of the fifth layer feature vector becomes . Then, the fifth layer feature vector after feature reduction is input into the fourth convolutional layer and the activation function to obtain the sixth deep layer feature vector , as follows:

[0069] (7)

[0070] wherein, is the activation function in torch.nn, is the 1x1 convolution operation. After the fourth convolutional layer and the activation function, the size of the sixth deep layer feature vector remains unchanged. The sixth deep layer feature vector is continuously operated and input into the fifth convolutional layer and the activation function to obtain the seventh deep layer feature vector , as follows:

[0071] (8)

[0072] wherein, is the activation function in torch.nn, is the 1x1 convolution operation. After the fifth convolutional layer and the activation function, the size of the seventh deep layer feature vector remains unchanged. Finally, the seventh deep layer feature vector is input into the sixth convolutional layer to obtain the extracted coherent speckle noise representation vector , as follows:

[0073] (9)

[0074] wherein, is the 1x1 convolution operation, is the extracted coherent speckle noise representation vector, and the vector size is .

[0075] 2. Coherent speckle noise reconstruction network

[0076] Through the above steps, the real coherent speckle noise representation vector is obtained, which can fully express the distribution state, degree and corresponding real coherent speckle noise generation process of the noise. Then, the coherent speckle noise reconstruction network is used to use the coherent speckle noise representation vector to complete the transmission of the real coherent speckle noise, and the real coherent speckle noise is transmitted from the original SAR image to the clean and noise-free remote sensing image, and a more reasonable synthetic data set is constructed. The specific process is as shown in Figure 3

[0077] Firstly, the SAR image is a single-channel gray image, and for the optical remote sensing image, the visible light image is converted into a single-channel gray image to simulate the SAR image. Then the optical remote sensing image is input into the shallow feature extraction module, and the original optical remote sensing image is converted into a high-dimensional shallow feature vector, as shown below:

[0078] (10)

[0079] wherein, is a 3x3 convolution operation, is the extracted high-dimensional shallow feature vector, is the input original noise-free optical remote sensing image.

[0080] Next, the extracted high-dimensional shallow feature vector and the extracted coherent speckle noise representation vector are input into the four representation vector guided generation modules in turn, so that the coherent speckle noise is fully transmitted to the feature vector of the optical remote sensing image, as shown below:

[0081] (11)

[0082] (12)

[0083] (13)

[0084] (14)

[0085] wherein, , , , are the same representation vector guided generation modules, , , , are the corresponding , , , ​The module generates optical remote sensing feature vectors containing speckle noise in turn. Here, the structures of the four representation vector guiding generation modules described above are all the same, and only one of them is described in detail below, and the corresponding structural diagram is shown in . Figure 4 .

[0086] The high-dimensional shallow feature vector and the extracted speckle noise representation vector are input into the feature refinement module, and the refined and optimized feature vector is obtained as follows:

[0087] (15)

[0088] (16)

[0089] wherein, , are 3x3 convolution operations respectively, is an activation function in torch.nn, , are the optical remote sensing feature vectors refined and optimized in turn.

[0090] Then, the speckle noise representation vector is input into the modulation module to perform noise vector modulation operation. First, the speckle noise representation vector is subjected to dimension expansion operation, and its dimension is expanded from to . Then, the expanded speckle noise representation vector is subjected to operation, and the modulation operation is completed as follows:

[0091] (17)

[0092] wherein, is a learnable weight parameter, and the initial value is set to a random number between 0 and 1 with a size of . The initial value can be set to 1, and the value of the parameter can be updated during the back propagation in the training process, and gradually becomes optimal with the training process. is a scale quantization parameter, and its value is determined by the input feature vector and the size of the selected convolution kernel. The convolution kernel size used in the present application is 3x3, and the vector dimension is 64, so 3x3x64=576.

[0093] is the speckle noise representation vector ​​The modulated weight is set according to the size of the selected convolution kernel and the dimension of the input feature vector. Then, the modulated weight and the simplified and optimized feature vector is input into the demodulation module to perform a coherent speckle noise transfer operation, specifically:

[0094] (18)

[0095] wherein, is a summation operation on the 2nd, 3rd and 4th dimensions of the vector respectively, is the feature vector after noise transfer output by the demodulation module.

[0096] The demodulated feature vector is input into a convolution layer and an activation function in sequence to obtain the final representation vector guiding the generation module output by the optical remote sensing feature vector containing coherent speckle noise , as shown below:

[0097] (19)

[0098] wherein, is a 3x3 convolution operation, is an activation function in torch.nn.

[0099] The above is the entire process of the representation vector guiding the generation module , and then is executed three times according to formulas (11)-(14) to obtain the optical remote sensing image feature vector with sufficient coherent speckle noise transfer. In order to fully transfer the noise, setting four times is a compromise between computational load and model performance. Setting less will reduce performance, and setting more will increase computational load.

[0100] Then, a convolution layer and an activation function are executed in succession on , as shown below:

[0101] (20)

[0102] wherein, is a 3x3 convolution operation, is an activation function in torch.nn, is the optical remote sensing image feature vector with real coherent speckle noise after optimization processing. Finally, is input into a convolution layer to obtain the final optical remote sensing image with real coherent speckle noise, as shown below:

[0103] (21)

[0104] wherein, is a 3x3 convolution operation, is the final optical remote sensing image with real speckle noise.

[0105] As Figure 5 shown, the left column of three images shows noise-free optical remote sensing images, the middle column of three images has SAR images with real speckle noise, and the right column of three images is the corresponding optical remote sensing image after speckle noise transmission after processing using the method of the present application. The figures shown are only a few data, and real generated data will use a large number of real SAR images to generate different types of data, which will include various terrain characteristics, including natural characteristics such as mountains, deserts, towns, and grasslands, and will also include some dynamic changes in terrain and geology, such as sea surface pollution, plant dynamic growth stages, etc. A large number of initial images with different characteristics are used to improve the applicability after processing by the method of the present application, and this can improve the training effect of the model as a training set.

[0106] To further improve the processing effect, environmental features are added as additional inputs in each encoding and decoding block for different environments, a skip connection is added, and the environmental features are directly transmitted to the corresponding decoding block. An adaptive feature fusion layer can also be set to dynamically adjust the weights of the UNet output features and the environmental features, and an attention mechanism is used to achieve adaptive fusion of the features.

[0107] In the polar ice and snow monitoring scenario, SAR images are often used to monitor glacier changes and sea ice coverage. However, SAR images in the polar environment are often affected by more complex speckle noise, which is caused by the special atmospheric conditions, temperature changes, and surface reflection characteristics in the polar region. In this case, traditional speckle noise encoding networks may not be able to effectively capture the special noise patterns in polar SAR images.

[0108] Further, an adaptive special terrain such as polar coherent spot noise coding network is proposed. The network adds a special environment feature extraction module such as a polar environment feature extraction module on the basis of the original UNet structure. This module contains multiple parallel spatial attention mechanisms, and each attention mechanism is used to extract different types of polar environment features (such as ice surface texture, snow depth, etc.). The coherent spot noise representation vector processed by the spatial attention mechanism has the specificity of the corresponding terrain. The training set established for the terrain features after processing this type of image can reflect higher authenticity. In specific implementation, a simple spatial attention module can usually be constructed using convolutional layers, pooling layers and activation functions. Then multiple spatial attention modules are applied in parallel to the input initial SAR image, and then the output feature maps are fused to obtain multiple attention-enhanced feature maps. Different fusion methods such as splicing fusion, addition fusion and weighted addition fusion can be used, and different fusion methods can be selected according to the situation. For example, for the situation that requires certain adaptive ability and needs to learn the importance of different attention mechanisms, the weighted addition fusion method can be selected for the situation that the importance of different attention mechanisms is uncertain.

[0109] The above is an exemplary description of the present application, and it should be noted that any simple modification, modification or other equivalent replacement that does not deviate from the core of the present application and can be made by those skilled in the art without creative labor falls within the protection scope of the present application.

Claims

1.A method for SAR image speckle noise processing based on deep learning, characterized in that, The method comprises the following steps: 1) converting a SAR image with real speckle noise into a high-dimensional deep feature vector, 2) inputting the high-dimensional deep feature vector into an average pooling layer for feature reduction after passing through multiple convolution layers and activation functions, and then passing through multiple convolution layers and activation functions again to obtain a speckle noise representation vector, 3) converting an optical remote sensing image into a high-dimensional shallow feature vector, 4) adjusting the dimension of the speckle noise representation vector and calculating the modulated weight, 5) inputting the modulated weight and the high-dimensional shallow feature vector into a demodulation module to obtain a demodulated feature vector, 6) inputting the demodulated feature vector into a convolution layer and an activation function to obtain an optical remote sensing feature vector containing speckle noise, 7) replacing the high-dimensional shallow feature vector with the optical remote sensing feature vector containing speckle noise, and repeating the iteration of steps 4) to 6) for a predetermined number of times to output an optical remote sensing image feature vector with sufficient speckle noise transmission, 8) inputting the optical remote sensing image feature vector with sufficient speckle noise transmission into a convolution layer to obtain an image with real speckle noise. 2.The deep learning-based SAR image speckle noise processing method of claim 1, wherein, In step 1), a network with UNet structure is used to obtain a high-dimensional deep feature vector. 3.The deep learning-based SAR image speckle noise processing method of claim 1, wherein, In step 2), the high-dimensional deep feature vector is input into an average pooling layer for feature reduction after passing through 3 convolution layers and activation functions, and then passing through 3 convolution layers and activation functions again to obtain a speckle noise representation vector. 4.The deep learning-based SAR image speckle noise processing method of claim 1, wherein, Step 4) further comprises the step of optimizing the shallow feature vector. 5.The deep learning-based SAR image speckle noise processing method of claim 4, wherein, The method for performing the refinement optimization in step 4) is to perform the refinement optimization on the shallow feature vector the method for performing the refinement optimization is to ， ， wherein, , are both 3x3 convolution operations, is an activation function in torch.nn, , are the optical remote sensing feature vectors after successive pruning optimizations, respectively. 6.The deep learning-based SAR image speckle noise processing method of claim 5, wherein, the dimension-adjusted coherent speckle noise representation vector in step 4) The method for modulating is that, ， wherein, is a learnable weight parameter, is a scale quantization parameter, is a weight of the modulating module after modulating the speckle noise representation vector E. 7.The deep learning-based SAR image speckle noise processing method of claim 6, wherein, The weights modulated in step 4) and the reduced optimized feature vectors are input to the following demodulation module, ， wherein, is a sum operation on the 2nd, 3rd, 4th dimension of the vector, is a demodulated noise transferred feature vector output by the demodulation module. 8.The deep learning-based SAR image speckle noise processing method of claim 1, wherein, In step 8), an optimization processing step of performing a convolution layer and an activation function on the optical remote sensing image feature vector with sufficient speckle noise transmission is further included.

Images