SAR (Synthetic Aperture Radar) image speckle suppression method based on depth denoising device prior, terminal equipment and storage medium

Abstract

The invention discloses an SAR (Synthetic Aperture Radar) image speckle suppression method based on depth de-noising device prior, terminal equipment and a storage medium, the method combines simulation data and a real SAR image for training, the spatial correlation characteristic of a real speckle is destroyed through random downsampling, and then the spatial correlation characteristic of the real speckle is destroyed through random downsampling; the output of an external de-noising device trained by simulation data is used as a supervision signal, and a speckle suppression network is trained; according to the method, the problem that deep learning is difficult to effectively apply due to the fact that a speckle suppression label does not exist in a real SAR image can be solved, spatially correlated speckles in the real SAR image can be effectively suppressed, the quality and the recognizability of the SAR image are improved, and a high-quality sample is provided for a downstream SAR image interpretation task.

Description

Technical Field

[0001] The present invention relates to the field of radar image intelligent processing and computer intelligence, and in particular to a SAR image coherent speckle suppression method based on a priori deep denoiser, a terminal device and a storage medium. Background Art

[0002] Synthetic Aperture Radar (SAR) can produce stable images under any weather and lighting conditions and is an important means of obtaining earth observation information. SAR uses a coherent imaging mechanism. When the radar beam is irradiated onto the rough surface of the ground, the signal returned in each resolution unit contains echoes from many scatterers. The inconsistency of the phase of the echoes from these scatterers causes random fluctuations in the echo intensity at the pixel level, forming coherent speckle noise in the SAR image. Coherent speckle will damage the target structure in the image and interfere with the interpretation of the SAR image. Therefore, suppressing the coherent speckle of SAR images to improve image clarity is of great significance to the interpretation of SAR images.

[0003] Research on SAR image speckle reduction algorithms has been going on for decades. In recent years, deep learning has become the mainstream method for SAR speckle reduction due to its powerful data fitting ability. Supervised learning methods require a large amount of labeled data for training. However, the labels corresponding to SAR speckle reduction (i.e., noise-free SAR images) do not exist. Therefore, supervised learning methods are generally trained using simulated data. Simulated noise cannot simulate the spatial correlation characteristics of real speckles, resulting in poor speckle reduction effects when supervised learning methods are applied to real SAR images.

[0004] The denoiser prior method is a traditional image restoration method. Its main idea is to use the output of an external denoiser as a constraint to guide the optimization process of image restoration. This method is applicable to most image restoration tasks. Combining the denoiser prior method with deep learning and using the output of the external denoiser as a label can alleviate the problem that supervised learning is difficult to apply in SAR speckle suppression due to the absence of labels. However, how to construct an external denoiser that is effective for SAR speckle suppression is a difficult problem that needs to be solved urgently.

[0005] Invention patent application CN118379218A discloses a pyramid-based SAR image speckle suppression method, which only uses simulated data for training, and the network cannot process spatially correlated speckles in real SAR images. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a SAR image speckle suppression method, terminal device and storage medium based on deep denoiser prior in view of the shortcomings of the existing technology, so as to solve the problem that supervised learning methods are difficult to handle real speckles, and can effectively suppress the speckles in SAR images and improve the clarity and recognizability of SAR images.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a SAR image speckle suppression method based on deep denoiser prior, comprising the following steps:

[0008] S1. Build a simulated speckle noise dataset and pre-train a deep learning denoiser.

[0009] S2, acquiring a SAR image, normalizing the SAR image, and constructing a training set using the normalized SAR image;

[0010] S3, sampling a SAR image from the training set, using the sampled SAR image as an input of a speckle suppression network to obtain a denoised SAR image, and randomly downsampling the denoised SAR image to obtain a denoised SAR image with a length and a width both reduced by half;

[0011] S4. Using the denoised SAR image with the length and width reduced by half as the input of the deep learning denoiser to obtain a secondary denoised image, using the secondary denoised image as a label, calculating a loss function, and updating the coherent speckle suppression network.

[0012] The present invention uses the output of a deep denoiser pre-trained on a simulation data set as a priori to guide the training of a coherent speckle suppression network. In addition, the spatial correlation characteristics of coherent speckles in SAR images are destroyed by a random downsampling method, so that the deep denoiser trained on the simulation data can effectively process coherent speckles, thereby generating effective supervisory information. The present invention can solve the problem that supervised learning methods are difficult to process real coherent speckles, can effectively suppress coherent speckles in SAR images, and improve the clarity and recognizability of SAR images.

[0013] The method of the present invention further comprises:

[0014] S5. Repeat steps S3 and S4 until the set number of training rounds is reached.

[0015] In the present invention, the simulated speckle noise data is expressed as: z=s·n; wherein n represents multiplicative gamma noise with a mean and a variance of 1, s represents a clean image, and z represents a simulated noise image.

[0016] The deep learning denoiser and the speckle suppression network both use the U-Net network architecture. The advantages of the U-Net network are that it has a simple structure, is easy to train, is suitable for image denoising tasks, and has excellent performance.

[0017] In the present invention, the specific implementation process of normalizing the SAR image includes: sorting the pixel values ​​of the SAR image in ascending order, taking the pixel value that is M% away from the maximum value as the upper limit, modifying all pixel values ​​greater than the upper limit to be equal to the upper limit value, and dividing all pixel values ​​of the SAR image by the upper limit value to obtain a normalized SAR image. Through normalization, the pixel values ​​of the input SAR image can be constrained to be within the same range, making the denoising performance of the network more stable.

[0018] In the present invention, the specific implementation process of the random downsampling includes: dividing the denoised SAR image into 2×2 small blocks along the pixel interval, randomly selecting a pixel in the small block, discarding the remaining pixels, combining all the retained pixels, and obtaining a denoised SAR image with the length and width reduced by half.

[0019] The expression of the loss function L is:

[0020]

[0021] Among them, y i represents the i-th SAR image, represents the i-th denoised image output by the speckle suppression network, λ is the weight parameter, is the i-th image obtained by random downsampling, Stopgrad indicates gradient interruption, D is the pre-trained external denoiser, Represents the output of the external denoiser, which is the deep denoiser prior.

[0022] As an inventive concept, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the steps of the above method.

[0023] As an inventive concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon; characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention uses simulated data and real SAR images in training, and through random downsampling, the present invention can more effectively suppress the coherent speckle in the real SAR image. The present invention destroys the spatial correlation characteristics of the coherent speckle in the SAR image through the random downsampling method, so that the external denoiser trained with simulated data can suppress the coherent speckle noise of correlation decoupling in SAR, thereby forming supervisory information; the present invention combines the denoiser prior with deep learning, and uses the denoiser prior as a training label, which effectively solves the problem of the non-existence of the coherent speckle suppression label of the SAR image, so that the deep learning method can effectively process the real coherent speckle with complex distribution in the SAR image, improve the quality of the SAR image, and has positive significance for the subsequent SAR image interpretation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0026] Figure 2 This is an example diagram of simulation data used for pre-training an external denoiser according to an embodiment of the present invention;

[0027] Figure 3 Schematic diagram of the training architecture of the speckle suppression network based on the deep denoiser prior;

[0028] Figure 4 Schematic diagram of the network structure used by the coherent speckle suppression network and the external denoiser. Both the coherent speckle suppression network and the external denoiser use Figure 4 The network architecture in

[0029] Figure 5 The results of SAR image speckle suppression are shown in Figure 2; (a) SAR image, (b) PPB, (c) SAR-BM3D, (d) Speckle2Void, (e) MERLIN, (f) deep denoiser prior;

[0030] Figure 6 The results of SAR image speckle suppression; (a) SAR image, (b) PPB, (c) SAR-BM3D, (d) Speckle2Void, (e) MERLIN, (f) deep denoiser prior. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Example 1

[0033] Figure 1 A method for suppressing coherent speckle in SAR images based on a deep denoiser prior is provided in an embodiment of the present invention, and specifically comprises the following steps:

[0034] Step 1: Add simulated noise based on the grayscale image of the ordinary optical image, and synthesize the simulated data set to train the external denoiser (deep learning denoiser);

[0035] Sub-step 1.1: Add simulated coherent speckle noise to the grayscale image of the ordinary optical image, which can be expressed as z=s·n, where n represents multiplicative gamma noise with a mean and variance of 1, s represents a clean image, and z represents a simulated noisy image. A total of 5000 images are simulated;

[0036] Sub-step 1.2: Use the simulated dataset to train a deep denoiser. The denoiser network structure is shown in Figure 4 , the loss function is Where D represents the denoiser, i represents the i-th training sample, and z i represents the i-th simulated noise image, s i represents the i-th clean image, the number of training rounds is 200, and the optimizer is Adam.

[0037] Step 2: Collect SAR images, normalize the SAR images and construct a training data set. The normalization method is to sort the pixel values ​​of the SAR images from small to large, take the value that is 1% away from the maximum value as the upper bound, truncate all pixel values ​​greater than the upper bound, that is, directly set the pixel values ​​that overflow the upper bound as the upper bound value, and divide the SAR image by the upper bound to obtain a normalized image. Through normalization, the input SAR image pixel values ​​can be constrained to be within the same range, making the denoising performance of the network more stable.

[0038] Step 3: Sample SAR images from the training set, randomly crop them into 128×128 image blocks, input them into the coherent speckle suppression network to obtain denoised SAR images, and randomly downsample the denoised SAR images to obtain denoised SAR images with their length and width reduced by half;

[0039] Sub-step 3.1: If Figure 3As shown in the figure, random downsampling is to divide the image into 2×2 small blocks along the pixel interval. Each small block contains 4 pixels. One pixel is randomly selected to be retained and the remaining 3 are discarded. All the retained pixels are combined to form a SAR image with the length and width reduced by half.

[0040] Step 4: Input the reduced denoised SAR image into an external denoiser with fixed parameters to obtain a secondary denoised image. The secondary denoised image is used as a label to calculate the loss function. The coherent speckle suppression network is updated by the gradient descent algorithm. The gradient of the secondary denoised image is interrupted to stabilize the training process.

[0041] Sub-step 4.1: fix the parameters of the deep denoiser trained by the simulation data. Since the spatial correlation characteristics of the coherent speckles in the reduced SAR image have been destroyed, the deep denoiser trained by the simulation data can effectively suppress the coherent speckles remaining in the reduced SAR image.

[0042] Sub-step 4.2: Use the secondary denoised image as the label to calculate the loss function L, and update the coherent speckle suppression network through the gradient descent algorithm. At this time, since the parameters of the deep denoiser are fixed, the deep denoiser is only used as prior information and is not updated.

[0043] Step 5: Repeat steps 3 and 4. When the number of training rounds reaches 200, terminate the training and obtain the coherent speckle suppression network.

[0044] Step 6: Test the performance of the speckle suppression network on the test set.

[0045] In the embodiment of the present invention, the deep learning denoiser and the speckle suppression network both adopt the U-Net network architecture. Figure 4 As shown in the figure, the blue box represents the intermediate features extracted by the network. The network includes 3 maximum pooling layers and 3 upsampling layers, and uses jump connections to splice the shallow features of the network with the deep features. Jump connections help gradient back propagation and make network training easier to converge. The advantages of this type of U-Net network are simple structure, easy training, suitable for image denoising tasks, and excellent performance.

[0046] The effect of the embodiment of the present invention is further illustrated by the following experiment on measured data:

[0047] 1. Experimental scenario

[0048] 1.1) Dataset: ① Simulation data. The simulation data uses 4744 optical images from the Waterloo Exploration database, which are converted into grayscale images and then added with simulated noise. ② Real SAR images. TerraSAR-X images open sourced from the official website of Airbus (AIRBUS) are used and cropped into 65,000 120×120 small images;

[0049] 1.2) Experimental environment: The network was built using the Pytorch deep learning framework, and an RTX4090 graphics card was used for training. The number of training rounds was 200 and the batch size was 1.

[0050] 1.3) Comparison methods: traditional SAR speckle suppression algorithms PPB and SAR-BM3D, and deep learning SAR speckle suppression methods Speckle2Void and MERLIN;

[0051] 1.4) Evaluation index: Three non-reference quality evaluation indexes are used, namely, Equivalent Number of Look (ENL), Mean Of Ratio (MOR) and Variance Of Ratio (VOR). Generally speaking, the stronger the algorithm's denoising ability, the smoother the image homogeneous area and the larger the ENL value. In addition, the ratio between the SAR image and its ideal denoised image should satisfy the assumption that both the mean and variance are 1. Therefore, the closer the MOR and VOR are to 1, the smaller the deviation of the coherent speckle suppression result.

[0052] 2. Experimental content

[0053] 2.1) Test the coherent speckle suppression capability of each method on the test set. Figure 5 and Figure 6 The partial denoising results of various comparison methods are shown. It can be seen from the results in the figure that compared with other methods, the SAR speckle suppression method based on deep denoiser prior in the embodiment of the present invention has a stronger ability to suppress speckle, and at the same time, this method is better than other methods in maintaining the image structure details. In general, this method suppresses speckle in SAR images well, which is conducive to the interpretation of SAR images.

[0054] 2.2) Table 1 gives Figure 5 and Figure 6 The evaluation index of the results of the comparison methods in the above examples, where the ENL value is obtained by intercepting Figure 5 and Figure 6 It can be seen that compared with other methods, the SAR coherent speckle suppression method based on deep denoiser prior in the embodiment of the present invention is Figure 5 and Figure 6 The highest ENL values ​​were obtained, indicating that this method has the best effect on the suppression of coherent speckle in the smooth area of ​​SAR images. In addition, the difference between the MOR and VOR of this method and 1 is the smallest, indicating that the coherent speckle suppression image produced by this method has the smallest deviation.

[0055] Table 1 Quantitative evaluation of speckle suppression results of each method

[0056]

[0057] Example 2

[0058] Embodiment 2 of the present invention provides a terminal device corresponding to the above-mentioned embodiment 1. The terminal device may be a processing device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of the above-mentioned embodiment.

[0059] The terminal device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method in the above-mentioned embodiment 1.

[0060] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0061] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0062] Example 3

[0063] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to the above embodiment 1, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the above embodiment 1 are implemented.

[0064] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0065] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0066] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0067] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0068] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0069] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for SAR image speckle suppression based on deep denoiser prior, characterized in that: The following steps are involved: S1. Build a simulated speckle noise dataset and pre-train a deep learning denoiser. S2, acquiring a SAR image, normalizing the SAR image, and constructing a training set using the normalized SAR image; S3, sampling a SAR image from the training set, using the sampled SAR image as an input of a speckle suppression network to obtain a denoised SAR image, and randomly downsampling the denoised SAR image to obtain a denoised SAR image with a length and a width both reduced by half; S4. Using the denoised SAR image with the length and width reduced by half as the input of the deep learning denoiser to obtain a secondary denoised image, using the secondary denoised image as a label, calculating a loss function, and updating the coherent speckle suppression network.

2. The SAR image speckle suppression method based on deep denoiser prior according to claim 1, characterized in that: Also includes: S5. Repeat steps S3 and S4 until the set number of training rounds is reached.

3. The SAR image speckle suppression method based on deep denoiser prior according to claim 1, characterized in that: The simulated speckle noise data is expressed as: z=s·n; wherein n represents multiplicative gamma noise with a mean and a variance of 1, s represents a clean image, and z represents a simulated noise image.

4. The SAR image speckle suppression method based on deep denoiser prior according to claim 1, characterized in that: The deep learning denoiser and speckle suppression network both adopt the U-Net network architecture.

5. The SAR image speckle suppression method based on deep denoiser prior according to claim 1, characterized in that: The specific implementation process of normalizing the SAR image includes: sorting the pixel values ​​of the SAR image in ascending order, taking the pixel value that is ranked M% away from the maximum value as the upper limit, modifying all pixel values ​​greater than the upper limit to be equal to the upper limit value, dividing all pixel values ​​of the SAR image by the upper limit value, and obtaining a normalized SAR image.

6. The SAR image speckle suppression method based on deep denoiser prior according to claim 1, characterized in that: The specific implementation process of the random downsampling includes: dividing the denoised SAR image into 2×2 small blocks along the pixel interval, randomly selecting a pixel in the small block, discarding the remaining pixels, combining all the retained pixels, and obtaining a denoised SAR image with the length and width reduced by half.

7. The SAR image speckle suppression method based on deep denoiser prior according to claim 1, characterized in that: The expression of the loss function L is: Among them, y i represents the i-th SAR image, represents the i-th denoised image output by the speckle suppression network, λ is the weight parameter, is the i-th image obtained by random downsampling, Stopgrad indicates gradient interruption, D is the pre-trained external denoiser, Represents the output of the external denoiser, which is the deep denoiser prior.

8. A terminal device comprising a memory, a processor and a computer program stored in the memory; characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images