Method and system for phase unwrapping detail loss optimization in deep learning, and storage medium

Abstract

The application discloses a detail loss optimization method and system for phase unwrapping in deep learning, and a storage medium. The method comprises the following steps: constructing a generative adversarial network for phase unwrapping, including a generator and a discriminator; the generator is used for unwrapping wrapped phase data and generating an image; the discriminator is used for discriminating the true and false results of the image generated by the generator; a hybrid loss function of the generative adversarial network is determined, and the generative adversarial network is trained to process phase unwrapping data. By setting the generative adversarial network with the hybrid loss function, the trained GAN retains the texture details of the phase unwrapping results as much as possible, effectively suppresses the speckle noise, and significantly reduces the speckle noise in the valley.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image unwrapping processing, and in particular to a detail loss optimization method and system for phase unwrapping in deep learning and a storage medium. BACKGROUND

[0002] At present, deep learning is widely used in phase unwrapping, which is a key step in interferometric synthetic aperture radar (InSAR) data processing. For example, one-step phase unwrapping is widely used due to its strong noise adaptability. This method regards the unwrapping process as a regression problem, and uses L1 or L2 loss function to constrain the network output to be close to the basic true value of the absolute phase. The one-step phase unwrapping method is mainly based on UNet and residual block, for example, Res-UNet, which uses L2 loss to regress the wrapped phase to the absolute phase.

[0003] However, the premise of using L2 loss is to assume that the sample is subject to Gaussian distribution. However, the wrapped phase or absolute phase can be non-Gaussian, which leads to over-smoothed results when using L2 loss. Although the root mean square error (RMSE) converges to a very low value, the texture in the rugged terrain after unwrapping can be blurred, and many noise points can be seen in the valley area. Since L1 is a global loss function, it also causes these problems, and the generated unwrapping image cannot well preserve the details. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a detail loss optimization method and system for phase unwrapping in deep learning and a storage medium, which optimizes and improves the problem of image detail preservation in existing phase unwrapping.

[0005] To achieve the above purpose, the present application provides the following technical solutions:

[0006] The detail loss optimization method for phase unwrapping in deep learning provided by the present application comprises the following steps:

[0007] Constructing a generative adversarial network for phase unwrapping: the generative adversarial network comprises a generator and a discriminator ; the generator is used for unwrapping wrapped phase data and generating an image; and the discriminator is used for discriminating the true or false result of the image generated by the generator.

[0008] A hybrid loss function of the generative adversarial network is determined, which includes an adversarial loss in a WGAN-GP, an L1 loss function and a multi-scale structural similarity (MS-SSIM) loss function, which are combined to ensure that the output of the neural network does not become blurred and to stabilize the training process.

[0009] The generative adversarial network is trained.

[0010] Phase unwrapping data is processed by the trained generative adversarial network.

[0011] Further, the loss function of the generative adversarial network includes a generator loss function.

[0012] The generator loss function is calculated according to the following formula:

[0013]

[0014] wherein, represents the generator loss; represents the L1 loss function; represents the mathematical expectation of the distribution function; represents the wrapped phase data;

[0015] represents the discrimination result of the wrapped phase data after being processed by the generator and the discriminator; represents the unwrapping result of the wrapped phase data after being processed by the generator; is the wrapped phase data, , , is a weight;

[0016] Further, the loss function of the generative adversarial network includes a discriminator loss function.

[0017] The discriminator loss function is calculated according to the following formula:

[0018]

[0019] wherein, represents the discriminator loss;

[0020] represents the output of the random noise sample after being processed by the discriminator; represents the output of the absolute phase label after being processed by the discriminator; represents the gradient operator;

[0021] represents the output of the random noise sample after being processed by the discriminator, and then the gradient is calculated and the two-norm is calculated.

[0022] ​​ an output of the generator;

[0023] an absolute phase label;

[0024] is output data of the generator, is label data of the absolute phase, is random noise sample data;

[0025] is a parameter.

[0026] Further, the generator loss function is obtained by comparing the output image processed by the wrapped phase data with the label image.

[0027] Further, the discrimination result of the discriminator is returned to the generator to participate in the calculation of the generator loss function.

[0028] Further, the training of the generative adversarial network includes training the discriminator Further, the training of the generative adversarial network includes training the discriminator The training is performed according to the following steps:

[0029] The discriminator The image after unwrapping and preserving details is judged as a true result, and vice versa.

[0030] Further, the generator adopts a Res-UNet phase unwrapping network structure.

[0031] The system for optimizing the detail loss of phase unwrapping in deep learning provided by the application comprises

[0032] A memory for storing a computer program;

[0033] A processor for executing the computer program to realize the steps of the above method.

[0034] The storage medium provided by the application, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the steps of the above method.

[0035] The application has the following beneficial effects:

[0036] The application provides a detail loss optimization method for phase unwrapping in deep learning, which comprises the following steps: constructing a generative adversarial network for phase unwrapping, including a generator and a discriminator; the generator is used for unwrapping wrapped phase data and generating images; the discriminator is used for discriminating the true and false results of the images generated by the generator; determining a hybrid loss function of the generative adversarial network, and processing phase unwrapping data by training the generative adversarial network. By setting the generative adversarial network with the hybrid loss function, the trained GAN makes the phase unwrapping result retain the texture details as much as possible, effectively suppresses the speckle noise, and significantly reduces the speckle noise in the valley.

[0037] Additional advantages, objects, and features of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following or can be learned by practice of the application. The objects and other advantages of the application can be realized and attained by the structure particularly pointed out in the description below. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to make the objectives, technical solutions and beneficial effects of the present application clearer, the present application provides the following drawings for illustration:

[0039] Figure 1 A deep learning step phase unwrapping effect analysis diagram.

[0040] Figure 2 A phase unwrapping diagram using an L2 loss function.

[0041] Figure 3 A generative adversarial network process diagram.

[0042] Figure 4 A Res-Unet network structure diagram.

[0043] Figure 5 A comparison diagram of different methods.

[0044] Figure 6 An error distribution comparison diagram. DETAILED DESCRIPTION

[0045] The application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the application and implement it, but the embodiments are not limiting the application.

[0046] Embodiment 1

[0047] The detail loss optimization method for phase unwrapping in deep learning provided in the embodiment optimizes the image detail retention problem in the prior art, and the specific case analysis of the existing image detail retention problem is as follows:

[0048] As Figure 1 shown, Figure 1 is a deep learning one-step phase unwrapping effect analysis diagram, two DEM scene data are unwrapped by using one-step phase unwrapping method, and the unwrapped phase and absolute phase error are compared:

[0049] Figure 1 The two scenes (each line is a scene) are based on the L2 loss function one-step phase unwrapping example. Figure 1 (a) is the real phase, Figure 1 (b) is the unwrapped phase, Figure 1 (c) reflects the unwrapping error, and it can be seen that the unwrapped phase texture is blurred and the ridge details are missing. Figure 1 (f) shows the speckle noise in the valley, because the wrapped phase contains noise, and the L2 loss can only pursue the overall optimal value. In these two cases, they will both cause details to be ignored. The loss of details will seriously affect the accuracy of the DEM generated using InSAR technology. When using the L1-based method, there is the same or similar problem of detail loss.

[0050] Therefore, the embodiment solves the related problems by designing the loss function, and the design process of the loss function in the embodiment is as follows.

[0051] The SSIM (structural similarity) image evaluation index is introduced, which is used to evaluate the similarity between two images. It is a perception model, and the formula is expressed as:

[0052] ;

[0053] ;

[0054] ;

[0055] ;

[0056] Among them, and are two images to be evaluated, reflects the brightness comparison part, reflects the contrast comparison part, reflects the structure comparison part, and respectively represent the average values of two images and , and are and the standard deviation of the two images, denotes the image and the covariance of the two images, is a constant added to avoid errors caused by a zero denominator, 、 respectively represent the proportion of different scores in the SSIM index.

[0057] Since the mean, standard deviation and covariance usually vary dramatically across the image, it is better to use a truncated image with a sliding window to calculate the SSIM index. In this embodiment, the SSIM index is traversed pixel by pixel across the entire image with a kernel size of 8x8, and the SSIM index is used to evaluate the superiority of the unwrapping effect, as shown in Figure 2 . Figure 2 is a schematic diagram of phase unwrapping using an L2 loss function, wherein (a) represents the absolute phase, and (b) represents a schematic diagram of the unwrapping result.

[0058] The RMSE of the two images has been as low as 0.5882, but it can be seen that there is obvious loss of details, so the RMSE may not be sufficient to evaluate the quality of unwrapping, but the SSIM index of the two is only 0.7226, which indicates that it is more suitable as an evaluation index than the RMSE.

[0059] Therefore, a multi-scale version of the SSIM is used as part of the loss function, which evaluates the structural similarity from different scales (i.e. image sizes) and avoids adjusting the standard deviation of the Gaussian filter, and the loss function is usually combined with the L1 loss to ensure better results.

[0060] ;

[0061] wherein, denotes a cross-scale structural similarity loss function;

[0062] denotes a cross-scale structural similarity index;

[0063] In addition, the generative adversarial network can effectively avoid the image becoming blurred when generating the image, so the GAN loss is added to solve the problem of detail blurring caused by the above phase unwrapping. In order to solve the problem of unstable gradient often encountered in the training process of the adversarial network, the improved loss function in WGAN-GP is used to stabilize the training.

[0064] Then, the final loss function includes WGAN-GP, L1 and :

[0065] wherein,

[0066] WGAN-GP refers to improved WGAN with gradient penalty;

[0067] L1 loss function refers to mean absolute error loss function;

[0068] The discriminator loss function and the discriminator loss function are calculated according to the following formula:

[0069]

[0070]

[0071] wherein, the generator loss; the discriminator loss; L1 loss function; the mathematical expectation of the distribution function; the wrapped phase data;

[0072] the wrapped phase data through the generator and then through the discriminator; the unwrapping result of the wrapped phase data through the generator; the output of the random noise sample through the discriminator; the output of the absolute phase label through the discriminator; the gradient operator;

[0073] the gradient of the random noise sample through the discriminator and then the two-norm;

[0074] the output of the generator;

[0075] the absolute phase label;

[0076] is the wrapped phase data, is the output data of the generator, is the label data of the absolute phase, and is a random sample data, is a parameter, , , is a weight;

[0077] In this embodiment, is set to 10, is one percent of is 0.16, is 0.0016,​​ is 0.824.

[0078] It should be noted that the weight distribution is related to a specific data set and is usually determined empirically.

[0079] The generative adversarial network in this embodiment is designed according to the following steps:

[0080] A generative adversarial network (GAN) is used for phase unwrapping, contains two structures and , is a generator for generating images, in this embodiment The function of the generator is to unwrap the wrapped phase data; is a discriminator for discriminating whether the image generated by the generator is "true" or "false", in this method, the function of the discriminator is to discriminate whether the generator has achieved unwrapping and preserved details, which can be simply regarded as a game process of the two networks.

[0081] The generative adversarial network of this embodiment is shown in Figure 3 , Figure 3 is the process of the generative adversarial network, in which Real represents "true"; Fake represents "false"; represents the discriminator;

[0082] The wrapped phase data is input to the generator, which performs phase unwrapping to obtain the output, which is compared with the label to participate in the loss of the generator, and both are input to the discriminator for unwrapping discrimination. "Unwrapping and preserving details" is "true", and vice versa, to train the discriminator. The discrimination result of the discriminator is also returned to the generator to participate in the loss.

[0083] As shown in Figure 4 , Res-UNet is a commonly used network structure for one-step phase unwrapping, and residual blocks are added after downsampling and upsampling to avoid learning difficulties caused by excessive network depth. This encoding-decoding structure is also applied in many image processing tasks. The specific details of the implementation of Res-UNet are shown in Figure 4 ,

[0084] Figure 4 is the Res-Unet network structure, the left side is the encoder, and the right side is the decoder. The encoder will pass through a residual block before downsampling, and the decoder will also be connected to a residual block after upsampling. The same level of the encoder and the decoder has a skip connection.

[0085] In the figure, Conv3x3, ReLU, BN represent 3x3 convolution, ReLU activation function, and batch normalization layer.

[0086] Resblock denotes a residual block;

[0087] Maxpool2x2 denotes 2x2 max pooling;

[0088] Dcov2x2, ReLU, BN denotes 2x2 deconvolution, ReLU activation function, batch normalization layer;

[0089] Concatenate denotes a skip connection;

[0090] The discriminator uses a Markov model in PatchGAN, and the specific parameters are shown in Table 1. Since the gradient penalty strategy of WGAN-GP is considered, the discriminator does not add a sigmoid activation function.

[0091] Table 1 Main parameters of the discriminator

[0092] Layer Name #Filters Output Size Conv 64 128×128×64 LReLU 128×128×64 BatchNorm 128×128×64 Conv 128 64×64×128 LReLU 64×64×128 BatchNorm 64×64×128 Conv 256 32×32×256 LReLU 32×32×256 BatchNorm 32×32×256 Conv 512 16×16×512 LReLU 16×16×512 BatchNorm 16×16×512 Conv 1 16×16×1

[0093] In the table, Conv denotes convolution;

[0094] LReLU denotes LeakyReLU activation function;

[0095] BatchNorm denotes a batch normalization layer;

[0096] Conv denotes convolution;

[0097] LReLU denotes LeakyReLU activation function;

[0098] BatchNorm denotes a batch normalization layer;

[0099] Conv denotes convolution;

[0100] LReLU denotes LeakyReLU activation function;

[0101] BatchNorm denotes a batch normalization layer;

[0102] Conv denotes convolution;

[0103] LReLU denotes LeakyReLU activation function;

[0104] BatchNorm denotes a batch normalization layer;

[0105] Conv denotes convolution;

[0106] In this embodiment, the designed generative adversarial network and loss function are used for image processing. Compared with the phase unwrapping effect of the traditional loss function, the specific circumstances are as follows:

[0107] The phase unwrapping performance of the improved loss function and the traditional loss function is compared. The training set is composed of 8000 images, and the test set is 800. The Adam optimizer is used, and the initial learning rate is set to 0.0001.

[0108] As Figure 5 shown, Figure 5 is a schematic diagram for comparison of different methods, and comparison of different methods. Each column is for the same study area. By row: Figure 5 (a) wrapped phase image, (b) absolute phase image, (c), (d), (e) are respectively the unwrapping results based on L2 loss function, L1 loss function and the method of the embodiment,

[0109] It can be found by comparison that the absolute phase image has obvious texture, but due to the influence of wrapped phase noise, the neural network based on L1 and L2 is difficult to restore the edge information, because they all calculate 2 loss pixel by pixel, in each scene, the method based on L1 and L2 loss will blur the texture near the ridge, and there are noise points in the valley, while the method of the embodiment well preserves the texture and suppresses the noise.

[0110] On the other hand, the adjacent pixels of the absolute phase image have strong correlation. By detecting whether the structural information of the image changes, the correlation can be used to display the key information of the surrounding area and perceive the approximate information of the image distortion. Therefore, the SSIM index can restore the edge details blurred by noise. The discriminator helps the network to avoid outputting blurred results and try to preserve the original details, as Figure 6 shown, Figure 6 shows the difference between the unwrapped phase and the true phase. In the three study areas, the method produces smaller and more uniform error distribution compared with using L1 and L2 loss function. Figure 6 is a comparison of error distribution, and by column, (a), (b), (c) respectively reflect Figure 5 the difference between (c), (d), (e) in

[0111] Table 2 shows the RMSE and SSIM values of using different loss functions in the same network. Although the RMSE of the method is slightly larger, the SSIM is the highest. It is worth noting that and different weights of

[0112] Table 2 Test set evaluation results

[0113] RMSE SSIM Res-UNet(l2) 1.1755 0.7029 Res-UNet(l1) 1.2374 0.7085 Our approach 1.2441 0.8031

[0114] In the table, Res-UNet(L2) represents Res-UNet based on L2 loss;

[0115] Res-UNet(L1) represents Res-UNet based on L1 loss;

[0116] Our approach represents the method herein;

[0117] RMSE represents root mean square error;

[0118] SSIM represents structural similarity index;

[0119] The above-described embodiments are merely preferred embodiments of the present application and are not intended to limit the present application. Any equivalent or modification of the present application made by those skilled in the art based on the teachings of the present application should be within the scope of the present application. The scope of the present application is defined by the appended claims.

Claims

1. A method for optimizing detail loss in phase unwrapping during deep learning, characterized by: The method comprises the following steps: constructing a generative adversarial network for phase unwrapping: the generative adversarial network comprises a generator G and a discriminator D; the generator for unwrapping the wrapped phase data and generating an image; the discriminator for discriminating the true or false result of the image generated by the generator determining a hybrid loss function of the generative adversarial network, the hybrid loss function comprising an improved adversarial loss in a WGAN-GP, an L1 loss function and a cross-scale structural similarity loss function MS-SSIM, the three combined to ensure that the output of the neural network does not become blurred and to stabilize the training process; training the generative adversarial network; processing phase unwrapping data by using the trained generative adversarial network; the loss function of the generative adversarial network comprises a generator loss function; the generator loss function is calculated according to the following formula: ； wherein, denotes the generator loss; denotes the LI loss function; denotes the mathematical expectation of the distribution function; denotes the wrapped phase data; denotes the discriminative result of the wrapped phase passing through the generator and then the discriminator; denotes the unwrapping result of the wrapped phase passing through the generator; is the wrapped phase data, , , is the weight; denotes the cross-scale structural similarity loss function.

2. The method of claim 1, wherein: the loss function of the generative adversarial network comprises a discriminator loss function; the discriminator loss function is calculated according to the following formula: ； wherein, represents the discriminator loss; denotes the output of a random noise sample through a discriminator; denotes the output of an absolute phase label through a discriminator; denotes a gradient operator; denotes the gradient of the discriminator with respect to the random noise sample and the 2-norm of the gradient. representing the output of the generator; denotes an absolute phase label; is output data of the generator, is label data of the absolute phase, is a random noise sample data; is a parameter.

3. The method for detail loss optimization in phase unwrapping in deep learning according to claim 1, wherein: the generator loss function is obtained by comparing the output image obtained by processing the wrapped phase data with the label image.

4. The method of claim 1, wherein: The discrimination result of the discriminator is returned to the generator to participate in the calculation of the generator loss function.

5. The method for detail loss optimization in phase unwrapping in deep learning according to claim 1, wherein: The training generative adversarial network includes a discriminator training the discriminator The training is performed in the following steps: The discriminator is trained to distinguish between images that have been disentangled and retain details and images that do not. The disentangled and detail-retaining images are classified as true results, and vice versa.

6. The method for detail loss optimization in phase unwrapping in deep learning according to claim 1, wherein: The generator adopts a Res-UNet phase unwrapping network structure.

7. A system for phase unwrapping optimization with detail loss in deep learning, characterized in that: comprise a memory for storing a computer program; a processor for executing the computer program to implement the steps of the detail loss optimization method for phase unwrapping in deep learning according to any one of claims 1 to 6.

8. A computer readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the steps of the detail loss optimization method for phase unwrapping in deep learning according to any one of claims 1 to 6.

Images