An interferometric phase unwrapping method based on multi-scale feature fusion enhanced network
By using a multi-scale feature fusion enhancement network, the problems of long training time, large dataset size, and low accuracy under low signal-to-noise ratio in existing technologies are solved, achieving fast and high-precision interferometric phase unwrapping and improving the effect of synthetic aperture radar interferometry.
Patent Information
- Application Number
- CN202411157082.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-22
AI Technical Summary
Existing deep learning-based interferometric phase unwrapping methods suffer from problems such as long training time, large training dataset size, poor robustness, and low accuracy under low signal-to-noise ratio conditions.
A multi-scale feature fusion enhancement network is adopted. By designing diverse residual modules and constructing multi-scale feature fusion modules, combined with the Adam gradient optimization algorithm, the network structure is optimized to improve the untangling accuracy.
Fast convergence and high-precision interferometric phase unwrapping were achieved under low signal-to-noise ratio conditions, improving the accuracy and robustness of synthetic aperture radar interferometry.
Smart Images

Figure CN118671764B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an interference phase unwrapping method based on a multi-scale feature fusion enhancement network, and belongs to the technical field of synthetic aperture radar (SAR) interferometry. Background Art
[0002] Synthetic Aperture Radar (InSAR) is one of the most promising space-based earth observation technologies. It is an extension of SAR technology. Its working principle is to use SAR to take two images of the same area, and obtain the interference fringe pattern of the area through interference. The interference fringe pattern contains the terrain information of the area. Affected by noise, phase discontinuity and rapid phase change, the phase unwrapping problem becomes extremely challenging, making interferometric phase unwrapping one of the key steps in interferometric measurement processing. It has broad application prospects in various fields, especially in geographic location detection, urban and rural planning, nuclear magnetic resonance imaging, disaster monitoring, disaster prevention and relief deployment, and military monitoring. Therefore, it is of great significance to study interferometric phase unwrapping.
[0003] There are two main traditional phase unwrapping methods: path tracking and minimum optimization. The path tracking method completes phase unwrapping by selecting a suitable path to integrate the phase, while the minimum optimization method minimizes the difference between the estimated gradient and the unwrapped phase gradient through the objective function to obtain the optimal unwrapped phase. Although the path tracking method is more computationally efficient, it has low robustness to noise, while the minimum norm method is just the opposite.
[0004] With the rise of artificial intelligence technology, in the field of interferometric phase unwrapping, deep learning-based algorithms such as U-net (U-type convolutional neural network), QGPU (quality-guided phase unwrapping) and PhaseNet (phase unwrapping neural network) have been applied to interferometric phase unwrapping, and have achieved excellent performance in interferometric phase unwrapping scenarios. The general implementation step is to restore the entangled phase that is disturbed by noise to the true unwrapped phase through the powerful learning ability of the neural network. However, these methods have the disadvantages of long network model training time, large training data set size, poor robustness and low accuracy under low signal-to-noise ratio conditions. Therefore, for the interferometric phase unwrapping scenario, how to propose an interferometric phase unwrapping method with fast convergence speed, strong real-time performance, strong robustness and high unwrapping accuracy under low signal-to-noise ratio conditions is a technical problem that needs to be solved urgently. Summary of the invention
[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide an interference phase unwrapping method based on a multi-scale feature fusion enhanced network. In view of the problems of long training time, large scale of training data set, poor robustness and low accuracy under low signal-to-noise ratio conditions of the existing network model, the purpose of the present invention is to propose an interference phase unwrapping method based on a multi-scale feature fusion enhanced network, which adopts technologies such as diversity residual module design, construction of multi-scale feature fusion module, neural network training and verification, to achieve interference phase unwrapping under low signal-to-noise ratio conditions and improve the accuracy of synthetic aperture radar interferometry measurement.
[0006] Preferably, the present invention provides an interferometric phase unwrapping method based on a multi-scale feature fusion enhanced network, comprising:
[0007] The pre-acquired interference entangled phase image affected by noise is input into the trained multi-scale feature fusion enhancement network, and the obtained interference unwrapped phase image is output.
[0008] Prioritize the training to obtain a multi-scale feature fusion enhancement network, including:
[0009] The pre-acquired data set is input into the constructed multi-scale feature fusion enhancement network, and the constructed multi-scale feature fusion enhancement network is optimized using the Adam gradient optimization algorithm until the loss function converges to a certain value, thereby obtaining the trained multi-scale feature fusion enhancement network.
[0010] Preferably, the constructed network structure based on multi-scale feature fusion enhancement includes an input layer Input, an encoder network Encoder, a multi-scale feature fusion module MFFM, a connection layer Cat, a decoder network Decoder and a linear activation convolution layer Conv2D, the input layer Input is connected to the encoder network Encoder, the encoder network Encoder is connected to the multi-scale feature fusion module MFFM and the connection layer Cat, the encoder network Encoder is connected to the decoder network Decoder, the connection layer Cat is connected to the decoder network Decoder, and the decoder network Decoder is connected to the linear activation convolution layer Conv2D.
[0011] Preferentially, the encoder network Encoder includes a first diversity residual module DRM, a second diversity residual module DRM, a third diversity residual module DRM, a fourth diversity residual module DRM, a first maximum pooling layer MaxPooling2D, a second maximum pooling layer MaxPooling2D, and a third maximum pooling layer MaxPooling2D, and the first diversity residual module DRM, the first maximum pooling layer MaxPooling2D, the second diversity residual module DRM, the second maximum pooling layer MaxPooling2D, the third diversity residual module DRM, the third maximum pooling layer MaxPooling2D, and the fourth diversity residual module DRM are connected in series in sequence;
[0012] The decoder network Decoder includes a first transposed convolution layer Conv2DTranspose, a second transposed convolution layer Conv2DTranspose, a third transposed convolution layer Conv2DTranspose, a fifth diversity residual module DRM, a sixth diversity residual module DRM, a seventh diversity residual module DRM, and a two-dimensional convolution layer Conv2D. The first transposed convolution layer Conv2DTranspose, the fifth diversity residual module DRM, the second transposed convolution layer Conv2DTranspose, the sixth diversity residual module DRM, the third transposed convolution layer Conv2DTranspose, the seventh diversity residual module DRM, and the first two-dimensional convolution layer Conv2D are connected in sequence.
[0013] Preferentially, the multi-scale feature fusion module MFFM includes a first multi-scale feature fusion module MFFM and a second multi-scale feature fusion module MFFM, the connection layer Cat includes a first connection layer Cat, a second connection layer Cat and a third connection layer Cat, the output end of the first diversity residual module DRM is connected to the input end of the first multi-scale feature fusion module MFFM, the output end of the second diversity residual module DRM is connected to the input end of the first multi-scale feature fusion module MFFM and the input end of the second multi-scale feature fusion module MFFM, the output end of the third diversity residual module DRM is connected to the input end of the third connection layer Cat and the second multi-scale feature fusion module MFFM, the first multi-scale feature fusion module MFFM is connected to the first connection layer Cat, the second multi-scale feature fusion module MFFM is connected to the second connection layer Cat, and the output end of the fourth diversity residual module DRM is connected to the first transposed convolution layer Conv2DTranspose.
[0014] Preferably, the first diversity residual module DRM, the second diversity residual module DRM, the third diversity residual module DRM, the fourth diversity residual module DRM, the fifth diversity residual module DRM, the sixth diversity residual module DRM, and the seventh diversity residual module DRM all include a diversity residual module input layer Input, a diversity residual module output layer Output, a two-dimensional convolution layer Conv2d1, a two-dimensional convolution layer Conv2d2, a two-dimensional convolution layer Conv2D3, a two-dimensional convolution layer Conv2D4, a two-dimensional convolution layer Conv2D5, a two-dimensional convolution layer Conv2D6, a two-dimensional convolution layer Conv2d7, a first depth separation convolution layer DSC1, a second depth separation convolution layer DSC2, a third depth separation convolution layer DSC3, a batch normalization and linear activation combination layer BN+RELU, a first dilated convolution DC1, a second dilated convolution DC2, a third dilated convolution DC3, and a fourth connection layer Cat,
[0015] The input layer of the diversity residual module Input is connected to the two-dimensional convolution layer Conv2d1 and the two-dimensional convolution layer Conv2d2. The two-dimensional convolution layer Conv2d1, the first depth separation convolution layer DSC1, the second depth separation convolution layer DSC2, the third depth separation convolution layer DSC3, the batch normalization and linear activation combination layer BN+RELU, the two-dimensional convolution layer Conv2D3, the fourth connection layer Cat and the two-dimensional convolution layer Conv2d7 are connected in sequence. The output end of the third depth separation convolution layer DSC3 is connected to the input end of the two-dimensional convolution layer Conv2D4, the input end of the two-dimensional convolution layer Conv2D5 and the output end of the two-dimensional convolution layer Conv2D6. The output of the two-dimensional convolution layer Conv2D4 is connected to the input of the first dilated convolution DC1, the output of the first dilated convolution DC1 and the output of the two-dimensional convolution layer Conv2D5 are added and fused and input into the second dilated convolution DC2, the output of the second dilated convolution DC2 and the output of the two-dimensional convolution layer Conv2D6 are added and fused and input into the third dilated convolution DC3, the outputs of the first dilated convolution DC1, the second dilated convolution DC2 and the third dilated convolution DC3 are connected to the fourth connection layer Cat, the output of the two-dimensional convolution layer Conv2d2 and the output of the two-dimensional convolution layer Conv2d7 are added and fused and input into the diversity residual module output layer Output.
[0016] Preferably, the first multi-scale feature fusion module MFFM and the second multi-scale feature fusion module MFFM include a low-level feature map, a high-level feature map, a second two-dimensional convolution layer Conv2D, a third two-dimensional convolution layer Conv2D, a fourth transposed convolution layer Conv2DTranspose, a global average pooling layer GlobalAveragePooling2D, a deformation Reshape, a fourth maximum pooling layer MaxPooling2D, a fourth two-dimensional convolution layer Conv2D, a fifth transposed convolution layer Conv2DTranspose, a fifth connection layer Cat and a multi-scale feature fusion module output layer Output,
[0017] The low-level feature map inputs the second two-dimensional convolution layer Conv2D and the fifth connection layer Cat. The high-level feature map inputs the global average pooling layer GlobalAveragePooling2D, the fourth maximum pooling layer MaxPooling2D and the fifth transposed convolution layer Conv2DTranspose. The output of the global average pooling layer GlobalAveragePooling2D is connected to the input of the deformation Reshape. The output of the fourth maximum pooling layer MaxPooling2D is connected to the input of the fourth two-dimensional convolution layer Conv2D. The deformation Res The output of hape and the output of the fourth two-dimensional convolutional layer Conv2D are added and fused, and then input into the fourth transposed convolutional layer Conv2DTranspose. The output of the fourth transposed convolutional layer Conv2DTranspose is connected to the third two-dimensional convolutional layer Conv2D. The output of the third two-dimensional convolutional layer Conv2D is multiplied and fused with the second two-dimensional convolutional layer Conv2D and the output of the fifth transposed convolutional layer Conv2DTranspose, and then input into the fifth connection layer Cat. The output of the fifth connection layer Cat is connected to the output layer Output of the multi-scale feature fusion module.
[0018] Preferably, the loss function is:
[0019] ,
[0020] in, It is the difference between the pixel value of the interference phase image in the training data set and the pixel value of the predicted interference phase image output by the multi-scale feature fusion enhancement network training. (i, j) represents the phase pixel coordinates, and b is a positive coefficient.
[0021] Prioritize pre-acquisition of datasets, including:
[0022] Create irregular and arbitrary shaped interference phase patterns by adding and subtracting several Gaussian functions of different shapes and positions;
[0023] Randomly select ramps along vertical and horizontal directions and add them to the created interference phase image to form an interference phase image with ramp phase;
[0024] The interference phase image phase is pixel-phase wrapped, and the wrapped phase The calculation formula is:
[0025] ,
[0026] Among them, exp is an exponential function with the natural constant e as the base, is the original true phase of the interference phase image pixel, is the spatial coordinate of the pixel in the interference phase pattern, ∠ is the sign of the angle;
[0027] Creating a preset number of interferometric phase map data sets based on the interferometric phase map of pixel phase wrapping;
[0028] The interference phase image in the interference phase image data set is randomly assigned with a preset Gaussian additive noise to construct an interference phase data set;
[0029] The interferometric phase dataset is divided into a training set and a test set.
[0030] Preferably, an interferometric phase image dataset with a size of 1400 is created, each interferometric phase image has a pixel size of 256×256 and a pixel value of -55 to 55; the interferometric phase images in the interferometric phase image dataset are randomly assigned Gaussian additive noise of 0dB, 5dB and 10dB.
[0031] Preferably, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the program.
[0032] Preferably, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the methods described in the first aspect.
[0033] The beneficial effects achieved by the present invention are:
[0034] The multi-scale feature fusion enhancement network structure constructed by the present invention includes an input layer Input, an encoder network Encoder, a multi-scale feature fusion module MFFM, a connection layer Cat, a decoder network Decoder and a linear activation convolution layer Conv2D. The input layer Input is connected to the encoder network Encoder, the encoder network Encoder is connected to the multi-scale feature fusion module MFFM and the connection layer Cat, the encoder network Encoder is connected to the decoder network Decoder, the connection layer Cat is connected to the decoder network Decoder, and the decoder network Decoder is connected to the linear activation convolution layer Conv2D. The present invention inputs the interference winding phase image under noise obtained in advance into the multi-scale feature fusion enhancement network obtained by training. The output is an interference unwrapping phase map with most of the noise removed. The improved network model has lower requirements for the data set, and network training and prediction can be performed with a smaller data set. The improved network model is more robust to the influence of noise. Even under low signal-to-noise ratio conditions, it can still achieve good unwrapping effects on the entangled interference phase, and the obtained unwrapping accuracy is higher (observing the noise map in the patent, you can see the influence of noise on the interference phase map). The result map obtained by the improved network model, that is, the predicted interference unwrapping phase map and the real interference phase map have extremely high similarity, that is, even in the presence of noise, high-precision unwrapping can be performed to obtain the unwrapping phase (the real phase map and predicted phase map in the patent map can be observed);
[0035] In view of the long training time, large scale of training data set, poor robustness and low accuracy of existing network models under low signal-to-noise ratio conditions, the present invention aims to propose an interferometric phase unwrapping method based on a multi-scale feature fusion enhanced network, which adopts technologies such as diversity residual module design, multi-scale feature fusion module construction, neural network training and verification, to achieve interferometric phase unwrapping under low signal-to-noise ratio conditions with a small data set, and improve the precision of synthetic aperture radar interferometry measurement. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0037] Figure 1 is a flow chart of some embodiments of the present application;
[0038] Figure 2 is a structural diagram of a multi-scale feature fusion enhancement network in some embodiments of the present application;
[0039] Figure 3 is a structural diagram of a diversity residual module in some embodiments of the present application;
[0040] Figure 4 is a structural diagram of a multi-scale feature fusion module in some embodiments of the present application;
[0041] Figure 5 is an interferometric entanglement phase diagram under noise in some embodiments of the present application;
[0042] Figure 6 is a real interference phase diagram in some embodiments of the present application;
[0043] Figure 7 is a diagram of the unwrapping result of the interference entanglement phase under noise in some embodiments of the present application; DETAILED DESCRIPTION
[0044] To facilitate the technical solution of the application, some concepts involved in the application are first explained below.
[0045] See also Figure 1 , the present application discloses an interference phase unwrapping method based on a multi-scale feature fusion enhanced network, comprising:
[0046] The pre-acquired interference entangled phase map is input into the multi-scale feature fusion enhancement network obtained by training, and the interference unwrapped phase map with most of the noise removed is output. The interference phase map obtained at the beginning of the present invention is an entangled phase, ranging from [-Π, Π], and the purpose of this network structure is to obtain an unwrapped phase from an entangled phase with noise. Compared with the input interference entangled phase map, the present invention finally obtains an unwrapped phase map with high unwrapping accuracy.
[0047] The training is based on a multi-scale feature fusion enhancement network, including:
[0048] The multi-scale feature fusion enhancement network constructed by inputting the pre-acquired training set is optimized using the Adam gradient optimization algorithm until the loss function converges to a certain value, thereby obtaining the trained multi-scale feature fusion enhancement network.
[0049] The present invention proposes an interference phase unwrapping method based on a multi-scale feature fusion enhanced network to process streams such as Figure 1 As shown, the specific implementation steps are as follows:
[0050] 1) Interference phase data preprocessing
[0051] By adding and subtracting several Gaussian functions of different shapes and positions, irregular and arbitrary shapes of interference phase images are created, and slopes and Gaussian additive noise along the vertical and horizontal directions are added to the interference phase image to create an interference phase image data set. The specific steps are as follows:
[0052] (1) Create irregular and arbitrary interferometric phase patterns by adding and subtracting several Gaussian functions of different shapes and positions.
[0053] (2) Randomly select ramps and add them to the interference phase image in the vertical and horizontal directions to form an interference phase image with a ramp phase.
[0054] (3) The interference phase pattern phase is twisted pixel by pixel, and the twisted phase The calculation formula is as follows:
[0055] ,
[0056] Among them, exp is an exponential function with the natural constant e as the base, is the original phase of the interference phase image pixel, (x, y) is the spatial coordinate of the pixel in the phase image, and ∠ is the sign of the angle.
[0057] (4) According to the above method, a data set of 1400 interferometric phase images is created. The pixel size of each interferometric phase image is 256×256, and the pixel value ranges from -55 to 55.
[0058] (5) The interference phase patterns in the interference phase pattern data set are randomly assigned Gaussian additive noises of 0 dB, 5 dB, and 10 dB to construct an interference phase data set.
[0059] 2) Enhanced network structure design based on multi-scale feature fusion
[0060] The network structure diagram based on multi-scale feature fusion enhancement is as follows Figure 2 As shown in the figure, it consists of an input layer Input (m, m), an encoder network (Encoder), a diversity residual module (DRM), a multi-scale feature fusion module (MFFM), a connection layer (Cat), and a decoder network (Decoder).
[0061] The encoder network (Encoder) includes four diversity residual modules DRM (c) and three maximum pooling layers MaxPooling2D (c, k, s, p) which are connected alternately in series. Specifically, the encoder network includes a first diversity residual module DRM, a second diversity residual module DRM, a third diversity residual module DRM, a fourth diversity residual module DRM, a first maximum pooling layer MaxPooling2D, a second maximum pooling layer MaxPooling2D, and a third maximum pooling layer MaxPooling2D. The first diversity residual module DRM, the first maximum pooling layer MaxPooling2D, the second diversity residual module DRM, the second maximum pooling layer MaxPooling2D, the third diversity residual module DRM, the third maximum pooling layer MaxPooling2D, and the fourth diversity residual module DRM are connected in series in series.
[0062] The output feature map X of the encoder is connected to the decoder. The decoder network (Decoder) includes three transposed convolutional layers Conv2DTranspose and three diversity residual modules (DRM) alternately connected in series. The three transposed convolutional layers Conv2DTranspose are the first transposed convolutional layer Conv2DTranspose, the second transposed convolutional layer Conv2DTranspose and the third transposed convolutional layer Conv2DTranspose. The three diversity residual modules are the fifth diversity residual module DRM, the sixth diversity residual module DRM and the seventh diversity residual module DRM. Figure 2As shown in the figure, they are connected in series, and then pass through a 3x3 two-dimensional convolution layer Conv2D (c, k, s, p) and a 1x1 linear activation convolution layer Conv2D (1,1,1,0) for output. The first diversity residual module DRM in the encoder network outputs feature map X1, the second diversity residual module DRM outputs feature map X2, the third diversity residual module DRM outputs feature map X3, and the fourth diversity residual module DRM outputs feature map X. Feature map X1 and feature map X2 are output through the first multi-scale feature fusion module MFFM to obtain output feature map M1, and feature map X2 and feature map X3 are output through the second multi-scale feature fusion module MFFM to obtain output feature map M2. Then, the feature map U2 output by the second transposed convolution layer Conv2DTranspose in the decoder network and the feature map M2 output by the second multi-scale feature fusion module MFFM are spliced through the connection layer (Cat), and the feature map U3 output by the third transposed convolution layer Conv2DTranspose and the feature map M1 output by the first multi-scale feature fusion module MFFM are spliced and output. Specifically, the decoder network Decoder includes a first transposed convolution layer Conv2DTranspose, a second transposed convolution layer Conv2DTranspose, a third transposed convolution layer Conv2DTranspose, a fifth diversity residual module DRM, a sixth diversity residual module DRM, a seventh diversity residual module DRM, and a two-dimensional convolution layer Conv2D. The first transposed convolution layer Conv2DTranspose, the fifth diversity residual module DRM, the second transposed convolution layer Conv2DTranspose, the sixth diversity residual module DRM, the third transposed convolution layer Conv2DTranspose, the seventh diversity residual module DRM, and the first two-dimensional convolution layer Conv2D are connected in sequence.
[0063] The multi-scale feature fusion enhancement module is mainly divided into the diversity residual module (DRM) and the multi-scale feature fusion module (MFFM). Figure 3As shown in the figure, the main structure of the network is a residual structure. The input feature map is divided into two branches, which pass through the 1x1 convolution layer Conv2D (c, k, s, p) and the 3×3 convolution layer Conv2D (c, k, s, p). The branch 3x3 convolution layer and the diversity convolution structure and 3×3 convolution layer are connected in series. Then, the output feature maps of the two branches are added and fused to obtain the output. The dotted part represents the structure of the diversity convolution, which includes three depth-separable convolutions, three dilated convolutions, multiple ordinary convolutions with different convolution kernels, and a connection layer. First, the input feature map passes through three 3×3 depth-separable convolutions, and then is divided into four branches through different convolution layers to obtain feature maps H1, H2, H3, and H4. The first three branch feature maps H1, H2, and H3 are then dilated convolutions with different dilation coefficients to obtain feature maps D1, D2, and D3. Figure 4 It can be seen that these three branches are not simply in parallel, but the output feature map D1 of the first dilated convolution DC1 (d, c, k, p) is fused with the feature map H2 after the second convolution, and then the feature map D2 is obtained through the second second dilated convolution DC2 (d, c, k, p). The output feature map D2 of the second second dilated convolution DC2 (d, c, k, p) is fused with the feature map H3 after the third convolution, and then the feature map D3 is obtained through the third third dilated convolution DC3 (d, c, k, p). Finally, the feature maps D1, D2, D3, and H4 obtained by the four branches are spliced and output to complete the feature extraction task of diversity convolution. Specifically, the first diversity residual module DRM, the second diversity residual module DRM, the third diversity residual module DRM, the fourth diversity residual module DRM, the fifth diversity residual module DRM, the sixth diversity residual module DRM, and the seventh diversity residual module DRM all include a diversity residual module input layer Input, a diversity residual module output layer Output, a two-dimensional convolution layer Conv2d1, a two-dimensional convolution layer Conv2d2, a two-dimensional convolution layer Conv2D3, a two-dimensional convolution layer Conv2D4, a two-dimensional convolution layer Conv2D5, a two-dimensional convolution layer Conv2D6, a two-dimensional convolution layer Conv2d7, a first depth separation convolution layer DSC1, a second depth separation convolution layer DSC2, a third depth separation convolution layer DSC3, a batch normalization and linear activation combination layer BN+RELU, a first dilated convolution DC1, a second dilated convolution DC2, a third dilated convolution DC3 and a fourth connection layer Cat,
[0064] The input layer of the diversity residual module Input is connected to the two-dimensional convolution layer Conv2d1 and the two-dimensional convolution layer Conv2d2. The two-dimensional convolution layer Conv2d1, the first depth separation convolution layer DSC1, the second depth separation convolution layer DSC2, the third depth separation convolution layer DSC3, the batch normalization and linear activation combination layer BN+RELU, the two-dimensional convolution layer Conv2D3, the fourth connection layer Cat and the two-dimensional convolution layer Conv2d7 are connected in sequence. The output end of the third depth separation convolution layer DSC3 is connected to the input end of the two-dimensional convolution layer Conv2D4, the input end of the two-dimensional convolution layer Conv2D5 and the input end of the two-dimensional convolution layer Conv2D6. The input end and the output end of the two-dimensional convolution layer Conv2D4 are connected to the input end of the first dilated convolution DC1. The output of the first dilated convolution DC1 and the output of the two-dimensional convolution layer Conv2D5 are added and fused and input into the second dilated convolution DC2. The output of the second dilated convolution DC2 and the output of the two-dimensional convolution layer Conv2D6 are added and fused and input into the third dilated convolution DC3. The output ends of the first dilated convolution DC1, the second dilated convolution DC2 and the third dilated convolution DC3 are connected to the fourth connection layer Cat. The output of the two-dimensional convolution layer Conv2d2 and the output of the two-dimensional convolution layer Conv2d7 are fused and input into the diversity residual module output layer Output.
[0065] Multi-scale feature fusion module (MFFM) such as Figure 4 As shown in the figure, the high-level feature map X2 is divided into three branches, one through the global average pooling layer (GlobalAveragePooling2D(c,3,1,1)) and deformation Reshape, the second branch through the maximum pooling layer MaxPooling2D (c, k, s, p) and the convolution layer Conv2D (c, k, s, p), the third branch through the transposed convolution layer (c, k, s, p), the first two branches are added and fused, and then through the transposed convolution layer Conv2DTranspose (c, k, s, p) and the convolution layer Conv2D (c, k, s, p), and then multiplied and fused with the low-level feature map through the convolution layer Conv2D (c, k, s, p), and then the feature map through the multiplication fusion and the third branch through the transposed convolution layer Conv2DTranspose (c, k, s, p), and finally the added and fused feature maps and low-level feature maps are concatenated to obtain the output result. Where m represents the pixel size of the image data, d represents the dilation coefficient, c represents the number of channels, k represents the kernel size, s represents the number of steps, and p represents the number of padding.
[0066] 3) Training multi-scale feature fusion enhancement network
[0067] Set relevant parameters, randomly divide the interference phase image data set with a scale of 1400 into training data set, verification data set and test data set according to 7:2:1, and send the training data set to the multi-scale feature enhancement network, train the multi-scale feature fusion enhancement network until the multi-scale feature fusion enhancement network converges, and save the training weights of the multi-scale feature fusion enhancement network. The specific steps are as follows:
[0068] (1) Set the training starting learning rate of the multi-scale feature fusion enhancement network to 0.0001, the maximum learning rate to 0.01, the training batch size to 4, and the number of training rounds to 100.
[0069] (2) During the training process, the K-means clustering algorithm is used to iteratively solve the weights of the multi-scale feature fusion enhancement network, and L2 norm regularization is used to prevent the network from overfitting.
[0070] (3) The Adam gradient optimization algorithm is used to optimize the network training of the multi-scale feature fusion enhancement network. The following new constructor is used as the loss function during the optimization process. The calculation formula is as follows:
[0071]
[0072] In the formula, It is the difference between the pixel value of the interference phase image in the training data set and the pixel value of the predicted interference phase image output by the multi-scale feature fusion enhancement network training. (i, j) represents the phase pixel coordinate, b is a positive coefficient, and b is 0.2.
[0073] (4) Repeat steps 2 and 3 until the network converges, and obtain the final multi-scale feature fusion enhanced network model and weights for interference phase unwrapping, and save the network training weights.
[0074] 5) Verify the enhanced network based on multi-scale feature fusion
[0075] Load the network training weights, send the verification data set to the trained network, verify the network, and obtain the interferometric phase unwrapping results.
[0076] In an embodiment of the present application, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the program.
[0077] In an embodiment of the present application, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.
[0078] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
[0079] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention invented herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not invented by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.
[0080] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above are only specific implementation methods of the present application and are not used to limit the protection scope of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.
Claims
1. An interferometric phase unwrapping method based on a multi-scale feature fusion enhanced network, characterized in that: include: The pre-acquired interference entanglement phase image is input into the trained multi-scale feature fusion enhancement network, and the interference unwrapped phase image is output; The training is based on a multi-scale feature fusion enhancement network, including: Input the pre-acquired data set into the constructed multi-scale feature fusion enhancement network, and use the Adam gradient optimization algorithm to optimize the constructed multi-scale feature fusion enhancement network until the loss function converges to a certain value, thereby obtaining the trained multi-scale feature fusion enhancement network; The constructed network structure based on multi-scale feature fusion enhancement includes an input layer Input, an encoder network Encoder, a multi-scale feature fusion module MFFM, a connection layer Cat, a decoder network Decoder and a linear activation convolution layer Conv2D. The input layer Input is connected to the encoder network Encoder, the encoder network Encoder is connected to the multi-scale feature fusion module MFFM and the connection layer Cat, the encoder network Encoder is connected to the decoder network Decoder, the connection layer Cat is connected to the decoder network Decoder, and the decoder network Decoder is connected to the linear activation convolution layer Conv2D. The encoder network Encoder includes a first diversity residual module DRM, a second diversity residual module DRM, a third diversity residual module DRM, a fourth diversity residual module DRM, a first maximum pooling layer MaxPooling2D, a second maximum pooling layer MaxPooling2D, and a third maximum pooling layer MaxPooling2D. The first diversity residual module DRM, the first maximum pooling layer MaxPooling2D, the second diversity residual module DRM, the second maximum pooling layer MaxPooling2D, the third diversity residual module DRM, the third maximum pooling layer MaxPooling2D, and the fourth diversity residual module DRM are connected in series in sequence; The decoder network Decoder includes a first transposed convolution layer Conv2DTranspose, a second transposed convolution layer Conv2DTranspose, a third transposed convolution layer Conv2DTranspose, a fifth diversity residual module DRM, a sixth diversity residual module DRM, a seventh diversity residual module DRM, and a first two-dimensional convolution layer Conv2D, and the first transposed convolution layer Conv2DTranspose, the fifth diversity residual module DRM, the second transposed convolution layer Conv2DTranspose, the sixth diversity residual module DRM, the third transposed convolution layer Conv2DTranspose, the seventh diversity residual module DRM, and the first two-dimensional convolution layer Conv2D are connected in sequence; The multi-scale feature fusion module MFFM includes a first multi-scale feature fusion module MFFM and a second multi-scale feature fusion module MFFM, the connection layer Cat includes a first connection layer Cat, a second connection layer Cat and a third connection layer Cat, the output end of the first diversity residual module DRM is connected to the input end of the first multi-scale feature fusion module MFFM, the output end of the second diversity residual module DRM is connected to the input end of the first multi-scale feature fusion module MFFM and the input end of the second multi-scale feature fusion module MFFM, the output end of the third diversity residual module DRM is connected to the input end of the third connection layer Cat and the second multi-scale feature fusion module MFFM, the first multi-scale feature fusion module MFFM is connected to the first connection layer Cat, the second multi-scale feature fusion module MFFM is connected to the second connection layer Cat, and the output end of the fourth diversity residual module DRM is connected to the first transposed convolution layer Conv2DTranspose; The first diversity residual module DRM, the second diversity residual module DRM, the third diversity residual module DRM, the fourth diversity residual module DRM, the fifth diversity residual module DRM, the sixth diversity residual module DRM, and the seventh diversity residual module DRM all include a diversity residual module input layer Input, a diversity residual module output layer Output, a two-dimensional convolution layer Conv2d1, a two-dimensional convolution layer Conv2d2, a two-dimensional convolution layer Conv2D3, a two-dimensional convolution layer Conv2D4, a two-dimensional convolution layer Conv2D5, a two-dimensional convolution layer Conv2D6, a two-dimensional convolution layer Conv2d7, a depthwise separable convolution layer DSC1, a depthwise separable convolution layer DSC2, a depthwise separable convolution layer DSC3, a batch normalization and linear activation combination layer BN+RELU, a first dilated convolution DC1, a second dilated convolution DC2, a third dilated convolution DC3, a fourth connection layer Cat, and an output layer Output of the diversity residual module, The input layer of the diversity residual module Input is connected to the two-dimensional convolution layer Conv2d1 and the two-dimensional convolution layer Conv2d2. The two-dimensional convolution layer Conv2d1, the depth-separable convolution layer DSC1, the depth-separable convolution layer DSC2, the depth-separable convolution layer DSC3, the batch normalization and linear activation combination layer BN+RELU, the two-dimensional convolution layer Conv2D3, the fourth connection layer Cat and the two-dimensional convolution layer Conv2d7 are connected in sequence. The output end of the depth-separable convolution layer DSC3 is connected to the input end of the two-dimensional convolution layer Conv2D4, the input end of the two-dimensional convolution layer Conv2D5 and the input end of the two-dimensional convolution layer Conv2D6. The output end of the two-dimensional convolution layer Conv2D4 is connected to the input end of the first dilated convolution DC1, the output of the first dilated convolution DC1 and the output of the two-dimensional convolution layer Conv2D5 are added and fused and input into the second dilated convolution DC2, the output of the second dilated convolution DC2 and the output of the two-dimensional convolution layer Conv2D6 are added and fused and input into the third dilated convolution DC3, the output ends of the first dilated convolution DC1, the second dilated convolution DC2 and the third dilated convolution DC3 are connected to the fourth connection layer Cat, the output of the two-dimensional convolution layer Conv2d2 and the output of the two-dimensional convolution layer Conv2d7 are added and fused and input into the output layer Output of the diversity residual module.
2. The interferometric phase unwrapping method based on a multi-scale feature fusion enhanced network according to claim 1, characterized in that: The first multi-scale feature fusion module MFFM and the second multi-scale feature fusion module MFFM include a low-level feature map, a high-level feature map, a second two-dimensional convolution layer Conv2D, a third two-dimensional convolution layer Conv2D, a fourth transposed convolution layer Conv2DTranspose, a global average pooling layer GlobalAveragePooling2D, a deformation Reshape, a fourth maximum pooling layer MaxPooling2D, a fourth two-dimensional convolution layer Conv2D, a fifth transposed convolution layer Conv2DTranspose, a fifth connection layer Cat and an output layer Output of the multi-scale feature fusion module. The low-level feature map inputs the second two-dimensional convolution layer Conv2D and the fifth connection layer Cat. The high-level feature map inputs the global average pooling layer GlobalAveragePooling2D, the fourth maximum pooling layer MaxPooling2D and the fifth transposed convolution layer Conv2DTranspose. The output of the global average pooling layer GlobalAveragePooling2D is connected to the input of the deformation Reshape. The output of the fourth maximum pooling layer MaxPooling2D is connected to the input of the fourth two-dimensional convolution layer Conv2D. The deformation Res The output of hape and the output of the fourth two-dimensional convolutional layer Conv2D are added and fused, and then input into the fourth transposed convolutional layer Conv2DTranspose. The output of the fourth transposed convolutional layer Conv2DTranspose is connected to the third two-dimensional convolutional layer Conv2D. The output of the third two-dimensional convolutional layer Conv2D is multiplied and fused with the second two-dimensional convolutional layer Conv2D and the output of the fifth transposed convolutional layer Conv2DTranspose, and then input into the fifth connection layer Cat. The output of the fifth connection layer Cat is connected to the output layer Output of the multi-scale feature fusion module.
3. The interferometric phase unwrapping method based on a multi-scale feature fusion enhanced network according to claim 1, characterized in that: The loss function is: , in, is the difference between the pixel value of the interference phase image in the training data set and the pixel value of the predicted interference phase image output by the multi-scale feature fusion enhancement network training, ( i,j ) represents the phase pixel coordinate, b Is a positive coefficient.
4. The interferometric phase unwrapping method based on a multi-scale feature fusion enhanced network according to claim 1, characterized in that: Pre-fetch the dataset, including: Create irregular and arbitrary shaped interference phase patterns by adding and subtracting several Gaussian functions of different shapes and positions; Randomly select ramps along vertical and horizontal directions and add them to the created interference phase image to form an interference phase image with ramp phase; The interference phase image phase is pixel-phase wrapped, and the wrapped phase The calculation formula is: , Among them, exp is an exponential function with the natural constant e as the base, is the original true phase of the interferometric phase image pixel, (x, y) is the spatial coordinate of the pixel in the interferometric phase image, and ∠ is the sign of the angle to be found; Creating a preset number of interferometric phase map data sets based on the interferometric phase map of pixel phase wrapping; The interference phase image in the interference phase image data set is randomly assigned with a preset Gaussian additive noise to construct an interference phase data set; The interferometric phase dataset is divided into a training set and a test set.
5. The interferometric phase unwrapping method based on a multi-scale feature fusion enhanced network according to claim 4, characterized in that: An interferometric phase image dataset with a size of 1400 is created. The pixel size of each interferometric phase image is 256×256 and the pixel value is -55 to 55. The interferometric phase images in the interferometric phase image dataset are randomly assigned Gaussian additive noise of 0dB, 5dB and 10dB.
