Phase unwrapping method based on interactive self-distillation and structural re-parameterization

Abstract

The invention relates to a phase unwrapping method based on interactive self-distillation and structural re-parameterization, which comprises the following steps: constructing a gradient segmentation model, and adding an interactive self-distillation mechanism in a semantic segmentation network; a Repvg module is adopted as a basic module of the network; a divider at the tail end of the network adopts an integrated architecture strong divider; the loss function adopts weighted cross entropy loss and weighted mean square error loss; training the gradient segmentation model on a pre-constructed data set by adopting an adaptive enhancement strategy to obtain a package counting gradient prediction model; and inputting the current wrapped phase diagram into the model, outputting to obtain corresponding transverse and longitudinal wrapped counting gradients, obtaining a wrapped counting diagram through a least square method based on discrete cosine variation, multiplying the wrapped counting by 2 pi, and adding the multiplied wrapped counting with the wrapped phase diagram to obtain an unwrapped phase. Compared with the prior art, end-to-end cross-resolution phase unwrapping can be realized, and high-precision and stable phase unwrapping can be carried out on images with different resolutions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional measurement, and particularly relates to a phase unwrapping method based on interactive self-distillation and structure reparameterization. BACKGROUND

[0002] Phase unwrapping is a key technique for processing periodic phase images and is a key step in the fields of interferometry, synthetic aperture radar and nuclear magnetic resonance. In these scenarios, the phase often contains important information. For example, in the fields of interferometry and synthetic aperture radar, the phase contains height information. In the field of nuclear magnetic resonance, the phase contains information about the uniformity of the magnetic field and physiological parameters. In many measurement systems, due to the periodic nature of phase measurement, the phase data in the obtained phase image is often wrapped within a limited range [-π, π]. This wrapped phase data needs to be processed by a specific image processing technique to recover the true phase, which is the phase unwrapping technique.

[0003] Traditional phase unwrapping methods recover the true phase by minimizing the influence of phase discontinuity on the unwrapping result, including path tracking methods and optimization methods. Among them, the path tracking method determines a better integration path to minimize the influence of discontinuous points on the whole. This method may produce unreasonable paths when dealing with complex wrapped phases, thereby affecting the accuracy of unwrapping. The optimization-based method uses different objective functions to minimize the difference between the unwrapped phase gradient and the wrapped phase gradient to obtain the unwrapped phase. This method tends to generate a smooth unwrapped phase, which leads to the fact that the difference between the unwrapped phase and the wrapped phase is not necessarily an integer multiple of 2π when dealing with high-noise images. This inconsistency causes errors at each point on the image. Although the traditional phase unwrapping method alleviates the influence of discontinuous phase to some extent, it still fails in the presence of severe noise or complex wrapped phase. In addition, the traditional algorithm is slow and cannot meet the real-time unwrapping requirements in many scenarios.

[0004] Currently, researchers find that phase unwrapping through neural networks is not constrained by Itoh condition and is still effective in the presence of noise. Therefore, the phase unwrapping method based on deep neural network has become the mainstream research direction in the field of phase unwrapping. The phase unwrapping method based on deep learning mainly includes regression method, wrapped count method and gradient method. Among them, the regression method is to directly map the wrapped phase to the unwrapped phase by the neural network. This method is the most direct, but the prediction accuracy cannot be guaranteed. The wrapped count method is to redefine the phase unwrapping problem as a semantic segmentation problem, and classify the wrapped count k(x, y) of each pixel point. This method is fast and is the most commonly used method at present. However, due to the continuous distribution of the wrapped count, the segmentation of the wrapped count needs to integrate the global feature information, and the requirement for the network receptive field is high. Therefore, this method is only suitable for low-resolution images. In addition, when facing samples with large differences in phase distribution characteristics and training set, it is usually difficult to ensure the generalization. The gradient method is to classify the wrapped count gradient of each pixel point, and then combine the post-processing step to complete the reconstruction of the unwrapped phase. The prediction of the gradient only depends on the local feature information, which to some extent relieves the problem of the receptive field. This method is closer to the essence of the phase solving process. However, the wrapped count gradient is a typical sparse semantic information. The number of points with wrapped count gradient +1 or-1 in a wrapped phase image is much smaller than the number of points with wrapped count gradient 0. This serious sample imbalance makes the network training very difficult. And when the number of phase periods in an image is certain, the sparsity of the wrapped count gradient will continue to increase with the increase of the resolution, and the proportion of points with gradient +1 or-1 in the whole image will continue to decrease. This makes the gradient prediction network training more difficult at high resolution. Therefore, the current network still only targets low-resolution images. And because the gradient predicted by these networks is not accurate enough, complex post-processing processes are followed by the networks, making the overall unwrapping speed slow. These factors limit the application of the gradient method.

[0005] In summary, the existing phase unwrapping method is difficult to realize cross-resolution phase unwrapping, and cannot perform high-precision and stable phase unwrapping on images of different resolutions. SUMMARY

[0006] The purpose of the present application is to overcome the defects of the prior art and provide a phase unwrapping method based on interactive self-distillation and structural reparameterization, which can realize end-to-end cross-resolution phase unwrapping.

[0007] The purpose of the present application can be realized by the following technical scheme: a phase unwrapping method based on interactive self-distillation and structural reparameterization, comprising the following steps:

[0008] The data set is divided into a training set, a validation set and a test set, each group of data including input and label, the input being a wrapped phase map with noise, and the label being a transverse and longitudinal wrapped count gradient map;

[0009] A gradient segmentation model based on a convolutional neural network is constructed, an interactive self-distillation mechanism is added in the semantic segmentation network to realize cross-level supervised learning and ensure the stability of semantic extraction at high resolution; a Repvgg (Re-parameterization Visual Geometry Group) module is used as the basic module of the network, which trains parameters in a double-branch form during training, and in the inference stage, the double-branch architecture is equivalent to a single-branch architecture through structure re-parameterization, reducing the model parameter quantity and memory consumption; the end of the network uses an integrated architecture to integrate three weak segmenters containing different dilated convolution into a strong segmenter to adapt to different spatial frequency semantic information; the loss function uses weighted cross-entropy loss and weighted mean square error loss to improve the gradient segmentation accuracy;

[0010] The constructed gradient segmentation model is trained on the data set, and an adaptive enhancement strategy is used during training to give higher weights to samples with high error rates to increase the importance of easy-to-mistake samples, and different weights are given to the correct rates of the three segmenters to achieve optimal integration effect; after training, a parameter-fixed wrapped count gradient prediction model is obtained;

[0011] The current wrapped phase map is input into the wrapped count gradient prediction model, and the corresponding transverse and longitudinal wrapped count gradient is output, and then the wrapped count map is obtained through the least square method based on discrete cosine change, and the wrapped count is multiplied by 2π and added to the wrapped phase map to obtain the unwrapped phase.

[0012] Further, the data set is specifically a data set generated by a simulation algorithm, and the process includes:

[0013] A Gaussian function with random mean and standard deviation is superimposed on a random slope ramp function to generate a real phase According to the formula The wrapped count k is obtained, wherein round is the rounding operation;

[0014] The wrapped count k is differentiated to obtain a transverse and longitudinal wrapped count gradient distribution map as a gradient label, and then is subjected to an inverse tangent transformation and added with different degrees of noise to obtain a wrapped phase with noise ψ as network input.

[0015] Further, the semantic segmentation network comprises an encoder, a decoder and an interactive self-distillation module, the encoder is configured to perform multiple times of down-sampling on an input parcel phase image, and after each down-sampling, feature extraction is performed through a Repvgg module;

[0016] The decoder is configured to perform multiple times of up-sampling on the input parcel phase image, and after each up-sampling, feature extraction is performed through a normal convolution layer;

[0017] The feature layers with the same resolution in the encoder and the decoder are connected through the interactive self-distillation module to perform bidirectional attention distillation.

[0018] Further, the interactive self-distillation module specifically takes the feature attention maps of the corresponding layers of the encoder and the decoder as interactive distillation targets in the training process, optimizes bidirectional Kullback-Leibler (KL) divergence, realizes cross-level supervised learning, and ensures the stability of semantic extraction at high resolution;

[0019] The working process of the interactive self-distillation module comprises:

[0020] Firstly, the feature attention map soft labels of the corresponding layers of the encoder and the decoder are extracted, and in the extraction process, firstly, the sum of the pixel values of each channel in each point of the feature map is calculated, and the calculation expression is as follows:

[0021]

[0022] Wherein, E (c,i,j) and D (c,i,j) respectively represent the value of the cth channel of the feature map of the encoder and the decoder at position (i,j);

[0023] Then, the Softmax function is applied along the spatial dimension to normalize and generate the attention map soft label:

[0024]

[0025] Wherein, H and W respectively represent the height and width of the image, and D respectively represent the value of the cth channel of the feature map of the encoder and the decoder at position (i,j);

[0026] Secondly, the bidirectional KL divergence loss is calculated to realize bidirectional distillation learning, and the calculation method is as follows:

[0027] KLloss=KL(E∣D)+KL(D||E)

[0028]

[0029] Wherein, KL(E||D) is the KL divergence from the encoder to the decoder, at this time the encoder as the teacher, the decoder as the student, KL(D||E) is the KL divergence from the decoder to the encoder, at this time the decoder as the teacher, the encoder as the student.

[0030] Further, the Repvgg module adopts a training-pushing decoupling mode: in training, it remains a double-branch architecture, the first branch consists of a 3x3 convolution layer and a batch normalization layer, and the second branch consists of a 1x1 convolution layer and a batch normalization layer.

[0031] In inference, the double-branch is equivalent to a single-branch through structure reparameterization technology, and the specific inference process includes:

[0032] First, the 1x1 convolution layer in the 1x1 convolution layer branch is changed to a 3x3 convolution layer through zero padding;

[0033] Second, the batch normalization layers inside the two branches are equivalent to the convolution layers in the respective branches and are combined into a new convolution layer with bias to obtain an equivalent convolution layer.

[0034] Third, the equivalent convolution layer parameters of the two branches are added and combined to obtain the combined single-branch convolution layer parameters, completing the equivalent conversion of the double-branch to the single-branch.

[0035] Further, the expression of the equivalent convolution layer is:

[0036]

[0037] Wherein, ω, b are the original convolution layer weight and bias, respectively, and p, μ, γ, σ are the BN (Batch Normalization) layer offset, mean, scaling, and standard deviation parameters, respectively. And Equivalent convolution layer weight and bias, respectively.

[0038] Further, the integrated segmenter at the end of the network is specifically a strong segmenter combined by weighting three sub-segmenters with different receptive fields, and the three sub-segmenters respectively adopt convolution layers with a dilation rate of 1, 2, and 4.

[0039] Further, the loss function includes a weighted cross-entropy loss and a weighted mean square error loss:

[0040]

[0041] Wherein, L wce is the weighted cross-entropy loss, and L wmse is the weighted mean square error loss, y c,i,ja one-hot encoding label for the point (i, j) in channel C, is the corresponding model output after softmax normalization, and the class weight corresponding to channel C is denoted as β c Channels 0, 1 and 2 represent classes 0, +1 and -1 respectively, so the values of β1 and β2 are larger, and the value of β0 is smaller, indicating that the loss of the gradient point is given a larger weight, and the loss of the non-gradient point is given a smaller weight.

[0042] Further, the adaptive enhancement strategy includes two parts of content:

[0043] The first part is to count the error rate ∈ of each sample in the current training round t i , ∈ i is the proportion of points classified incorrectly by the i-th sample in the whole graph, and then according to ∈ i , the current sample weight is updated to the sample weight in the t+1 round

[0044]

[0045] In this way, the sample with a high error rate is given a larger weight, so that the network pays more attention to this sample in the next round of training;

[0046] The second part is to update the weight of the sub-segmentor in the t-th round to the weight of the sub-segmentor in the t+1-th round according to the classification accuracy R k of each sub-segmentor

[0047]

[0048] In this way, the sub-segmentor with a high accuracy is given a larger weight, and the sub-segmentor with a low accuracy is given a smaller weight, and the final gradient segmentation result of the model is the weighted output result of each sub-segmentor.

[0049] Further, the solving process of the unwrapping phase includes:

[0050] After obtaining the transverse and longitudinal wrapping count gradients of the output, a least squares mathematical model between the ideal wrapping count value and the wrapping count gradient is established:

[0051]

[0052] where k i,j represents the ideal wrapping count value at point (i, j), gradx(i, j) is the wrapping count gradient in the x direction at point (i, j), and grady(i, j) represents the wrapping count gradient in the y direction.

[0053] The corresponding Poisson equation is derived:

[0054] (k i+1,j -2k i,j +k i-1,j )+(k i,j+1 -2k i,j +k i,j-1 )

[0055] =gradx(i,j)-gradx(i-1,j)+grady(i,j)-grady(i,j-1)=ρ i,j

[0056] Perform discrete cosine transform (DCT) on both sides of the Poisson equation to obtain the package count value k i,j and ρ i,j Relationship in the DCT domain:

[0057]

[0058] in, represents k i,j The value in the DCT domain, represents ρ i,j The value in the DCT domain, n and m represent the horizontal and vertical coordinates in the DCT domain, N represents the number of columns of the entire image, and M represents the number of rows of the entire image;

[0059] Through Perform inverse DCT transform to obtain the required package count value k i,j , the package count value k i,j Multiply by 2π and add the input wrapped phase ψ to get the unwrapped phase

[0060] Compared with the prior art, the present invention has the following advantages:

[0061] The present invention constructs a gradient segmentation model based on a convolutional neural network and trains it using a dataset to obtain a package count gradient prediction model. Specifically, the gradient segmentation model adds an interactive self-distillation mechanism to the semantic segmentation network. Through mutual supervision between shallow and deep layers within the network during training, it achieves cross-layer representation learning and promotes fine-grained segmentation of package count gradients under cross-resolution conditions. Furthermore, an integrated segmenter based on adaptive enhancement is used to achieve stable segmentation of semantic information at different spatial frequencies. This ensures high-precision segmentation of package count gradients across resolutions to obtain accurate package count gradients. A simple least-squares solution is then required to recover the unwrapped phase, reliably achieving end-to-end cross-resolution phase unwrapping.

[0062] In the present application, the semantic segmentation network comprises an encoder, a decoder and an interactive self-distillation module, the encoder and the decoder are connected through the interactive self-distillation module between the feature layers with the same resolution to perform bidirectional attention distillation, the interactive self-distillation mechanism takes the feature attention map of the corresponding layers of the encoder and the decoder as the interactive distillation target in the training process, optimizes the bidirectional KL divergence, realizes the cross-level supervised learning, and the symmetric distillation process enables the network to perform internal cross-layer supervision, enhances the perception ability of the shallow structure, and strengthens the detail information in the deep feature map, so that the stable semantic information extraction ability can be maintained under different resolution inputs.

[0063] The present application adopts a Repvgg module as the basic module of the network, which trains parameters in a double-branch form during the training process, and this double-branch structure is beneficial to improve the feature extraction capability of the model; in the inference stage, the double-branch structure is equivalent to a single-branch structure through structure reparameterization, which can greatly reduce the model parameter quantity and memory consumption while keeping the model accuracy unchanged, significantly improve the inference speed, and be beneficial to efficiently and accurately predict the package counting gradient.

[0064] The present application adopts a strong segmenter at the end of the network, which is obtained by weighted combination of three sub-segmenters with different receptive fields, and the three sub-segmenters respectively adopt convolution layers with hole rates of 1, 2 and 4. This integrated structure with multiple size receptive fields is beneficial to the adaptation of the segmenter to the semantic features under various spatial frequencies extracted by the front end of the network, and is beneficial to the network to maintain stable phase unwrapping effect for package phase maps with different spatial frequencies. When segmenting semantic information, the strong segmenter can adapt to a wider range of spatial frequencies, avoiding the frequency bias problem that may occur in a single segmenter, and realizing accurate and generalized segmentation of multi-resolution semantic information.

[0065] The present application considers that for all pixel points in the image, the number of classes +1 and-1 is much less than the number of class 0. Therefore, the loss function in the training process is designed to include weighted cross-entropy loss and weighted mean square error loss, by giving different weights to each class channel, the model pays more attention to the segmentation loss of +1 and-1 classes in the training process, which can effectively solve the problem of class imbalance, and ensure reliable training of the package counting gradient prediction model. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 The present application is a method flowchart;

[0067] Figure 2 The present application is a process diagram for phase unwrapping;

[0068] Figure 3A structural schematic diagram of the semantic segmentation network in the embodiment;

[0069] Figure 4 A working mechanism schematic diagram of the interactive self-distillation mechanism in the embodiment;

[0070] Figure 5 A structural schematic diagram of the network end strong segmenter in the embodiment;

[0071] Figure 6 A flowchart of the adaptive enhancement strategy in the network training process. DETAILED DESCRIPTION

[0072] The application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] EMBODIMENT

[0074] As Figure 1 shown, a phase unwrapping method based on interactive self-distillation and structure reparameterization includes the following steps:

[0075] The data set is divided into a training set, a validation set and a test set, each group of data including an input and a label, the input being a wrapped phase map with noise, and the label being a horizontal and vertical wrapping count gradient map;

[0076] A gradient segmentation model based on a convolutional neural network is constructed, an interactive self-distillation mechanism is added in the semantic segmentation network, cross-level supervised learning is realized, and the stability of semantic extraction at high resolution is ensured; a Repvgg module is used as the basic module of the network, which trains parameters in a double-branch form during the training process, and in the inference stage, the double-branch architecture is equivalently merged into a single-branch architecture through structure reparameterization, reducing the model parameter quantity and memory consumption; the segmenter at the end of the network adopts an integrated architecture, integrating three weak segmenters containing different dilated convolution into a strong segmenter for adapting to semantic information of different spatial frequencies; the loss function adopts a weighted cross-entropy loss and a weighted mean square error loss to improve the gradient segmentation accuracy;

[0077] The constructed gradient segmentation model is trained on the data set, and an adaptive enhancement strategy is adopted during the training process, giving higher weights to samples with high error rates to increase the importance of easy-to-mistake samples, and giving different weights to the correct rates of the three segmenters to achieve optimal integration effect; after training, a parameter-fixed wrapping count gradient prediction model is obtained;

[0078] The current wrapped phase map is input into the wrapping count gradient prediction model, and the corresponding horizontal and vertical wrapping count gradient is output, and then the wrapping count graph is obtained through the least square method based on discrete cosine change, and the unwrapped phase is obtained by adding the wrapping count multiplied by 2π to the wrapped phase map.

[0079] The embodiment applies the above scheme, as shown in Figure 2 The semantic segmentation network is a fast high-resolution phase unwrapping network based on interactive self-distillation and structure reparameterization. The wrapped phase map is input into the phase unwrapping network. The network outputs the lateral and longitudinal wrapped count gradient maps. Then, the least square algorithm based on discrete cosine transform is used to reconstruct the wrapped count map. The unwrapped phase map is obtained by multiplying the wrapped count map by 2π and adding the wrapped phase map.

[0080] Specifically, the embodiment first constructs a data set through a simulation algorithm. A Gaussian function with random mean and standard deviation is superimposed on a random slope slope function to generate a real phase According to the formula The wrapped count k is obtained, where round is the rounding operation; the wrapped count k is differentiated to obtain the lateral and longitudinal wrapped count gradient distribution maps, which are used as gradient labels. Then, the is subjected to an inverse tangent transformation and added with different degrees of noise to obtain the noisy wrapped phase ψ, which is used as the network input. The data set constructed in the embodiment contains 12,000 samples. Each sample contains a wrapped phase map, lateral and longitudinal wrapped count gradient labels, a wrapped count map, and an unwrapped phase true value map. 80% of the samples are used for training, 10% of the samples are used for verification, and 10% of the samples are used for testing.

[0081] Figure 3 As shown in the structural diagram of the semantic information extraction part, the wrapped phase map is first input into this part after being input into the network to output the semantic information feature map. This part is composed of an encoder, a decoder, and an interactive self-distillation module. In the embodiment, the encoder performs 5 times of downsampling on the picture, each time of downsampling is completed through a max pooling layer, and after each downsampling, a Repvgg module is used for feature extraction. The decoder performs 5 times of upsampling on the picture, each time of upsampling is completed through bilinear interpolation, and after each upsampling, a normal convolution layer is used for further feature extraction. The feature layers with the same resolution in the encoder and the decoder are connected by the interactive self-distillation module for bidirectional attention distillation.

[0082] As shown in the structural diagram of the semantic information extraction part, the wrapped phase map is first input into this part after being input into the network to output the semantic information feature map. This part is composed of an encoder, a decoder, and an interactive self-distillation module. In the embodiment, the encoder performs 5 times of downsampling on the picture, each time of downsampling is completed through a max pooling layer, and after each downsampling, a Repvgg module is used for feature extraction. The decoder performs 5 times of upsampling on the picture, each time of upsampling is completed through bilinear interpolation, and after each upsampling, a normal convolution layer is used for further feature extraction. The feature layers with the same resolution in the encoder and the decoder are connected by the interactive self-distillation module for bidirectional attention distillation. Figure 4 As shown in the structural diagram of the semantic information extraction part, the wrapped phase map is first input into this part after being input into the network to output the semantic information feature map. This part is composed of an encoder, a decoder, and an interactive self-distillation module. In the embodiment, the encoder performs 5 times of downsampling on the picture, each time of downsampling is completed through a max pooling layer, and after each downsampling, a Repvgg module is used for feature extraction. The decoder performs 5 times of upsampling on the picture, each time of upsampling is completed through bilinear interpolation, and after each upsampling, a normal convolution layer is used for further feature extraction. The feature layers with the same resolution in the encoder and the decoder are connected by the interactive self-distillation module for bidirectional attention distillation.

[0083] First, the feature attention map soft labels of the corresponding layers in the encoder and the decoder are extracted. In the extraction process, first, the sum of the pixel values of each channel in the feature map is calculated. The calculation expression is as follows:

[0084]

[0085] wherein E (c,i,j) and D (c,i,j) denote the value of the c-th channel in the feature map of the encoder and the decoder at position (i,j), respectively;

[0086] After that, the Softmax function is applied along the spatial dimension for normalization to generate the attention map soft label:

[0087]

[0088] wherein H and W denote the height and width of the image, respectively, denote the value of the attention map soft label of the encoder and the decoder at position (i,j), respectively;

[0089] The second step is to calculate the bidirectional KL divergence loss to achieve bidirectional distillation learning, and the calculation method is as follows:

[0090] KLloss = KL(E||D) + KL(D||E)

[0091]

[0092]

[0093] wherein KL(E||D) is the KL divergence from the encoder to the decoder, at this time the encoder is the teacher and the decoder is the student, and KL(D||E) is the KL divergence from the decoder to the encoder, at this time the decoder is the teacher and the encoder is the student.

[0094] KL(E||D) is used to reduce the detail distortion of deep features, and KL(D||E) is used to enhance the context perception ability of shallow layers, so that fine segmentation and dense prediction can be achieved on large size images.

[0095] In this embodiment, the network basic module Repvgg module adopts a training-pushing decoupling mode, which maintains a double-branch architecture during training, the first branch is composed of a 3x3 convolution layer and a batch normalization layer, and the second branch is composed of a 1x1 convolution layer and a batch normalization layer. This double-branch architecture is beneficial to improve the ability of the model to extract features; during inference, the double-branch is equivalent to a single-branch through structure reparameterization technology. The first step of the specific inference process, the 1x1 convolution layer in the 1x1 convolution layer branch is changed to a 3x3 convolution layer through zero padding; the second step, the batch normalization layers inside the two branches are equivalent to a new convolution layer with bias, which is combined with the convolution layers in each branch, and the expression is as follows:

[0096]

[0097] where ω, b are the original convolutional layer weights and bias, p, μ, γ, σ are BN layer offset, mean, scale, standard deviation parameters, respectively, and are the equivalent convolutional layer weights and bias, respectively; the third step is to add and merge the equivalent convolutional layer parameters of the two branches to obtain the merged single-branch convolutional layer parameters, thus completing the equivalent conversion from double branches to single branch, the model accuracy remains unchanged, the parameter amount and memory consumption are greatly reduced, and the inference speed is significantly improved.

[0098] Figure 5 As shown in the structural diagram of the strong segmenter (adaptive integrated segmenter) at the end of the network in the embodiment, the semantic information feature map output by the semantic segmentation network is input into the strong segmenter, and finally the horizontal and vertical package counting gradient maps are output. The adaptive integrated segmenter is composed of three branches, each branch is composed of two hollow convolutional layers, and the hollow rates of the hollow convolutional layers of the three branches are different. The hollow rate of the first branch is 2, the hollow rate of the second branch is 1, and the hollow rate of the third branch is 4. The final gradient segmentation result is obtained after the results output by the three branches are integrated by weighting. The integrated architecture integrates the three segmenters with different receptive fields into a strong segmenter. This integrated architecture with multiple sizes of receptive fields is conducive to the adaptation of the segmenter to the semantic features under various spatial frequencies extracted by the front-end part of the network, and is conducive to the network to maintain stable phase unwrapping effect on package phase maps with different spatial frequencies.

[0099] In this embodiment, the package counting gradient segmentation task is regarded as a 3-class segmentation problem. The horizontal and vertical gradients each have three classes: 0, 1, and -1, where the -1 class is expressed as the second class in the network. Therefore, the output of the gradient prediction network contains a total of 6 channels, the first three channels are for the three classes of horizontal gradient, and the last three channels are for the three classes of vertical gradient. For all pixel points in the image, the number of classes +1 and -1 is much smaller than the number of class 0. In order to solve the problem of class imbalance, a weighted loss function is proposed. By assigning different weights to each class channel, the model pays more attention to the segmentation loss of classes +1 and -1 during the training process. The loss function adopts a weighted cross-entropy loss and a weighted mean square error loss, both of which give larger weights to gradient points, so as to help the network pay more attention to the segmentation accuracy of gradient points during the training process. The weighted cross-entropy loss L wce and the weighted mean square error loss L wmse are represented as follows:

[0100]

[0101] where y c,i,j is the one-hot encoding label of point (i, j) in channel C, and is the corresponding model output after softmax normalization, and the category weight corresponding to channel C is expressed as β c , channels 0, 1, and 2 represent categories 0, +1, and -1, respectively, so the values ​​of β1 and β2 are large, and the value of β0 is small, indicating that a larger weight is given to the loss of gradient points, and a smaller weight is given to the loss of non-gradient points.

[0102] In addition, an adaptive enhancement strategy is used during the model training process to enhance the model training effect and provide appropriate weights for each sub-segmenter at the end of the network, such as Figure 6 As shown, the adaptive enhancement strategy consists of two parts. The first part is to count the error rate ∈ of each sample in the current training round t. i ,∈ i The proportion of points misclassified by the i-th sample to the entire image, and then according to ∈ i The current sample weight Update to the sample weight of round t+1

[0103]

[0104] This gives higher error rate samples a greater weight, so that the network pays more attention to the sample in the next round of training;

[0105] The second part is based on the classification accuracy R of each sub-segmenter k , the weight of the sub-splitter in round t Update to the weight of round t+1

[0106]

[0107] In this way, the sub-segmenters with high accuracy are given greater weights, and the sub-segmenters with low accuracy are given smaller weights. The final gradient segmentation result of the model is the weighted output result of each sub-segmenter.

[0108] After the package count gradient prediction model is trained through the above process, the current package phase map is input into the package count gradient prediction model, and the corresponding horizontal and vertical package count gradients are output. Then, based on the least squares method of discrete cosine change, the gradient segmentation result output by the network is converted into the final unwrapped phase. Specifically:

[0109] After obtaining the output transverse and longitudinal package count gradients, a least squares mathematical model is established between the ideal package count value and the package count gradient:

[0110]

[0111] Among them, ki,j represents the ideal package count value at point (i, j), gradx(i, j) is the package count gradient along the x direction at point (i, j), and grady(i, j) represents the package count gradient along the y direction;

[0112] The corresponding Poisson equation is derived:

[0113] (k i+1,j -2k i,j +k i-1,j )+(k i,j+1 -2k i,j +k i,j-1 )

[0114] =grady(i,j)-gradx(i-1,j)+grady(i,j)-grady(i,j-1)=ρ i,j

[0115] Perform discrete cosine transform (DCT) on both sides of the Poisson equation to obtain the package count value k i，j and ρ i，j Relationship in the DCT domain:

[0116]

[0117] in, represents k i,j The value in the DCT domain, Represents ρ i,j The value in the DCT domain, n and m represent the horizontal and vertical coordinates in the DCT domain, N represents the number of columns of the entire image, and M represents the number of rows of the entire image;

[0118] Through Perform inverse DCT transform to obtain the required package count value k i,j , the package count value k i,j Multiply by 2π and add the input wrapped phase ψ to get the unwrapped phase

[0119] To verify the effectiveness of this solution, this example conducted comparative experiments with several commonly used phase unwrapping networks, including REDN, DeepLabv3+, PhaseNet2.0, VDENet, and PhaseNet. This included comparative analysis at different resolutions, different phase cycle densities, and with discontinuous phases. All methods were trained with the same dataset and learning rate, and the same epochs were selected for comparison. The results are shown in Table 1:

[0120] Table 1

[0121]

[0122]

[0123] Four sub-test sets with different resolutions are constructed for comparative testing, which are 256x256, 512x512, 768x768 and 1024x1024. The average RMSE of each method under each sub-test set is shown in Table 1. It can be seen that the RMSE of the six methods is small at the resolution of 256x256. Among them, the effect of the scheme is the best, and the average RMSE is 0.3254, and the average RMSE of the second-best REDN is 0.5185. Compared with REDN, the average RMSE of the scheme is reduced by 37.24%. When the resolution is increased to 512x512, 768x768 and 1024x1024, the average RMSE of the scheme under the three resolutions is 0.3492, 0.3503 and 0.3429 respectively, and always maintains high accuracy. Obviously, the scheme can maintain excellent performance under multiple resolutions, while the other five methods are only effective at low resolution, and their errors increase sharply when the resolution increases. Through the experiment verification, it is proved that the scheme significantly improves the multi-resolution adaptability of the network, can realize high-precision gradient prediction in the cross-resolution scene, and realize high-precision and stable phase unwrapping for images of different resolutions.

Claims

1. A phase unwrapping method based on interactive self-distillation and structural reparameterization, characterized in that: The following steps are involved: The dataset is divided into training set, validation set and test set. Each set of data includes input and label. The input is the parcel phase map with noise, and the label is the horizontal and vertical parcel count gradient map. A gradient segmentation model based on a convolutional neural network is constructed. An interactive self-distillation mechanism is added to the semantic segmentation network to achieve cross-level supervised learning and ensure the stability of semantic extraction at high resolution. The Repvgg module is used as the basic module of the network. This module trains parameters in a dual-branch form during training. During the inference phase, the dual-branch architecture is equivalently merged into a single-branch architecture through structural reparameterization, reducing the number of model parameters and video memory consumption. The segmenter at the end of the network adopts an integrated architecture, integrating three weak segmenters containing convolutions with different void ratios into a strong segmenter to adapt to semantic information of different spatial frequencies. The loss function uses weighted cross entropy loss and weighted mean square error loss to improve the accuracy of gradient segmentation. The constructed gradient segmentation model is trained on the dataset. During the training process, an adaptive boosting strategy is used to assign higher weights to samples with high error rates to increase the emphasis on error-prone samples. Different weights are assigned according to the accuracy of each of the three segmenters to achieve the optimal integration effect. After the training is completed, a package count gradient prediction model with fixed parameters is obtained. The current package phase map is input into the package count gradient prediction model, and the corresponding horizontal and vertical package count gradients are output. The package count map is then obtained through the least squares method based on discrete cosine variation. The package count is multiplied by 2π and then added to the package phase map to obtain the unfolded phase.

2. The phase unwrapping method based on interactive self-distillation and structure reparameterization according to claim 1, characterized in that: The data set is specifically a data set generated by a simulation algorithm, the process of which includes: The true phase is generated by superimposing a Gaussian function with random mean and standard deviation on a ramp function with random slope. According to the formula Get the package count k, where round is the rounding operation; Difference the package count k to obtain the gradient distribution map of the horizontal and vertical package counts as gradient labels. Perform an inverse tangent transform and add different degrees of noise to obtain the noisy wrapped phase ψ as the network input.

3. The phase unwrapping method based on interactive self-distillation and structure reparameterization according to claim 1, characterized in that: The semantic segmentation network includes an encoder, a decoder and an interactive self-distillation module. The encoder is used to downsample the input wrapped phase map multiple times, and each downsampling is followed by feature extraction by the Repvgg module. The decoder is used to perform multiple upsampling on the input wrapped phase image, and extract features through a common convolutional layer after each upsampling; The feature layers with the same resolution in the encoder and decoder are connected through an interactive self-distillation module to perform bidirectional attention distillation.

4. The phase unwrapping method based on interactive self-distillation and structural reparameterization according to claim 3, characterized in that: The interactive self-distillation module specifically uses the feature attention maps of the corresponding layers of the encoder and decoder as interactive distillation targets during training, optimizes the bidirectional KL divergence, realizes cross-level supervised learning, and ensures the stability of semantic extraction at high resolution; The working process of the interactive self-distillation module includes: The first step is to extract the soft labels of the feature attention maps of the corresponding layers in the encoder and decoder. The first step in the extraction process is to calculate the sum of the pixel values ​​of each channel at each point in the feature map. The calculation expression is as follows: Among them, E (c,i,j) and D (c,i,j) Represents the value of the c-th channel at position (i, j) in the feature graph of the encoder and decoder respectively; Then apply the Softmax function along the spatial dimension for normalization to generate the attention map soft label: Where H and W represent the height and width of the image respectively. are the values ​​of the soft labels of the encoder and decoder attention maps at position (i, j); The second step is to calculate the bidirectional KL divergence loss to achieve bidirectional distillation learning. The calculation method is as follows: KLloss = KL(E|D) + KL(D||E) Among them, KL(E||D) is the KL divergence from the encoder to the decoder, in this case the encoder acts as the teacher and the decoder acts as the student, and KL(D||E) is the KL divergence from the decoder to the encoder, in this case the decoder acts as the teacher and the encoder acts as the student.

5. The phase unwrapping method based on interactive self-distillation and structure reparameterization according to claim 1, characterized in that: The Repvgg module adopts a training-inference decoupling mode: during training, it maintains a dual-branch architecture, the first branch consists of a 3×3 convolutional layer and a batch normalization layer, and the second branch consists of a 1×1 convolutional layer and a batch normalization layer; During reasoning, the two branches are equivalently merged into a single branch through structural reparameterization technology. The specific reasoning process includes: In the first step, the 1×1 convolution layer in the 1×1 convolution layer branch is converted to a 3×3 convolution layer by zero padding; In the second step, the batch normalization layers in the two branches are equivalently merged with the convolutional layers in their respective branches into a new convolutional layer with a bias to obtain an equivalent convolutional layer; In the third step, the equivalent convolutional layer parameters of the two branches are added and merged to obtain the merged single-branch convolutional layer parameters, completing the equivalent conversion from dual branches to single branch.

6. The phase unwrapping method based on interactive self-distillation and structure reparameterization according to claim 5, characterized in that: The expression of the equivalent convolutional layer is: Among them, ω and b are the weight and bias of the original convolution layer respectively, p, μ, γ, and σ are the offset, mean, scale, and standard deviation parameters of the batch normalization layer respectively. and are equivalent to the weights and biases of the convolutional layers, respectively.

7. The phase unwrapping method based on interactive self-distillation and structure reparameterization according to claim 1, characterized in that: The integrated segmenter at the end of the network specifically combines three sub-segmenters with different receptive fields into a strong segmenter by weighted combination, and the three sub-segmenters respectively use convolutional layers with void rates of 1, 2, and 4.

8. The phase unwrapping method based on interactive self-distillation and structure reparameterization according to claim 1, characterized in that: The loss function includes weighted cross entropy loss and weighted mean square error loss: Among them, H and W represent the height and width of the image respectively, L wce is the weighted cross entropy loss and L wmse is the weighted mean square error loss, y c,i,j is the one-hot encoding label of the point (i, j) in channel C, is the corresponding model output after softmax normalization, and the category weight corresponding to channel C is expressed as β c , channels 0, 1, and 2 represent categories 0, +1, and -1, respectively, so the values ​​of β1 and β2 are large, and the value of β0 is small, indicating that a larger weight is given to the loss of gradient points, and a smaller weight is given to the loss of non-gradient points.

9. The phase unwrapping method based on interactive self-distillation and structure reparameterization according to claim 1, characterized in that: The adaptive enhancement strategy includes two parts: The first part is to count the error rate of each sample in the current training round t, i ,∈ i The proportion of points misclassified by the i-th sample to the entire image, and then according to ∈ i The current sample weight Update to the sample weight of round t+1 This gives higher error rate samples a greater weight, so that the network pays more attention to the sample in the next round of training; The second part is based on the classification accuracy R of each sub-segmenter k , the weight of the sub-splitter in round t Update to the weight of round t+1 In this way, the sub-segmenters with high accuracy are given greater weights, and the sub-segmenters with low accuracy are given smaller weights. The final gradient segmentation result of the model is the weighted output result of each sub-segmenter.

10. The phase unwrapping method based on interactive self-distillation and structure reparameterization according to claim 1, characterized in that: The solution process of the unwrapped phase includes: After obtaining the output transverse and longitudinal package count gradients, a least squares mathematical model is established between the ideal package count value and the package count gradient: Among them, k i,j represents the ideal package count value at point (i, j), gradx(i, j) is the package count gradient along the x direction at point (i, j), and grady(i, j) represents the package count gradient along the y direction; The corresponding Poisson equation is derived: (k i+1,j -2k i,j +k i-1,j )+(k i,j+1 -2k i,j +k i,j-1 ) =gradx(i,j)-gradx(i-1,j)+grady(i,j)-grady(i,j-1)=ρ i,j Perform discrete cosine transform DCT on both sides of the Poisson equation to obtain the package count value k i,j and ρ i,j Relationship in the DCT domain: in, represents k i,j The value in the DCT domain, represents ρ i,j The value in the DCT domain, n and m represent the horizontal and vertical coordinates in the DCT domain, N represents the number of columns of the entire image, and M represents the number of rows of the entire image; Through Perform inverse DCT transform to obtain the required package count value k i,j , the package count value k i,j Multiply by 2π and add the input wrapped phase ψ to get the unwrapped phase

Images