A method for constructing an interference image phase unwrapping model, an unwrapping method and a device

Abstract

The present invention discloses a construction method, an unwrapping method and a device for an interference image phase unwrapping model. The construction method comprises: collecting a sample initial phase map containing various surface defects of an object, processing the sample initial phase map, and obtaining a training set; constructing a ResNet-Unet-cGAN network model, wherein the ResNet-Unet-cGAN network model comprises a generator and a discriminator; training the generator and the discriminator based on the training set, updating the parameters of the generator and the discriminator by using a gradient descent method until the generator and the discriminator converge, and obtaining an optimal ResNet-Unet-cGAN network model; the optimal ResNet-Unet-cGAN network model is used to input an image to be unwrapped and output a single-channel unwrapped image; and the method can improve the reliability of the unwrapping result.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for constructing an interference image phase unwrapping model, an unwrapping method and a device. Background Art

[0002] Fringe projection technology is a commonly used method for collecting images in plane inspection. It has the characteristics of simple structure, non-contact, high precision, and can quickly obtain the structural characteristics of surface defects of objects, such as convex hulls, depressions, and damage. Therefore, it has been widely used in industrial quality inspection, medical device monitoring, and cultural heritage protection.

[0003] Based on the multiple interference phase images collected by fringe projection technology, the wrapped phase image is calculated by the phase shift method, and the absolute phase is restored by phase unwrapping to enhance the visualization effect of defects. The traditional branch cutting method can obtain a more accurate unwrapped image. However, when the noise range is too large, it is difficult to quickly place a more accurate branch cut line on the residual point, which leads to redundancy in the unwrapping process and a large area of ​​unwrapped area in the processing result.

[0004] Since deep learning has outstanding advantages in feature extraction and data analysis, a reasonable network model can quickly obtain accurate and useful information in the input image, thereby automatically completing the image conversion task. In the study of phase unwrapping, the main network models are based on CNN and GAN. The CNN-based phase unwrapping network extracts image feature information through a complex network structure, learns the rules of phase unwrapping, and fuses to obtain the absolute phase image after unwrapping. However, when the data set is insufficient or the sample features are single, the generalization ability of the network model is low, and the unwrapping result is unreliable. For example, the patent text CN116664419A provides an InSAR phase unwrapping method and system for a multi-scale feature fusion denoising CNN network. The noisy real InSAR interferometric phase image is input into the InSAR phase unwrapping model of the trained multi-scale feature fusion denoising CNN network. The phase unwrapping network model uses the denoising network DnCNN as the framework, and extracts multi-scale features by setting dilated convolution and deformable convolution, fuses the extracted multi-scale feature information, and uses the residual module to perform phase unwrapping to restore the feature information; outputs the unwrapped phase image. The GAN-based phase disentanglement network uses a game-adversarial model between the generator network and the discriminator network to generate images close to the absolute phase. However, the network model has problems with insufficient convergence and stability during training, making it difficult to generate the expected image effect. Summary of the invention

[0005] The present invention provides a method for constructing an interference image phase unwrapping model, an unwrapping method and a device, which can improve the reliability of the unwrapping result.

[0006] A method for constructing an interference image phase unwrapping model, comprising:

[0007] Collecting initial phase images of samples containing various surface defects of objects, and processing the initial phase images of the samples to obtain a training set;

[0008] Construct an initial ResNet-Unet-cGAN network model, which includes a generator and a discriminator;

[0009] The generator and the discriminator are trained based on the training set, and the parameters of the generator and the discriminator are updated using the gradient descent method until the generator and the discriminator converge, so as to obtain an optimal ResNet-Unet-cGAN network model, wherein the optimal ResNet-Unet-cGAN network model is used to input an image to be disentangled and output a single-channel disentangled image.

[0010] Furthermore, the sample initial phase map is processed to obtain a training set, including:

[0011] Extracting a channel wrapped phase image according to the phase relationship between the sample initial phase images;

[0012] Determine whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point, mark the residual point and the polarity of the charge carried by the residual point, and generate a one-channel positive and negative residual point distribution map;

[0013] Merge the one-channel wrapped phase image and the one-channel positive and negative residual point distribution image to obtain a two-channel residual wrapped phase image;

[0014] Performing phase unwrapping on the positive and negative residual point distribution diagram of the one channel to obtain an absolute phase diagram of the one channel;

[0015] The training set is constructed according to the two-channel residual wrapped phase map and the one-channel absolute phase map.

[0016] Further, extracting a channel wrapped phase map according to the phase relationship between the sample initial phase maps includes:

[0017] For the sample initial phase images of the surface defects of the same object, a first sample initial phase image, a second sample initial phase image, a third sample initial phase image and a fourth sample initial phase image having the same phase offset are selected;

[0018] Calculating a first pixel difference between the fourth sample initial phase image and the second sample initial phase image, and a second pixel difference between the third sample initial phase image and the first sample initial phase image;

[0019] The inverse tangent function value of the quotient of the first pixel difference and the second pixel difference is used as the pixel value of the one-channel wrapped phase image to obtain the one-channel wrapped phase image.

[0020] Further, determining whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point includes:

[0021] Identify the wrapping phase in the one-channel wrapping phase image, and calculate the sum of phase differences for each pixel in the wrapping phase in a preset window according to a preset direction;

[0022] If the sum of the phase differences is 0, the corresponding pixel is determined to be a non-residual point; if the sum of the phase differences is not 0, the corresponding pixel is determined to be a residual point;

[0023] If the sum of the phase differences is greater than 0, the polarity of the charge carried by the corresponding residual point is determined to be positive, and if the sum of the phase differences is less than 0, the polarity of the charge carried by the corresponding residual point is determined to be negative.

[0024] Furthermore, the generator is an improved Unet network, and the improved Unet network includes an encoder and a decoder;

[0025] The encoder comprises a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit and a fifth feature extraction unit connected in sequence;

[0026] The first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit each include a convolution layer, four BasicBlock residual modules, two BottleNeck residual modules, an activation layer and a maximum pooling layer; the fifth feature extraction unit is an improved SPPCSPC module.

[0027] Further, the decoder includes a first feature fusion unit, a second feature fusion unit, a third feature fusion unit, a fourth feature fusion unit and a fifth feature fusion unit connected in sequence;

[0028] The fifth feature extraction unit is connected to the first feature fusion unit, the first feature fusion unit is jump-connected to the fourth feature extraction unit, the second feature fusion unit is jump-connected to the third feature extraction unit, the third feature fusion unit is jump-connected to the second feature extraction unit, the fourth feature fusion unit is jump-connected to the first feature extraction unit, and the fifth feature fusion unit is the output unit of the encoder;

[0029] The first feature fusion unit, the second feature fusion unit, the third feature fusion unit and the fourth feature fusion unit all include a transposed convolution layer, a convolution layer and an activation layer, and the fifth feature fusion unit consists of a convolution layer.

[0030] Further, the improved SPPCSPC module includes a first branch and a second branch, wherein the first branch includes a first CBS module, a four-layer BasicBlock residual module, a second CBS module, a pyramid pooling module, a first splicing layer, a third CBS module, a two-layer BottleNeck residual module, a second splicing layer, and a fourth CBS module connected in sequence;

[0031] The second branch includes a fifth CBS module, and the fifth CBS module is connected to the second splicing layer;

[0032] The BasicBlock residual module includes two stacked convolutional layers and one concatenation layer, and the BottleNeck residual module includes three stacked convolutional layers and one concatenation layer.

[0033] Furthermore, the discriminator is a PatchGAN module, and the PatchGAN module includes four CBS modules, a convolutional layer and an activation layer connected in sequence.

[0034] Further, the generator and the discriminator are trained based on the training set, and the parameters of the generator and the discriminator are updated by using a gradient descent method until the generator and the discriminator converge, including:

[0035] Input the two-channel residual wrapped phase image into the generator, and output a predicted single-channel unwrapped image;

[0036] The predicted single-channel unwrapped image and the two-channel residual wrapped phase map are spliced, and the one-channel absolute phase map and the two-channel residual wrapped phase map are spliced, and then input into the discriminator, and the discrimination result is output;

[0037] The loss function value and the gradient are calculated based on the discrimination result, and the parameters of the generator and the discriminator are updated along the gradient descent direction until the convergence condition is met.

[0038] A phase unwrapping method for interference images uses the optimal ResNet-Unet-cGAN network model in the above method to unwrap the image to be unwrapped and output a single-channel unwrapped image.

[0039] A device for constructing an interference image phase unwrapping model, comprising:

[0040] An acquisition module is used to acquire an initial phase image of a sample containing various surface defects of an object, and process the initial phase image of the sample to obtain a training set;

[0041] A construction module is used to construct an initial ResNet-Unet-cGAN network model, which includes a generator and a discriminator.

[0042] A training module is used to train the generator and the discriminator based on the training set, and update the parameters of the generator and the discriminator using the gradient descent method until the generator and the discriminator converge, so as to obtain an optimal ResNet-Unet-cGAN network model; the optimal ResNet-Unet-cGAN network model is used to input the image to be disentangled and output the single-channel disentangled image.

[0043] Furthermore, the acquisition module processes the sample initial phase map to obtain a training set, including:

[0044] Extracting a channel wrapped phase image according to the phase relationship between the sample initial phase images;

[0045] Determine whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point, mark the residual point and the polarity of the charge carried by the residual point, and generate a one-channel positive and negative residual point distribution map;

[0046] Merge the one-channel wrapped phase image and the one-channel positive and negative residual point distribution image to obtain a two-channel residual wrapped phase image;

[0047] Performing phase unwrapping on the positive and negative residual point distribution diagram of the one channel to obtain an absolute phase diagram of the one channel;

[0048] The training set is constructed according to the two-channel residual wrapped phase map and the one-channel absolute phase map.

[0049] Furthermore, the acquisition module extracts a channel wrapped phase map according to the phase relationship between the sample initial phase maps, including:

[0050] For the sample initial phase images of the surface defects of the same object, a first sample initial phase image, a second sample initial phase image, a third sample initial phase image and a fourth sample initial phase image having the same phase offset are selected;

[0051] Calculating a first pixel difference between the fourth sample initial phase image and the second sample initial phase image, and a second pixel difference between the third sample initial phase image and the first sample initial phase image;

[0052] The inverse tangent function value of the quotient of the first pixel difference and the second pixel difference is used as the pixel value of the one-channel wrapped phase image to obtain the one-channel wrapped phase image.

[0053] Furthermore, the acquisition module determines whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point, including:

[0054] Identify the wrapping phase in the one-channel wrapping phase image, and calculate the sum of phase differences for each pixel in the wrapping phase in a preset window according to a preset direction;

[0055] If the sum of the phase differences is 0, the corresponding pixel is determined to be a non-residual point; if the sum of the phase differences is not 0, the corresponding pixel is determined to be a residual point;

[0056] If the sum of the phase differences is greater than 0, the polarity of the charge carried by the corresponding residual point is determined to be positive, and if the sum of the phase differences is less than 0, the polarity of the charge carried by the corresponding residual point is determined to be negative.

[0057] Furthermore, the generator is an improved Unet network, and the improved Unet network includes an encoder and a decoder;

[0058] The encoder comprises a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit and a fifth feature extraction unit connected in sequence;

[0059] The first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit each include a convolution layer, four BasicBlock residual modules, two BottleNeck residual modules, an activation layer and a maximum pooling layer; the fifth feature extraction unit is an improved SPPCSPC module.

[0060] Further, the decoder includes a first feature fusion unit, a second feature fusion unit, a third feature fusion unit, a fourth feature fusion unit and a fifth feature fusion unit connected in sequence;

[0061] The fifth feature extraction unit is connected to the first feature fusion unit, the first feature fusion unit is jump-connected to the fourth feature extraction unit, the second feature fusion unit is jump-connected to the third feature extraction unit, the third feature fusion unit is jump-connected to the second feature extraction unit, the fourth feature fusion unit is jump-connected to the first feature extraction unit, and the fifth feature fusion unit is the output unit of the encoder;

[0062] The first feature fusion unit, the second feature fusion unit, the third feature fusion unit and the fourth feature fusion unit all include a transposed convolution layer, a convolution layer and an activation layer, and the fifth feature fusion unit consists of a convolution layer.

[0063] Further, the improved SPPCSPC module includes a first branch and a second branch, wherein the first branch includes a first CBS module, a four-layer BasicBlock residual module, a second CBS module, a pyramid pooling module, a first splicing layer, a third CBS module, a two-layer BottleNeck residual module, a second splicing layer, and a fourth CBS module connected in sequence;

[0064] The second branch includes a fifth CBS module, and the fifth CBS module is connected to the second splicing layer;

[0065] The BasicBlock residual module includes two stacked convolutional layers and one concatenation layer, and the BottleNeck residual module includes three stacked convolutional layers and one concatenation layer.

[0066] Furthermore, the discriminator is a PatchGAN module, and the PatchGAN module includes four CBS modules, a convolutional layer and an activation layer connected in sequence.

[0067] Furthermore, the training module trains the generator and the discriminator based on the training set, and updates the parameters of the generator and the discriminator using a gradient descent method until the generator and the discriminator converge, including:

[0068] Input the two-channel residual wrapped phase image into the generator, and output a predicted single-channel unwrapped image;

[0069] The predicted single-channel unwrapped image and the two-channel residual wrapped phase map are spliced, and the one-channel absolute phase map and the two-channel residual wrapped phase map are spliced, and the two channels are input into the discriminator, and the discrimination result is output;

[0070] The loss function value and the gradient are calculated based on the discrimination result, and the parameters of the generator and the discriminator are updated along the gradient descent direction until the convergence condition is met.

[0071] The method for establishing an interference image phase unwrapping model, the unwrapping method and the device provided by the present invention have at least the following beneficial effects:

[0072] (1) In the process of constructing the training set, for the initial phase image, the phase relationship is used to extract a channel wrapped phase image, and then the residual point is judged to obtain a channel positive and negative residual point distribution map, and then the one-channel wrapped phase image and the one-channel positive and negative residual point distribution map are merged. The generated two-channel residual wrapped phase image can highlight the location and regional information of most defects in the initial phase image; based on the training set, the constructed ResNet-Unet-cGAN network model is trained to improve the generalization ability of the model. Through the mutual confrontation between the model generator and the discriminator, the output result of the generator is optimized, and finally a valid image after disentanglement is generated;

[0073] (2) Introducing the residual point distribution information into the phase unwrapping task of deep learning can quickly complete the automatic unwrapping of the wrapped phase image, significantly improving the efficiency and accuracy of phase unwrapping, enhancing the visualization effect of surface defects of objects, and greatly improving the accuracy and missed detection rate of defect detection;

[0074] (3) The input image of the ResNet-Unet-cGAN network model incorporates the distribution map of positive and negative residual points into the wrapped phase map, which not only provides the model generator with the structural features and local details of the initial phase map, but also helps the model discriminator to more specifically judge the "true or false" of the generated image;

[0075] (4) In the improved U-Net module encoder network, multi-layer BasicBlock modules are used to replace the original convolutional layers of U-Net as the core part to deepen the network depth and enhance the model's learning of local image detail information. This not only effectively extracts image features and optimizes network model performance, but also avoids problems such as gradient vanishing and gradient explosion during training. In addition, multi-layer BottleNeck modules are connected after the BasicBlock module to further deepen the network depth and reduce the amount of calculation, thus maintaining the model stability and ensuring the accuracy of the model.

[0076] (5) The fifth submodule of the encoder adopts an improved SPPCSPC module, which continues to use the BasicBlock module and the BottleNeck module to extract deep semantic features. At the same time, it utilizes pooling windows of multiple scales to increase the local receptive field and adaptively extract target features of different sizes, effectively improving the model's ability to process input images of different resolutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 A flow chart of an embodiment of a method for constructing an interference image phase unwrapping model provided by the present invention.

[0078] Figure 2A flowchart of an embodiment of a generator in the method for constructing an interference image phase unwrapping model provided by the present invention.

[0079] Figure 3 A flowchart of another embodiment of a generator in the method for constructing an interference image phase unwrapping model provided by the present invention.

[0080] Figure 4 A structural schematic diagram of an embodiment of the SPPCSPC module in the method for constructing the interference image phase unwrapping model provided by the present invention.

[0081] Figure 5 A structural schematic diagram of an embodiment of a BasicBlock residual module in the method for constructing an interference image phase unwrapping model provided by the present invention.

[0082] Figure 6 A structural schematic diagram of an embodiment of a BottleNeck residual module in the method for constructing an interference image phase unwrapping model provided by the present invention.

[0083] Figure 7 A schematic structural diagram of an embodiment of a discriminator in the method for constructing an interference image phase unwrapping model provided by the present invention.

[0084] Figure 8 A schematic structural diagram of an embodiment of a device for constructing an interference image phase unwrapping model provided by the present invention. DETAILED DESCRIPTION

[0085] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0086] refer to Figure 1 In some embodiments, a method for constructing an interference image phase unwrapping model is provided, comprising:

[0087] S1, collecting sample initial phase images containing various surface defects of objects, processing the sample initial phase images, and obtaining a training set;

[0088] S2. Build an initial ResNet-Unet-cGAN network model, which includes a generator and a discriminator.

[0089] S3. Train the generator and the discriminator based on the training set, and use the gradient descent method to update the parameters of the generator and the discriminator until the generator and the discriminator converge, so as to obtain an optimal ResNet-Unet-cGAN network model, wherein the optimal ResNet-Unet-cGAN network model is used to input the image to be disentangled and output a single-channel disentangled image.

[0090] Specifically, in step S1, a fringe projection method with a fixed phase offset is used to obtain a plurality of groups of sample initial phase images of surface defects of the object.

[0091] Furthermore, in step S1, the sample initial phase map is processed to obtain a training set, including:

[0092] S11, extracting a channel wrapped phase image according to the phase relationship between the sample initial phase images;

[0093] S12, determining whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point, marking the residual point and the polarity of the charge carried by the residual point, and generating a one-channel positive and negative residual point distribution map;

[0094] S13, performing channel merging on the one-channel wrapped phase image and the one-channel positive and negative residual point distribution image to obtain a two-channel residual wrapped phase image;

[0095] S14, performing phase unwrapping on the positive and negative residual point distribution diagram of the one channel to obtain an absolute phase diagram of the one channel;

[0096] S15. Construct the training set according to the two-channel residual wrapped phase map and the one-channel absolute phase map.

[0097] Furthermore, in step S11, a channel wrapped phase map is extracted according to the phase relationship between the sample initial phase maps, including:

[0098] S11a, for the sample initial phase images of the surface defects of the same object, select a first sample initial phase image, a second sample initial phase image, a third sample initial phase image, and a fourth sample initial phase image having the same phase offset;

[0099] S11b, calculating a first pixel difference between the fourth sample initial phase image and the second sample initial phase image, and a second pixel difference between the third sample initial phase image and the first sample initial phase image;

[0100] S11c, using the arctangent function value of the quotient of the first pixel difference and the second pixel difference as the pixel value of a one-channel wrapped phase image to obtain the one-channel wrapped phase image.

[0101] Specifically, assuming that the phase offsets among the first sample initial phase map, the second sample initial phase map, the third sample initial phase map, and the fourth sample initial phase map are all π / 2, the following formula is used to extract a channel wrapped phase map:

[0102] ; (1)

[0103] Among them, I 1 (x, y) represents the pixel values ​​of the initial phase image of the first sample, I 2 (x, y) represents the pixel values ​​of the initial phase image of the second sample, I 3 (x, y) represents the pixel values ​​of the initial phase image of the third sample, I 4 (x, y) represents the pixel values ​​of the fourth sample initial phase map, Represents the individual pixel values ​​of a one-channel wrapped phase image.

[0104] Further, in step S12, determining whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point includes:

[0105] S12a, identifying the wrapping phase in the one-channel wrapping phase image, and calculating the sum of phase differences for each pixel in the wrapping phase in a preset window according to a preset direction;

[0106] S12b, if the sum of the phase differences is 0, determining the corresponding pixel as a non-residual point; if the sum of the phase differences is not 0, determining the corresponding pixel as a residual point;

[0107] S12c. If the sum of the phase differences is greater than 0, the polarity of the charge carried by the corresponding residual point is determined to be positive; if the sum of the phase differences is less than 0, the polarity of the charge carried by the corresponding residual point is determined to be negative.

[0108] Specifically, the pixels in a one-channel wrapped phase image store phase information. Due to the discontinuity of phase data or the influence of noise, there will be gradient jumps. According to the pixel relationship, some residual points are calculated. These residual points are likened to points with charges, which has become a term definition in this technical field. Among them, positive residual points are regarded as positively charged, and negative residual points are regarded as negatively charged. Polarity can be considered as an attribute of residual points.

[0109] The preset window may be 2*2, and the preset direction may be counterclockwise.

[0110] Further, in step S14, phase unwrapping is performed on the positive and negative residual point distribution diagram of the channel to obtain an absolute phase diagram of the channel, including:

[0111] In the distribution diagram of positive and negative residual points of the one channel, a residual point is randomly selected as the starting point, and the residual points with opposite charge polarity to the previous one are searched in turn to connect, and the branch tangent is constructed, and the accumulated charge is stopped when it is 0; all residual points are traversed to form multiple branch tangents; finally, the local phase gradient difference is used to bypass the branch tangent and perform integral unwrapping on the element. In this way, phase unwrapping is realized and the one-channel absolute phase diagram is generated.

[0112] Specifically, in step S2, the generator is an improved Unet network, and the improved Unet network includes an encoder and a decoder.

[0113] refer to Figure 2 The encoder includes a first feature extraction unit 101, a second feature extraction unit 102, a third feature extraction unit 103, a fourth feature extraction unit 104 and a fifth feature extraction unit 105 which are connected in sequence.

[0114] refer to Figure 3 The first feature extraction unit 101, the second feature extraction unit 102, the third feature extraction unit 103 and the fourth feature extraction unit 104 each include a 3*3 convolution layer Conv, four BasicBlock residual modules, two BottleNeck residual modules, one activation layer and one maximum pooling layer; the fifth feature extraction unit 105 is an improved SPPCSPC module.

[0115] Specifically, the first feature extraction unit 101, the second feature extraction unit 102, the third feature extraction unit 103, the fourth feature extraction unit 104 and the fifth feature extraction unit 105 perform feature extraction on the input image from shallow to deep. The introduced BasicBlock residual module and BottleNeck residual module can deepen the network depth and enhance the model's learning of local detail information of the image. It not only effectively extracts image features and optimizes network model performance, but also avoids problems such as gradient vanishing and gradient explosion during training, thereby improving the accuracy and stability of the model.

[0116] refer to Figure 2 , the decoder includes a first feature fusion unit 201, a second feature fusion unit 202, a third feature fusion unit 203, a fourth feature fusion unit 204 and a fifth feature fusion unit 205 connected in sequence;

[0117] The fifth feature extraction unit 105 is connected to the first feature fusion unit 201, the first feature fusion unit 201 is jump-connected to the fourth feature extraction unit 104, the second feature fusion unit 202 is jump-connected to the third feature extraction unit 103, the third feature fusion unit 203 is jump-connected to the second feature extraction unit 102, the fourth feature fusion unit 204 is jump-connected to the first feature extraction unit 101, and the fifth feature fusion unit 205 is the output unit of the encoder.

[0118] Furthermore, the first feature fusion unit 201, the second feature fusion unit 202, the third feature fusion unit 203 and the fourth feature fusion unit 204 all include a transposed convolution layer SkipConnect, a convolution layer Conv and an activation layer, and the fifth feature fusion unit is composed of a 3*3 convolution layer Conv.

[0119] Specifically, the transposed convolution layer of the first feature fusion unit 201 is connected to the fifth feature extraction unit 105, namely, the SPPCSPC module. The transposed convolution layer of the first feature fusion unit 201 is also jump-connected to the activation layer of the fourth feature extraction unit 104. The transposed convolution layer of the second feature fusion unit 202 is jump-connected to the activation layer of the third feature extraction unit 103. The transposed convolution layer of the third feature fusion unit 203 is jump-connected to the activation layer of the second feature extraction unit 102. The transposed convolution layer of the fourth feature fusion unit 204 is jump-connected to the activation layer of the first feature extraction unit 101.

[0120] The jump connection between the transposed convolution layer and the activation layer forms a skip connection layer, which directly transfers the activation layer in the feature extraction unit to the output of the transposed convolution layer.

[0121] Further, refer to Figure 4 The improved SPPCSPC module includes a first branch and a second branch, wherein the first branch includes a first CBS module 301, a four-layer BasicBlock residual module 302, a second CBS module 303, a pyramid pooling module 304, a first splicing layer 305, a third CBS module 306, a two-layer BottleNeck residual module 307, a second splicing layer 308, and a fourth CBS module 309 connected in sequence;

[0122] Among them, the pyramid pooling module includes three maximum pooling layers MaxPool of different scales, and the pooling window sizes are 5*5, 9*9, and 13*13 respectively.

[0123] The second branch includes a fifth CBS module 310 connected to the second splicing layer connection 308 .

[0124] The fifth feature extraction module introduces the BasicBlock module and the BottleNeck module, which can extract deep semantic features. At the same time, it uses pooling windows of various scales to increase the local receptive field and adaptively extract target features of different sizes, effectively improving the model's processing capabilities for input images of different resolutions.

[0125] Further, refer to Figure 5 The BasicBlock residual module includes two stacked 3*3 convolutional layers Conv and a concatenation layer Concat.

[0126] refer to Figure 6 The BottleNeck residual module includes three stacked convolutional layers Conv and a concatenation layer Concat, where the first convolutional layer is 1*1 and the other two convolutional layers are 3*3.

[0127] Further, refer to Figure 7 , the discriminator is a PatchGAN module, and the PatchGAN module includes four CBS modules connected in sequence, a convolution layer Conv and an activation layer.

[0128] Furthermore, in step S3, the generator and the discriminator are trained based on the training set, and the parameters of the generator and the discriminator are updated using the gradient descent method until the generator and the discriminator converge, including:

[0129] S31, inputting the two-channel residual wrapped phase image into the generator, and outputting a predicted single-channel unwrapped image;

[0130] S32, splicing the predicted single-channel unwrapped image and the two-channel residual wrapped phase map, and splicing the one-channel absolute phase map and the two-channel residual wrapped phase map, inputting them into a discriminator, and outputting a discrimination result;

[0131] S33. Calculate the loss function value and gradient based on the discrimination result, and update the parameters of the generator and the discriminator along the gradient descent direction until the convergence condition is met.

[0132] Specifically, when training the ResNet-Unet-cGAN network model, the above two-channel residual wrapped phase image is input into the generator of the model as the conditional image of the ResNet-Unet-cGAN network model, and the generator outputs a predicted single-channel disentangled image. The closer the predicted single-channel disentangled image is to the above-mentioned real one-channel absolute phase image, the better the generator effect.

[0133] The predicted single-channel unwrapped image obtained by the generator is spliced ​​with the two-channel residual wrapped phase map, and the above-mentioned one-channel absolute phase map is spliced ​​with the two-channel residual wrapped phase map, and they are input into the discriminator respectively. The discriminator outputs the probability value of the predicted single-channel unwrapped image being "real". When the output value tends to 0, it means that the predicted single-channel unwrapped image is "false"; when the output value tends to 1, it means that the predicted single-channel unwrapped image is "true".

[0134] Through the training and learning of the ResNet-Unet-cGAN network model, the discriminator continuously distinguishes the "true" and "false" of the predicted single-channel disentangled image, thereby prompting the continuous optimization of the generator. The generated predicted single-channel disentangled image is constantly close to the one-channel absolute phase map, and the characteristics of the two are gradually similar, thereby optimizing the discriminator's ability to distinguish "true" and "false" images. In this way, adversarial training is repeated. When the probability value of the predicted single-channel disentangled image generated by the discriminator output is "real" approaches 0.5, it means that the performance of the generator and the discriminator has reached expectations. At this time, the generated predicted single-channel disentangled image is close to the one-channel absolute phase map, and the model training is completed.

[0135] The ResNet-Unet-cGAN network model training adopts the adversarial loss function, which can be expressed as:

[0136] ; (2)

[0137] Among them, z is the noise vector, which represents the random input of the generator; V(D, G) represents the loss function; y is the conditional vector, which represents the above two-channel residual wrapped phase map; x is the real sample, which represents the above one-channel absolute phase map; Represents the generated result, that is, predicting a single-channel disentangled image, Indicates the judgment result; Represents the loss of the true absolute phase of a channel, Represents the loss of the generated predicted single-channel disentangled image. The adversarial loss function is divided into two parts: the discriminator network and the generator network. By increasing the probability that the generated result is judged as unreal, the generated result is promoted to be closer to reality, and the parameters of the network model are optimized, so as to improve the accuracy of the generated result close to the real image.

[0138] In some embodiments, an interference image phase unwrapping method is also provided, which uses the optimal ResNet-Unet-cGAN network model in the above method to unwrap the image to be unwrapped and outputs a single-channel unwrapped image.

[0139] refer to Figure 8 In some embodiments, an interference image phase unwrapping device is provided, comprising:

[0140] The acquisition module 100 is used to acquire sample initial phase images containing various surface defects of objects, and process the sample initial phase images to obtain a training set;

[0141] A construction module 200 is used to construct an initial ResNet-Unet-cGAN network model, where the initial ResNet-Unet-cGAN network model includes a generator and a discriminator;

[0142] The training module 300 is used to train the generator and the discriminator based on the training set, and update the parameters of the generator and the discriminator using the gradient descent method until the generator and the discriminator converge, so as to obtain the optimal ResNet-Unet-cGAN network model, and the optimal ResNet-Unet-cGAN network model is used to input the image to be disentangled and output the single-channel disentangled image.

[0143] Furthermore, the acquisition module 100 processes the sample initial phase map to obtain a training set, including:

[0144] Extracting a channel wrapped phase image according to the phase relationship between the sample initial phase images;

[0145] Determine whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point, mark the residual point and the polarity of the charge carried by the residual point, and generate a one-channel positive and negative residual point distribution map;

[0146] Merge the one-channel wrapped phase image and the one-channel positive and negative residual point distribution image to obtain a two-channel residual wrapped phase image;

[0147] Performing phase unwrapping on the positive and negative residual point distribution diagram of the one channel to obtain an absolute phase diagram of the one channel;

[0148] The training set is constructed according to the two-channel residual wrapped phase map and the one-channel absolute phase map.

[0149] Furthermore, the acquisition module 100 extracts a channel wrapped phase map according to the phase relationship between the sample initial phase maps, including:

[0150] For the sample initial phase images of the surface defects of the same object, a first sample initial phase image, a second sample initial phase image, a third sample initial phase image and a fourth sample initial phase image having the same phase offset are selected;

[0151] Calculating a first pixel difference between the fourth sample initial phase image and the second sample initial phase image, and a second pixel difference between the third sample initial phase image and the first sample initial phase image;

[0152] The inverse tangent function value of the quotient of the first pixel difference and the second pixel difference is used as the pixel value of the one-channel wrapped phase image to obtain the one-channel wrapped phase image.

[0153] Furthermore, the acquisition module 100 determines whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point, including:

[0154] Identify the wrapping phase in the one-channel wrapping phase image, and calculate the sum of phase differences for each pixel in the wrapping phase in a preset window according to a preset direction;

[0155] If the sum of the phase differences is 0, the corresponding pixel is determined to be a non-residual point; if the sum of the phase differences is not 0, the corresponding pixel is determined to be a residual point;

[0156] If the sum of the phase differences is greater than 0, the polarity of the charge carried by the corresponding residual point is determined to be positive, and if the sum of the phase differences is less than 0, the polarity of the charge carried by the corresponding residual point is determined to be negative.

[0157] Furthermore, the generator is an improved Unet network, and the improved Unet network includes an encoder and a decoder;

[0158] The encoder comprises a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit and a fifth feature extraction unit connected in sequence;

[0159] The first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit each include a convolution layer, four BasicBlock residual modules, two BottleNeck residual modules, an activation layer and a maximum pooling layer; the fifth feature extraction unit is an improved SPPCSPC module.

[0160] Further, the decoder includes a first feature fusion unit, a second feature fusion unit, a third feature fusion unit, a fourth feature fusion unit and a fifth feature fusion unit connected in sequence;

[0161] The fifth feature extraction unit is connected to the first feature fusion unit, the first feature fusion unit is jump-connected to the fourth feature extraction unit, the second feature fusion unit is jump-connected to the third feature extraction unit, the third feature fusion unit is jump-connected to the second feature extraction unit, the fourth feature fusion unit is jump-connected to the first feature extraction unit, and the fifth feature fusion unit is the output unit of the encoder;

[0162] The first feature fusion unit, the second feature fusion unit, the third feature fusion unit and the fourth feature fusion unit all include a transposed convolution layer, a convolution layer and an activation layer, and the fifth feature fusion unit consists of a convolution layer.

[0163] Further, the improved SPPCSPC module includes a first branch and a second branch, wherein the first branch includes a first CBS module, a four-layer BasicBlock residual module, a second CBS module, a pyramid pooling module, a first splicing layer, a third CBS module, a two-layer BottleNeck residual module, a second splicing layer, and a fourth CBS module connected in sequence;

[0164] The second branch includes a fifth CBS module, and the fifth CBS module is connected to the second splicing layer;

[0165] The BasicBlock residual module includes two stacked convolutional layers and one concatenation layer, and the BottleNeck residual module includes three stacked convolutional layers and one concatenation layer.

[0166] Furthermore, the discriminator is a PatchGAN module, and the PatchGAN module includes four CBS modules, a convolutional layer and an activation layer connected in sequence.

[0167] Furthermore, the training module 300 trains the generator and the discriminator based on the training set, and updates the parameters of the generator and the discriminator using a gradient descent method until the generator and the discriminator converge, including:

[0168] Input the two-channel residual wrapped phase image into the generator, and output a predicted single-channel unwrapped image;

[0169] The predicted single-channel unwrapped image and the two-channel residual wrapped phase map are spliced, and the one-channel absolute phase map and the two-channel residual wrapped phase map are spliced, and the two channels are input into the discriminator, and the discrimination result is output;

[0170] The loss function value and the gradient are calculated based on the discrimination result, and the parameters of the generator and the discriminator are updated along the gradient descent direction until the convergence condition is met.

[0171] The method for establishing the interference image phase unwrapping model based on deep learning, the unwrapping method and the device provided in the above embodiments have at least the following beneficial effects:

[0172] (1) In the process of constructing the training set, for the initial phase image, the phase relationship is used to extract a channel wrapped phase image, and then the residual point is judged to obtain a channel positive and negative residual point distribution map, and then the one-channel wrapped phase image and the one-channel positive and negative residual point distribution map are merged. The generated two-channel residual wrapped phase image can highlight the location and regional information of most defects in the initial phase image; based on the training set, the constructed ResNet-Unet-cGAN network model is trained to improve the generalization ability of the model. Through the mutual confrontation between the model generator and the discriminator, the output result of the generator is optimized, and finally a valid image after disentanglement is generated;

[0173] (2) Introducing the residual point distribution information into the phase unwrapping task of deep learning can quickly complete the automatic unwrapping of the wrapped phase image, significantly improving the efficiency and accuracy of phase unwrapping, enhancing the visualization effect of surface defects of objects, and greatly improving the accuracy and missed detection rate of defect detection;

[0174] (3) The input image of the ResNet-Unet-cGAN network model incorporates the distribution map of positive and negative residual points into the wrapped phase map, which not only provides the model generator with the structural features and local details of the initial phase map, but also helps the model discriminator to more specifically judge the "true or false" of the generated image;

[0175] (4) In the improved U-Net module encoder network, multi-layer BasicBlock modules are used to replace the original convolutional layers of U-Net as the core part to deepen the network depth and enhance the model's learning of local image detail information. This not only effectively extracts image features and optimizes network model performance, but also avoids problems such as gradient vanishing and gradient explosion during training. In addition, multi-layer BottleNeck modules are connected after the BasicBlock module to further deepen the network depth and reduce the amount of calculation, thus maintaining the model stability and ensuring the accuracy of the model.

[0176] (5) The fifth submodule of the encoder adopts an improved SPPCSPC module, which continues to use the BasicBlock module and BottleNeck to extract deep semantic features. At the same time, it uses pooling windows of multiple scales to increase the local receptive field and adaptively extract target features of different sizes, effectively improving the model's ability to process input images of different resolutions.

[0177] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for constructing an interference image phase unwrapping model, characterized in that: include: Collecting initial phase images of samples containing various surface defects of objects, and processing the initial phase images of the samples to obtain a training set; Construct an initial ResNet-Unet-cGAN network model, which includes a generator and a discriminator; Training the generator and the discriminator based on the training set, updating the parameters of the generator and the discriminator using the gradient descent method until the generator and the discriminator converge, and obtaining an optimal ResNet-Unet-cGAN network model, wherein the optimal ResNet-Unet-cGAN network model is used to input the image to be disentangled and output a single-channel disentangled image; The sample initial phase map is processed to obtain a training set, including: Extracting a channel wrapped phase image according to the phase relationship between the sample initial phase images; Determine whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point, mark the residual point and the polarity of the charge carried by the residual point, and generate a one-channel positive and negative residual point distribution map; Merge the one-channel wrapped phase image and the one-channel positive and negative residual point distribution image to obtain a two-channel residual wrapped phase image; Performing phase unwrapping on the positive and negative residual point distribution diagram of the one channel to obtain an absolute phase diagram of the one channel; The training set is constructed according to the two-channel residual wrapped phase map and the one-channel absolute phase map.

2. The method according to claim 1, characterized in that: Extracting a channel wrapped phase image according to the phase relationship between the sample initial phase images, including: For the sample initial phase images of the surface defects of the same object, a first sample initial phase image, a second sample initial phase image, a third sample initial phase image and a fourth sample initial phase image having the same phase offset are selected; Calculating a first pixel difference between the fourth sample initial phase image and the second sample initial phase image, and a second pixel difference between the third sample initial phase image and the first sample initial phase image; The inverse tangent function value of the quotient of the first pixel difference and the second pixel difference is used as the pixel value of the one-channel wrapped phase image to obtain the one-channel wrapped phase image.

3. The method according to claim 1, characterized in that Determining whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point includes: Identify the wrapping phase in the one-channel wrapping phase image, and calculate the sum of phase differences for each pixel in the wrapping phase in a preset window according to a preset direction; If the sum of the phase differences is 0, the corresponding pixel is determined to be a non-residual point; if the sum of the phase differences is not 0, the corresponding pixel is determined to be a residual point; If the sum of the phase differences is greater than 0, the polarity of the charge carried by the corresponding residual point is determined to be positive, and if the sum of the phase differences is less than 0, the polarity of the charge carried by the corresponding residual point is determined to be negative.

4. The method according to claim 1, characterized in that The generator is an improved Unet network, and the improved Unet network includes an encoder and a decoder; The encoder comprises a first feature extraction unit, a second feature extraction unit, a third feature extraction unit, a fourth feature extraction unit and a fifth feature extraction unit connected in sequence; The first feature extraction unit, the second feature extraction unit, the third feature extraction unit and the fourth feature extraction unit each include a convolution layer, four BasicBlock residual modules, two BottleNeck residual modules, an activation layer and a maximum pooling layer; the fifth feature extraction unit is an improved SPPCSPC module.

5. The method according to claim 4, characterized in that The decoder comprises a first feature fusion unit, a second feature fusion unit, a third feature fusion unit, a fourth feature fusion unit and a fifth feature fusion unit connected in sequence; The fifth feature extraction unit is connected to the first feature fusion unit, the first feature fusion unit is jump-connected to the fourth feature extraction unit, the second feature fusion unit is jump-connected to the third feature extraction unit, the third feature fusion unit is jump-connected to the second feature extraction unit, the fourth feature fusion unit is jump-connected to the first feature extraction unit, and the fifth feature fusion unit is the output unit of the encoder; The first feature fusion unit, the second feature fusion unit, the third feature fusion unit and the fourth feature fusion unit all include a transposed convolution layer, a convolution layer and an activation layer, and the fifth feature fusion unit consists of a convolution layer.

6. The method according to claim 4, characterized in that The improved SPPCSPC module includes a first branch and a second branch, wherein the first branch includes a first CBS module, a four-layer BasicBlock residual module, a second CBS module, a pyramid pooling module, a first splicing layer, a third CBS module, a two-layer BottleNeck residual module, a second splicing layer, and a fourth CBS module connected in sequence; The second branch includes a fifth CBS module, and the fifth CBS module is connected to the second splicing layer; The BasicBlock residual module includes two stacked convolutional layers and one concatenation layer, and the BottleNeck residual module includes three stacked convolutional layers and one concatenation layer.

7. The method according to claim 1, characterized in that The generator and the discriminator are trained based on the training set, and the parameters of the generator and the discriminator are updated by using a gradient descent method until the generator and the discriminator converge, including: Input the two-channel residual wrapped phase image into the generator, and output a predicted single-channel unwrapped image; The predicted single-channel unwrapped image and the two-channel residual wrapped phase map are spliced, and the one-channel absolute phase map and the two-channel residual wrapped phase map are spliced, and the two channels are input into the discriminator, and the discrimination result is output; The loss function value and the gradient are calculated based on the discrimination result, and the parameters of the generator and the discriminator are updated along the gradient descent direction until the convergence condition is met.

8. A method for phase unwrapping of an interference image, characterized in that: The image to be untangled is untangled using the optimal ResNet-Unet-cGAN network model in any one of the methods described in claims 1 to 7, and a single-channel untangled image is output.

9. A device for constructing an interference image phase unwrapping model based on deep learning, characterized in that: include: An acquisition module is used to acquire an initial phase image of a sample containing various surface defects of an object, and process the initial phase image of the sample to obtain a training set; The construction module is used to initially construct the ResNet-Unet-cGAN network model. The initial ResNet-Unet-cGAN network model includes a generator and a discriminator. A training module, used to train the generator and the discriminator based on the training set, and update the parameters of the generator and the discriminator using the gradient descent method until the generator and the discriminator converge to obtain an optimal ResNet-Unet-cGAN network model; The optimal ResNet-Unet-cGAN network model is used to input the image to be disentangled and output the single-channel disentangled image; The acquisition module processes the sample initial phase map to obtain a training set, including: Extracting a channel wrapped phase image according to the phase relationship between the sample initial phase images; Determine whether a pixel in the one-channel wrapped phase image is a residual point and the polarity of the charge carried by the residual point, mark the residual point and the polarity of the charge carried by the residual point, and generate a one-channel positive and negative residual point distribution map; Merge the one-channel wrapped phase image and the one-channel positive and negative residual point distribution image to obtain a two-channel residual wrapped phase image; Performing phase unwrapping on the positive and negative residual point distribution diagram of the one channel to obtain an absolute phase diagram of the one channel; The training set is constructed according to the two-channel residual wrapped phase map and the one-channel absolute phase map.

Images