A deep learning-based digital holographic wrapped phase distortion compensation method

Abstract

The application discloses a digital holographic wrapped phase distortion compensation method based on deep learning. Random Zernike polynomial coefficients and corresponding wrapped phase maps are generated by a computer, and are used as learning labels and network training neural network models respectively; a digital holographic light path is built to record a hologram of a sample to be measured, and after numerical reconstruction, the wrapped phase map is input into the trained neural network model to output Zernike polynomial coefficients to reconstruct a phase distortion distribution, and the object light reconstruction amplitude is compensated in a spatial domain; after compensation, the wrapped phase map is subjected to phase filtering and unwrapping, and the unwrapped phase is subjected to Zernike polynomial fitting based on background segmentation to compensate residual distortion. The method only trains the network with a simulation data set, compensates most of the distortion before phase unwrapping, improves the reliability of the wrapped phase data, greatly improves the phase recovery accuracy, and has the advantages of fast calculation speed and accurate distortion compensation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital holography, and particularly relates to a digital holographic wrapped phase distortion compensation method based on deep learning. BACKGROUND

[0002] When measuring the microstructure surface topography by using digital holography technology, the off-axis optical path structure, the mismatch of spherical curvature between the object beam and the reference beam, and the optical distortion introduced by the system building, etc., make the phase information after holographic reconstruction not only contain the phase of the sample to be measured, but also contain a large amount of tilt, bending and high-order phase distortion. Only by accurately compensating these phase distortions can the three-dimensional profile of the sample to be measured be effectively recovered. In the wrapped phase map obtained by the arctangent transformation, these phase distortions are manifested as dense tilt and bending fringes modulated by the phase of the sample to be measured. The more phase distortions introduced by the system, the denser the wrapped phase fringes.

[0003] When the structure edge of the sample to be measured is on the dense fringe area, fringe dislocation is prone to occur. Both the fringe dislocation and the coherent noise introduced by the coherent light source can reduce the data reliability of the wrapped phase map and cause errors in the phase unwrapping process. The traditional numerical compensation method of phase distortion is to compensate the continuous phase after phase unwrapping. The inaccuracy of phase unwrapping directly restricts the effectiveness of phase distortion compensation.

[0004] Although the wrapped phase map can be denoised and smoothed before phase unwrapping, for the dense fringe wrapped phase map, inappropriate phase filtering may cause fringe damage, aggravate fringe dislocation or excessively smooth the sample structure edge, thereby introducing new phase errors. SUMMARY

[0005] In order to solve the above technical problems, the present application provides a digital holographic wrapped phase distortion compensation method based on deep learning, which directly establishes a network model of wrapped phase map input and Zernike polynomial output, and automatically compensates most of the distortion components in the wrapped phase map before phase unwrapping. The method has the advantages of fast calculation speed, high precision and strong robustness, and does not require any manual intervention, input initial parameters and limitation of sample types.

[0006] The technical scheme of the present application is as follows:

[0007] The method comprises two stages of network training and holographic measurement, and is divided into the following steps:

[0008] a. The steps of the network training stage are as follows:

[0009] A simulated wrapped phase map data is generated by a computer to train a neural network model, and a trained neural network model is obtained;

[0010] b. holographic measurement stage

[0011] The trained neural network model is used to process the to-be-tested sample to obtain a three-dimensional profile distribution of the to-be-tested sample.

[0012] The network training stage specifically comprises:

[0013] Step one: a computer generates a large number of random Zernike polynomial coefficients A according to a preset numerical range, uses the Zernike polynomial coefficients A to fit a plurality of continuous two-dimensional surfaces as phase distortions by Zernike polynomials, simulates the phase distortions, and superimposes the phase distortions on a preset microstructure phase of the same type as the to-be-tested sample to construct a microstructure phase distortion distribution. A two-dimensional surface is a phase distortion, and a phase distortion is superimposed to obtain a microstructure phase distortion distribution Different two-dimensional surfaces correspond to different microstructure phase distortion distributions

[0014] The preset microstructure phase is also simulated by a computer.

[0015] Step two: the microstructure phase distortion distribution is converted into a complex exponential, and the phase angle of the complex exponential is calculated to obtain a simulated wrapped phase map φ with a value in the range of [-π, π];

[0016] Step three: a neural network model is established, the simulated wrapped phase map φ is taken as the input of the neural network model, the corresponding Zernike polynomial coefficient A is taken as the label of the neural network model, the neural network model is trained, and a trained neural network model is obtained.

[0017] The steps of the holographic measurement stage are:

[0018] S1: a digital holographic optical path is built to measure the to-be-tested sample, record the hologram of the to-be-tested sample, perform numerical reconstruction according to the hologram to obtain the object light complex amplitude U of the to-be-tested sample, calculate the wrapped phase map of the object light complex amplitude U, and input the wrapped phase map into the trained neural network model to output the Zernike polynomial coefficient A c ;

[0019] S2: the Zernike polynomial coefficient A c is used to fit a phase distortion The conjugate complex index of the fitted phase distortion is multiplied by the object light complex amplitude U to compensate for most of the phase distortions in the object light complex amplitude U, and a pre-compensation wrapped phase map that has been compensated for most of the phase distortions is obtained.

[0020] S3: Perform phase filtering and phase unwrapping on the pre-compensated wrapped phase map to obtain a continuous phase distribution containing only a small portion of residual phase distortion. For continuous phase distribution Edge enhancement and local adaptive thresholding are performed to obtain a binary mask that only represents the background region;

[0021] S4: Extracting continuous phase using a binarization mask Phase data from the mid-background region was used to construct a system of Zernike polynomial equations, and the Zernike polynomial coefficients A of the residual distortion were solved. r Using the residual distortion Zernike polynomial coefficients A r The residual distortion phase distribution was obtained by Zernike polynomial fitting.

[0022] S5: In continuous phase distribution Subtract phase distribution The true phase of the sample under test is recovered, and the wavelength is converted from the true phase to output the three-dimensional contour distribution of the sample under test.

[0023] In particular, the above-mentioned processing of this invention performs deep learning-based wrapping phase distortion compensation before phase unwrapping. This can eliminate most of the phase distortion and the stripe pattern modulated on the sample structure, avoid phase anomalies such as stripe breakage and misalignment during phase filtering, protect the edge information of the sample structure while suppressing coherent noise, and greatly improve the phase data quality, thus realizing automatic and accurate compensation of digital holographic phase distortion.

[0024] Performing distortion compensation on the wrapped phase map directly before unwrapping is of great significance.

[0025] In step one, phase distortions constructed by randomly generating Zernike polynomial coefficients are superimposed on various microstructure phase models, with the overall phase truncated within the range of [-π, π] to simulate the actual acquired digital holographic wrapped phase map, forming training samples for inputting into the neural network model. During network training, no actual digital holographic wrapped phase map or Zernike polynomial coefficients are required as input.

[0026] The neural network model in this invention is any neural network model with residual structure or its variant used for classification. The network only needs to be trained once, and then the trained network can be used to perform regression analysis on unknown samples an unlimited number of times.

[0027] The neural network model described is ResNet50, which can be implemented using frameworks such as TensorFlow or PyTorch.

[0028] In S1, the digital holographic optical path is any holographic optical path, such as coaxial, off-axis, microscopic, multi-wavelength, or super-resolution structure, and includes tilt distortion introduced by off-axis interference, secondary phase distortion introduced by the microscope objective, or higher-order phase distortion introduced by optical path construction.

[0029] The sample to be tested can be any microstructure used for holographic imaging, and can be a transmissive or reflective sample. Transmissive samples include biological cells, biological tissue structures, and microlens arrays, while reflective samples include MEMS micro / nano structures, integrated circuit chips, and silicon wafers.

[0030] In S1, numerical reconstruction includes a first step of phase shifting or spatial filtering and a second step of Fresnel diffraction, convolution, angular spectral analysis or compressed sensing reconstruction.

[0031] In S3, edge enhancement is achieved by superimposing the gradient of the continuous phase and the gradient binarization result, while local adaptive threshold segmentation is achieved by calculating the threshold of each pixel in the continuous phase separately and performing binarization processing.

[0032] This invention generates random Zernike polynomial coefficients and corresponding wrapped phase maps using a computer, which are used as learning labels and network inputs to create a dataset for training a neural network model. A digital holographic optical path recording sample hologram is constructed, and after numerical reconstruction, its wrapped phase map is input into the trained network. The output Zernike polynomial coefficients reconstruct the phase distortion distribution, compensating for the object light complex amplitude in the spatial domain. Phase filtering and unwrapping are performed on the compensated wrapped phase map, and Zernike polynomial fitting based on background segmentation is applied to the unwrapped phase to compensate for residual distortion.

[0033] This invention pre-establishes a network model with the input of the wrapped phase map and the output of the Zernike polynomial, and automatically compensates for most of the distortion components in the wrapped phase map before unwrapping the phase, thus solving the problem that it is difficult to compensate for the phase distortion components in the wrapped phase map before unwrapping the phase.

[0034] The technical solution of the present invention achieves at least the following beneficial technical effects:

[0035] (1) Reliable phase data.

[0036] This invention establishes a mapping relationship between the wrapped phase map and the Zernike polynomial coefficients directly by training a neural network. It pre-compensates for distortion before unwrapping the phase, reduces the density of the wrapped phase fringe, and avoids problems such as fringe breakage, dislocation, and excessive smoothing of sample structure edges during the phase filtering process, thus effectively improving the reliability of the wrapped phase data.

[0037] (2) Simple dataset creation.

[0038] Compared to other methods that use deep learning to compensate for phase distortion, this invention uses only a large number of computer-generated random Zernike polynomial coefficients and corresponding wrapped phase maps to create a dataset for training the neural network. During network training, there is no need to construct a digital holographic optical path to record the actual microstructure hologram, numerically reconstruct the wrapped phase map, or go through the complex process of solving Zernike polynomial coefficients (including phase filtering, phase unwrapping, background segmentation, and Zernike polynomial fitting) to create the dataset, greatly simplifying the creation of the network training dataset.

[0039] (3) Ultra-fast compensation speed.

[0040] Compared to traditional phase distortion compensation methods, this invention requires no manual intervention, no input of initial parameters, and no restrictions on sample type, thus improving the computational efficiency of phase distortion compensation. This invention can be used not only in conventional digital holographic systems but also in multi-wavelength digital holographic systems, synthetic aperture super-resolution holographic systems, and large field-of-view stitched digital holographic microscopy systems to simultaneously handle multiple distortion compensation tasks.

[0041] In summary, the method of this invention trains the network using only a simulation dataset, compensates for most distortions before unwrapping the phase data, improves the reliability of the wrapped phase data, and greatly improves the accuracy of phase recovery. It has the advantages of fast computation speed and accurate distortion compensation. Attached Figure Description

[0042] Figure 1 This is a flowchart of the deep learning-based digital holographic wrapping phase distortion compensation method provided in this embodiment of the invention;

[0043] Figure 2 This is a diagram of the neural network (ResNet) structure used in the embodiments of the present invention;

[0044] Figure 3 The figure shows the experimental results of a specific example of an embodiment of the present invention. Detailed Implementation

[0045] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0046] Examples of embodiments of the present invention Figure 1 As shown in the flowchart, the specific steps are as follows:

[0047] (1) Network training phase

[0048] Based on the input data size of the neural network (a ResNet with 20 convolutional layers), the length and width of the two-dimensional plane are set to lie on the X-axis and Y-axis respectively, with values ​​within the range of [-1, 1], and the number of sampling points is M×M; a randomly generated set of real numbers is used as the coefficients of the Zernike polynomial A = [a0 a1 a2 … a n ] T a0 a1 a2 … a n Let represent the coefficients of the Zernike polynomial for the first n terms, and T denote the matrix transpose.

[0049] A continuous two-dimensional surface was obtained by fitting Zernike polynomials to represent the phase distortion; the phase distortion was then superimposed with the simulated microstructure phase to simulate the phase distortion distribution of the microstructure.

[0050] Therefore, the simulated package phase diagram φ is specifically represented as follows:

[0051]

[0052] Where (x,y) are coordinate points on a two-dimensional plane, 1≤x≤M, 1≤y≤M; `x` represents an imaginary number, and `angle` represents the operation of taking the complex phase angle. `φ(x,y)` represents the wrapped phase at coordinates (x,y) on the wrapped phase map.

[0053] A training set is generated using a large number of simulated package phase maps φ and corresponding Zernike polynomial coefficients A as network input and labels, and then... Figure 2 The ResNet convolutional neural network training example shown has its initial parameters set to a learning rate of 0.0003, an Adam optimizer, a root mean square error loss function, and a cosine annealing function to decay the learning rate. The larger the amount of data in the training set, the less likely the network is to overfit.

[0054] (2) Holographic Measurement Stage

[0055] A digital holographic coaxial optical path is constructed, and the positive first-order term OR* in the hologram of the sample under test is extracted using the phase-shifting method; or a digital holographic off-axis optical path is constructed, and the positive first-order term OR* in the hologram is extracted using a spatial filtering algorithm; then, the object optical complex amplitude U of the sample under test is reconstructed using holographic reconstruction algorithms such as Fresnel diffraction, convolution, angular spectrum, or compressed sensing; the phase map angle(U) of the sample under test is input into a trained neural network model, which automatically outputs a set of Zernike polynomial coefficients A. c This is used to approximate phase distortion.

[0056] Next, using the Zernike polynomial coefficients A cFitting phase distortion Its conjugate complex exponent Multiply by the optical complex amplitude U of the sample to be tested to cancel out most of the phase distortion components in the optical complex amplitude, and calculate the pre-compensated wrap phase map, i.e.

[0057] Figure 3 (a) is the microstructure encapsulation phase map obtained from actual holographic measurement. Figure 3 (b) is a graph of the coefficients of the nine Zernike polynomials output by the neural network. Figure 3 (c) is the wrap-around phase map after neural network pre-compensation.

[0058] Then, the pre-compensated wrapped phase map is subjected to phase filtering based on sine and cosine transform and phase unwrapping based on least squares iteration to obtain a continuous phase containing only a small amount of residual distortion. Calculate continuous phase The gradient is calculated, and the gradient is binarized and then compared with... Adding them together yields a continuous phase Ф with enhanced edge features. c ; Calculate Ф c An adaptive threshold for each pixel, and for Ф c Local adaptive thresholding is performed to obtain a binary mask that only represents the background region.

[0059] Figure 3 (d) is the continuous phase map after edge enhancement. Figure 3 (e) is for Figure 3 (d) After local adaptive thresholding, a binary mask is obtained, where black pixels represent the background region.

[0060] Finally, continuous phase is extracted using a binarized mask. Phase data of the mid-background region; Zernike polynomial fitting is performed only on this background phase to obtain the Zernike polynomial coefficients A of the residual distortion. r and phase distribution exist Subtract The true phase of the sample under test is recovered, and finally the three-dimensional contour distribution of the sample is output.

[0061] Figure 3 (f) shows the recovered true phase distribution of the sample to be tested. It can be seen that the sample phase obtained by the present invention can accurately characterize the surface morphology distribution of the microstructure. The background area is flat and the microstructure outline is clear, which confirms the effectiveness of the present invention.

[0062] As can be seen from this implementation, the present invention can address the limitations of existing unwrapped continuous phase distortion compensation methods, which are constrained by coherent noise levels and phase data reliability. By placing distortion compensation before the wrapped phase and combining deep learning methods in artificial intelligence, most distortions in the wrapped phase image can be compensated without first unwrapping, reducing the fringe density of the wrapped phase, avoiding problems such as fringe breakage, dislocation, and excessive smoothing during phase filtering, and improving the reliability of phase data and the accuracy of holographic measurement.

Claims

1. A deep learning-based digital holographic wrapping phase distortion compensation method, characterized in that, The method comprises two phases: network training and holographic measurement, and consists of the following steps: a. The steps in the network training phase are as follows: The neural network model is trained by autonomously generating simulated package phase map data using a computer, and a trained neural network model is obtained. b. Holographic Measurement Stage The three-dimensional contour distribution of the test sample is obtained by processing the test sample using a trained neural network model. The steps in the holographic measurement stage are as follows: S1: Construct a digital holographic optical path to measure and record the hologram of the sample under test. Based on the hologram, perform numerical reconstruction to obtain the object complex amplitude U of the sample under test. Calculate the enveloping phase map of the object complex amplitude U and input it into the trained neural network model to output the Zernike polynomial coefficients Ac. S2: The phase distortion φac is fitted using Zernike polynomial coefficients Ac, and the conjugate complex exponent exp(-jφac) of the fitted phase distortion φac is multiplied by the complex amplitude U of the object beam to obtain a pre-compensated wrap phase map that has compensated for most of the phase distortion. S3: Perform phase filtering and phase unwrapping on the pre-compensated wrapped phase map to obtain a continuous phase distribution φc containing only a small portion of phase distortion. Perform edge enhancement and local adaptive threshold segmentation on the continuous phase distribution φc to obtain a binary mask that only represents the background region. S4: Use a binarized mask to extract the phase data of the background region in the continuous phase φc, construct a Zernike polynomial equation system based on the phase data of the background region, solve for the Zernike polynomial coefficients Ar of the residual distortion, and use the Zernike polynomial coefficients Ar of the residual distortion to perform Zernike polynomial fitting to obtain the residual distortion phase distribution φr. S5: Subtract the phase distribution φr from the continuous phase distribution φc to recover the true phase of the sample under test. Perform wavelength conversion on the true phase to output the three-dimensional contour distribution of the sample under test.

2. The deep learning-based digital holographic wrapping phase distortion compensation method according to claim 1, characterized in that: The network training phase specifically includes: Step 1: Generate a large number of random Zernike polynomial coefficients using a computer. A Using Zernike polynomial coefficients A Several continuous two-dimensional surfaces were fitted using Zernike polynomials to represent phase distortions. These phase distortions were then superimposed onto the microstructure phase of the same type as that of the sample under test to construct a microstructure phase distortion distribution. φ ; Step 2: Distribute the phase distortion of the microstructure φ Convert to a complex exponent, calculate the phase angle of the complex exponent to obtain the value in [- π , π Simulated package phase map within the range ϕ ; Step 3: Establish a neural network model and simulate the phase map. ϕ As input to the neural network model, the corresponding Zernike polynomial coefficients A As a label for the neural network model, it is used to train the neural network model and obtain a trained neural network model.

3. The deep learning-based digital holographic wrapping phase distortion compensation method according to claim 1, characterized in that: The neural network model described is ResNet50.

4. The deep learning-based digital holographic wrapping phase distortion compensation method according to claim 1, characterized in that: In S1, the digital holographic optical path is a coaxial, off-axis, microscopic, multi-wavelength, or super-resolution structure.

5. The deep learning-based digital holographic wrapping phase distortion compensation method according to claim 1, characterized in that: The sample to be tested can be any microstructure used for holographic imaging, and can be a transmissive or reflective sample.

6. The deep learning-based digital holographic wrapping phase distortion compensation method according to claim 1, characterized in that: In S1, numerical reconstruction includes a first step of phase shifting or spatial filtering and a second step of Fresnel diffraction, convolution, angular spectral analysis or compressed sensing reconstruction.

7. The deep learning-based digital holographic wrapping phase distortion compensation method according to claim 1, characterized in that: In S3, edge enhancement is achieved by superimposing the gradient of the continuous phase and the gradient binarization result, while local adaptive threshold segmentation is achieved by calculating the threshold of each pixel in the continuous phase separately and performing binarization processing.

Images