A Phase Aberration Correction Algorithm Optimized Based on Neural Network

Abstract

The present invention discloses a phase aberration correction algorithm optimized based on a neural network, which relates to the field of digital holographic measurement. Its technical features are as follows: After measurement and processing using digital holography, an unwrapped object phase image with phase aberration is obtained. A neural network model is established, the output of the network is determined as a set of vectors, and it is transformed into a fitting phase according to the Zernike aberration model. The difference between the object phase image with phase aberration and the fitting phase is calculated as the residual object phase image, and a multi-domain loss constraint is constructed to constrain and promote the optimization of network parameters until an object phase image with phase aberration correction is obtained. The advantage of this method is that it is not affected by the assumption that the object phase is a perturbation, does not require complex parameter adjustment or an additional image segmentation algorithm, gets rid of the defects of the deep learning algorithm's dependence on a large amount of data and poor generalization ability, is applicable to various single or multiple objects, and can achieve fast fitting and accurate correction of phase aberration.

Description

Technical Field

[0001] The present invention belongs to the field of digital holographic measurement, and particularly relates to a method for correcting phase aberration in digital holographic measurement. Background Art

[0002] Digital holography can accurately obtain the phase distribution on the path of the object light wave through two steps of interference recording and analog reproduction. However, due to problems such as off-axis angle, path perturbation, and inherent aberration of optical lenses, the recorded object phase image will contain superimposed background phase aberration. Especially in a digital holographic microscopy measurement system, the secondary phase aberration caused by a high-magnification objective lens is particularly serious, greatly affecting the accuracy of three-dimensional shape measurement of the object. Therefore, a phase aberration correction method must be used to compensate for the aberration.

[0003] The phase aberration correction techniques in digital holography can be mainly divided into two categories: physical compensation method and numerical compensation method. Among them, the physical compensation method requires introducing additional optical elements in the measurement optical path and precise calibration, increasing the inherent cost, complexity, and measurement requirements of the system, and can only be used to compensate for the inherent aberration brought by optical elements. The numerical compensation method can be mainly divided into the double-exposure method and the numerical fitting method. The double-exposure method assumes that the system is constant, and records a background hologram to cancel the system aberration, but cannot correct the phase aberration caused by perturbation during long-term observation. The numerical fitting method constructs an aberration model and uses an optimization algorithm for numerical fitting to compensate for the phase aberration. Common algorithms include the least squares method, the least squares method based on image segmentation, the quasi-Newton method, the deep learning method, etc.

[0004] The least squares method uses the constructed aberration polynomial model to perform least squares fitting on the object phase image with phase aberration to solve for the optimal coefficients, but it needs to be based on the assumption that the object phase is a small perturbation relative to the aberration phase, and its application range is limited. The least squares method based on image segmentation combines image segmentation technology to eliminate the object phase information to reduce errors, but the image segmentation algorithm requires additional parameter adjustment costs and running time. The quasi-Newton method can solve for the coefficients using the constructed objective function, but has defects such as professional parameter adjustment, long iterative optimization time, and limited accuracy. The deep learning method pre-trains network parameters through the collected data set to directly correct the phase aberration or extract the phase aberration for compensation, but this method relies on a large amount of data collected in the early stage and long-term training, and cannot be generalized to all cases of samples. Summary of the Invention

[0005] The present invention provides a phase aberration correction algorithm optimized based on a neural network. By constructing multi-domain loss constraints and using the combination of a neural network model and a Zernike polynomial aberration model to optimize and solve the Zernike coefficients, the final correction effect is remarkable. Compared with the existing algorithms, the present invention has a short optimization time and high accuracy, without the assumption that the object phase is a perturbation and complex parameter tuning, and without pre-training network parameters in advance. It can be applied to the fast and accurate correction of the phase images of various objects with phase aberration in single or multiple images, which is beneficial to ensuring the real-time performance and accuracy of the digital holographic measurement system.

[0006] The specific technical solution of the present invention is as follows:

[0007] A phase aberration correction algorithm optimized based on a neural network, characterized by including the following steps:

[0008] S1. Using digital holography, obtain the unwrapped object phase image P with phase aberration.

[0009] S2. Establish a neural network model, set a non-zero constant vector as the network input, determine the output of the network as a set of vectors, marked as C = {c1, c2,..., cN}, where N - 1 is the number of coefficients required for fitting the Zernike aberration model. Initialize the network parameters, use the output vector of the network as the coefficients of the Zernike polynomial, and calculate the fitted phase B using the polynomial model.

[0010] S3. Take the difference between the object phase image P with phase aberration and the fitted phase B as the residual object phase image, construct multi-domain loss constraints, including mean absolute error loss constraint, wrapped gradient loss constraint, and local L1 loss constraint, to promote network iterative optimization. Further set the convergence condition and determine whether the residual object phase image meets the convergence condition.

[0011] S4. If the residual object phase image does not meet the convergence condition, perform backpropagation, adjust the neural network parameters, obtain the optimized output vector C, and continue to loop; if the residual object phase image meets the convergence condition, stop backpropagation, save the network output, and the object phase image F corresponding to the phase aberration correction of P can be obtained.

[0012] The object in step S1 is any target with a clear contour.

[0013] The neural network model in step S2 is any neural network model with an output of a set of vectors, and the framework is Pytorch or Tensorflow. The neural network model is either arbitrarily initialized or pre-trained.

[0014] The network input in step S2 is a non-zero constant vector or matrix of any shape, and its shape depends on the requirements of the neural network model.

[0015] The aberration model in step S2 is an aberration model that arbitrarily fits the phase through polynomials, including but not limited to the Zernike aberration model, the standard polynomial aberration model, and the Legendre polynomial aberration model.

[0016] The convergence condition in step S3 is only used to stop the iteration, including but not limited to the manual judgment of the residual object phase image.

[0017] The beneficial effect of the present invention is to combine the optimization ability of the neural network with the problem of numerical compensation of phase aberration, and propose a phase aberration correction algorithm based on neural network optimization. This method is not affected by the assumption that the object phase must be perturbed, nor does it require an additional image segmentation algorithm. It can realize the rapid fitting and accurate correction of phase aberration through the self-constructed multi-domain loss constraint and the optimization ability of the neural network, without complex parameter tuning. This method gets rid of the defects of the traditional deep learning algorithm that depends on a large amount of data for long-term training and has poor generalization ability, and is applicable to the correction of various object phase images with single or multiple phase aberrations, with short calculation time and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flowchart of a phase aberration correction algorithm based on neural network optimization;

[0019] Figure 2 is a schematic diagram of the loss function calculation and network parameter optimization in the embodiment;

[0020] Figure 3 is an optical path diagram of a phase aberration correction algorithm based on neural network optimization in the embodiment;

[0021] Figure 4 is a 3D residual phase diagram of the fitted phase and the reference phase aberration of the present invention and the traditional method in the embodiment;

[0022] Figure 5 is a 2D phase diagram of the phase correction of hela cells of the present invention and the traditional method in the embodiment.

[0023] Figure 1 In the figure: The dashed line indicates continuous iterative optimization that does not meet the convergence condition, and the solid line indicates the realization of phase aberration correction that meets the convergence condition.

[0024] Figure 3 In the figure: 1 - laser, 2 - collimating lens, 3 - non-polarizing beam splitter prism, 4 - high-power microscope objective, 5 - cell sample, 6 - mirror, 7 - collimating lens, 8 - mirror, 9 - non-polarizing beam splitter prism, 10 - CCD camera, 11 - computer. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The present invention will be further described in conjunction with embodiments and the accompanying drawings:

[0026] Embodiment 1: The working process of implementing the phase aberration correction algorithm based on neural network optimization according to the present invention is as follows:

[0027] Adopt as Figure 3 shown off-axis digital holographic microscopy measurement system, including: laser 1, collimating lens 2, non-polarizing beam splitter prism 3, high-power microscope objective 4, cell sample 5, mirror 6, collimating lens 7, mirror 8, non-polarizing beam splitter prism 9, CCD camera 10, computer 11.

[0028] The laser 1 emits a laser of 532.8 nm, which is expanded and collimated after passing through the collimating lens 2, and is divided into two laser beams in different directions by the non-polarizing beam splitter prism 3: transmitted light and reflected light. The transmitted light irradiates on the cell sample 5 after passing through the high-power microscope objective 4, and carries the cell phase information after passing through the cell sample 5. Then, the transmitted light magnified by the high-power microscope objective 4 is reflected by the mirror 6 and collimated by the collimating lens 7, and then enters the non-polarizing beam splitter prism 9. The reflected light directly enters the non-polarizing beam splitter prism 9 after passing through the mirror 8 and interferes with the transmitted light. The laser passing through the non-polarizing beam splitter prism 9 is photographed and saved by the CCD camera 10, and is transmitted and stored in the computer 11.

[0029] Measure the hela cell sample, perform numerical reconstruction and phase unwrapping, segment the measurement result, and take out the hela cell phase as the object phase as Figure 5 (a). Using the Zernike polynomial aberration model of order 1 to 3, generate a reference phase aberration with a phase range similar to the object phase range. Superimpose the object phase and the reference phase aberration to obtain the object phase image P with phase aberration, and the number of pixels is 512×512 pixels.

[0030] According to as Figure 1 and Figure 2 shown algorithm flow to perform numerical fitting of phase aberration. Figure 2 The fully connected neural network in contains an input layer, two hidden layers and an output layer. The input layer and the output layer each contain 10 neural nodes, and each hidden layer contains 100 neural nodes. Establish a combination of a neural network model and a Zernike polynomial aberration model, and initialize the network parameters in the kaiming_uniform manner. The initial learning rate of the network is set to 0.02, and it is set to perform a 20-step decrease to 80% of the original when the learning rate is greater than 0.001. The optimizer is Adagrad, and the activation function is tanh. Use the output vector C of the network to be transformed into the fitting phase B according to the Zernike aberration model. Among them, the specific calculation formula for transforming into the fitting phase B according to the Zernike aberration model is

[0031]

[0032] The residual object phase image is obtained by subtracting the fitting phase B from the object phase P with phase aberration, and the loss calculation and gradient derivation are performed on the residual object phase image using the constructed multi-domain loss constraint. Among them, the specific calculation formula of the mean absolute error loss constraint Loss1 in the multi-domain loss constraint is

[0033]

[0034] where i and j respectively represent the row and column numbers of the two-dimensional image, and |·| represents taking the absolute value; the specific calculation formula of the wrapped gradient loss constraint Loss2 is

[0035] G = |Grad(Wrap(P - B))|, (3)

[0036]

[0037] where Wrap(a) = ((a + π) % 2π - π) represents wrapping the phase a, and % represents taking the remainder, represents taking the absolute gradient matrix G of the phase a along the x and y directions. The specific calculation formula of the local L1 loss constraint Loss3 is

[0038]

[0039] where Mean(a) = a / (i × j) represents taking the average value of the phase a, and Slide() is the window sliding function. Equation (5) represents calculating the mean of the L1 norm within a k × k window and then sliding the window with a step size of k to obtain the local L1 norm matrix L. The convergence condition is set to that the sparsity of the absolute gradient matrix G or the local L1 norm matrix L exceeds 70%, and its specific calculation formula:

[0040]

[0041] where num(a < b) represents the number of elements in matrix a that are less than the value b, and numel(a) represents the total number of elements in matrix a.

[0042] The algorithm automatically performs the convergence condition judgment. When the convergence condition is not met, it continues to loop and adjusts the network parameters to optimize the output vector; when the convergence condition is met, it stops the network optimization process and saves the network output to obtain the object phase image F with phase aberration correction as Figure 5 (b).

[0043] To further verify the accuracy and rapidity of the phase aberration correction of the algorithm involved in the present invention, this algorithm was compared with the least squares method, the least squares method based on image segmentation, and the quasi - Newton method. All four algorithms were written in the Python language and run on a computer with Windows 10, Intel(R) Core i7 - 10700K CPU, Nvidia Geforce GTX 1080Ti, and RAM 32.0GB. The initial vector values of all algorithms were set to 1 and run 10 times on the computer to ensure the reliability of the results. Figure 4 3D residual phase diagrams of the fitting phase and the reference phase aberration for all algorithms, where Figure 4 (a) is the algorithm involved in the present invention; Figure 4 (b) is the quasi - Newton method; Figure 4 (c) is the least squares method; Figure 4 (d) is the least squares method based on image segmentation; the image segmentation method uses the level set method, the X and Y axes represent the horizontal and vertical pixel numbers of the image, and the Z axis represents the phase value. It can be seen from Figure 4 that the residual between the fitting phase obtained by the algorithm involved in the present invention and the reference phase aberration is the smallest, and its 3D phase diagram is the flattest. Further, the mean square error value and the peak - to - valley value were used to calculate the residual phase diagram, and the structural similarity was used to calculate the fitting phase and the reference phase aberration to quantitatively evaluate the specific performance of different algorithms, as shown in Table 1. It can be seen that although the least squares method takes the shortest time of 0.6770 s, the corresponding mean square error value and peak - to - valley value reach as high as 0.8474 and 3.8451, and the structural similarity value is only 0.9288 at the lowest. In contrast, the algorithm involved in the present invention achieved the best results with a second - shortest time of 2.0827 s, with a mean square error value and peak - to - valley value of 0.0666 and 0.5010, and a structural similarity of 0.9996.

[0044] Figure 5 2D phase diagrams of the phase of hela cells after correction by different algorithms, where Figure 5 (a) is the initial object phase, Figure 5 (b) is the correction result of the algorithm involved in the present invention, Figure 5 (c) is the correction result of the quasi - Newton method, Figure 5 (d) is the correction result of the least squares method, Figure 5 (e) is the correction result of the least squares method based on image segmentation. It can be seen that the correction effect of the algorithm involved in the present invention is intuitively the closest to the initial object phase and has the best effect.

[0045] Table 1 Quantitative index performance of different algorithms

[0046]

Claims

1. A phase aberration correction algorithm optimized based on a neural network, characterized in that, It includes the following steps: S1. Using digital holography, obtain the unwrapped object phase image P with phase aberration; S2. Establish a neural network model, set a non-zero constant vector as the network input, and determine the output of the network as a set of vectors, labeled as C = {c1, c2, …, c N}, where N - 1 is the number of coefficients required for Zernike aberration model fitting. Initialize the network parameters, use the output vector of the network as the coefficients of the Zernike polynomial, and calculate the fitted phase B using the polynomial model; S3. Take the difference between the object phase image P with phase aberration and the fitted phase B as the residual object phase image, construct a multi-domain loss constraint, which includes mean absolute error loss constraint, wrapped gradient loss constraint and local L1 loss constraint, promote the network iterative optimization, further set the convergence condition, and judge whether the residual object phase image meets the convergence condition; S4. If the residual object phase image does not meet the convergence condition, perform backpropagation, adjust the neural network parameters, obtain the optimized output vector C, and continue to loop; if the residual object phase image meets the convergence condition, stop backpropagation, save the network output, and the object phase image F with phase aberration correction corresponding to P can be obtained.

2. The phase aberration correction algorithm optimized based on a neural network according to claim 1, wherein: The object in step S1 is any target with a clear contour.

3. The phase aberration correction algorithm optimized based on a neural network according to claim 1, wherein: The neural network model in step S2 is any neural network model whose output is a set of vectors, with the framework of Pytorch or Tensorflow. The neural network model is either arbitrarily initialized or pre-trained.

4. The phase aberration correction algorithm optimized based on a neural network according to claim 1, wherein: The network input in step S2 is a non-zero constant vector or matrix of any shape, and its shape depends on the requirements of the neural network model.

5. The phase aberration correction algorithm optimized based on a neural network according to claim 1, characterized in that: The aberration model in step S2 is any aberration model that fits the phase through polynomials, including but not limited to Zernike aberration model, standard polynomial aberration model and Legendre polynomial aberration model.

6. The phase aberration correction algorithm optimized based on a neural network according to claim 1, characterized in that: The convergence condition in step S3 is only used to stop the iteration, including but not limited to the manual judgment of the residual object phase image.

Images