Optical synthetic aperture digital holographic microscopy wrapped phase aberration compensation method based on deep learning

Abstract

The invention discloses an optical synthetic aperture digital holographic microscopy wrapped phase aberration compensation method based on deep learning. The method comprises the following steps: firstly, training a deep learning network model according to a simulation wrapped phase diagram and a corresponding Zernike polynomial coefficient, and then sequentially carrying out comprehensive processing such as multi-direction illumination processing and phase imaging processing on a to-be-detected sample to obtain a sub-aperture wrapped phase diagram; and inputting the sub-aperture wrapped phase diagram into the trained deep learning network model for processing, and comprehensively processing the processing result to obtain the super-resolution sample contour of the to-be-tested sample. According to the method, the phase aberration of the multi-direction sub-aperture complex amplitude can be quickly and accurately compensated, reliable aperture synthesis and resolution isotropic enhancement are realized, subsequent residual aberration compensation is not needed, the aperture synthesis process is greatly simplified while the phase data quality is improved, the time cost is saved, and the method is suitable for large-scale popularization and application. The method has great potential in digital holographic microscopic quantitative measurement application.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical digital holography, and in particular relates to a method for compensating phase aberration of optical synthetic aperture digital holographic microscopy based on deep learning. Background Art

[0002] Synthetic aperture digital holographic microscopy (SA-DHM) can break through the mutual constraints between resolution and field of view in digital holographic microscopy (DHM) through multi-directional oblique illumination and numerical synthetic aperture. However, the sensitivity of quantitative phase to subtle changes in the light field makes aperture synthesis and phase reconstruction susceptible to optical aberrations, limiting the signal-to-noise ratio, image quality and spatial resolution of SA-DHM imaging. Due to the off-axis structure, the mismatch of the spherical curvature of the object beam and the reference beam, and the optical aberrations of the device, the complex amplitude of the object light reconstructed by holography contains oblique aberrations, secondary aberrations and higher-order aberrations, which reduces the ability of the microscope to image fine spatial features. When multi-directional illumination is achieved using beam deflection devices such as galvanometers, spatial light modulators (SLMs) or DMDs, random phase aberrations are introduced by slight mismatches of different oblique beams in the experimental layout. In addition, secondary phase aberrations will expand the sub-aperture spectrum in each illumination direction, and it is difficult to achieve accurate frequency shift and aperture synthesis by directly using spectrum centering. Therefore, the phase aberration in each oblique illumination sub-aperture spectrum must be accurately compensated before the aperture is synthesized in order to achieve accurate and reliable aperture synthesis and effectively improve image quality and spatial resolution.

[0003] In order to reduce aberration interference, physical compensation is generally required in the construction of the optical path, and numerical compensation is performed when the object light information is reconstructed. For example, a telecentric microstructure is designed in the object light path, the same micro-objective lens or electrically adjustable lens is introduced in the reference light path to physically offset the secondary aberration of the object light wave, or an additional object-free hologram is recorded. However, these physical methods require complex adjustments or post-processing operations for numerical aberration compensation. Traditional numerical compensation methods include principal component analysis, polynomial fitting, alternating direction method, and multi-parameter optimization method. Almost all of these methods require reprocessing after phase unwrapping, which not only takes a long time to calculate but may also introduce unwrapping errors, resulting in cumbersome and low-accuracy multi-directional sub-aperture spectrum synthesis process, and high requirements for the overlapping area of each sub-aperture spectrum. Although the current method of using aberration compensation using a neural network model can quickly compensate for continuous phase aberrations, it still has problems such as cumbersome aperture synthesis, low spectrum splicing accuracy, and subsequent residual aberration compensation. In addition, the data set is made using real sample data and manually set labels, and the data volume is small, which is not conducive to the generalization of the model. Conventional data augmentation methods such as rotation, shearing, and scaling will destroy the mapping relationship between the actual phase aberration and the Zernike polynomial coefficients, causing the model to learn incorrect features. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a method for compensating for phase aberrations of optical synthetic aperture digital holographic microscopy based on deep learning. The method of the present invention can quickly complete the compensation for phase aberrations of all directions and baseline unification before aperture spectrum synthesis, realize reliable synthetic aperture and phase reconstruction, and effectively improve the spatial resolution and phase image quality of DHM. The present invention relaxes the strict correction requirements for optical system aberrations, and does not require subsequent residual aberration compensation operations. It has the advantages of fast calculation speed, high compensation accuracy and strong robustness, and does not require any manual intervention, input of initial parameters, and restriction of sample types.

[0005] The technical solution of the optical synthetic aperture digital holographic microscopy package phase aberration compensation method of the present invention is as follows:

[0006] S1. Perform diversified processing on the preset samples to obtain simulation samples, and randomly generate Zernike polynomial coefficients. Perform comprehensive processing on the simulation samples and Zernike polynomial coefficients to obtain the simulation wrapped phase map and the corresponding new Zernike polynomial coefficients respectively. Take the simulation wrapped phase map as input and the new Zernike polynomial coefficients corresponding to the simulation wrapped phase map as learning labels, build and train a deep learning network model, and obtain a trained deep learning network model.

[0007] S2. Perform multi-directional illumination processing and phase imaging processing on the sample to be tested in turn to obtain holograms of the sample in multiple directions. Perform Fourier transform processing, sub-aperture spectrum extraction processing, inverse Fourier transform processing and numerical reconstruction processing on the hologram in each direction in turn to obtain a sub-aperture wrapped phase map in the corresponding direction.

[0008] S3. All sub-aperture wrapped phase images are input into the trained deep learning network model for processing, and the processed results are sequentially subjected to phase aberration compensation, baseline unification, weighting, inverse Fourier transform, phase unwrapping and bit-to-wavelength conversion to obtain the super-resolution sample profile of the sample to be tested.

[0009] The obtained super-resolution sample profile is used for real-time monitoring of biological cells and three-dimensional profile measurement of micro-nano devices.

[0010] The step S1 is specifically as follows:

[0011] S11. For several preset samples, elliptic function, Gaussian function and random matrix amplification method are used to obtain elliptical feature samples, smooth transition feature samples and random feature samples respectively, and the elliptical feature samples, smooth transition feature samples and random feature samples are aggregated to obtain diversified samples, and the diversified samples are used as simulation samples.

[0012] S12. Obtain the phase distribution of the simulation samples based on the simulation samples.

[0013] The simulation samples can be obtained by methods such as the method of drawing single-height samples, the method of randomly superimposing Gaussian functions, the method of randomly arranging ellipsoidal functions, and random matrix amplification to obtain the phase distribution of the simulation samples.

[0014] S13. Preset the range of Zernike polynomial coefficients and randomly generate several real numbers as Zernike polynomial coefficients.

[0015] S14. Use the Zernike polynomial fitting method to fit the obtained Zernike polynomial coefficients into a continuous three-dimensional surface, and use the obtained three-dimensional surface as the phase difference of the simulation samples.

[0016] S15. Superimpose the phase distribution of the simulation samples and the phase difference of the simulation samples to obtain the phase difference distribution of the simulation samples.

[0017] S16. Process the obtained phase difference distribution of the simulation samples and the corresponding Zernike polynomial coefficients using the data interpolation enhancement method to obtain a new phase difference distribution of the simulation samples and corresponding new Zernike polynomial coefficients.

[0018] S17. Convert the new phase difference distribution of the simulation samples into complex exponential form and extract the phase from it to generate a simulation wrapped phase map.

[0019] S18. Construct a deep learning network model, use the simulation wrapped phase map as the input, use the Zernike polynomial coefficients corresponding to the simulation wrapped phase map as the learning labels, and train the deep learning network model to obtain a trained deep learning network model.

[0020] The preset samples in step S11 are transmission-type samples or reflection-type samples.

[0021] The transmission-type samples include microfluidic chips, polystyrene microspheres, biological cells, biological tissue structures, and microlens arrays. The reflection-type samples include MEMS chips, micro-nano devices, integrated circuit chips, and silicon wafers.

[0022] The data interpolation enhancement method in step S16 is set according to the following formula:

[0023]

[0024] A = αA1+(1 - α)A2

[0025] Where, represents the new phase difference distribution of the simulation samples, and respectively represent two phase aberration distributions arbitrarily selected from the phase difference distribution of the obtained simulation samples, A represents the obtained new Zernike polynomial coefficients, and A1 and A2 respectively represent and the corresponding Zernike polynomial coefficients, and α represents a preset proportional coefficient.

[0026] The deep learning network model in the step S18 adopts a ConvNeXt network model.

[0027] The step S2 is specifically as follows:

[0028] S21. Use a synthetic aperture digital holographic microscopy system to perform multi-directional illumination processing and phase imaging processing on the sample to be measured in sequence, and obtain holograms of multiple directions of the sample to be measured.

[0029] S22. Perform Fourier transform processing on the hologram of each direction to convert the hologram from the spatial domain to the frequency domain.

[0030] S23. Perform sub-aperture spectrum extraction processing on the hologram of each direction in the frequency domain to obtain the sub-aperture spectrum corresponding to the hologram of each direction.

[0031] S24. Perform inverse Fourier transform processing on each obtained sub-aperture spectrum to convert the sub-aperture spectrum from the frequency domain back to the spatial domain, and obtain the complex amplitude distribution corresponding to the sub-aperture spectrum.

[0032] S25. Perform numerical reconstruction processing on the corresponding sub-aperture according to each complex amplitude distribution to obtain the corresponding sub-aperture complex amplitude.

[0033] S26. Extract phase information from the sub-aperture complex amplitude of each direction and generate a corresponding sub-aperture wrapped phase map.

[0034] The numerical reconstruction processing in the step S25 is sequentially performed spectrum extraction processing and reproduction processing. The spectrum extraction processing adopts the phase method or the spectrum filtering method. The reproduction processing adopts inverse Fourier transform, Fresnel diffraction method, angular spectrum method or compressive sensing.

[0035] The step S3 is specifically as follows:

[0036] S31. Input all the sub-aperture wrapped phase maps into the trained deep learning network model for processing, and predict to obtain the Zernike polynomial coefficients of several samples to be measured.

[0037] S32. Respectively fit the corresponding phase aberration according to each predicted Zernike polynomial coefficient.

[0038] S33. Obtain the conjugate complex function of the corresponding phase aberration for each phase aberration respectively.

[0039] S34. Multiply the conjugate complex function of each phase aberration by the sub-aperture complex amplitude obtained in the corresponding step S25 to obtain several complex amplitudes after aberration compensation.

[0040] S35. Perform baseline unification processing on each complex amplitude after aberration compensation respectively to obtain the corresponding sub-aperture spectrum.

[0041] S36. Weight all the obtained sub-aperture spectra to obtain the synthetic aperture spectrum.

[0042] S37. Perform inverse Fourier transform processing on the synthetic aperture spectrum to obtain the synthetic aperture complex amplitude.

[0043] S38. Perform phase unwrapping processing on the synthetic aperture complex amplitude to obtain the super-resolution unwrapped phase map, realizing the elimination of the periodic wrapping of the phase for subsequent three-dimensional contour calculation.

[0044] S39. According to the super-resolution unwrapped phase map, perform bit-wavelength conversion processing to obtain the super-resolution sample contour of the sample to be measured.

[0045] The baseline unification processing in the step S35 is specifically as follows:

[0046] D1. Extract the phase information from the compensated sub-aperture spectrum, and use the phase unwrapping method to eliminate the phase jump to obtain continuous phase information.

[0047] D2. Take the median of the continuous phase information as the translational aberration, obtain the conjugate function of the translational aberration, and multiply the conjugate function of the translational aberration by the sub-aperture spectrum to obtain the corresponding sub-aperture spectrum.

[0048] The corresponding phase aberration is obtained by using the Zernike polynomial fitting method in the step S32.

[0049] The phase unwrapping processing in the step S38 adopts the iterative least squares phase unwrapping algorithm based on the reliability mask.

[0050] The technical solution of the present invention has the following beneficial technical effects:

[0051] 1. Isotropically enhanced resolution. The present invention can break through the mutual restriction between the resolution and the field of view in the traditional digital holographic microscopy system through multi-directional oblique illumination and numerical synthetic aperture. Move the sample high-frequency information lost due to the inability to exceed the system cut-off frequency during normal incidence to the system frequency band-pass through the frequency shift method, and synthesize the multi-directional sub-aperture spectra to achieve isotropic enhancement of the resolution.

[0052] 2. Reliable phase data. The present invention directly establishes the mapping relationship between the simulated wrapped phase map and the Zernike polynomial coefficients by training a deep learning network model, pre-compensates the phase aberration before synthetic aperture and phase unwrapping, effectively improving the reliability of the wrapped phase data, the splicing accuracy of the sub-aperture spectra, and the quality of the super-resolution image.

[0053] 3. Simple dataset production. Compared with other methods using deep learning to compensate for phase aberration, the present invention only uses a computer to generate a large number of diverse samples, random Zernike polynomial coefficients and the corresponding wrapped phase maps to train the deep learning network. When training the network, there is no need to build a complex tilted illumination digital holographic optical path to record multiple sample holograms in different directions, numerically reconstruct the wrapped phase maps, and perform a complex process of solving the Zernike polynomial coefficients to produce the dataset, greatly simplifying the difficulty of dataset production.

[0054] 4. Ultra-fast compensation speed. Compared with traditional phase aberration compensation methods, the present invention greatly simplifies the multi-directional sub-aperture spectrum synthesis steps, relaxes the strict requirements for correcting the aberration of the optical system, and does not require subsequent residual aberration compensation operations. For a large number of aberration compensation tasks of synthetic aperture spectra, the present invention has the advantages of fast operation speed, high compensation accuracy, and strong robustness, without any manual intervention, input of initial parameters, and sample type restrictions, improving the efficiency of phase aberration compensation and aperture synthesis.

[0055] 5. The present invention can not only be used in synthetic aperture super-resolution digital holographic systems, but also be extended to conventional digital holographic systems, multi-wavelength digital holographic systems, multi-field stitching digital holographic systems, and structured illumination digital holographic systems to simultaneously process multiple aberration compensation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flowchart of the method of the present invention;

[0057] Figure 2 is an experimental result diagram of an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The following describes the present invention in more detail with reference to the drawings and embodiments, but the present invention is not limited thereto. For those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and retouches can be made, and these improvements and retouches are also regarded as within the protection scope of the present invention. The content not described in detail in this specification belongs to the prior art well-known to those of ordinary skill in the art.

[0059] Taking the frog cleavage stage cells as an example, the specific steps of the embodiment of the present invention are as follows:

[0060] As Figure 1As shown, in S1, a preset frog cleavage stage cell sample is diversified to obtain a simulation sample, and Zernike polynomial coefficients are randomly generated. The simulation sample and the Zernike polynomial coefficients are comprehensively processed to obtain a simulated wrapped phase diagram and corresponding new Zernike polynomial coefficients respectively. The simulated wrapped phase diagram is used as the input, and the new Zernike polynomial coefficients corresponding to the simulated wrapped phase diagram are used as the learning label to construct and train a deep learning network model to obtain a trained deep learning network model.

[0061] S11. For a preset number of frog cleavage stage cell samples, an elliptical function, a Gaussian function, and a random matrix amplification method are respectively used to obtain an elliptical feature sample, a smooth transition feature sample, and a random feature sample. The elliptical feature sample, the smooth transition feature sample, and the random feature sample are summarized to obtain a diversified sample, and the diversified sample is used as the simulation sample.

[0062] The preset samples in step S11 are transmission type samples or reflection type samples. The transmission type samples include a microfluidic chip, polystyrene microspheres, biological cells, life tissue structures, and a microlens array. The reflection type samples include a MEMS chip, micro-nano devices, an integrated circuit chip, and a silicon wafer.

[0063] Specifically:

[0064] First, a two-dimensional plane with a size of M×M pixels is defined. This plane will serve as the basis for generating and displaying samples.

[0065] Based on the two-dimensional plane, an elliptical function is used to generate a sample with elliptical features from the preset sample data.

[0066] Based on the two-dimensional plane, a Gaussian function is used to generate a sample with smooth transition features from the preset sample data.

[0067] Based on the two-dimensional plane, a random matrix amplification method is used to generate a sample with random features from the preset sample data.

[0068] The sample with elliptical features, the sample with smooth transition, and the sample with random features are summarized to obtain a diversified sample.

[0069] S12. According to the simulation sample, the phase distribution of the simulation sample is obtained. The simulation sample can obtain the phase distribution of the simulation sample by methods such as drawing a single height sample method, a random superposition Gaussian function method, a random arrangement ellipsoidal function method, and a random matrix amplification.

[0070] S13. According to the preset range of Zernike polynomial coefficients for the simulation sample, several real numbers are randomly generated as Zernike polynomial coefficients; A′ = [a1 a2…a n ​T , a1, a2, …, a n respectively represent the Zernike polynomial coefficients of the first n terms, and T represents the matrix transpose.

[0071] S14. Use the Zernike polynomial fitting method for the obtained Zernike polynomial coefficients to fit several continuous three-dimensional surfaces, and use each obtained three-dimensional surface as the phase difference of each simulation sample respectively.

[0072] S15. Superimpose the phase distribution of the simulation sample and the corresponding phase difference of the simulation sample to obtain the phase difference distribution of the simulation sample. The phase difference distribution contains the information of the original phase distribution of the simulation sample and the new phase distribution after superimposing the phase aberration.

[0073] S16. Process the obtained phase difference distribution of the simulation sample and the corresponding Zernike polynomial coefficients by using the data interpolation enhancement method to obtain the new phase difference distribution of the simulation sample and the corresponding new Zernike polynomial coefficients.

[0074] The data interpolation enhancement method in step S16 is set according to the following formula:

[0075]

[0076] A = αA1 + (1 - α)A2

[0077] where represents the new phase difference distribution of the simulation sample, and respectively represent two phase aberration distributions arbitrarily selected from the obtained phase difference distribution of the simulation sample, A represents the obtained new Zernike polynomial coefficients, and A1 and A2 respectively represent and the corresponding Zernike polynomial coefficients, and α represents a preset proportional coefficient.

[0078] S17. Convert the new phase difference distribution of the simulation sample into the complex exponential form and extract the phase from it to generate the simulation wrapped phase diagram.

[0079] In specific implementation:

[0080] Assume that the new phase difference distribution of the simulation sample is φ(x, y), where (x, y) represents the spatial coordinates. It can be converted into the complex exponential form U(x, y):

[0081] U(x, y) = e jφ(x,y)

[0082] where j is the imaginary unit, satisfying j 2 = -1.

[0083] This conversion is based on Euler's formula for complex numbers \(e\) jθ \(= \cos\theta + j\sin\theta\), which encodes the phase information in the form of complex numbers.

[0084] The phase value is restricted to the range \((-\pi, \pi]\) because the phase is a periodic function with a period of \(2\pi\). When taking the phase of the complex exponential form \(U(x, y)\), the phase value is automatically truncated to the interval \([-\pi, \pi]\) to obtain the wrapped phase \(\varphi'(x, y)\):

[0085] \(\varphi'(x, y)=\text{angle}[U(x, y)]

[0086] where \(\text{angle}[\cdot]\) represents taking the phase angle of a complex number.

[0087] The wrapped phase \(\varphi'(x, y)\) is the wrapped phase value corresponding to each spatial position \((x, y)\). Arranging these wrapped phase values according to the spatial coordinates generates the simulated wrapped phase diagram.

[0088] S18. Construct a deep learning network model, use the simulated wrapped phase diagram as the input, and use the Zernike polynomial coefficients corresponding to the simulated wrapped phase diagram as the learning labels to train the deep learning network model to obtain a trained deep learning network model.

[0089] The deep learning network model in step S18 uses the ConvNeXt network model.

[0090] S2. Perform multi-directional illumination processing and phase imaging processing on the sample to be measured in sequence to obtain the oblique illumination holograms of multiple directions of the sample to be measured. Perform Fourier transform processing, sub-aperture spectrum extraction processing, inverse Fourier transform processing, and numerical reconstruction processing on each oblique illumination hologram in sequence to obtain the sub-aperture wrapped phase diagram corresponding to the corresponding direction.

[0091] S21. Use a synthetic aperture digital holographic microscopy system to perform multi-directional illumination processing and phase imaging processing on the sample to be measured in sequence to obtain the oblique illumination holograms of multiple directions of the sample to be measured.

[0092] S22. Perform Fourier transform processing on each oblique illumination hologram to convert the oblique illumination hologram from the spatial domain to the frequency domain.

[0093] S23. Perform sub-aperture spectrum extraction processing on each oblique illumination hologram in the frequency domain to obtain the sub-aperture spectrum corresponding to each oblique illumination hologram.

[0094] S24. Perform an inverse Fourier transform on each obtained sub-aperture spectrum to convert the sub-aperture spectrum from the frequency domain back to the spatial domain, and obtain the complex amplitude distribution corresponding to the sub-aperture spectrum.

[0095] S25. Perform numerical reconstruction processing on the corresponding sub-aperture according to each complex amplitude distribution to obtain the corresponding sub-aperture complex amplitude, which contains amplitude and phase information.

[0096] The numerical reconstruction processing is sequentially performed spectral extraction processing and reproduction processing. The spectral extraction processing adopts the phase method or the spectral filtering method. The reproduction processing adopts the inverse Fourier transform, the Fresnel diffraction method, the angular spectrum method or compressive sensing.

[0097] The sub-aperture complex amplitude is set according to the following formula:

[0098]

[0099] where, O R1 (x,y) and O R2 (x,y) respectively represent the sub-aperture complex amplitudes of two oblique illumination holograms in the horizontal direction, O R3 (x,y) and O R4 (x,y) respectively represent the sub-aperture complex amplitudes of two oblique illumination holograms in the vertical direction, P1′(x,y), P2′(x,y), P3′(x,y) and P4′(x,y) respectively represent four phase aberration factors after compensating for the partial tilt aberration with the spectrum centered, O represents the object light complex amplitude, represents the convolution operator, M represents the transverse magnification of the digital holographic microscopy system, h(x,y) represents the optical transfer function of the imaging system, x and y represent spatial coordinates, f1 and f2 respectively represent two spatial frequencies introduced by the oblique illumination in the horizontal direction, f3 and f4 respectively represent two spatial frequencies introduced by the oblique illumination in the vertical direction, i represents the imaginary unit, π represents a constant, and exp() represents the exponential function.

[0100] S26. Extract the phase information from the sub-aperture complex amplitudes in each direction and generate the corresponding sub-aperture wrapped phase map.

[0101] The synthetic aperture digital holographic microscopy system specifically is:

[0102] The illumination source is a He-Ne laser. The beam emitted by the laser first passes through a collimating and beam expanding system BEC to expand and collimate the incident light. A beam splitter divides the collimated beam into two beams. One beam is reflected by a mirror M1 to a digital micromirror device DMD for diffraction, and the other is reflected by a mirror M2. The illumination angle on the sample to be measured can be adjusted by the DMD. The DMD acts as a sinusoidal grating, diffracting to generate multi-level beams. The lens L and the condenser CD are combined into a modulated illumination 4f system, where the filter F can screen the beams. The microscope objective MO and the sleeve lens TL are combined to form a telecentric microscopic system. The incident beam forms an object beam after diffraction by the sample. The BS2 is used to interfere the object beam and the reference beam to form a digital oblique illumination hologram, which is collected by a CMOS camera.

[0103] S3. Input the sub-aperture wrapped phase map into the trained deep learning network model for processing, and perform phase aberration compensation processing, baseline unification processing, weighting processing, inverse Fourier transform processing, phase unwrapping processing, and wavelength conversion processing in sequence according to the processing results to obtain the super-resolution sample profile of the sample to be measured.

[0104] S31. Input all the sub-aperture wrapped phase maps into the trained deep learning network model for processing, and predict the Zernike polynomial coefficients of several samples to be measured.

[0105] S32. Fit the corresponding phase aberration according to each predicted Zernike polynomial coefficient; use the Zernike polynomial fitting method to obtain the corresponding phase aberration.

[0106] S33. Obtain the conjugate complex function of the corresponding phase aberration according to each phase aberration for phase compensation.

[0107] S34. Multiply the conjugate complex function of each phase aberration by the sub-aperture complex amplitude obtained in step S25 to obtain several complex amplitudes after aberration compensation, canceling the phase aberration in the sub-aperture.

[0108] S35. Perform baseline unification processing on each complex amplitude after aberration compensation to obtain the corresponding sub-aperture spectrum.

[0109] The baseline unification processing is specifically as follows:

[0110] D1. Extract the phase information from the compensated sub-aperture spectrum, and use the phase unwrapping method to eliminate the phase jump to obtain continuous phase information.

[0111] In specific implementation, the phase unwrapping method is processed using the unwrap function.

[0112] D2. Take the median of the continuous phase information as the translational aberration, obtain the conjugate function of the translational aberration, and multiply the conjugate function of the translational aberration by the sub-aperture spectrum to compensate for the phase deviation, thereby obtaining the corresponding sub-aperture spectrum.

[0113] S36. Perform weighted processing on all the obtained sub-aperture spectra to obtain a synthetic aperture spectrum;

[0114] The synthetic aperture spectrum is set according to the following formula:

[0115]

[0116] Where, represents the synthesized aperture spectrum, M′ represents the scaling factor or magnification factor, represents the spectrum of the original image or signal in the spatial frequency domain,, H CTF ( ) represents the composite coherent transfer function of the imaging system after synthetic aperture, H( ) represents the basic contrast transfer function of the imaging system after synthetic aperture, f1′, f2′, f3′ and f4′ all represent frequency offsets, used to adjust the position of the basic contrast transfer function in the spatial frequency domain, and u and v represent the coordinates of spatial frequency.

[0117] S37. Perform inverse Fourier transform processing on the synthetic aperture spectrum to obtain a synthetic aperture complex amplitude, realizing the conversion of frequency domain information back to spatial domain information.

[0118] S38. Perform phase unwrapping processing on the synthetic aperture complex amplitude to obtain a super-resolution unwrapped phase map, realizing the elimination of the periodic wrapping of the phase for subsequent three-dimensional contour calculation. The phase unwrapping processing uses an iterative least squares phase unwrapping algorithm based on a reliability mask.

[0119] Performing phase unwrapping processing on the synthetic aperture complex amplitude includes first obtaining the synthetic aperture wrapped phase and then unwrapping the aperture wrapped phase.

[0120] S39. According to the super-resolution unwrapped phase map, perform bit-wavelength conversion processing to obtain the super-resolution sample contour of the sample to be measured.

[0121] The bit-wavelength conversion processing uses the bit-wavelength conversion formula Where h' represents the height or thickness of the sample to be measured, that is, the height value in the super-resolution sample contour information of the sample to be measured, λ represents the wavelength of light, represents the unwrapped phase of the synthetic aperture, Δn represents the difference in refractive index between the sample to be measured and air, and π represents a constant.

[0122] The obtained super-resolution sample contour is used for real-time monitoring of biological cells and three-dimensional contour measurement of micro-nano devices.

[0123] Such asFigure 2 As shown Figure 2 (a1)-(a4) in it are sub-aperture wrapped phase diagrams obtained from the four sub-aperture spectra of frog egg cleavage stage cells in four illumination directions; Figure 2 (b1)-(b4) in it are four sub-aperture wrapped phase diagrams after deep learning phase aberration compensation; Figure 2 (c) in it is the synthetic aperture spectrum; Figure 2 (d) in it is the unwrapped phase diagram after directly performing synthetic aperture spectrum without phase aberration compensation; Figure 2 (e) in it is the unwrapped phase diagram after numerical reconstruction and phase unwrapping of figure (c).

[0124] Thus Figure 2 it can be seen that the present invention places phase aberration compensation before synthetic aperture and phase unwrapping, combines deep learning methods, directly establishes the mapping relationship between multi-directional sub-aperture wrapped phase diagrams and Zernike polynomial coefficients, pre-compensates phase aberration before synthetic aperture and phase unwrapping, effectively improves the reliability of wrapped phase data, the splicing accuracy of sub-aperture spectra, and the quality of super-resolution images. It simplifies the synthetic aperture spectrum synthesis steps in multiple directions, relaxes the strict correction requirements for optical system aberrations, and eliminates the need for subsequent residual aberration compensation operations.

[0125] Generally speaking, the method of the present invention uses a DMD to build a synthetic aperture digital holographic microscopy system, obtains a set of complementary apertures containing different spatial frequency contents under multiple oblique beam illuminations, and can achieve super-resolution quantitative phase imaging by using synthetic aperture technology. Only using the simulation data set to train the network, directly establishing the mapping relationship between multi-directional sub-aperture wrapped phase diagrams and Zernike polynomial coefficients, and quickly completing the phase aberration compensation of sub-aperture wrapped phase images in each direction before aperture synthesis, it has an ultra-fast operation speed, reliable phase data, and isotropically enhanced resolution.

Claims

1. A method for compensating wrapped phase aberration in optical synthetic aperture digital holographic microscopy based on deep learning, characterized in that, Including the following steps: S1. Diversify a preset sample to obtain a simulated sample, randomly generate Zernike polynomial coefficients, comprehensively process the simulated sample and the Zernike polynomial coefficients to obtain a simulated wrapped phase map and corresponding new Zernike polynomial coefficients respectively, use the simulated wrapped phase map as the input, and the corresponding new Zernike polynomial coefficients as the learning label, construct and train a deep learning network model to obtain a trained deep learning network model; S2. Perform multi-directional illumination processing and phase imaging processing on the sample to be measured in sequence to obtain holograms of multiple directions of the sample to be measured, and perform Fourier transform processing, sub-aperture spectrum extraction processing, inverse Fourier transform processing and numerical reconstruction processing on the hologram of each direction in sequence to obtain a sub-aperture wrapped phase map of the corresponding direction; S3. Input all sub-aperture wrapped phase maps into the trained deep learning network model for processing, and perform phase aberration compensation processing, baseline unification processing, weighting processing, inverse Fourier transform processing, phase unwrapping processing and wavelength conversion processing on the processing results in sequence to obtain the super-resolution sample profile of the sample to be measured.

2. The method for compensating the wrapped phase aberration of an optical synthetic aperture digital holographic microscope based on deep learning according to claim 1, characterized in that, The specific steps of step S1 are as follows: S11. Respectively use the elliptic function, Gaussian function and random matrix amplification method for several preset samples to obtain an elliptical feature sample, a smooth transition feature sample and a random feature sample, summarize the elliptical feature sample, the smooth transition feature sample and the random feature sample to obtain a diversified sample, and use the diversified sample as the simulated sample; S12. Obtain the phase distribution of the simulated sample according to the simulated sample; S13. Preset the range of Zernike polynomial coefficients, and randomly generate several real numbers as Zernike polynomial coefficients; S14. Use the Zernike polynomial fitting method to fit the obtained Zernike polynomial coefficients into a continuous three-dimensional surface, and use the obtained three-dimensional surface as the phase difference of the simulated sample; S15. Superimpose the phase distribution of the simulated sample and the phase difference of the simulated sample to obtain the phase difference distribution of the simulated sample; S16. Use the data interpolation enhancement method to process the obtained phase difference distribution of the simulated sample and the corresponding Zernike polynomial coefficients to obtain a new phase difference distribution of the simulated sample and corresponding new Zernike polynomial coefficients; S17. Convert the new phase difference distribution of the simulated sample into a complex exponential form and extract the phase from it to generate a simulated wrapped phase map; S18. Construct a deep learning network model, use the simulated wrapped phase map as the input, and use the Zernike polynomial coefficients corresponding to the simulated wrapped phase map as the learning label, and train the deep learning network model to obtain a trained deep learning network model.

3. A method for compensating phase aberration of a wrapped phase in an optical synthetic aperture digital holographic microscope based on deep learning according to claim 2, characterized in that: The preset sample in step S11 is a transmissive sample or a reflective sample.

4. A method for compensating wrapped phase aberration in optical synthetic aperture digital holographic microscopy based on deep learning according to claim 2, characterized in that: The data interpolation enhancement method in step S16 is set according to the following formula: φ = αφ1 + (1 - α)φ2 A = αA1 + (1 - α)A2 where φ represents the new phase difference distribution of the simulation sample, φ1 and φ2 respectively represent two arbitrarily selected phase aberration distributions from the obtained phase difference distributions of the simulation sample, A represents the obtained new Zernike polynomial coefficients, A1 and A2 respectively represent the Zernike polynomial coefficients corresponding to φ1 and φ2, and α represents a preset proportional coefficient.

5. A method for compensating wrapped phase aberration in optical synthetic aperture digital holographic microscopy based on deep learning according to claim 2, characterized in that: The deep learning network model in step S18 adopts the ConvNeXt network model.

6. A method for compensating the wrapped phase aberration of an optical synthetic aperture digital holographic microscope based on deep learning according to claim 1, characterized in that, Step S2 is specifically as follows: S21. Use the synthetic aperture digital holographic microscopy system to perform multi-directional illumination processing and phase imaging processing on the sample to be measured in sequence, and obtain holograms of multiple directions of the sample to be measured; S22. Perform Fourier transform processing on the hologram of each direction to transform the hologram from the spatial domain to the frequency domain; S23. Perform sub-aperture spectrum extraction processing on the hologram of each direction in the frequency domain to obtain the sub-aperture spectrum corresponding to the hologram of each direction; S24. Perform inverse Fourier transform processing on the obtained sub-aperture spectrum of each direction to transform the sub-aperture spectrum from the frequency domain back to the spatial domain, and obtain the complex amplitude distribution corresponding to the sub-aperture spectrum; S25. Perform numerical reconstruction processing on the corresponding sub-aperture according to each complex amplitude distribution to obtain the corresponding sub-aperture complex amplitude; S26. Extract the phase information from the sub-aperture complex amplitude of each direction and generate the corresponding sub-aperture wrapped phase map.

7. A method for compensating wrapped phase aberration in optical synthetic aperture digital holographic microscopy based on deep learning according to claim 6, characterized in that: The numerical reconstruction processing in step S25 is sequentially performed spectrum extraction processing and reconstruction processing; The spectrum extraction processing adopts the phase method or the spectrum filtering method; The reconstruction processing adopts inverse Fourier transform, Fresnel diffraction method, angular spectrum method or compressive sensing.

8. A method for compensating wrapped phase aberration in optical synthetic aperture digital holographic microscopy based on deep learning according to claim 1, characterized in that, Step S3 is specifically as follows: S31. Input all sub-aperture wrapped phase maps into the trained deep learning network model for processing, and predict the Zernike polynomial coefficients of several samples to be measured; S32. Fit the corresponding phase aberration according to each predicted Zernike polynomial coefficient respectively; S33. Obtain the conjugate complex function of the corresponding phase aberration according to each phase aberration respectively; S34. Multiply the conjugate complex function of each phase aberration by the sub-aperture complex amplitude obtained in the corresponding step S25 to obtain several complex amplitudes after aberration compensation; S35. Perform baseline unification processing on each complex amplitude after aberration compensation respectively to obtain the corresponding sub-aperture spectrum; S36. Weight all the obtained sub-aperture spectra to obtain the synthetic aperture spectrum; S37. Perform an inverse Fourier transform on the synthetic aperture spectrum to obtain the synthetic aperture complex amplitude; S38. Perform phase unwrapping on the synthetic aperture complex amplitude to obtain a super-resolved unwrapped phase map; S39. According to the super-resolved unwrapped phase map, perform a bit-wavelength conversion process to obtain the super-resolved sample contour of the sample to be measured.

9. A method for compensating the wrapped phase aberration of an optical synthetic aperture digital holographic microscope based on deep learning according to claim 8, characterized in that, The baseline unification process in step S35 is specifically as follows: D1. Extract phase information from the compensated sub-aperture spectrum, and use a phase unwrapping method to eliminate phase jumps to obtain continuous phase information; D2. Take the median of the continuous phase information as the translational aberration, obtain the conjugate function of the translational aberration, and multiply the conjugate function of the translational aberration by the sub-aperture spectrum to obtain the corresponding sub-aperture spectrum.

10. A method for compensating wrapped phase aberration in optical synthetic aperture digital holographic microscopy based on deep learning according to claim 8, characterized in that: In step S32, a Zernike polynomial fitting method is used to obtain the corresponding phase aberration; The phase unwrapping process in step S38 uses an iterative least squares phase unwrapping algorithm based on a reliability mask.

Images