A large-aberration high-robustness correction method based on fourier ptychographic microscopy

Abstract

The application provides a large-aberration high-robustness correction method based on Fourier ptychographic microscopy. The method models the propagation of light waves in a transparent thick medium layer in sections, constructs forward and reverse propagation aperture function models, and uses two propagation distance factors in the two aperture function models to calculate a comprehensive aperture aberration function. Under the condition of ring-shaped matching illumination based on an LED array, combined with a nonlinear optimization algorithm, intensity constraints are applied to the collected images, and the two propagation distance factors are jointly optimized to realize aberration correction under the condition of a thick medium layer. The method solves the problem of inaccurate sample phase recovery caused by large aberration introduced by a transparent thick medium layer, accurately compensates complex high-order aberration through a small number of parameters without other prior information, and promotes the engineering application of FPM high-throughput imaging technology in actual scenes.

Description

Technical Field

[0001] This invention belongs to the field of optical microscopy imaging technology, specifically a robust correction method for large aberrations based on Fourier layer microscopy, which can quickly correct system aberrations under thick dielectric layer conditions and obtain clear high-resolution images. Background Technology

[0002] From the forefront of scientific research to quality control in industrial production, microscopic imaging technology plays an irreplaceable role in many key fields such as medicine, biology, and materials science. However, traditional microscopic imaging systems are largely limited by classical optical theory and physical limits, making it difficult to achieve both high resolution and a large field of view, which severely restricts the development of various fields. To address this challenge, mechanical scanning and image stitching techniques, synthetic aperture technology, and spatial layering imaging technology have been invented. Although these have expanded the system's spatial basis of image processing (SBP) to some extent, they all essentially rely on high-precision physical displacement or complex interference optical paths, placing extremely high demands on hardware, resulting in high hardware costs and low imaging efficiency. In addition, aberrations during the imaging process are difficult to correct effectively. Driven by the urgent need for the invention and practical application of various new imaging technologies, Fourier layered microscopy (FPM) has emerged. This technology integrates the concepts of synthetic aperture and phase recovery. It uses plane waves from multiple angles to illuminate the sample, then images it through a low numerical aperture objective lens. The frequency information introduced by each imaging is then synthesized in the frequency domain to obtain a high-resolution image with a large field of view. The mathematical constraints of the algorithm replace the precision requirements of the hardware, and the system aberrations can also be accurately compensated through post-calculation.

[0003] In optical design, the thickness of a standard coverslip (0.17 mm) is typically factored into the aberration compensation path to ensure ideal imaging quality within the working distance. However, in high-throughput imaging, samples are usually placed in containers for observation. These containers are typically made of millimeter-thick glass or plastic, much thicker than a standard coverslip. For Fourier stacked imaging systems, the light wave field carrying object information must pass through this transparent, thick dielectric layer before entering the objective for imaging. Due to refractive index mismatch and long-distance propagation, significant aberrations are introduced, directly affecting the accuracy of phase recovery. These aberrations are further amplified, especially when using high numerical aperture objectives, limiting the performance of computational imaging methods in practical applications. Thanks to the unique algorithms of Fourier stacked microscopy (FPM), high-quality image reconstruction and system aberration correction can still be achieved through joint optimization, even under non-ideal optical conditions. The most representative algorithms are EPRY and AO-QPI. The EPRY algorithm was the first to achieve joint recovery of the sample spectrum and the system pupil function within the FPM framework, but it heavily relies on spectral overlap and redundancy, resulting in low reconstruction efficiency. AO-QPI, as a computational adaptive optics correction method, introduces Zernike polynomials to parametrically model pupil aberrations. While it can compensate for dynamic aberrations in real time, its high parameter dimensionality and limited ability to compensate for complex high-order aberrations generated under thick dielectric conditions are limitations. Summary of the Invention

[0004] The purpose of this invention is to propose a robust correction method for large aberrations based on Fourier layered microscopy, which solves the problem of inaccurate phase recovery in existing Fourier layered imaging systems when imaging samples with transparent thick dielectric layers, and achieves accurate aberration correction under thick dielectric layer conditions.

[0005] The technical solution to achieve the purpose of this invention is: a robust correction method for large aberrations based on Fourier layered microscopy, comprising:

[0006] Step 1: Use an LED array to provide ring-matched illumination to the sample, and use a Fourier layered microscopy system to acquire bright field intensity maps under illumination at different angles.

[0007] Step 2: Obtain the initial complex amplitude distribution of the sample through the deconvolution algorithm, and then perform Fourier transform on the initial complex amplitude distribution to obtain the initial spectrum of the sample's complex amplitude;

[0008] Step 3: Construct a segmented propagation model within a transparent thick dielectric layer, and construct two aperture function models corresponding to the forward and backward propagation processes respectively. Use the two propagation distance factors in the two aperture function models to calculate a comprehensive aperture aberration function model, i.e., the pupil function.

[0009] Step 4: Apply intensity constraints to the frequency domain sub-aperture region corresponding to each brightfield intensity map in sequence, and alternately update the sample spectrum and pupil function; wherein, the update of the pupil function is achieved by updating the propagation distance factor;

[0010] Step 5: Repeat step 4, and determine the convergence condition at the end of each iteration until the error converges. At the same time, output the high-resolution sample reconstruction map and system optical aberration.

[0011] Compared with the prior art, the present invention has the following significant advantages: (1) Compared with the existing aberration correction technology, the present invention has a low parameter dimension that needs to be optimized, low data redundancy requirements, and high correction efficiency. (2) The aberration correction process caused by the transparent thick medium layer is relatively independent, and the original multi-dimensional aberration correction problem is divided into multiple steps, which improves the correction speed. Attached Figure Description

[0012] Figure 1 This is a flowchart of a robust correction method for large aberrations based on Fourier layered microscopy.

[0013] Figure 2 This is a schematic diagram of the propagation model of the present invention.

[0014] Figure 3 This is a simulation imaging result image, in which, Figure 3 (a) in the image is the phase map of the input sample. Figure 3 (b) in the image is the intensity map of the collected sample. Figure 3 (c) in the image is the recovered sample phase map. Figure 3 In the figure, (d) represents the input aberration. Figure 3 (e) in the figure represents the recovery of aberrations. Detailed Implementation

[0015] The present invention will now be described in further detail with reference to the accompanying drawings.

[0016] like Figure 1 As shown, a robust correction method for large aberrations based on Fourier layered microscopy includes the following steps:

[0017] Step 1: Use an LED array for ring-matched illumination of the sample, and acquire data using a Fourier transform in-situ microscopy system. Brightfield luminous intensity diagrams under illumination from different angles ;

[0018] The specific implementation process is as follows: Illumination beads with a numerical aperture equal to the objective lens numerical aperture are sequentially lit, and a bright-field light intensity map is captured using a camera. ,in Represents spatial domain coordinates;

[0019] Step 2: Obtain the initial complex amplitude distribution of the sample using the deconvolution algorithm. Then, a Fourier transform is performed to obtain the initial spectrum of the complex amplitude of the sample. ;

[0020] The specific implementation process is as follows: only the LED bead located in the center of the LED array is lit, and intensity images are captured under normal incident illumination conditions. ;

[0021] Perform a Fourier transform on each original brightfield intensity image acquired in step 1 to obtain its corresponding intensity spectrum distribution. ,in Represents frequency domain coordinates;

[0022] Calculate the phase transfer function corresponding to each illumination angle using the known parameters of the imaging system. ;

[0023] The initial estimate of the sample phase in the frequency domain is calculated using the Tikhonov regularized deconvolution formula, specifically expressed as follows:

[0024]

[0025] In the formula Indicates conjugate. For regularization parameters;

[0026] The formula for calculating the initial complex amplitude distribution of the sample is:

[0027]

[0028] In the formula, for Fourier transform;

[0029] The initial spectrum of the sample is:

[0030]

[0031] Step 3: Construct a segmented propagation model within the transparent thick dielectric layer. Since the location of the transparent thick dielectric layer, i.e., the protective medium of the sample, does not affect the algorithm's flow, we consider the process of the light wave field carrying sample information passing through the transparent thick dielectric layer after passing through the sample or being reflected from the sample surface. For example... Figure 2 In the figure, the blue arrows indicate the direction of light propagation, and the blue areas represent the transparent thick dielectric layer. The propagation of the light wave field carrying sample information within the thick dielectric layer is a forward propagation process, which can be modeled using an aperture function. To describe this, the dashed line represents the reverse propagation process of the light wave field carrying sample information, which propagates to the object-side focal plane of the objective lens to form a virtual image. This is described using an aperture function model. To describe it, a combined aperture aberration function model is calculated using two propagation distance factors from two aperture function models. That is, the pupil function of the system;

[0032] The specific implementation process is as follows: Constructing a spatial frequency coordinate grid. Calculate its normalized frequency coordinates The specific expression for normalization is:

[0033]

[0034] In the formula, The numerical aperture of the objective lens. To determine the wavelength of the illumination light, the frequency outside the aperture is set to 0, i.e., when... season ;

[0035] During the propagation of light, the phase changes of different frequency components can be determined by the phase propagation factor. To describe it, its expression is:

[0036]

[0037] In the formula, The refractive index of the corresponding propagation medium;

[0038] Construct aperture function models corresponding to forward and backward propagation, with the specific expressions as follows:

[0039]

[0040]

[0041] In the formula, The imaginary unit, and These are the forward propagation distance factor and the backward propagation distance factor, respectively, which correspond to the respective distances in a real system, such as... Figure 2 As shown, For a circular low-pass filter, its expression is:

[0042]

[0043] Pupil function on the objective lens focal plane This can be derived by quotient from the two-aperture function model, and the specific expression is:

[0044]

[0045] The amplitude component of the final pupil function satisfies the physical constraint of being fully passable inside the aperture and cut off outside the aperture, while the optical aberration, as the phase component, is characterized by a formula with two propagation distance factors.

[0046] Step 4: Apply intensity constraints to the sub-spectral regions corresponding to each input image in turn, and alternately update the estimated sample spectrum. With pupil function The pupil function is updated by updating the propagation distance factor. accomplish.

[0047] The specific implementation process is as follows: Update the image according to the acquisition sequence of the original bright-field light intensity map, and calculate the frequency domain offset caused by the tilted illumination based on the spatial position of the corresponding LED beads. Find the sub-spectrum corresponding to each brightfield intensity map. The specific expression is:

[0048] ;

[0049] Calculate the corresponding amplitude update factor:

[0050]

[0051] In the formula, The intensity distribution of the actual acquired image. To update the intensity distribution of the sub-aperture before updating, the complex amplitude distribution of the sub-aperture is used. The square of the modulus is calculated;

[0052] Sub-aperture complex amplitude distribution Multiplying by the amplitude update factor to apply an intensity constraint, and then performing a Fourier transform, yields the updated sub-spectrum, specifically expressed as:

[0053]

[0054] The complete spectrum is estimated using the sub-spectrum updated at each step. To update, all sub-spectrums are substituted into the next iteration, as shown in the following formula:

[0055]

[0056] In the formula, To update the step size;

[0057] Synchronous update of pupil function under thick dielectric layer conditions The sample spectrum is specifically achieved by updating two propagation distance factors. These factors are initialized to 0 and updated using gradient descent. For each propagation distance factor... The update formula is given by the following equation:

[0058]

[0059]

[0060] In the formula, Indicates the update step size. This represents the estimated sub-spectrum of the image. The specific form is as follows:

[0061]

[0062] Step 5: Repeat Step 4, checking the convergence condition at the end of each iteration, until the [number]th iteration... After multiple iterations, the error converges, and a high-resolution sample reconstruction image is output. Pupil function characterizing system optical aberrations ;

[0063] Specifically, when the root mean square error of the system optical aberration tends to level off with the increase of the iteration rounds, that is, when the root mean square error of the system optical aberration no longer changes significantly between adjacent iteration rounds, the error is judged to converge and the iteration is stopped. In practical applications, the number of iteration rounds can also be preset. When the number of iteration rounds reaches the preset value and the system optical aberration no longer changes significantly, the error is judged to converge. After stopping the iteration, a high-resolution sample reconstruction image and system optical aberration are output, where the system optical aberration is characterized by the phase part of the pupil function obtained after the iteration convergence.

[0064] The specific implementation process is as follows: In simulation experiments, the trend of the root mean square error (RMSE) of aberrations is used to determine convergence. After multiple iterations, when the RMSE value tends to level off, the algorithm is considered to have converged, and iteration can be stopped. In practical applications, a sufficient number of iterations is set based on experience. When the aberration value no longer changes significantly, convergence is considered, and the results can be output.

[0065] This invention corrects complex high-order aberrations introduced by the transparent thick dielectric layer by segmenting the propagation of light in a thick, transparent dielectric layer and constructing two aperture function models. Under annular matched illumination conditions, the two propagation distance factors are jointly optimized. This method requires correction of parameters with low dimensionality and decouples the aberrations of the transparent thick dielectric layer from other aberrations, allowing for independent correction. This significantly improves the speed and efficiency of the entire aberration correction process, expands the practical application conditions of FPM technology, and provides impetus for the development of FPM high-throughput imaging technology towards greater efficiency and accuracy.

[0066] The simulation imaging results in this invention are as follows: Figure 3As shown, (a) is the input sample phase map, (b) is the sample intensity map acquired, (c) is the recovered sample phase map, (d) is the input aberration, and (e) is the recovered aberration. A comparison of (a) and (c) shows that (c) has clearer details, better contrast, and no obvious artifacts, indicating that this method is relatively accurate in correcting large aberrations introduced by thick dielectric layers.

Claims

1. A robust correction method for large aberrations based on Fourier layered microscopy, characterized in that, include: Step 1: Use an LED array to provide ring-matched illumination to the sample, and use a Fourier layered microscopy system to acquire bright field intensity maps under illumination at different angles. Step 2: Obtain the initial complex amplitude distribution of the sample through the deconvolution algorithm, and then perform Fourier transform on the initial complex amplitude distribution to obtain the initial spectrum of the sample's complex amplitude; Step 3: Construct a segmented propagation model within a transparent thick dielectric layer, and construct two aperture function models corresponding to the forward and backward propagation processes respectively. Use the two propagation distance factors in the two aperture function models to calculate a comprehensive aperture aberration function model, i.e., the pupil function. Step 4: Apply intensity constraints to the frequency domain sub-aperture region corresponding to each brightfield intensity map in sequence, and alternately update the sample spectrum and pupil function; wherein, the update of the pupil function is achieved by updating the propagation distance factor; Step 5: Repeat step 4, and determine the convergence condition at the end of each iteration until the error converges. At the same time, output the high-resolution sample reconstruction map and system optical aberration.

2. The robust correction method for large aberrations based on Fourier layered microscopy according to claim 1, characterized in that, The ring-shaped matching illumination mentioned in step 1 means that the illumination numerical aperture of the lit LED beads is equal to the numerical aperture of the objective lens, and they are distributed in a ring shape.

3. The robust correction method for large aberrations based on Fourier layered microscopy according to claim 1, characterized in that, The specific method for obtaining the initial spectrum of the sample in step 2 is as follows: Only the LED bead located in the center of the LED array is lit, and intensity images are captured under normal incident illumination conditions. ; Perform a Fourier transform on each original brightfield intensity image acquired in step 1 to obtain its corresponding intensity spectrum distribution. In the formula, This is the sequence number of the bright field intensity diagram. , This represents the total number of brightfield intensity images acquired in step 1. Spatial domain coordinates, These are frequency domain coordinates; Calculate the phase transfer function corresponding to each illumination angle using the known parameters of the imaging system. ; The initial estimate of the sample phase in the frequency domain is calculated using the Tikhonov regularized deconvolution formula, specifically expressed as follows: ； In the formula Indicates conjugate. For regularization parameters, This represents the modulo operation. This is an initial estimate of the sample phase in the frequency domain; The formula for calculating the initial complex amplitude distribution of the sample is: ； In the formula, for Fourier transform, The imaginary unit, Represented by natural constant An exponential function with base 0. Intensity images acquired under normal incident illumination conditions; The initial spectrum of the sample is: ； In the formula, This represents the Fourier transform operator.

4. The robust correction method for large aberrations based on Fourier layered microscopy according to claim 1, characterized in that, In step 3, two aperture function models corresponding to the forward and backward propagation processes are constructed. and A combined aperture aberration function model is calculated using two propagation distance factors from two aperture function models. The specific method is as follows: Constructing a spatial frequency coordinate grid Calculate the normalized frequency coordinates The specific expression for normalization is: ； In the formula, The numerical aperture of the objective lens. To determine the wavelength of the illumination light, the frequency outside the aperture is set to 0, i.e., when... season ; During the propagation of light, the phase changes of different frequency components are determined by the phase propagation factor. To describe, the expression is: ； In the formula, This represents the propagation distance of the corresponding propagation segment. The refractive index of the corresponding propagation medium; Construct aperture function models corresponding to forward and backward propagation, with the specific expressions as follows: ； ； In the formula, The imaginary unit, and These are the forward propagation distance factor and the backward propagation distance factor, respectively. For a circular low-pass filter, the expression is: ； In the formula, The refractive index of the medium through which the material propagates forward, i.e., the refractive index of the transparent thick medium layer; The refractive index of the medium through which the reverse propagation occurs; These correspond to the light wave propagation distances of the forward and reverse propagation segments, respectively. This represents a circular domain function, which takes the value 1 when the independent variable is not greater than 1, and 0 otherwise. Pupil function on the focal plane of the objective lens The specific expression is obtained by quotienting using the two-aperture function model: ； The amplitude component of the final pupil function satisfies the physical constraint of being fully passable inside the aperture and cut off outside the aperture, while the optical aberration, as the phase component, is characterized by a formula with two propagation distance factors.

5. The robust correction method for large aberrations based on Fourier layered microscopy according to claim 1, characterized in that, The specific process of updating the sample spectrum in step 4 is as follows: Update sequentially according to the order in which the bright field intensity maps were collected in step 1; Based on the spatial position of the LED beads corresponding to the current bright field intensity map, calculate the frequency domain offset caused by tilted illumination, and determine the frequency domain sub-aperture region corresponding to the current bright field intensity map. For each sub-aperture update, calculate the corresponding amplitude update factor: ； In the formula, The intensity distribution of the actual acquired image. To update the intensity distribution of the previous sub-aperture; Sub-aperture complex amplitude distribution Multiplying by the amplitude update factor to apply an intensity constraint, and then performing a Fourier transform, yields the updated sub-spectrum, specifically expressed as: ； In the formula, This represents the Fourier transform operator; Each updated sub-spectrum is used to update the spectral components within the corresponding frequency domain sub-aperture region in the complete sample spectrum estimate. One iteration is completed after all sub-spectrums have been updated. The iteration formula is as follows: ； In the formula, The update step size for updating the sample spectrum; To describe the pupil function, for conjugate, for The maximum value in the frequency domain.

6. The robust correction method for large aberrations based on Fourier layered microscopy according to claim 5, characterized in that, In step 4, the pupil function is updated synchronously with the sample spectrum. The update of the pupil function is achieved by updating two propagation distance factors. The specific process is as follows: Both propagation distance factors are initialized to 0 and updated using gradient descent. For each propagation distance factor, the update formula for each iteration is given by the following equation: ； ； In the formula, Indicates the update step size. This represents the estimated sub-spectrum of the image. The specific form is as follows: ； In the formula, The first The values ​​of the forward and backward propagation distance factors before each iteration update. The first The value after round of iteration updates; The update step size for updating the propagation distance factor; The amplitude update factor as described in claim 5; The sub-spectrum as described in claim 5; for The spatial domain sub-aperture complex amplitude distribution obtained by inverse Fourier transform express The conjugate; Indicates taking the imaginary part; Represents the coordinates of each point in the spatial domain Summation; This represents the Fourier transform operator. This represents the inverse Fourier transform operator; These are the forward propagation distance factor and the backward propagation distance factor, respectively, which correspond to the equivalent propagation distances of the forward propagation segment and the backward propagation segment; The numerical aperture of the objective lens; For normalized frequency coordinates, when season ; middle Calculate the refractive index of the propagation medium. Update volume Pick ,calculate Update volume Pick ; The refractive index of the medium through which the material propagates forward, i.e., the transparent, thick dielectric layer. It is the refractive index of the medium through which the reverse propagation occurs.

7. The robust correction method for large aberrations based on Fourier layered microscopy according to claim 1, characterized in that, In step 5, the specific method for determining the convergence condition and outputting the result is as follows: Repeat step 4 for multiple iterations. At the end of each iteration, calculate the root mean square error of the system optical aberrations obtained in the current iteration. When the root mean square error of the system optical aberration tends to level off with the increase of the iteration rounds, that is, when the root mean square error of the system optical aberration no longer changes significantly between adjacent iteration rounds, the error is judged to converge and the iteration stops. In practical applications, the number of iteration rounds can also be preset. When the number of iteration rounds reaches the preset value and the system optical aberration no longer changes significantly, the error is judged to converge. After stopping the iteration, a high-resolution sample reconstruction image and system optical aberration are output, where the system optical aberration is characterized by the phase part of the pupil function obtained after the iteration convergence.

8. The robust correction method for large aberrations based on Fourier layered microscopy according to claim 4, characterized in that, The forward propagation is the process of light waves carrying sample information propagating inside a transparent thick dielectric layer, and the backward propagation is the process of light waves carrying sample information propagating to the object-side focal plane of the objective lens to form a virtual image. The transparent thick medium layer is a transparent glass or plastic layer with a thickness on the order of millimeters, used to support the sample or as a protective medium for the sample. The amplitude portion of the pupil function satisfies the physical constraint of being fully open inside the aperture and closed outside the aperture. The optical aberration of the system is characterized by the phase portion containing the two propagation distance factors and is corrected by the joint optimization of the two propagation distance factors.

Images