Large-depth-of-field real-time drift correction method and system based on programmable point spread function

Abstract

The invention discloses a large-depth-of-field real-time drift correction method and system based on a programmable point spread function. According to the invention, wide-range and high-precision drift correction is realized by dynamically switching different PSF modes, and the problem that the axial correction range and precision cannot be obtained at the same time in the prior art is solved. The system comprises a marker detection module, a PSF dynamic switching module and a closed-loop control module, and an optimal PSF mode is automatically selected according to the drift distance detected in real time: astigmatic PSF, 1 [mu] m lt is started when z is smaller than or equal to 1 [mu] m; z < = 2 [mu] m, and switching to a double-helix PSF, 2 [mu] m lt; bessel PSF, zgt is adopted when z is smaller than or equal to 3 microns; the clover PSF is activated when the particle size is 3 microns. Through self-adaptive switching of the PSF, full-closed-loop real-time correction is achieved within the axial range of 6 microns, the drift suppression error is smaller than or equal to 2 nm, and the method is suitable for living cell long-term super-resolution imaging.

Description

Technical Field

[0001] This invention relates to the field of super-resolution technology, specifically to a drift correction method and system that achieves high-precision real-time drift correction over a large depth of field (6μm axial range) through adaptive switching of the PSF (point spread function). Background Technology

[0002] The stability of the imaging system is a prerequisite for obtaining high-quality microscopic data. Any slight disturbance from the environment or the instrument will cause unexpected movement of the observed sample in three-dimensional space, known as sample drift. For ultra-high resolution microscopes aiming for resolutions below 10 nanometers, even a small amount of drift will accumulate over time, leading to increased positioning deviation and directly affecting the quality of the super-resolution image.

[0003] To overcome the effects of drift, existing technologies mostly employ post-processing drift correction techniques, analyzing the positional changes of markers or sample features in a sequence of images and estimating and compensating for the drift trajectory after imaging. Current mainstream post-processing correction techniques generally use a single, fixed PSF (Pressure Field Sample), making it difficult to balance the "wide range" and "high precision" of correction: on the one hand, correction schemes using high-precision PSFs often cannot cover significant axial drift due to limited depth of field; on the other hand, correction schemes using wide-axis-range PSFs are limited by their light intensity distribution characteristics, resulting in insufficient positioning accuracy and failing to meet the requirements of ultra-high resolution imaging.

[0004] In applications such as long-term live-cell imaging, the axial position of a sample undergoes significant and irregular changes over time, with its drift trajectory often spanning a large axial range of several micrometers. Because existing post-processing correction techniques use a fixed PSF mode, they cannot dynamically adjust based on real-time drift, resulting in poor drift correction performance in scenarios with large axial ranges. Currently, there is a lack of a drift correction method and system that can dynamically switch to the optimal PSF based on the actual drift of the sample, thereby simultaneously achieving a wide axial range and high correction accuracy. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a real-time drift correction method and system for large depth of field based on a programmable point extension function.

[0006] A first aspect of the present invention provides a real-time drift correction method for large depth of field based on a programmable point spread function, the method comprising the following steps:

[0007] S1. Acquire images of markers in the sample in real time using a high frame rate camera;

[0008] S2. Based on the current point spread function pattern, the image is processed using the maximum likelihood estimation algorithm to calculate the three-dimensional drift of the markers;

[0009] S3. Based on the axial drift in the three-dimensional drift, the optimal point spread function is determined and switched according to the Cramer-Rao lower bound optimization algorithm, and the corresponding phase mask is loaded by the programmable phase modulator to generate the optimal point spread function.

[0010] S4. Drive the triaxial nano-displacement stage to perform real-time closed-loop compensation of the sample position according to the axial drift amount;

[0011] Steps S3 and S4 are executed in parallel, and the optimal point spread function is an astigmatic point spread function in the near-focal region, a double helix or Bessel point spread function in the intermediate region, and a cloverleaf point spread function in the far-focal region, so as to achieve real-time drift correction with a positioning accuracy better than 1nm within a 6μm axial range.

[0012] A second aspect of the present invention provides a large depth-of-field real-time drift correction system based on a programmable point spread function, comprising: The marker detection module is used to acquire images of markers in samples in real time using a high frame rate camera; The PSF dynamic switching module has a built-in programmable phase modulator, which is used to switch and generate the optimal point spread function based on the Cramer-Rao lower bound optimization algorithm according to the axial drift amount. The closed-loop control module is used to receive the three-dimensional drift calculated by the maximum likelihood estimation algorithm and drive the three-axis nano-displacement stage to perform real-time closed-loop compensation of the sample position. The optimal point spread function is an astigmatic point spread function in the near-focal region, a double helix or Bessel point spread function in the intermediate region, and a cloverleaf point spread function in the far-focal region, so as to achieve real-time drift correction with a positioning accuracy better than 1 nm within a 6 μm axial range.

[0013] Compared with the prior art, the present invention has the following outstanding advantages:

[0014] 1) Balancing wide calibration range and high precision: An adaptive PSF switching strategy optimized by CRLB (Cramax-Rao lower bound) achieves optimal adaptation across different axial ranges. Near-focal region utilizes astigmatic PSF for high-precision calibration within <1 nm, while far-focal region uses cloverleaf PSF to cover a wide drift range of |z|>3μm. The intermediate range is connected and transitioned by double-helix PSF and Bessel PSF, overcoming the shortcomings of existing calibration techniques where "high precision comes with a narrow range, and a wide range comes with low precision," thus meeting the requirements of ultra-high resolution imaging.

[0015] 2) Real-time dynamic adaptation capability: The system captures drift data in real time throughout the process, dynamically switches PSF and performs correction. The response speed is synchronized with the imaging process. It can accurately match the scenario of irregular changes in the axial position of the sample in live cell imaging. Even if the drift trajectory crosses multiple axial intervals, it can maintain the continuity and stability of correction.

[0016] 3) High compatibility and scene adaptability: The system adopts a modular design, and core components such as programmable phase modulators and gold nanoparticle labels can be directly integrated into existing super-resolution microscopes (such as SMLM and MINFLUX) without complex hardware modifications; at the same time, it is adapted to long-term imaging scenarios of live cells, ensuring the accuracy of calibration without interfering with the activity of biological samples, and taking into account both technical practicality and biocompatibility. Attached Figure Description

[0017] Figure 1 This is a flowchart of the workflow of this embodiment;

[0018] Figure 2 This is a schematic diagram of a three-dimensional real-time drift correction system based on a programmable PSF.

[0019] Figure 3 These are simulation diagrams of several PSFs in this embodiment;

[0020] Figure 4 This is an image showing the imaging results and selection of markers in this embodiment;

[0021] Figure 5 This is a graph showing the long-term standard deviation data of the XYZ triaxial axes based on astigmatism PSF in this embodiment;

[0022] Figure 6 This is a graph showing the long-term standard deviation data of the XYZ triaxial axes based on the double-helix PSF in this embodiment;

[0023] Figure 7 This is a graph showing the long-term standard deviation data of the XYZ triaxial axes based on the Bessel beam PSF in this embodiment;

[0024] Figure 8 This is a graph showing the long-term standard deviation data of the XYZ triaxial axes based on the cloverleaf PSF in this embodiment. Detailed Implementation

[0025] The present invention will be described in detail below with reference to the embodiments and accompanying drawings; however, the technical solutions of the present invention are not limited to these embodiments.

[0026] This application provides a real-time drift correction method for large depth of field based on a programmable point spread function, which includes the following steps:

[0027] S1. Capture images of sample markers in real time using a high frame rate camera;

[0028] S2. Based on the PSF mode, select the localization algorithm and calculate the drift of the marker from the image.

[0029] S3. Based on the axial drift, switch the PSF using the Cramer-Rhodes Lower Bound (CRLB) optimization algorithm;

[0030] S4. Based on the axial drift, control the triaxial nano-displacement stage for fine-tuning to correct sample drift in real time.

[0031] Furthermore, in step S2, the localization algorithm is a maximum likelihood estimation (MLE) based on a specific PSF (such as astigmatic PSF, double helix PSF, etc.) to calculate the three-dimensional coordinates of the marker.

[0032] Furthermore, in step S3, the CRLB optimization algorithm is used to calculate the theoretical lower limit of positioning accuracy for each PSF mode.

[0033] Furthermore, in step S3, the switching process includes the following steps:

[0034] S301. Calculate the ideal phase mask required for the target PSF;

[0035] S302. Superimpose the ideal phase mask with the system's inherent aberration compensation;

[0036] S303. Convert the synthesized phase mask into a grayscale signal recognizable by a spatial light modulator (SLM);

[0037] S304. Apply the signal to the SLM surface to complete wavefront modulation;

[0038] S305. SLM modulates the beam wavefront in real time to generate the target PSF on the camera.

[0039] Furthermore, steps S3 and S4 are performed in parallel to ensure that switching the PSF does not affect drift correction.

[0040] Furthermore, the method also integrates a correction log management function to back up drift correction data, and works with an interactive interface to monitor the system's compensation accuracy and current operating mode.

[0041] This application provides a system for real-time drift correction over large depth of field based on a programmable point spread function, comprising:

[0042] 1) Marker detection module, used to capture marker images;

[0043] 2) PSF dynamic switching module, used to quickly switch between different PSF modes;

[0044] 3) Closed-loop control module, used to compensate for sample drift in real time.

[0045] Furthermore, the marker detection module incorporates a laser light source (near-infrared band) to excite and illuminate the marker, and a high frame rate camera to continuously acquire images at high speed.

[0046] Furthermore, the PSF dynamic switching module incorporates a programmable phase modulator (SLM) for rapidly generating and switching different PSF modes; and a processor with a graphics processing unit (GPU) to meet the parallel and rapid computation of the MLE algorithm and phase mask.

[0047] Furthermore, the closed-loop control module receives sample drift data, drives a three-axis nanometer displacement stage to compensate for sample drift, and performs closed-loop control on the sample position.

[0048] Furthermore, the device also integrates a correction log management module to back up drift correction data; and works with an interactive interface to monitor the system's compensation accuracy and current operating mode.

[0049] Example:

[0050] Corresponding to the above method, this application embodiment constructs a drift correction device, the workflow of which is as follows: Figure 1 For details, see the device. Figure 2 The drift correction device includes: a laser source 1, a polarizing beam splitter 2, a mirror 3, a quarter-wave plate 4, a field lens 5, a mirror 6, an objective lens 7, a sample 8, a spatial light modulator 9, a mirror 10, a relay lens 11, a filter 12, an imaging lens 13, a filter 14, and a high frame rate camera 15; in addition, there is a laser fiber 16, a collimator 17, and a three-axis nano-displacement stage 18. The laser source is in the near-infrared band; the objective lens is a high numerical aperture objective lens; the sample consists of an upper cover glass and a lower biological cell layer, all mechanically fixed and clamped on the three-axis nano-displacement stage 18.

[0051] Regarding the drift correction of the device in this embodiment, the specific approach is as follows: After the laser is incident at the center, a scattered signal is generated and returns to the spatial light modulator; the spatial light modulator modulates the wavefront by loading a phase mask corresponding to the PSF, mapping the Z-axis displacement into identifiable features in the XY plane image. Based on this, the modulated PSF is fitted using the maximum likelihood estimation algorithm to calculate the three-dimensional coordinates of the marker, thus completing the drift correction.

[0052] Furthermore, regarding the drift correction of the device in this embodiment, the specific method is as follows: the laser source 1 is output through the laser fiber 16 and collimated by the collimator 17, with the beam diameter preferably being 2-5 mm. The collimated beam is first reflected by the polarizing beam splitter 2 and then reflected by the reflecting mirror 3, at which point the beam is in a vertically linearly polarized state.

[0053] The beam is then incident on the quarter-wave plate 4, converted into circularly polarized light, and then converged by the field lens 5, focusing at the center of the focal plane of the objective lens 7. The reflecting mirror 6 deflects and guides the converged beam. The beam entering the objective lens 7 forms a focused illumination region at the center of the sample surface. When this illumination light irradiates the gold nanoparticles in the sample, it excites a strong zero-angle signal and a weak scattered signal.

[0054] These signals return from the back focal plane of the objective lens and pass through mirror 6, field lens 5, quarter-wave plate 4, mirror 3, polarization beam splitter 2, spatial light modulator 9, mirror 10, relay lens 11, filter 12, imaging lens 13 and filter 14, and are finally collected by high frame rate camera 15.

[0055] Furthermore, the infrared beam passes through the quarter-wave plate 4 twice during its round-trip propagation, causing the polarization direction to rotate by 90°, turning it into horizontally linearly polarized light, which can then be transmitted through the polarization beam splitter 2.

[0056] Furthermore, filter 14 serves to filter stray light.

[0057] Furthermore, the relay lens 11 and the field lens 5 form a focal conjugate, conjugating the objective lens pupil plane focal point to the filter 12. This filter is a ring structure with a solid center and hollow edges, which can filter out strong zero-angle signals while allowing weak scattered signals to pass through. The contour and high-frequency information contained in the weak scattered signals support high-precision calculations for XY direction drift correction. Finally, the processing module calls the MLE algorithm to calculate the coordinates of the gold nanoparticle markers, ultimately achieving drift correction.

[0058] Preferably, the Maximum Likelihood Estimation (MLE) in this embodiment is a statistical method for determining the coordinates of markers. Its principle is based on finding a set of XYZ coordinates such that the probability of observed image data occurring is maximized under these coordinates. During imaging, sample drift causes the image position to shift over time. MLE can effectively detect and correct this drift, thereby ensuring imaging accuracy.

[0059] Furthermore, the specific steps of MLE are as follows:

[0060] PSF Model Construction: Based on the currently loaded phase mask, generate theoretical light intensity distribution data of the corresponding PSF at different axial depths, and establish a parameterized theoretical model that maps three-dimensional spatial coordinates to two-dimensional image morphology, which serves as the benchmark for subsequent positioning and correction.

[0061] Constructing the likelihood function: Calculate the probability of each pixel's actual light intensity value and theoretical light intensity value point by point and multiply them together to construct a log-likelihood function to characterize the degree of matching.

[0062] Log-likelihood maximization: Numerical optimization algorithms such as the Newton-Raphson algorithm are used to calculate the gradient direction of the likelihood function. The estimated values ​​of the 3D coordinates are continuously corrected through iterative iteration until the likelihood function converges to its maximum value. The parameters corresponding to this maximum value are the true positions of the markers.

[0063] Drift correction: The calculated three-dimensional coordinates are compared with the reference coordinates recorded at the initial time to obtain the original relative displacement deviation. Then, filtering is performed to eliminate random noise, and finally the drift vector of the sample is obtained.

[0064] Once the sample drift during imaging is accurately estimated, drift correction can be achieved using a triaxial nanostage with nanometer-level positioning accuracy. It is worth noting that the performance of the drift correction system is directly limited by the positioning accuracy and movement speed of the stage.

[0065] Preferably, the gold nanoparticle sample preparation steps in this embodiment are as follows: First, the culture dish is treated to prevent detachment. 400 μL of a 0.1 mg / mL polylysine aqueous solution is added to the culture dish. After standing for 3 hours, the solution is removed, and the culture dish is dried. Polylysine can enhance the adsorption performance of the culture dish slide and prevent the gold nanoparticles from falling off. Then, the gold nanoparticle stock solution is diluted 1000 times with distilled water and injected into the culture dish that has undergone the anti-detachment treatment. It is then left to stand for 2 hours. During the standing period, the gold nanoparticles sink and adhere to the surface of the bottom glass slide of the culture dish due to gravity.

[0066] Figure 3 The following are simulation diagrams of several PSFs in this embodiment. From left to right, they are the XY / XZ sections of the astigmatic PSF, Bessel PSF, double helix PSF, and cloverleaf PSF.

[0067] Figure 4 For the image of the 160nm gold nanoparticles captured in this embodiment, one of them is selected for drift correction. This can achieve long-term three-dimensional drift correction. Alternatively, a larger area (containing images of multiple gold nanoparticles) can be selected as the anti-drift calculation object, at the cost of reduced calculation speed. The area needs to be selected according to the actual situation.

[0068] Figures 5 to 8The drift correction curves are presented sequentially using astigmatic PSF, double-helix PSF, Bezier PSF, and cloverleaf PSF. These results were obtained over an 800-second experimental period. As shown in the figure, at a depth of |z| ≤ 1 μm, the standard deviations of the XYZ axes for drift correction using astigmatic PSF are 1.062 nm, 0.925 nm, and 1.030 nm, respectively. At a depth of 1 μm < |z| ≤ 2 μm, switching to double-helix PSF yields standard deviations of 1.025 nm, 0.935 nm, and 0.931 nm. At a depth of 2 μm < |z| ≤ 3 μm, using Bezier PSF yields standard deviations of 1.055 nm, 0.905 nm, and 0.962 nm. When |z| > 3 μm, activating cloverleaf PSF yields standard deviations of 0.890 nm, 0.755 nm, and 1.018 nm.

[0069] In summary, this application captures marker images through a marker detection module and calculates sample drift using the MLE algorithm. The PSF dynamic switching module determines the current optimal PSF based on CRLB optimization according to the sample drift and generates the corresponding phase mask by loading it through a programmable phase modulator. Simultaneously, the closed-loop control module drives the displacement stage to perform fine-tuning based on the drift, forming a fully closed-loop real-time correction process of "detection-judgment-switching and correction". This achieves three-dimensional drift correction that balances wide axial range and high precision, significantly improving the performance and reliability of drift correction.

[0070] It is obvious that those skilled in the art can make various adjustments and modifications to this invention without departing from the core concept and scope of this invention. If such adjustments and modifications meet the protection scope defined by the claims of this invention and the criteria for identifying equivalent technical solutions, this invention also covers such adjustments and modifications.

Claims

1. A real-time drift correction method for large depth of field based on a programmable point spread function, characterized in that, The method includes the following steps: S1. Acquire images of markers in the sample in real time using a high frame rate camera; S2. Based on the current point spread function pattern, the image is processed using the maximum likelihood estimation algorithm to calculate the three-dimensional drift of the markers; S3. Based on the axial drift in the three-dimensional drift, the optimal point spread function is determined and switched according to the Cramer-Rao lower bound optimization algorithm, and the corresponding phase mask is loaded by the programmable phase modulator to generate the optimal point spread function. S4. Drive the triaxial nano-displacement stage to perform real-time closed-loop compensation of the sample position according to the axial drift amount; Steps S3 and S4 are executed in parallel, and the optimal point spread function is an astigmatic point spread function in the near-focal region, a double helix or Bessel point spread function in the intermediate region, and a cloverleaf point spread function in the far-focal region, so as to achieve real-time drift correction with a positioning accuracy better than 1nm within a 6μm axial range.

2. The method according to claim 1, characterized in that, The switching process in step S3 includes: S301. Calculate the ideal phase mask required for the target point spread function; S302. Superimpose the ideal phase mask with the system's inherent aberration compensation; S303. Convert the synthesized phase mask into a grayscale signal recognizable by the spatial light modulator; S304. The grayscale signal is loaded onto the surface of the spatial light modulator to complete wavefront modulation; S305. A spatial light modulator modulates the beam wavefront in real time to generate the target point spread function on the camera.

3. The method according to claim 1 or 2, characterized in that, The marker is a gold nanoparticle. Its scattering signal is filtered out by a ring filter to remove strong zero-angle signals before entering a high frame rate camera to improve lateral positioning accuracy.

4. The method according to claim 3, characterized in that, The maximum likelihood estimation algorithm constructs a log-likelihood function based on the observed pixel counts and the Poisson statistical assumption, and finds the XYZ coordinates to maximize the log-likelihood function through iterative search to obtain the three-dimensional coordinates of the marker.

5. The method according to claim 3, characterized in that, The Cramer-Rao lower bound optimization algorithm calculates the theoretical lower bound of positioning accuracy for each candidate point spread function and selects the point spread function with the smallest theoretical lower bound as the current optimal mode.

6. The method according to claim 1, characterized in that, It also includes a calibration log management step: recording the drift amount, the point extension function mode used, and the compensation displacement at each moment in real time for subsequent accuracy analysis and fault tracing.

7. A large depth-of-field real-time drift correction system based on a programmable point spread function, characterized in that, include: The marker detection module is used to acquire images of markers in samples in real time using a high frame rate camera; The PSF dynamic switching module has a built-in programmable phase modulator, which is used to switch and generate the optimal point spread function based on the Cramer-Rao lower bound optimization algorithm according to the axial drift amount. The closed-loop control module is used to receive the three-dimensional drift calculated by the maximum likelihood estimation algorithm and drive the three-axis nano-displacement stage to perform real-time closed-loop compensation of the sample position. The optimal point spread function is an astigmatic point spread function in the near-focal region, a double helix or Bessel point spread function in the intermediate region, and a cloverleaf point spread function in the far-focal region, so as to achieve real-time drift correction with a positioning accuracy better than 1 nm within a 6 μm axial range.

8. The system according to claim 7, characterized in that, The PSF dynamic switching module includes: The phase mask calculation unit is used to generate the ideal phase mask required for the target point spread function and superimpose it with the system's inherent aberration compensation. The grayscale signal conversion unit converts the synthesized phase mask into a grayscale signal that can be recognized by the spatial light modulator; A spatial light modulator receives the grayscale signal and modulates the beam wavefront in real time to form the target point spread function on a high frame rate camera.

9. The system according to claim 7, characterized in that, The marker detection module includes: Near-infrared laser light source is used to excite gold nanoparticle labels; A high frame rate camera, with a frame rate ≥100 Hz, is used to acquire images of markers; A ring filter, located in the imaging optical path, is used to filter out zero-order strong light and transmit scattered signals to improve lateral positioning accuracy.

10. The system according to claim 7, characterized in that, It also includes a calibration log management module, which records the drift amount, the point extension function mode used, and the compensation displacement at each moment in real time, and provides an interactive interface to display the current compensation accuracy and working mode.

Images