Axial stable shooting system and shooting method for single molecule positioning microscope

Abstract

The invention discloses an axial stable shooting system and shooting method for a single molecule positioning microscope, and belongs to the technical field of microscopic imaging. The system comprises a sample axial drift real-time compensation light path, an infrared laser emits linear polarization parallel light, the linear polarization parallel light passes through a collimating lens L1, a half-wave plate H1 and a beam splitting cube BS1 to form two beams of parallel light, the two beams of parallel light are reflected by a polarization beam splitting cube PBS, then enter an objective lens through a half-wave plate H2, a lens L2, a beam splitting cube BS2 and a dichroscope DM1, and are focused at the position of a rear focal plane of the objective lens; the two beams of light reflected by the sample are transmitted out from a polarization beam splitting cube PBS along the original path, and enter a focus-locked camera camZ for imaging through a lens L3 and a band-pass optical filter F1. And during shooting, recording the difference delta x between the distance D of two light spots of a new image every time and the initial distance D0 in real time, controlling the axial position of the objective table to compensate by using an incremental PID (Proportion Integration Differentiation) algorithm, and controlling the delta x to be less than or equal to an allowable error. According to the system and the method, the problem of poor axial stability precision of the sample of the single-molecule positioning microscope is successfully solved.

Description

Technical Field

[0001] This invention belongs to the field of microscopic imaging technology, and particularly relates to an axially stabilized imaging system and method for single-molecule positioning microscopes that solves the problem of sample drift during imaging. Background Technology

[0002] Single-molecule localization microscopy plays an indispensable role in the study of organelle structure and function due to its resolution of up to tens of nanometers. The technology's fundamental imaging principle of "trading time for space" means that constructing a single super-resolution image requires thousands to tens of thousands of raw image frames. The image acquisition process for single-molecule localization microscopy typically takes tens of minutes, and the sample being imaged is inevitably affected by temperature and environmental factors during this extended acquisition process, resulting in drift. While lateral drift can be corrected using software algorithms in subsequent image processing, axial drift can only be compensated for and adjusted by the microscope's focusing system. Currently, commercial inverted microscopes can be equipped with optional focusing modules, such as Nikon's PFS (PerfectFocus System) and Olympus' ZDC (Z-Drift Compensator), with a focusing accuracy of approximately ±100 nm. This accuracy is generally sufficient for common scenarios where axial accuracy requirements are not high. However, in the field of single-molecule localization imaging, a focusing accuracy of ±100 nm is insufficient because the depth of field of single-molecule localization imaging microscopes is very small; ±100 nm will cause blurring of the single-molecule image, thus reducing its localization accuracy. Therefore, it is of great significance to develop a technique for single-molecule localization microscopy that can ensure that the axial drift of the sample is kept within 5 nm during long-term imaging. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides an axial stabilization imaging system for single-molecule positioning microscopes. This system can be directly applied to inverted microscopes to perform real-time axial drift compensation for the photographed samples, thus solving the problem of poor sample axial stabilization accuracy currently faced by single-molecule positioning microscopes.

[0004] This invention is achieved through the following technical solution: An axially stabilized imaging system for a single-molecule positioning microscope includes an optical path for real-time axial drift compensation of the sample in an inverted microscope and a focus-locking camera camZ. Linearly polarized parallel light of a certain wavelength is emitted from an infrared laser. After collimation by lens L1, polarization direction adjustment by half-wave plate H1, and beam splitting by beam splitter BS1, two parallel beams are incident on the polarization beam splitter PBS. After being reflected by the polarization beam splitter PBS, the beams pass through half-wave plate H2 and are then focused by lens L2 onto the back focal plane of the microscope objective. The two laser beams focused by lens L2 first pass through beam splitter BS2, then are reflected from dichroic mirror DM1 and enter the objective. After exiting the objective, the beams are reflected back to the objective by the sample. The returning two beams sequentially pass through the objective, dichroic mirror DM1, beam splitter BS2, and lens L2. After polarization direction adjustment by half-wave plate H2, they are transmitted from the polarization beam splitter PBS, collimated by lens L3, and have stray light eliminated by bandpass filter F1 before entering the focus-locking camera camZ for imaging.

[0005] Furthermore, the beam splitter BS1 splits the incident light into a projected light and a reflected light. These two beams are reflected by two mirrors and then incident together onto the polarization beam splitter PBS.

[0006] Furthermore, the half-wave plate H1 can be positioned between the lens L1 and the beam splitter BS1, or between the beam splitter BS1 and the polarizing beam splitter PBS. The beam is adjusted to S-polarized light by the half-wave plate H1, and is reflected after being incident on the polarizing beam splitter PBS.

[0007] Furthermore, the beam returning from the sample is adjusted to P-polarized light by the half-wave plate H2, and then transmitted after being incident on the polarized beam splitter PBS.

[0008] Furthermore, the infrared laser is preferably a near-infrared laser, and the emitted light wavelength is different from the light wavelength used for eyepiece imaging. The linearly polarized parallel light emitted by the infrared laser is first collimated by lens L1 to obtain a parallel beam of appropriate diameter.

[0009] Based on the above-described axially stabilized imaging system, the present invention also provides an axially stabilized imaging method for single-molecule positioning microscopy, comprising the following steps: 1) Place the sample on the three-dimensional nanostage of the microscope, adjust the axial position of the objective lens to complete the focus, turn on the axial stabilization imaging system, and use the focus-locking camera camZ to acquire an image containing two light spots; 2) The image is transmitted to the computer, and the drift between the sample and the objective lens is calculated through image processing. Then, based on this drift, a PID algorithm is used to control the three-dimensional nanostage for axial compensation. Specifically, the distance between the two light spots in the image is obtained by calculating their center positions. When the sample axial locking operation is initiated, the distance between the two light spots in the acquired image is recorded by the computer program. D 0; Calculate the distance between the two light spots each time a new image is generated. D and D The difference Δ to 0 x , with Δ x As input error, an incremental PID algorithm is used to control the axial position of the triaxial nanostage for compensation until Δ x Less than or equal to the allowable error Δ e until.

[0010] In the above-described axially stabilized imaging method, the specific flow of the computer program is as follows: a) Obtain the original image and perform binarization. Then, perform morphological opening and closing operations on the binary image to filter out noise and holes. b) Calculate the number and area of ​​all connected components. If the number of connected components is less than two, exit and wait for the next frame image to be input. If the number of connected components is greater than or equal to two, select the two connected components S1 and S2 with the largest area. If the area of ​​connected components S1 and S2 is greater than or equal to a preset threshold, then continue to the next step; otherwise, exit and wait for the next frame image to be input. c) Calculate the centroid coordinates of connected components S1 and S2; d) Using the centroid coordinates of connected domains S1 and S2 as the center, extract two rectangular image blocks B1 and B2 of size W*H pixels from the image; calculate the center coordinates of image blocks B1 and B2 respectively using an elliptic Gaussian fitting model; preferably, the maximum likelihood estimation method based on Poisson distribution noise is used to calculate the parameters of the elliptic Gaussian fitting model. e) Calculate the distance between image blocks B1 and B2 based on their center coordinates. D Thus, the axial drift error of the sample is obtained. ,in D 0 represents the initial centroid distance recorded when focus lock is engaged; f) Determine the axial drift error Δ x Is the absolute value less than or equal to the preset allowable value Δ? e If the requirements are not met, an incremental PID algorithm is used to control the axial position of the displacement stage.

[0011] The beneficial effects of this invention are as follows: Compared with the current commercial focus-locking modules which generally have a resolution of around 200 nm, the axial stabilization imaging system and imaging method provided by this invention can achieve a focus-locking accuracy of 5 nm, and can maintain the stability of the sample axial plane for more than 50 minutes of imaging time, successfully solving the problem of poor sample axial stability accuracy faced by existing single-molecule positioning microscopes. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the axially stabilized imaging system constructed according to an embodiment of the present invention, wherein: L1, L2, L3, L4 and L5 are lenses, H1 and H2 are half-wave plates, M is a mirror, BS1 and BS2 are beam splitters, PBS is a polarizing beam splitter, F1 and F2 are bandpass filters, camZ and camXY are cameras, and DM1 is a dichroic mirror.

[0013] Figure 2 This is an image formed on camera camZ without using a polarized light path to eliminate interference light in an embodiment of the present invention.

[0014] Figure 3 This is an image captured on camera camZ after using a polarized light path to eliminate interference light in an embodiment of the present invention.

[0015] Figure 4 This is a diagram showing the relationship between the center distance of the two light spots and the axial position of the displacement stage in Experiment 1 of this invention.

[0016] Figure 5 The images shown are (a) and (b) of the axial displacement of the displacement stage versus time, recorded in Experiment 2 of this invention under the focus-locked state. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and should not be construed as limiting the invention.

[0018] The axially stabilized imaging system for the single-molecule positioning microscope provided in this embodiment is as follows: Figure 1As shown, the system includes a 940 nm near-infrared laser, a collimating lens L1, a half-wave plate H1, a reflecting mirror M, a beam splitter cube BS1, a polarizing beam splitter cube PBS, a half-wave plate H2, a lens L2, a beam splitter cube BS2, a focusing lens L3, a bandpass filter F1, and a camera camZ. The light emitted from the 940 nm laser first passes through the collimating lens L1 to obtain a parallel beam of suitable diameter. The beam then passes through the half-wave plate H1 (adjusting polarization) and reaches the beam splitter cube BS1, where it splits into a projected beam and a reflected beam. These two beams are reflected by the two reflecting mirrors M of the beam splitter cube BS1 and then both incident on the polarizing beam splitter cube PBS. The polarizing beam splitter cube PBS separates the two beams along vertical and horizontal polarization directions. The beam along the vertical polarization direction enters the half-wave plate H2, while the beam along the horizontal polarization direction... The polarizing beam splitter PBS is transmitted along the original direction; the beam emitted from the half-wave plate H2 is focused by the lens L2 onto the back focal plane of the objective lens. During this time, the beam passes through the beam splitter BS2 and is reflected from the dichroic mirror DM1 into the objective lens. After exiting the objective lens, the beam is reflected back to the objective lens from the bottom of the sample culture dish. The reflected beam passes sequentially through the objective lens, dichroic mirror DM1, beam splitter BS2, lens L2, half-wave plate H2, polarizing beam splitter PBS, lens L3, bandpass filter F1, and finally enters the focus-locking camera camZ.

[0019] In addition to the axial stabilization shooting system Figure 1 The optical path used for eyepiece imaging is also shown, including an 850 nm LED, lens L5, a sample culture dish, an objective lens, a dichroic mirror DM1, a beam splitter BS2, a reflecting mirror M, lens L4, a bandpass filter F2, and a camera camXY. The 850 nm LED generates incoherent light with a center wavelength of 850 nm. This light is collimated into parallel light by lens L5 and transmitted to illuminate the sample. The light transmitted through the sample is collected by the objective lens and then reflected by the dichroic mirror DM1, the beam splitter BS2, and the reflecting mirror M before being focused onto the camera camXY by lens L4. The bandpass filter F2 between L4 and camXY filters out light with wavelengths other than 850 nm.

[0020] The steps to achieve axially stabilized shooting are as follows: (1) A near-infrared laser with a wavelength of 940 nm emits linearly polarized parallel light with a diameter of 2 mm. The laser passes through a collimating lens L1 and a half-wave plate H1, and then through a beam splitter BS1 combined with a reflector M to generate two outgoing light spots. These two light spots are reflected by the polarization beam splitter PBS and focused by the lens onto the back focal plane of the objective lens. (2) The two laser spots transmitted through the objective lens will be reflected back to the objective lens on the surface of the sample culture dish. The two spots will pass through the half glass plate H2 along the incident path direction, and then be transmitted through the polarization beam splitter cubic PBS and converged on the sensor plane of the camera camZ by the focusing lens L3 and the bandpass filter F1. (3) The image acquired by the camera camZ is transmitted to the computer. After image processing, the drift between the sample and the objective lens can be calculated. Finally, the PID algorithm is used to control the stage for axial compensation based on the drift.

[0021] In step (1), adjusting the half-wave plate H1 can change the polarization direction of the laser, which can reduce the transmitted light of the laser through the polarization beam splitter cube PBS. Reducing the transmitted light can effectively eliminate the interference spot generated by the light reflected back to the camera from the surface of the polarization beam splitter cube PBS. Figure 2 and Figure 3 The two images are the images taken on camera camZ before and after using polarization to eliminate interference light. It can be seen that after the stray light is eliminated by the polarization optical path (including polarization beam splitter PBS, half-wave plates H1 and H2), the image obtained by the camera only has two target light spots to be measured.

[0022] In step (2), after exiting the objective lens, the two laser beams are reflected off the bottom surface of the sample culture dish and then collected again by the objective lens. When the relative distance between the sample culture dish and the objective lens changes, the relative distance between the two laser beams reflected off the bottom surface of the culture dish and finally focused on the sensor target surface of the camera camZ will also shift. Thus, the axial drift between the objective lens and the sample culture dish can be determined based on the change in the distance between the laser beams. The half-wave plate H2 filters out other stray light whose polarization direction is inconsistent with the target laser beam, which can effectively improve the signal-to-background ratio of the image. The bandpass filter F1 is a bandpass filter with a wavelength of 940 nm, which is used to filter out lasers of other wavelengths reflected into the camera camZ.

[0023] The operation in step (3) involves a camera, a computer, and a triaxial nanostage. First, the camera camZ acquires the original image containing two light spots, such as... Figure 2 As shown; then the center positions of the two light spots are calculated to obtain the distance between them; when the axial locking of the sample culture dish is activated, the program will record the distance at this time. D 0, the distance between the two light spots is calculated each time a new image is generated. D and D The difference Δ to 0 x , with Δ x As input error, an incremental PID algorithm is used to control the axial position of the triaxial nanostage for compensation until Δ x Less than or equal to the allowable error Δe That's it. The specific program flow is as follows: 1. Acquire the original image and perform binarization. Then, perform morphological opening and closing operations on the binary image to filter out noise and holes.

[0024] 2. Calculate the number and area of ​​all connected components. If the number of connected components is less than two, exit and wait for the next frame. If the number of connected components is greater than or equal to two, select the two connected components S1 and S2 with the largest areas. If the areas of connected components S1 and S2 are greater than or equal to a preset threshold (set based on the average size of the actual areas of the two connected components S1 and S2, and the threshold is slightly smaller than the average size of S1 and S2), then continue to the next step; otherwise, exit and wait for the next frame.

[0025] 3. Calculate the centroid coordinates of connected components S1 and S2 according to the following formula, where C x and C y These are the x and y coordinates of the centroid, respectively. i Indicates the pixel index in the connected component. n This represents the total number of pixels in the connected component. x i and y i They represent the first and second connected components, respectively. i The x and y coordinates of each pixel represent p i The first connected component i The pixel value of each pixel (the pixel value is 1 or 0).

[0026] 4. After obtaining the centroid coordinates of connected components S1 and S2, extract two rectangular image blocks B1 and B2 of size W*H pixels from the image, centered on their centroid coordinates. Calculate the center coordinates of image blocks B1 and B2 using an elliptic Gaussian fitting model, as shown in the following formula: in, A 0 and L These are the light spot intensity and the background intensity, respectively. x and y These are the x and y coordinates of the pixels in the image patch that are involved in the fitting process; x c and y c These are the center coordinates obtained after fitting, and their initial values ​​are respectively... C x and Cy ; The parameters of the elliptic Gaussian fitting model are... Elliptic Gaussian fitting is an optimization problem. Here, we use the maximum likelihood estimation method based on Poisson noise distribution to calculate its parameters, and the formula is shown below: in, n Pixel number; N Number of pixels; The model is an elliptic Gaussian fitting model. z n For the first n grayscale value of each pixel; For parameters The maximum likelihood estimate is calculated below.

[0027] The center coordinates of the two image blocks B1 and B2 can be calculated by maximum likelihood estimation.

[0028] 5. Calculate the distance between image blocks B1 and B2 based on their center coordinates. D Therefore, the axial drift error of the sample can be obtained. ,in D 0 represents the initial centroid distance recorded when focus lock is enabled.

[0029] 6. Determine the axial drift error Δ x If the absolute value is less than or equal to a preset allowable value, and the requirement is not met, an incremental PID algorithm is used to control the axial position of the displacement stage. The formula for the control algorithm is shown below, where... K p , K i and K d These represent the proportional coefficient, integral coefficient, and differential coefficient, respectively, Δ x ( k ) indicates the first k Frame axis drift error, u ( k ) represents the output of the control algorithm, which is used to drive the stage for correction.

[0030] Experiment 1: A standard cell microtube sample was placed on the triaxial nanostage of the microscope. The objective lens was adjusted along its axial direction to achieve focus, with the focal plane of the objective lens focused on the plane containing the fluorescent microspheres. The stage was controlled to scan along its axial direction, and the distance between the centers of the two light spots in the camZ camera was recorded simultaneously. Figure 4This diagram illustrates the relationship between the center-to-center distance and axial position of two light spots. The horizontal axis represents the axial displacement stage sensor reading, and the vertical axis represents the center-to-center distance of the two light spots in the camZ image from the focus-locked camera. The straight line in the diagram is obtained using the least squares method for fitting, and its analytical expression is: It can be seen that for every 1 pixel change in the distance between the two light spots in the image, the sample shifts 250 nm along the axial direction. Figure 4 The linearity of the distance between the centers of the two light spots as the axial position changes is well demonstrated. The linearity is calculated using the coefficient of determination. A coefficient of determination close to 1 indicates that the linearity of the fitted line is better. The calculated coefficient of determination is 0.99, which proves that its linearity is excellent.

[0031] Experiment 2: A standard cell microtube sample was placed on the triaxial nanostage of the microscope. The objective lens was adjusted axially to achieve focus, with the focal plane focused on the plane containing the fluorescent microspheres. The axial stabilization system was then activated to ensure proper operation, and 60,000 frames of focus-locking experimental data were recorded. Figure 5 In the middle (a), the vertical axis represents the deviation of the center distance between the two light spots in the locked-focus image relative to the moment when the locked-focus state was started (in pixels), and the horizontal axis represents the image frame number. The time difference between two adjacent frames is 50 milliseconds, and the overall experimental recording time is 50 minutes. Figure 5 The data fluctuations in (a) have remained largely within the area covered by the gray shadow. Within the pixel range, according to Figure 4 The calibration results show that the fluctuation range corresponds to -2.5~2.5 nm. Therefore, it can be considered that the long-term stability accuracy of the focus-locking system designed in this invention can reach 5 nm. Figure 5 (b) shows the readings of the objective lens displacement stage sensor recorded synchronously when capturing the locked-focus image. The measurement accuracy of the objective lens displacement stage sensor used is 1 nm. As can be seen from the figure, the sample continuously drifted axially throughout the experiment, and the drift distance was compensated by 400 nm in 50 minutes. It can be seen that the locking-focus system plays a very important role in single-molecule positioning microscopy that requires long-term imaging.

Claims

1. An axial-stabilized imaging system for a single molecule localization microscope, comprising: The application relates to an optical path for real-time compensation of axial drift of a sample during shooting and a focus camera camZ applied to an inverted microscope, wherein linearly polarized parallel light emitted by an infrared laser is collimated by a lens L1, the polarization direction is adjusted by a half-wave plate H1, and the light is split by a beam splitting cube BS1, so that two beams of light are formed and enter a polarization beam splitting cube PBS in parallel, the light reflected by the polarization beam splitting cube PBS passes through a half-wave plate H2, is focused on the back focal plane of a microscope objective by a lens L2, and is focused by the lens L2, passes through a beam splitting cube BS2, is reflected from a dichroic mirror DM1 into the objective, and is reflected by the sample back into the objective, and the two beams of light returned from the objective pass through the objective, the dichroic mirror DM1, the beam splitting cube BS2 and the lens L2 in sequence, the polarization direction is adjusted by the half-wave plate H2, and then the light is transmitted out of the polarization beam splitting cube PBS, collimated by a lens L3 and enters the focus camera camZ after eliminating stray light by a band-pass filter F1.

2. The system for stabilizing the axial shooting according to claim 1, wherein The beam splitting cube BS1 splits the incident light into projection light and reflected light, and the two beams of light are reflected by two mirrors and enter the polarization beam splitting cube PBS in parallel.

3. The system of claim 1, wherein, The half-wave plate H1 is arranged between the lens L1 and the beam splitting cube BS1 or between the beam splitting cube BS1 and the polarization beam splitting cube PBS.

4. The system of claim 1, wherein, The light beam from the infrared laser is adjusted to S-polarized light by the half-wave plate H1, is reflected by the polarization beam splitting cube PBS after being incident on the polarization beam splitting cube PBS, and is adjusted to P-polarized light by the half-wave plate H2 after being returned from the sample, and is transmitted out of the polarization beam splitting cube PBS after being incident on the polarization beam splitting cube PBS.

5. The system of claim 1, wherein, The infrared laser is a near-infrared laser.

6. An axial-stabilized imaging method for a single molecule localization microscope, characterized in that, The axial stable shooting system is realized by using the axial stable shooting system according to any one of claims 1-5, and comprises the following steps: 1) placing a sample on a three-dimensional nanometer object table of the microscope, adjusting the axial position of the objective to complete focusing, starting the axial stable shooting system, and collecting an image containing two light spots by the focus camera camZ; 2) The image is transmitted to the computer, and the drift between the sample and the objective is calculated by image processing. Then, the three-dimensional nano stage is controlled by PID algorithm to compensate the axial shift. Specifically, the distance between the two spots in the image is calculated by the center positions of the two spots. When the axial locking of the sample is started, the distance between the two spots in the image is recorded by the computer program D 0; the distance between the two spots in the image is calculated every time a new image is generated D and D 0 is recorded as Δ x . With Δ x as the input error, the three-axis nano stage is controlled by incremental PID algorithm to compensate the axial shift until Δ x is less than or equal to the allowed error Δ e .

7. The method of stabilizing an image in an axial direction according to claim 6, wherein The flow of the computer program in step 2) is as follows: a) acquiring an original image and performing binaryzation processing, then performing morphological opening operation and closing operation processing on the binary image to filter out noise points and holes; b) calculating the number and area of all connected domains, if the number of connected domains is less than two, then the next frame of image is transmitted, if the number of connected domains is greater than or equal to two, then the two connected domains S1 and S2 with the largest area are selected, if the areas of the connected domains S1 and S2 are greater than or equal to a preset threshold, then the next step is continued, otherwise, the next frame of image is transmitted; c) calculating the centroid coordinates of the connected domains S1 and S2 according to the following formula: In the above formula, C x and C y These are the x and y coordinates of the centroid, respectively. i Indicates the pixel index in the connected component. n This represents the total number of pixels in the connected component. x i and y i They represent the first and second connected components, respectively. i The x and y coordinates of each pixel represent p i The first connected component i The pixel value of each pixel; d) taking the centroid coordinates of the connected domains S1 and S2 as the center to cut two rectangular image blocks B1 and B2 with the size of W*H pixels in the image, and using an elliptical Gaussian fitting model to calculate the center coordinates of the image blocks B1 and B2 respectively, and the formula is as follows: wherein, A 0 and L are the spot light intensity and the background intensity, respectively; x and y are the horizontal and vertical coordinates of the pixels involved in the fitting, respectively; x c and y c are the fitted center coordinates, whose initial values are C x and C y ; is the parameter of the elliptical Gaussian model, i.e. ; e) calculating the distance between the image blocks B1 and B2 from their center coordinates D thereby obtaining the sample axial drift error wherein D 0 is the initial centroid distance recorded when the lock focus is switched on; f) determining whether the absolute value of the axial drift error Δ x is less than or equal to a preset allowable value Δ e If the requirement is not met, the axial position of the displacement table is controlled using an incremental PID algorithm, the formula is as follows: wherein, K p , K i and K d denote a proportional coefficient, an integral coefficient and a derivative coefficient, respectively, Δ x k denotes the axial drift error of the first k frame, u k denotes the control algorithm output quantity for driving the stage to make a correction.​​ 8. The method of stabilizing an image in an axial direction according to claim 7, wherein d) using a maximum likelihood estimation method based on Poisson distribution noise to calculate the parameters of the elliptical Gaussian fitting model, and the formula is as follows: wherein, n is a pixel index number; N is a pixel number; is an elliptical Gaussian fitting model; z n is a gray value of the n th pixel; is a maximum likelihood estimate value calculated at the parameter .

Images