Fpm super-resolution microscope system based on a rotating lens and marker positioning

Abstract

The application discloses an FPM super-resolution microscope system based on a rotating lens and a mark positioning, which comprises a rotating mechanism with a pitch angle adjusting part, a stage with a positioning mark part and loaded with a sample, and an objective lens and an electronic eyepiece integratedly installed on the rotating mechanism and rotating with the rotating mechanism; wherein: the central axis of the rotating mechanism vertically penetrates the center of the stage, the rotating mechanism rotates around the central axis to make the objective lens and the electronic eyepiece make a circular motion with the central axis as the center and form an imaging field of view, and at least four positioning marks in the positioning mark part are located in the imaging field of view. Through the designed rotating mechanism carrying the objective lens and the electronic eyepiece, the electronic eyepiece can rotate around the central axis in the vertical direction of the observation target, while keeping shooting the observation target on the stage, and a series of target images under different shooting angles are obtained.

Description

Technical Field

[0001] This invention relates to the technical field of FPM super-resolution microscopy, and more specifically to an FPM super-resolution microscopy system based on rotating lenses and marker positioning. Background Technology

[0002] Existing Fourier Transform (FPM) super-resolution microscopy systems employ programmable LED arrays for illumination at different angles. Based on the fact that LEDs at different angles and positions produce different imaging results, multiple low-resolution images from different angles are acquired. In the spatial domain, the recorded light intensity information is used as amplitude constraints, and in the frequency domain, the system's cutoff frequency is used as constraints. Through alternating iterations in the spatial and frequency domains, these images are fused in the spectrum to obtain a high-resolution image with a large field of view and quantitative phase. Therefore, FPM technology uses a phase retrieval iterative algorithm to find a unique complex solution that satisfies the constraints of multiple far-field diffraction intensity images in an overlapping scanning mode, resulting in the super-resolution imaging result.

[0003] Existing FPM technology uses programmable LED arrays for illumination at different angles. The light intensity emitted by the LED beads is angle-dependent, decreasing rapidly as the angle increases. This inconsistency in light intensity across different illumination angles leads to a decline in reconstructed image quality. Therefore, during phase iterative inversion calculations, light intensity correction is necessary for images captured under different illumination angles. In practice, setting lighting conditions is quite stringent. Besides being susceptible to interference from natural light, there are also interference factors affecting algorithm performance, such as occlusion and astigmatism caused by unoptimized structures. Consequently, existing technologies are primarily used in scientific research, with limited commercialization potential.

[0004] Furthermore, FPM super-resolution synthesis essentially still utilizes redundant information to recover missing information from high-resolution images. Existing FPM technology maintains a constant lens acquisition angle, and the redundant information it utilizes mainly consists of the overlap of different spectral positions, i.e., it primarily uses frequency domain information. There is no suitable spatial domain redundant information to utilize, so overall, the amount of missing information that can be supplemented in high-resolution images is limited. The super-resolution magnification and image quality that can be recovered are also limited. Summary of the Invention

[0005] To address the problems of the existing technology, this invention provides an FPM super-resolution microscope system based on a rotating lens and marker positioning. Building upon the base and illumination design of a traditional optical microscope, it employs an adjustable pitch-angle rotating mechanism and a stage with positioning marks to create a novel FPM super-resolution microscope. Positioning marks on the stage ensure that at least four marks are within the imaging field of view of the objective lens. The objective lens and electronic eyepiece are integrated into a two-degree-of-freedom rotating mechanism. The rotation axis is located vertically above the center of the stage. The electrically controlled rotation axis allows the lens and electronic eyepiece mounted on the rotating arm to perform circular motion. Simultaneously, the rotating arm is designed in an arc shape, allowing adjustment of the mounting position to change the pitch angle of the observed target, achieving two-degree-of-freedom rotational control of both pitch and circumferential angles.

[0006] To achieve the above objectives, the present invention employs an FPM super-resolution microscope system based on a rotating lens and marker positioning, comprising:

[0007] A rotating mechanism with pitch adjustment;

[0008] A stage with positioning markers and carrying samples;

[0009] The objective lens and electronic eyepiece are integrated and mounted on the aforementioned rotating mechanism and rotate with the rotating mechanism; wherein:

[0010] The central axis of the aforementioned rotating mechanism passes vertically through the center of the stage. The rotating mechanism rotates around the central axis to make the objective lens and the electronic eyepiece move in a circle around the central axis and form an imaging field of view. At least four positioning marks in the aforementioned positioning mark portion are located within the aforementioned imaging field of view.

[0011] As a further optimization of the above solution, the pitch angle adjustment part includes an electrically controlled rotating shaft and a rotating arm, wherein the rotating arm is arc-shaped and angle-adjustable and is mounted on the electrically controlled rotating shaft.

[0012] As a further optimization of the above solution, one end of the rotating arm is mounted on the electrically controlled rotating shaft and rotates with the electrically controlled rotating shaft. An arc-shaped hole is opened on the side of the rotating arm. The objective lens and the electronic eyepiece are arranged on the support frame. The sliding part on the support frame is slidably connected to the arc-shaped hole. A positioning part is provided between the sliding part and the arc-shaped hole.

[0013] As a further optimization of the above scheme, the sample is rectangular, and the positioning mark part includes eight positioning marks, which are equally spaced on the outer edge of the sample.

[0014] Includes the following steps:

[0015] S1. Acquire at least one set of raw low-resolution images using an FPM super-resolution microscopy system based on rotating lens and marker positioning;

[0016] S2. Match the positioning markers based on the preset image matching algorithm and obtain the imaging image of the corresponding vertical viewpoint;

[0017] S3. Based on the imaging image, a high-resolution complex amplitude in the spatial domain is synthesized using an image super-resolution reconstruction algorithm;

[0018] S4. The above-mentioned high-resolution complex amplitude in the spatial domain is based on the preset FPM algorithm to obtain a high-resolution object intensity image.

[0019] As a further optimization of the above scheme, the preset image matching algorithm in step S1 includes the following steps:

[0020] S101. Identify the location information of the positioning markers in each frame of the original low-resolution image;

[0021] S102. Calculate the position information of the above positioning marks and obtain the projection transformation matrix of the corresponding vertical viewpoint;

[0022] S103. Pixel equivalent transformation of each frame of the original low-resolution image based on near-end imaging, and resolution magnification of each frame of the original low-resolution image to obtain the corresponding vertical view imaging image.

[0023] The FPM super-resolution microscope system based on rotating lens and marker positioning of the present invention has the following advantages:

[0024] This invention relates to an FPM super-resolution microscope system based on a rotating lens and marker positioning. The rotating lens achieves super-resolution microscopic imaging using FPM technology. The rotating lens design allows for tilted acquisition of the target surface, creating different spectral information similar to that of an LED array illuminated at different angles. Furthermore, it provides a larger pixel equivalent for imaging at close range. Simultaneously, because the lens rotates around its central axis to acquire target surface information from different angles, it obtains redundant information in the spatial domain; for example, it captures more pixels per unit length at close range along a circumference. Therefore, the rotating lens design can acquire redundant information in both the spatial and frequency domains, resulting in a greater super-resolution multiplier and superior quality.

[0025] Furthermore, the positioning markers are designed on the stage to keep them within the imaging field of view during lens rotation. Similar to the light intensity correction required by existing FPM technology, the use of spatial redundancy information also requires a spatial position matching process. To avoid hysteresis or backlash errors caused by mechanical movement, a positioning marker design was chosen instead of a precision rotation control system. An image feature matching algorithm is used to obtain the positional relationship between the rotated image sequences, thereby making more effective use of spatial redundancy information.

[0026] Specific embodiments of the present invention are disclosed in detail with reference to the following description and accompanying drawings, indicating how the principles of the present invention can be adopted. It should be understood that the embodiments of the present invention are not limited in scope as a result, and the embodiments of the present invention include many changes, modifications and equivalents. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the front view structure of an FPM super-resolution microscope system based on rotating lenses and marker positioning.

[0028] Figure 2 A top view of the FPM super-resolution microscope system based on rotating lens and marker positioning.

[0029] Figure 3 This is a schematic diagram of image transformation processing in this invention;

[0030] Figure 4 This is a top view of the rotating mechanism in this invention.

[0031] Figure 5 This is a schematic diagram of the main structure of the rotating arm in this invention;

[0032] Figure 6 This is a schematic diagram of the workflow of an FPM super-resolution microscope system based on rotating lenses and marker positioning.

[0033] In the figure: 1. Rotation mechanism; 2. Positioning mark part; 3. Sample; 4. Stage; 5. Objective lens; 6. Electronic eyepiece; 7. Imaging field of view; 8. Support frame; 9. Positioning component; 11. Electrically controlled rotating shaft; 12. Rotating arm; 21. Positioning mark; 81. Sliding component; 121. Arc-shaped hole. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of the invention.

[0035] It should be noted that when an element is referred to as "set on" or "provided with" another element, it can be directly on the other element or there may be an intermediate element. When an element is referred to as "connected to" or "connected to" another element, it can be directly connected to the other element or there may be an intermediate element at the same time. "Fixed connection" means fixed connection. There are many ways of fixed connection, which are not within the scope of protection of this document. The terms "vertical", "horizontal", "left", "right" and similar expressions used in this document are only for illustrative purposes and do not represent the only implementation method.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in the specification herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0037] Please refer to the instruction manual appendix. Figure 1-6 This invention provides a technical solution: an FPM super-resolution microscope system based on rotating lenses and marker positioning, comprising:

[0038] Rotating mechanism 1 with pitch adjustment section;

[0039] A stage 4 with positioning marker 2 and carrying sample 3;

[0040] Objective lens 5 and electronic eyepiece 6 are integrated and mounted on the aforementioned rotating mechanism 1 and rotate with the rotating mechanism 1; wherein:

[0041] The central axis of the aforementioned rotating mechanism 1 passes vertically through the center of the stage 4. The aforementioned rotating mechanism 1 rotates around the central axis so that the objective lens 5 and the electronic eyepiece 6 make circular motion around the central axis and form an imaging field of view 7. At least four positioning marks 21 in the aforementioned positioning mark portion 2 are located within the aforementioned imaging field of view 7.

[0042] The pitch angle adjustment part includes an electrically controlled rotating shaft 11 and a rotating arm 12. The rotating arm 12 is arc-shaped and its angle is adjustable, and it is mounted on the electrically controlled rotating shaft 11.

[0043] For the pitch angle adjustment of the objective lens 5 and the electronic eyepiece 6 mentioned above, this embodiment adopts a manual control method, wherein:

[0044] In the manual control mode, one end of the aforementioned rotating arm 12 is mounted on the electrically controlled rotating shaft 11 and rotates with the electrically controlled rotating shaft 11. An arc-shaped hole 121 is provided on the side of the rotating arm 12. The objective lens 5 and the electronic eyepiece 6 are arranged on the support frame 8. The sliding member 81 on the support frame 8 is slidably connected to the arc-shaped hole 121. A positioning member 9 is provided between the sliding member 81 and the arc-shaped hole 121, so that the pitch angle can be manually adjusted by manually adjusting the position of the sliding member 81 on the arc-shaped hole 121. Of course, the position changes of the aforementioned objective lens 5 and electronic eyepiece 6 can also be achieved by other means.

[0045] For the aforementioned positioning element 9, preferably, a clamping bolt is selected. The bolt of the clamping bolt includes a threaded part and a sliding part. The outer end of the clamping bolt is a clamping head with a diameter greater than the width of the arc-shaped hole. The sliding part is the sliding element 81 in the above embodiment. The threaded part is threadedly connected to the support frame 8. When it is necessary to position the positioning element 9, the positioning element 9 is rotated inward so that the clamping head of the positioning element 9 is pressed against the rotating arm 12 to achieve the purpose of positioning the aforementioned support frame 8.

[0046] Of course, the pitch angle adjustment of the aforementioned objective lens 5 and electronic eyepiece 6 can also be designed as an automatic control structure.

[0047] The sample 3 is rectangular, and the positioning mark part 2 includes eight positioning marks 21, which are arranged at equal intervals on the outer edge of the sample 3.

[0048] refer to Figure 6 The workflow of the FPM super-resolution microscope system based on rotating lens and marker positioning described above is as follows:

[0049] 1) Power on the system, place the target sample 3 (in this embodiment, the sample 3 is a glass slide), connect the electronic eyepiece 6, and turn on the light source;

[0050] 2) Adjust the lens magnification, pitch angle, and field of view; set the camera frame rate parameters and rotation mechanism speed.

[0051] 3) Drive the rotation, and the camera acquires a sequence of images;

[0052] 4) FPM compositing is performed using computer-based software;

[0053] 5) Obtain the final super-resolution image and display it on the software interface.

[0054] In summary, this invention utilizes a designed rotating mechanism to mount the objective lens 5 and the electronic eyepiece 6. The electronic eyepiece 6 can rotate around the vertical axis of the observed target while simultaneously capturing images of the target on the stage 4, thus obtaining a series of target images at different shooting angles. Furthermore, positioning marks 21 are designed on the stage 4 for positioning and matching between images at different angles, thereby forming a sequence of low-resolution images containing different spatial and frequency domain redundancy information generated from images captured at different angles. Finally, computer software uses the Fourier Transform Imaging (FPM) algorithm to generate high-resolution images with a wide field of view, achieving super-resolution microscopy.

[0055] The FPM super-resolution microscopy method based on rotating lens and marker positioning is characterized by the following steps:

[0056] S1. An FPM super-resolution microscope system based on rotating lens and marker positioning acquires at least one set of raw low-resolution images; preferably, multiple sets of raw low-resolution images correspond to images taken at different angles of tilt.

[0057] The preset image matching algorithm includes the following steps:

[0058] S101. Identify the position information of the positioning markers in each frame of the original low-resolution image. Since the position information of the positioning markers is obtained by the FPM super-resolution microscope system based on rotating lens and marker positioning in this invention, there are at least four positions of the positioning markers in this step S101.

[0059] S102. Calculate the position information of the above positioning marks and obtain the projection transformation matrix of the corresponding vertical viewpoint. ;

[0060] S103, see details Figure 3 ,Should Figure 3 The image transformation process is shown, which involves pixel equivalent transformation of each frame of the original low-resolution image based on near-end imaging, and then resolution magnification of each frame of the original low-resolution image to obtain the corresponding vertical view imaging image.

[0061] S2. Match the positioning markers based on the preset image matching algorithm and obtain the imaging image of the corresponding vertical viewpoint;

[0062] S3. Based on the imaging image, a high-resolution complex amplitude in the spatial domain is synthesized using an image super-resolution reconstruction algorithm; preferably, the initial value of this high-resolution complex amplitude in the spatial domain is an initial value. ;

[0063] S4. The above-mentioned high-resolution complex amplitude in the spatial domain is based on the preset FPM algorithm to obtain a high-resolution object intensity image.

[0064] The preset FPM algorithm includes the following steps:

[0065] S401. The above-mentioned high-resolution complex amplitude in the spatial domain is obtained by using a preset aperture truncation algorithm to obtain the simulated spectrum. The simulated spectrum is then propagated to the sensor plane through Fourier transform and simulated pattern information is generated.

[0066] Furthermore, in step S401, the aforementioned simulated spectrum is... Each camera position Each has its own analog spectrum. .

[0067] The preset aperture interception algorithm in step S401 above is as follows:

[0068]

[0069] in: This is denoted as the lens aperture. It is denoted as the high-resolution complex amplitude in the spatial domain.

[0070] S402. Based on the simulated pattern information, retain the phase information in the simulated pattern information on the sensor plane, replace the simulated intensity information in the simulated pattern information with the acquired intensity information and obtain the actual pattern information, and propagate the actual pattern information to the Fourier domain through Fourier transform.

[0071] S403. Propagate the above actual pattern information to the Fourier domain through Fourier transform and replace the corresponding position spectrum, and update the Fourier domain based on the preset update method.

[0072] In step S403 above, the actual pattern information is propagated to the Fourier domain through Fourier transform, and the specific formula is as follows:

[0073]

[0074] in: Record the acquisition intensity information at the corresponding location.

[0075] S404. The above Fourier domain is iteratively updated using a preset iterative algorithm until a preset number of iterations is reached or the error is less than a preset threshold, at which point the iteration stops. The difference between continuous estimates is used to update the Fourier domain. and And use similar update steps to update the aperture function;

[0076] The preset iterative algorithm in step S404 above is:

[0077]

[0078] .

[0079] S405. Obtain the spectrum after iteration and perform inverse Fourier transform to obtain the complex amplitude information of the reconstructed object in the spatial domain. Based on the square of the modulus of the complex amplitude information of the reconstructed object, obtain the high-resolution object intensity image.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An FPM super-resolution microscope system based on rotating lens and positioning markers, characterized in that, include: A rotating mechanism with pitch adjustment; A stage with positioning markers and carrying the target sample; The objective lens and the electronic eyepiece are integrated and mounted on the aforementioned rotating mechanism and rotate with the rotating mechanism; wherein: the central axis of the aforementioned rotating mechanism passes vertically through the center of the stage, the aforementioned rotating mechanism rotates around the central axis so that the objective lens and the electronic eyepiece make circular motion around the central axis and form an imaging field of view, the electronic eyepiece takes pictures of the target sample on the stage, and obtains a series of target images at different shooting angles, and at least four positioning marks in the aforementioned positioning mark portion are located within the aforementioned imaging field of view. The pitch angle adjustment part includes an electrically controlled rotating shaft and a rotating arm. The rotating arm is arc-shaped and its angle is adjustable, and it is mounted on the electrically controlled rotating shaft. One end of the aforementioned rotating arm is mounted on an electrically controlled rotating shaft and rotates with the electrically controlled rotating shaft. An arc-shaped hole is provided on the side of the rotating arm. The objective lens and electronic eyepiece are arranged on a support frame. The sliding component on the support frame is slidably connected to the arc-shaped hole. A positioning component is provided between the sliding component and the arc-shaped hole. The FPM super-resolution microscope system based on rotating lens and positioning markers enables tilted acquisition of target samples, obtaining redundant information in both spatial and frequency domains.

2. The FPM super-resolution microscope system based on rotating lens and positioning markers according to claim 1, characterized in that: The target sample is rectangular, and the positioning mark portion includes eight positioning marks, which are equally spaced on the outer edge of the target sample.

Images