Bimodal microscopic imaging system and method

Abstract

The invention provides a bimodal microscopic imaging system and method, and relates to the technical field of microscopic imaging, and the system comprises a first imaging module which is configured to obtain a three-dimensional refractive index distribution image of a sample based on an optical diffraction tomography principle; the second imaging module is configured to acquire a fluorescence image of the sample based on a confocal microscopic imaging principle; the control module comprises a time sequence control unit and an image fusion unit; the time sequence control unit is configured to control the data acquisition time sequence of the first imaging module and the second imaging module; the image fusion unit is configured to process the three-dimensional refractive index distribution image and the fluorescence image to generate a fused image. According to the invention, unmarked three-dimensional form imaging and specific fluorescence imaging of living cells are realized, distribution of organelles in the three-dimensional form of the cells can be accurately positioned through bimodal data fusion, and linkage analysis of cell functions and forms is carried out.

Description

Technical Field

[0001] This application relates to the field of microscopic imaging technology, and in particular to a dual-modal microscopic imaging system and method. Background Technology

[0002] Optical microscopy, as an important observational tool, has facilitated human exploration of life activities. Traditional wide-field optical microscopes support the observation of most common life science samples, but their resolution cannot overcome the physical limitation of the Abbe diffraction limit, nor can they perform three-dimensional imaging of samples. Traditional microscopes can only observe cellular-scale structures and struggle to observe smaller subcellular scale structures, namely organelles, making it difficult to further improve the resolution and contrast of their imaging results. Summary of the Invention

[0003] This application provides a dual-modal microscopy imaging system and method to solve the technical problem that traditional microscopes are unable to observe smaller subcellular sizes, resulting in difficulty in improving the resolution and contrast of imaging results.

[0004] In a first aspect, this application provides a dual-modal microscopic imaging system, comprising: The first imaging module is configured to acquire a three-dimensional refractive index distribution image of the sample based on the principle of optical diffraction tomography. The second imaging module is configured to acquire fluorescence images of the sample based on the principle of confocal microscopy. The control module includes a timing control unit and an image fusion unit; the timing control unit is configured to control the data acquisition timing of the first imaging module and the second imaging module; the image fusion unit is configured to process the three-dimensional refractive index distribution image and the fluorescence image to generate a fused image.

[0005] Secondly, this application also provides a dual-modal microscopic imaging method, comprising: The three-dimensional refractive index distribution image of the sample is obtained based on the first imaging module; The fluorescence image of the sample is acquired based on the second imaging module; The control module controls the data acquisition timing of the first imaging module and the second imaging module. The control module processes the three-dimensional refractive index distribution image and the fluorescence image to generate a fused image.

[0006] The dual-modal microscopy imaging system and method provided in this application combine the advantages of ODT and confocal microscopy, while overcoming the disadvantages of ODT's lack of specificity and confocal microscopy's slow imaging speed and susceptibility to photobleaching. It achieves label-free three-dimensional morphological imaging and specific fluorescence imaging of live cells, enabling cell analysis from both structural and functional dimensions. Through dual-modal data fusion, the distribution of organelles in the three-dimensional morphology of cells can be accurately located, allowing for the linked analysis of cell function and morphology. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the structure of the dual-modal microscopic imaging system provided in the embodiments of this application; Figure 2 This is one of the optical path schematic diagrams of the dual-modal microscopic imaging system provided in the embodiments of this application; Figure 3 This is the second schematic diagram of the optical path of the dual-modal microscopic imaging system provided in the embodiments of this application; Figure 4 This is the third schematic diagram of the optical path of the dual-modal microscopic imaging system provided in the embodiments of this application; Figure 5 This is the fourth schematic diagram of the optical path of the dual-modal microscopic imaging system provided in the embodiments of this application; Figure 6 This is a schematic diagram of the timing control of the dual-modal microscopic imaging system provided in the embodiments of this application; Figure 7 This is a schematic diagram of the image registration process of the dual-modal microscopic imaging system provided in the embodiments of this application; Figure 8 This is a schematic flowchart of the dual-modal microscopic imaging method provided in the embodiments of this application; Figure label: First imaging module 1 for D-type mirror beam combining; First imaging module 2 for cubic crystal beam combining; Second imaging module 3 for laser scanning; Second imaging module 4 for single-disc confocal focusing; Second imaging module 5 for dual-disc confocal focusing; First objective lens; First dichroic mirror; First reflecting mirror; First laser source 1-1; First fiber optic flange 1-2; First lens 1-3; Second reflecting mirror 1-4; First dual-axis scanning galvanometer 1-5; Second dichroic mirror 1-6; Second lens 1-7; Second objective lens 1-8; Third lens 1-9; Fourth lens 1-10 Second fiber optic flange 1-11; Fifth lens 1-12; Third reflector 1-13; Sixth lens 1-14; D-type mirror 1-15; Seventh lens 1-16; First camera 1-17; Second laser source 2-1; Third fiber optic flange 2-2; Eighth lens 2-3; Fourth reflector 2-4; Second dual-axis scanning galvanometer 2-5; Third dichroic mirror 2-6; Ninth lens 2-7; Third objective lens 2-8; Tenth lens 2-9; Fourth fiber optic flange 2-10; Eleventh lens 2-11; Fifth reflector 2-12; Cubic crystal 2-13; Twelfth lens 2-14; Thirteenth lens 2-15; Second camera 2-16; Third laser source 3-1; Fifth fiber optic flange 3-2; Fourteenth lens 3-3; First pinhole 3-4; Collimating lens 3-5; First filter 3-6; Fourth dichroic mirror 3-7; Scanning galvanometer 3-8; Fifteenth lens 3-9; Sixteenth lens 3-10; Second filter 3-11; Seventeenth lens 3-12; Second pinhole 3-13; Third camera 3-14; Fourth laser source 4-1; Sixth fiber optic flange 4-2; 18th lens 4-3; 5th dichroic mirror 4-4; 1st pinhole turntable 4-5; 19th lens 4-6; 20th lens 4-7; 21st lens 4-8; 3rd filter 4-9; 4th camera 4-10; 5th laser source 5-1; 7th fiber optic flange 5-2; 22nd lens 5-3; microlens array turntable 5-4; 6th dichroic mirror 5-5; 2nd pinhole turntable 5-6; 23rd lens 5-7; 24th lens 5-8; 25th lens 5-9; 4th filter 5-10; 5th camera 5-11. Detailed Implementation

[0008] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0009] Optical diffraction tomography (ODT) is a label-free, super-resolution three-dimensional microscopy technique. By combining quantitative phase and optical field diffraction models, it uses digital holography from different illumination angles to record the amplitude and phase information of the emitted (transmitted) light field, enabling continuous sampling of the sample's spatial frequency domain. Subsequently, by combining backscattering or inverse diffraction processes with ODT algorithms, the three-dimensional distribution of the sample's refractive index is precisely reconstructed. It has already achieved non-invasive, label-free three-dimensional imaging of living cells at high spatiotemporal resolution and long-term imaging to observe life processes. However, ODT also has its limitations. It is a non-specific imaging technique, only able to acquire the three-dimensional refractive index distribution of the sample, thus allowing for the identification and analysis of biological structures based on morphological features. When discovering organelle structures and life activities at higher resolutions that are not yet fully understood, further accurate analysis is difficult, and the lack of chemical specificity in morphological characterization limits its persuasiveness.

[0010] Confocal microscopy is a high-resolution fluorescence imaging technique based on the point scanning principle. It possesses chemical specificity, enabling the specific labeling of chemical molecules. By placing a confocal pinhole in front of the detector, it effectively suppresses out-of-focus blur signals, thereby significantly improving image contrast and signal-to-noise ratio, and also possesses optical tomography capabilities. Laser scanning confocal microscopy (LSCM) and spinning-disk confocal microscopy (SDCM) are two mainstream implementation methods.

[0011] Laser scanning confocal microscopy employs a single-point beam for point-to-point scanning, achieving high spatial resolution. However, its imaging rate is limited by the scanning mechanism, making it difficult to meet the observation requirements of high-speed, dynamic biological processes. To improve imaging speed, rotating disk confocal microscopy emerged, using a microlens disk to achieve multi-point parallel scanning, significantly improving temporal resolution. However, while this parallel strategy increases speed, it also leads to a decrease in spatial resolution (especially axial resolution) and may introduce artifacts due to uneven distribution of illumination spots.

[0012] Optical diffraction tomography (ODT) and confocal microscopy offer complementary advantages. Confocal microscopy allows for fluorescent labeling of specific molecules, overcoming the lack of specificity in ODT. Furthermore, ODT enables label-free 3D morphological imaging at high temporal resolution, mitigating the phototoxicity and photobleaching associated with fluorescent labeling in confocal imaging. Combining full-time ODT imaging with interval-based laser scanning confocal microscopy effectively addresses the short imaging time issues caused by fluorescence phototoxicity and photobleaching. Both ODT and rotating confocal microscopy offer high temporal resolution, enabling continuous imaging of live cells with both label-free and specific capabilities. The acquired label-free and fluorescent dual-modal data can be further registered in 3D to obtain fused label-free and fluorescent dual-modal 3D data, aiding in the discovery and verification of previously unknown life phenomena.

[0013] Figure 1 This is a schematic diagram of the structure of the dual-modal microscopic imaging system provided in the embodiments of this application, as shown below. Figure 1 As shown, the dual-modal microscopy imaging system includes: a first imaging module, a second imaging module, and a control module. The first imaging module is configured to acquire a three-dimensional refractive index distribution image of the sample based on the principle of optical diffraction tomography. The second imaging module is configured to acquire a fluorescence image of the sample based on the principle of confocal microscopy. The control module includes a timing control unit and an image fusion unit. The timing control unit is configured to control the data acquisition timing of the first and second imaging modules. The image fusion unit is configured to process the three-dimensional refractive index distribution image and the fluorescence image to generate a fused image, which is then used for quantitative analysis of cell structure and function.

[0014] The first imaging module is based on the principle of optical diffraction tomography. It obtains the phase information of the sample through multi-angle illumination and off-axis holographic detection, and reconstructs the three-dimensional refractive index distribution image of the sample by combining tomographic reconstruction algorithm.

[0015] In some embodiments, the first imaging module specifically includes: The scanning illumination unit is configured to illuminate the sample from multiple angles using a scanning coherent beam. An imaging detection unit includes a beam combiner and an image detector; the beam combiner is configured to split a coherent beam into an object beam and a reference beam, and combine the object beam and the reference beam after passing through the sample to form interference fringes; the image detector is configured to acquire the interference fringes and generate a holographic image. The reconstruction unit is configured to reconstruct the three-dimensional refractive index distribution of the sample based on the holographic image.

[0016] Specifically, depending on the type of beam combining element, the first imaging module may include: a first imaging module 1 for D-type mirror beam combining or a first imaging module 2 for cubic crystal beam combining.

[0017] D-type mirror beam combiners and cubic crystal beam combiners each have their advantages and application considerations. D-type mirror beam combiners achieve beam combining by depositing a high-reflectivity film of a specific wavelength on the substrate. The optical path setup is flexible, and the beam combining angle and position can be finely adjusted using a precision adjustment frame. For imaging systems with large fields of view and large light spots, there is no need to increase the size of the element, resulting in lower cost and wider applicability. Cubic crystal beam combiners, on the other hand, utilize the polarization beam splitter inside the crystal for beam combining. They have more reflective interfaces, which can easily introduce stray light. However, for high-power lasers, cubic crystal parallel beam combining has a lower damage threshold requirement compared to D-type mirror focusing beam combining.

[0018] After the holographic image is acquired, the reconstruction module uses the ODT reconstruction algorithm, employing the Rytov approximation and a Fourier diffraction theorem algorithm based on angular spectrum propagation theory, to quickly reconstruct the three-dimensional refractive index distribution map of the cell at the current moment. This result directly reflects the overall three-dimensional morphology of the cell, its dry mass distribution, and the label-free structural information of its internal organelles.

[0019] The second imaging module is based on the principle of confocal microscopy and achieves confocal fluorescence imaging of the sample through a galvanometer or rotating pinhole array, which has chemical specificity.

[0020] In some embodiments, depending on the type of confocal imaging, the second imaging module may specifically include: a laser scanning second imaging module 3 and a turntable confocal second imaging module. The turntable confocal second imaging module may be further divided into a single turntable confocal second imaging module 4 or a dual turntable confocal second imaging module 5.

[0021] In some embodiments, the dual-modal microscopy imaging system further includes a first objective lens 6, a first dichroic mirror 7, and a first reflecting mirror 8. The first objective lens 6, the first dichroic mirror 7, and the first reflecting mirror 8 are key components for optical path coupling and sample collection. The first imaging module and the second imaging module achieve optical path coupling through the same set of objectives and dichroic mirrors, enabling seamless switching or synchronous acquisition on the same platform.

[0022] The following section introduces several ways to combine the first imaging module and the second imaging module.

[0023] Figure 2 This is one of the optical path schematic diagrams of the dual-modal microscopic imaging system provided in the embodiments of this application, such as... Figure 2 As shown, the dual-modal microscopic imaging system includes: a first imaging module 1 for D-type mirror beam combining, a second imaging module 4 for single-rotor confocal focusing, a first objective lens 6, a first dichroic mirror 7, a first reflecting mirror 8, and a control module ( Figure 2 (Not shown in the image).

[0024] The optical path of the first imaging module 1 of the D-type mirror beam combiner includes: a first laser source 1-1, a first fiber optic flange 1-2, a first lens 1-3, a second reflector 1-4, a first dual-axis scanning galvanometer 1-5, a second dichroic mirror 1-6, a second lens 1-7, a second objective lens 1-8, a third lens 1-9, a fourth lens 1-10, a second fiber optic flange 1-11, a fifth lens 1-12, a third reflector 1-13, a sixth lens 1-14, a D-type mirror 1-15, a seventh lens 1-16, and a first camera 1-17.

[0025] The scanning illumination unit comprises a first laser source 1-1, a first fiber optic flange 1-2, a first lens 1-3, a second reflector 1-4, a first dual-axis scanning galvanometer 1-5, a second dichroic mirror 1-6, a second lens 1-7, and a second objective lens 1-8. A 532nm wavelength laser beam is emitted from the first laser source 1-1, enters the first fiber optic flange 1-2, exits from the second fiber optic flange 1-11, is collimated by the first lens 1-3, and then reflected by the second reflector 1-4 into the first dual-axis scanning galvanometer 1-5. The scanning galvanometer 1-5 controls the deflection of the laser beam in the X and Y directions through a fast-deflecting mirror. After the laser beam is emitted, it is reflected by the second dichroic mirror 1-6 into the vertically downward optical path. It passes through the first relay objective system composed of the second lens 1-7 and the lower second objective lens 1-8. That is, the first dual-axis scanning galvanometer 1-5 is located on the front focal plane of the first relay objective system. The emitted parallel beam is focused by the second lens 1-7, and the focal point is located on the rear focal plane of the second objective lens 1-8. Then, it is emitted as parallel light from the second objective lens 1-8 to illuminate the sample.

[0026] When imaging the sample, the sample surface needs to be placed on the front focal plane of the second objective lens 1-8, so that the plane containing the sample is conjugate to the plane containing the first biaxial scanning galvanometer 1-5. Therefore, the deflection change of the first biaxial scanning galvanometer 1-5 corresponds to the xy-axis movement of the sample plane, realizing the function of acquiring sample information from different illumination angles. After the light beam passes through the sample, the parallel light and the scattered light from the sample structure are collected by the first objective lens 6, and after exiting, they are reflected by the first dichroic mirror 7 into the imaging detection unit.

[0027] The imaging detection unit consists of the third lens 1-9, the fourth lens 1-10, the second fiber optic flange 1-11, the fifth lens 1-12, the third reflecting mirror 1-13, the sixth lens 1-14, the D-type mirror 1-15, the seventh lens 1-16, and the first camera 1-17. The transmitted signal light first passes through the second relay objective system composed of the first objective lens 6 and the third lens 1-9, where it is re-collimated into parallel light; this beam is called the object beam. It then passes through the third relay objective system composed of a pair of lenses (the fourth lens 1-10 and the seventh lens 1-16). The rear focal plane of the second relay objective system is located at the front focal plane of the third relay objective system, and the first camera 1-17 is located at the rear focal plane of the third relay objective system, making the photosensitive plane of the first camera 1-17 and the sample plane conjugate. A second optical fiber is needed as a reference light, which is emitted from the second optical fiber flange 1-11, collimated by the fifth lens 1-12, reflected by the third mirror 1-13, and focused by the sixth lens 1-14 near the Fourier surface of the third relay objective system. A D-type mirror 1-15 is placed at this location to combine the object light and the reference light. The combined light is then imaged by the seventh lens 1-16 and converged on the sensor of the first camera 1-17 to form interference fringes, forming an off-axis hologram to record the phase information of the light beam.

[0028] After data acquisition, the control module invokes the ODT reconstruction algorithm, utilizing the Rytov approximation and a Fourier diffraction theorem algorithm based on angular spectrum propagation theory to quickly reconstruct the three-dimensional refractive index distribution map of the cell at the current moment. This result directly reflects the overall three-dimensional morphology of the cell, its dry mass distribution, and the label-free structural information of its internal organelles.

[0029] After passing through the first dichroic mirror 7, the fluorescence is reflected by the first reflecting mirror 8 and enters the second imaging module 4 of the single-disc confocal focusing system. The optical path of the second imaging module 4 of the single-disc confocal focusing system includes: a fourth laser source 4-1, a sixth fiber optic flange 4-2, an eighteenth lens 4-3, a fifth dichroic mirror 4-4, a first pinhole rotating disk 4-5, a nineteenth lens 4-6, a twentieth lens 4-7, a twenty-first lens 4-8, a third filter 4-9, and a fourth camera 4-10. The fourth laser source 4-1 can use different wavelength channels as needed. After the laser is emitted from the fourth laser source 4-1, it enters the sixth fiber flange 4-2, exits from the sixth fiber flange 4-2, is collimated by the eighteenth lens 4-3, and is reflected by the fifth dichroic mirror 4-4 onto the first pinhole turntable 4-5 for confocal focusing. The laser emitted from the pinhole on the first pinhole turntable 4-5 is focused on the sample by the nineteenth lens 4-6, the first reflecting mirror 8, and the first objective lens 6. The fourth relay objective lens system, composed of the nineteenth lens 4-6 and the first objective lens 6, excites the sample structure at the focal point to generate a fluorescence signal. The signal light is then collected by the fourth relay objective lens system. The effective signal emitted from the sample focal plane can pass smoothly through the original pinhole on the first pinhole turntable 4-5. Interference light emitted from above or below the sample focal plane is mostly blocked by the first pinhole turntable 4-5 and cannot pass through because it is not focused on the pinhole plane. Subsequently, the light propagates through the fifth relay objective system, which consists of the twentieth lens 4-7 and the twentieth lens 4-8, and the third filter 4-9, and is finally captured by the fourth camera 4-10. The image captured by the fourth camera 4-10 is a clear image on the focal plane.

[0030] When imaging the sample, the sample surface is located at the front focal plane of the fourth relay objective system, and the pinholes on the first pinhole turntable 4-5 are located at the rear focal plane of the fourth relay objective system. Each fluorescence excitation point and its corresponding pinhole are conjugate to each other, satisfying the confocal condition. Only the light emitted from that point and focused onto the corresponding detection pinhole is received by the fourth camera 4-10. The pinhole array on the first pinhole turntable 4-5 is arranged at regular intervals in an Archimedean spiral. Rotating the first pinhole turntable 4-5 enables rapid point scanning imaging of the sample surface. After passing through the pinholes on the first pinhole turntable 4-5, the fluorescence passes through the fifth relay objective system, which consists of a pair of lenses (the twentieth lens 4-7 and the twenty-first lens 4-8), making the turntable pinholes and the camera surface conjugate to each other.

[0031] Figure 3 This is a second schematic diagram of the optical path of the dual-modal microscopic imaging system provided in the embodiments of this application, as shown below. Figure 3 As shown, the dual-modal microscopic imaging system includes: a first imaging module 2 for cubic crystal beam combining, a second imaging module 4 for single-disc confocal focusing, a first objective lens 6, a first dichroic mirror 7, a first reflecting mirror 8, and a control module ( Figure 3 (Not shown in the image).

[0032] The optical path of the first imaging module 2 of the cubic crystal beam combiner includes: a second laser source 2-1, a third fiber optic flange 2-2, an eighth lens 2-3, a fourth reflector 2-4, a second dual-axis scanning galvanometer 2-5, a third dichroic mirror 2-6, a ninth lens 2-7, a third objective lens 2-8, a tenth lens 2-9, a fourth fiber optic flange 2-10, an eleventh lens 2-11, a fifth reflector 2-12, a cubic crystal 2-13, a twelfth lens 2-14, a thirteenth lens 2-15, and a second camera 2-16.

[0033] The scanning illumination unit comprises a second laser source 2-1, a third fiber optic flange 2-2, an eighth lens 2-3, a fourth reflector 2-4, a second dual-axis scanning galvanometer 2-5, a third dichroic mirror 2-6, a ninth lens 2-7, and a third objective lens 2-8. A 532nm wavelength laser beam is emitted from the second laser source 2-1, enters the third fiber optic flange 2-2, exits from the fourth fiber optic flange 2-10, is collimated by the eighth lens 2-3, and then reflected by the fourth reflector 2-4 into the second dual-axis scanning galvanometer 2-5. Mirror 2-5 controls the deflection of the laser beam in the X and Y directions through a fast-deflecting mirror. After the laser beam is emitted, it is reflected by the long-wavelength third dichroic mirror 2-6 into the vertically downward optical path. After passing through the first relay objective system composed of the ninth lens 2-7 and the lower third objective lens 2-8, the second dual-axis scanning galvanometer 2-5 is located on the front focal plane of the first relay objective system. The emitted parallel beam is focused by the ninth lens 2-7, with the focal point located on the rear focal plane of the third objective lens 2-8. The parallel beam is then emitted from the third objective lens 2-8 to illuminate the sample.

[0034] When imaging the sample, the sample surface needs to be placed on the front focal plane of the third objective lens 2-8, so that the plane containing the sample is conjugate to the plane containing the second biaxial scanning galvanometer 2-5. Therefore, the deflection change of the second biaxial scanning galvanometer 2-5 corresponds to the xy-axis movement of the sample plane, realizing the function of acquiring sample information from different illumination angles. After the light beam passes through the sample, the parallel light and the scattered light from the sample structure are collected by the first objective lens 6, and after exiting, they are reflected by the first dichroic mirror 7 into the imaging detection unit.

[0035] The imaging and detection unit consists of the tenth lens 2-9, lens 2-10, fourth fiber optic flange 2-10, eleventh lens 2-11, fifth reflecting mirror 2-12, cubic crystal 2-13, twelfth lens 2-14, thirteenth lens 2-15, and second camera 2-16. The transmitted signal light first passes through the lenses of the second relay objective system, composed of the first objective lens 6 and the tenth lens 2-9, and is re-collimated into parallel light; this beam is called the object beam. It then passes through the third relay objective system, composed of the twelfth lens 2-14 and the thirteenth lens 2-15. The rear focal plane of the second relay objective system is located at the front focal plane of the third relay objective system, and the second camera 2-16 is located at the rear focal plane of the third relay objective system, making the photosensitive plane of the second camera 2-16 and the sample plane conjugate. A second optical fiber is needed as a reference beam, which is emitted from the fourth fiber flange 2-10, collimated by the eleventh lens 2-11, and reflected by the fifth mirror 2-12. The collimated parallel light is reflected by the cubic crystal 2-13, and the object beam and reference beam are combined. The combined light is then imaged by the third relay objective lens system composed of the twelfth lens 2-14 and the thirteenth lens 2-15. The light converges on the sensor of the second camera 2-16 to form interference fringes, forming an off-axis hologram to record the phase information of the beam.

[0036] After data acquisition, the control module invokes the ODT reconstruction algorithm, utilizing the Rytov approximation and a Fourier diffraction theorem algorithm based on angular spectrum propagation theory to quickly reconstruct the three-dimensional refractive index distribution map of the cell at the current moment. This result directly reflects the overall three-dimensional morphology of the cell, its dry mass distribution, and the label-free structural information of its internal organelles.

[0037] After passing through the first dichroic mirror 7, the fluorescence is reflected by the first reflecting mirror 8 and enters the second imaging module 4 of the single-disc confocal focusing system. The optical path of the second imaging module 4 of the single-disc confocal focusing system includes: a fourth laser source 4-1, a sixth fiber optic flange 4-2, an eighteenth lens 4-3, a fifth dichroic mirror 4-4, a first pinhole rotating disk 4-5, a nineteenth lens 4-6, a twentieth lens 4-7, a twenty-first lens 4-8, a third filter 4-9, and a fourth camera 4-10. The fourth laser source 4-1 can use different wavelength channels as needed. After the laser is emitted from the fourth laser source 4-1, it enters the sixth fiber flange 4-2, exits from the sixth fiber flange 4-2, is collimated by the eighteenth lens 4-3, and is reflected by the fifth dichroic mirror 4-4 onto the first pinhole turntable 4-5 for confocal focusing. The laser emitted from the pinhole on the first pinhole turntable 4-5 is focused on the sample by the nineteenth lens 4-6, the first reflecting mirror 8, and the first objective lens 6. The fourth relay objective lens system, composed of the nineteenth lens 4-6 and the first objective lens 6, excites the sample structure at the focal point to generate a fluorescence signal. The signal light is then collected by the fourth relay objective lens system and subsequently propagates to the camera surface through the fifth relay objective lens system, composed of the twentieth lens 4-7 and the twenty-first lens 4-8, and the third filter 4-9. Finally, it is captured by the fourth camera 4-10, which captures a clear image on the focal plane.

[0038] When imaging the sample, the sample surface is located at the front focal plane of the fourth relay objective system, and the pinholes on the first pinhole turntable 4-5 are located at the rear focal plane of the fourth relay objective system. Each fluorescence excitation point and its corresponding pinhole are conjugate to each other, satisfying the confocal condition. Only the light emitted from that point and focused onto the corresponding detection pinhole is received by the fourth camera 4-10. The pinhole array on the first pinhole turntable 4-5 is arranged at regular intervals in an Archimedean spiral. Rotating the first pinhole turntable 4-5 enables rapid point scanning imaging of the sample surface. After passing through the pinholes on the first pinhole turntable 4-5, the fluorescence then passes through the fifth relay objective system, which consists of a pair of lenses (the twentieth lens 4-7 and the twenty-first lens 4-8), ensuring that the turntable pinholes and the camera surface are conjugate to each other.

[0039] Figure 4 This is the third schematic diagram of the optical path of the dual-modal microscopic imaging system provided in the embodiments of this application, as shown below. Figure 4 As shown, the dual-modal microscopic imaging system includes: a first imaging module 1 for D-type mirror beam combining, a second imaging module 5 for dual-rotor confocal focusing, a first objective lens 6, a first dichroic mirror 7, a first reflecting mirror 8, and a control module ( Figure 4 (Not shown in the image).

[0040] The optical path of the first imaging module 1 of the D-type mirror beam combiner includes: a first laser source 1-1, a first fiber optic flange 1-2, a first lens 1-3, a second reflector 1-4, a first dual-axis scanning galvanometer 1-5, a second dichroic mirror 1-6, a second lens 1-7, a second objective lens 1-8, a third lens 1-9, a fourth lens 1-10, a second fiber optic flange 1-11, a fifth lens 1-12, a third reflector 1-13, a sixth lens 1-14, a D-type mirror 1-15, a seventh lens 1-16, and a first camera 1-17.

[0041] The scanning illumination unit comprises a first laser source 1-1, a first fiber optic flange 1-2, a first lens 1-3, a second reflector 1-4, a first dual-axis scanning galvanometer 1-5, a second dichroic mirror 1-6, a second lens 1-7, and a second objective lens 1-8. A 532nm wavelength laser beam is emitted from the first laser source 1-1, enters the first fiber optic flange 1-2, exits from the second fiber optic flange 1-11, is collimated by the first lens 1-3, and then reflected by the second reflector 1-4 into the first dual-axis scanning galvanometer 1-5. The scanning galvanometer 1-5 controls the deflection of the laser beam in the X and Y directions through a fast-deflecting mirror. After the laser beam is emitted, it is reflected by the second dichroic mirror 1-6 into the vertically downward optical path. It passes through the first relay objective system composed of the second lens 1-7 and the lower second objective lens 1-8. That is, the first dual-axis scanning galvanometer 1-5 is located on the front focal plane of the first relay objective system. The emitted parallel beam is focused by the second lens 1-7, and the focal point is located on the rear focal plane of the second objective lens 1-8. Then, it is emitted as parallel light from the second objective lens 1-8 to illuminate the sample.

[0042] When imaging the sample, the sample surface needs to be placed on the front focal plane of the second objective lens 1-8, so that the plane containing the sample is conjugate to the plane containing the first biaxial scanning galvanometer 1-5. Therefore, the deflection change of the first biaxial scanning galvanometer 1-5 corresponds to the xy-axis movement of the sample plane, realizing the function of acquiring sample information from different illumination angles. After the light beam passes through the sample, the parallel light and the scattered light from the sample structure are collected by the first objective lens 6, and after exiting, they are reflected by the first dichroic mirror 7 into the imaging detection unit.

[0043] The imaging detection unit consists of the third lens 1-9, the fourth lens 1-10, the second fiber optic flange 1-11, the fifth lens 1-12, the third reflecting mirror 1-13, the sixth lens 1-14, the D-type mirror 1-15, the seventh lens 1-16, and the first camera 1-17. The transmitted signal light first passes through the second relay objective system composed of the first objective lens 6 and the third lens 1-9, where it is re-collimated into parallel light; this beam is called the object beam. It then passes through the third relay objective system composed of a pair of lenses (the fourth lens 1-10 and the seventh lens 1-16). The rear focal plane of the second relay objective system is located at the front focal plane of the third relay objective system, and the first camera 1-17 is located at the rear focal plane of the third relay objective system, making the photosensitive plane of the first camera 1-17 and the sample plane conjugate. A second optical fiber is needed as a reference light, which is emitted from the second optical fiber flange 1-11, collimated by the fifth lens 1-12, reflected by the third mirror 1-13, and focused by the sixth lens 1-14 near the Fourier surface of the third relay objective system. A D-type mirror 1-15 is placed at this location to combine the object light and the reference light. The combined light is then imaged by the seventh lens 1-16 and converged on the sensor of the first camera 1-17 to form interference fringes, forming an off-axis hologram to record the phase information of the light beam.

[0044] After data acquisition, the control module invokes the ODT reconstruction algorithm, utilizing the Rytov approximation and a Fourier diffraction theorem algorithm based on angular spectrum propagation theory to quickly reconstruct the three-dimensional refractive index distribution map of the cell at the current moment. This result directly reflects the overall three-dimensional morphology of the cell, its dry mass distribution, and the label-free structural information of its internal organelles.

[0045] After passing through the first dichroic mirror 7, the fluorescence is reflected by the first reflecting mirror 8 and enters the second imaging module 5, which uses a dual-disc confocal focusing system. The optical path of the second imaging module 5 includes: a fifth laser source 5-1, a seventh fiber optic flange 5-2, a twenty-second lens 5-3, a microlens array disk 5-4, a sixth dichroic mirror 5-5, a second pinhole disk 5-6, a twenty-third lens 5-7, a twenty-fourth lens 5-8, a twenty-fifth lens 5-9, a fourth filter 5-10, and a fifth camera 5-11. The fifth laser source 5-1 can use different wavelength channels as needed. After being emitted from the fifth laser source 5-1, the laser enters the seventh fiber optic flange 5-2, exits from the seventh fiber optic flange 5-2, and is collimated by the twenty-second lens 5-3. The collimated light first illuminates the microlens array disk 5-4, where each microlens focuses the beam into a bright focal point. These focal points fall precisely on the pinholes of the second pinhole disk 5-6, greatly improving illumination efficiency. The light source, passing through a pinhole array, is focused onto the sample via a fourth relay objective system consisting of the twenty-third lens 5-7 and the first objective lens 6. This excites the sample structure at the focal point to generate a fluorescence signal, which is then captured by the fourth relay objective system consisting of the twenty-third lens 5-7 and the first objective lens 6.

[0046] The signal light returns along its original path and passes through the second pinhole turntable 5-6 again. The valid signal emitted from the sample's focal plane can pass smoothly through the original pinhole on the second pinhole turntable 5-6. Interference light emitted from above or below the sample's focal plane is mostly blocked by the second pinhole turntable 5-6 because it is not focused on the pinhole plane. The signal light passing through the turntable then passes through the twenty-fourth lens 5-8 and the twenty-fifth lens 5-9, and through the fourth filter 5-10 to select the fluorescence channel. Finally, it is captured by the fifth camera 5-11, which captures the clear image on the focal plane.

[0047] Figure 5 This is the fourth schematic diagram of the optical path of the dual-modal microscopic imaging system provided in the embodiments of this application, as shown below. Figure 5 As shown, the dual-modal microscopic imaging system includes: a first imaging module 1 for D-type mirror beam combining, a second imaging module 3 for laser scanning, a first objective lens 6, a first dichroic mirror 7, a first reflecting mirror 8, and a control module ( Figure 5 (Not shown in the image).

[0048] The optical path of the first imaging module 1 of the D-type mirror beam combiner includes: a first laser source 1-1, a first fiber optic flange 1-2, a first lens 1-3, a second reflector 1-4, a first dual-axis scanning galvanometer 1-5, a second dichroic mirror 1-6, a second lens 1-7, a second objective lens 1-8, a third lens 1-9, a fourth lens 1-10, a second fiber optic flange 1-11, a fifth lens 1-12, a third reflector 1-13, a sixth lens 1-14, a D-type mirror 1-15, a seventh lens 1-16, and a first camera 1-17.

[0049] The scanning illumination unit comprises a first laser source 1-1, a first fiber optic flange 1-2, a first lens 1-3, a second reflector 1-4, a first dual-axis scanning galvanometer 1-5, a second dichroic mirror 1-6, a second lens 1-7, and a second objective lens 1-8. A 532nm wavelength laser beam is emitted from the first laser source 1-1, enters the first fiber optic flange 1-2, exits from the second fiber optic flange 1-11, is collimated by the first lens 1-3, and then reflected by the second reflector 1-4 into the first dual-axis scanning galvanometer 1-5. The scanning galvanometer 1-5 controls the deflection of the laser beam in the X and Y directions through a fast-deflecting mirror. After the laser beam is emitted, it is reflected by the second dichroic mirror 1-6 into the vertically downward optical path. It passes through the first relay objective system composed of the second lens 1-7 and the lower second objective lens 1-8. That is, the first dual-axis scanning galvanometer 1-5 is located on the front focal plane of the first relay objective system. The emitted parallel beam is focused by the second lens 1-7, and the focal point is located on the rear focal plane of the second objective lens 1-8. Then, it is emitted as parallel light from the second objective lens 1-8 to illuminate the sample.

[0050] When imaging the sample, the sample surface needs to be placed on the front focal plane of the second objective lens 1-8, so that the plane containing the sample is conjugate to the plane containing the first biaxial scanning galvanometer 1-5. Therefore, the deflection change of the first biaxial scanning galvanometer 1-5 corresponds to the xy-axis movement of the sample plane, realizing the function of acquiring sample information from different illumination angles. After the light beam passes through the sample, the parallel light and the scattered light from the sample structure are collected by the first objective lens 6, and after exiting, they are reflected by the first dichroic mirror 7 into the imaging detection unit.

[0051] The imaging detection unit consists of the third lens 1-9, the fourth lens 1-10, the second fiber optic flange 1-11, the fifth lens 1-12, the third reflecting mirror 1-13, the sixth lens 1-14, the D-type mirror 1-15, the seventh lens 1-16, and the first camera 1-17. The transmitted signal light first passes through the second relay objective system composed of the first objective lens 6 and the third lens 1-9, where it is re-collimated into parallel light; this beam is called the object beam. It then passes through the third relay objective system composed of a pair of lenses (the fourth lens 1-10 and the seventh lens 1-16). The rear focal plane of the second relay objective system is located at the front focal plane of the third relay objective system, and the first camera 1-17 is located at the rear focal plane of the third relay objective system, making the photosensitive plane of the first camera 1-17 and the sample plane conjugate. A second optical fiber is needed as a reference light, which is emitted from the second optical fiber flange 1-11, collimated by the fifth lens 1-12, reflected by the third mirror 1-13, and focused by the sixth lens 1-14 near the Fourier surface of the third relay objective system. A D-type mirror 1-15 is placed at this location to combine the object light and the reference light. The combined light is then imaged by the seventh lens 1-16 and converged on the sensor of the first camera 1-17 to form interference fringes, forming an off-axis hologram to record the phase information of the light beam.

[0052] After data acquisition, the control module invokes the ODT reconstruction algorithm, utilizing the Rytov approximation and a Fourier diffraction theorem algorithm based on angular spectrum propagation theory to quickly reconstruct the three-dimensional refractive index distribution map of the cell at the current moment. This result directly reflects the overall three-dimensional morphology of the cell, its dry mass distribution, and the label-free structural information of its internal organelles.

[0053] The second imaging module 3 for laser scanning includes a third laser source 3-1, a fifth fiber optic flange 3-2, a fourteenth lens 3-3, a first pinhole 3-4, a collimating lens 3-5, a first filter 3-6, a fourth dichroic mirror 3-7, a scanning galvanometer 3-8, a fifteenth lens 3-9, a sixteenth lens 3-10, a second filter 3-11, a seventeenth lens 3-12, a second pinhole 3-13, and a third camera 3-14. The scanning galvanometer 3-8 can perform two-dimensional planar scanning of the sample. Compared with confocal scanning with a turntable, the scanning galvanometer 3-8 can only perform single-point scanning, and the time required for a single imaging scan is longer. However, it avoids the blurred background signal caused by aperture crosstalk effect and has higher axial resolution.

[0054] The dual-modal imaging system provided in this application is not a simple optical combination of two subsystems, optical diffraction tomography (ODT) and confocal imaging. Instead, it achieves deep hardware-level coupling and functional complementarity through a series of key optical component settings and collaborative design. These special designs include: Shared High Numerical Aperture Objective and Customized Dichroic Mirror: At the heart of the system is a shared high numerical aperture (NA) (e.g., NA ≥ 1.2) first objective 6, which serves simultaneously as the collecting objective for the ODT and the excitation / collecting objective for confocal imaging. To achieve this functionality, a core first dichroic mirror 7 is specifically designed with precisely optimized characteristics: high reflectivity for the ODT probe laser (e.g., 532 nm), high transmittance for the confocal excitation laser (e.g., 488 nm, 561 nm), and high transmittance for the confocal fluorescence emission signal. This design ensures that the optical paths from the two subsystems can be coupled with maximum efficiency and minimal crosstalk, providing the physical basis for synchronous / alternating acquisition.

[0055] An integrated scanning and imaging relay system: The scanning galvanometer plane of the ODT and the scanning unit (galvanometer or turntable) of the confocal imaging system are both conjugate to the sample surface through a precisely defined objective relay system. This not only ensures the accuracy of their respective imaging, but more importantly, it allows the two subsystems to have a completely consistent spatial reference and field of view. This means that, at the hardware level, the three-dimensional space reconstructed by the ODT and the two-dimensional image acquired by the confocal imaging system inherently have a very high spatial correspondence, laying a solid foundation for subsequent high-precision software registration and avoiding the complex and time-consuming image searching and matching process.

[0056] A time-controlled laser and detection system: All lasers (ODT laser, confocal multi-wavelength laser) and cameras in the system are time-managed through a unified control module. This integrated control enables precise switching between the two acquisition modes, which is key to achieving advanced imaging modes such as "ODT full-time and confocal equal-interval," thereby balancing the contradiction between spatially specific information and fluorescence phototoxicity in long-term live-cell imaging.

[0057] The control module will be described in detail below.

[0058] The control module includes a timing control unit and an image fusion unit; the timing control unit is configured to control the data acquisition timing of the first imaging module and the second imaging module; the image fusion unit is configured to process the three-dimensional refractive index distribution image and the fluorescence image to generate a fused image.

[0059] The timing control unit is responsible for coordinating the acquisition timing of ODT and confocal imaging to achieve synchronous or alternating acquisition of dual-modal data; the image fusion unit uses image processing algorithms to register and fuse the three-dimensional refractive index distribution image of ODT with the fluorescence image of confocal imaging to generate a fused image, thereby simultaneously obtaining the structural and chemical information of the sample under the same field of view.

[0060] The control module is based on a main control computer and sends high-precision timing trigger signals through a digital I / O board (such as National Instruments' PCIe-6321). For the first imaging module, the trigger signal controls the switching on and off of the laser, the movement of the scanning galvanometer, and the exposure of the camera. For the second imaging module, the trigger signal controls the switching on and off of its laser, the reading of the rotary encoder (for SDCM) or the synchronization of the scanning galvanometer (for LSCM), and the exposure of its camera.

[0061] In some embodiments, the timing control unit includes: The first timing control subunit is configured to control the first imaging module and the second imaging module to image synchronously; or... The second timing control subunit is configured to control the first imaging module to perform full-time imaging and the second imaging module to perform equal-interval imaging.

[0062] Specifically, label-free and fluorescent dual-modal imaging data are of great significance for biological research. In practical biological research, label-free imaging of live cells can be performed first to observe the life activities of live cells from the perspective of three-dimensional refractive index distribution; then, targeted fluorescent labeling can accurately reveal the location, quantity, activity, and interactions of specific biomolecules (such as proteins, deoxyribonucleic acid, and ions).

[0063] Figure 6 This is a timing control schematic diagram of the dual-modal microscopic imaging system provided in the embodiments of this application, as shown below. Figure 6 As shown, simultaneous imaging using ODT and confocal dual-modal imaging can be employed; ODT full-time imaging and confocal equal-interval imaging solve the problem of short imaging time caused by confocal fluorescence phototoxicity and photobleaching.

[0064] Simultaneous imaging acquires accurate dual-modal images of live cells over a period of time, avoiding errors in subsequent image registration caused by cell movement and deformation. It is suitable for directly observing the immediate changes in cell physical properties (ODT) when a molecular event (confocal) occurs, providing the strongest evidence for mechanistic studies. However, it introduces significant phototoxicity and photobleaching, and cannot perform long-term imaging. Interval imaging uses ODT for continuous, uninterrupted imaging as the monitoring thread, while the confocal laser is only briefly activated at specific time points (e.g., every five minutes). Although some changes in fluorescence signal are lost, morphological features can be tracked based on label-free ODT data. This minimizes interference with live cells and enables ultra-long-term imaging ranging from hours to days, suitable for observing long-term cellular changes such as the cell cycle.

[0065] In the "ODT full-time, confocal equal-interval" mode, the main control program continuously runs the ODT acquisition cycle while maintaining an internal timer. When the timer reaches the preset confocal imaging interval (e.g., 5 minutes), the program inserts a confocal acquisition sequence after the current ODT acquisition cycle ends: first, the ODT laser is turned off and the confocal laser is turned on, triggering the confocal camera to complete single-frame or multi-frame acquisition; then, the confocal laser is turned off and the ODT laser is turned on again, resuming ODT full-time acquisition.

[0066] In some embodiments, the image fusion unit includes: The registration subunit is configured to use a hybrid registration strategy that combines coarse registration based on feature points with fine registration based on intensity information to spatially align the three-dimensional refractive index distribution image and the fluorescence image. The fusion subunit is configured to fuse and visualize the three-dimensional refractive index distribution image acquired by the registered first imaging module and the fluorescence image acquired by the second imaging module.

[0067] Specifically, by acquiring single-layer two-dimensional images of the same standard microsphere sample in the first imaging module based on ODT and the second imaging module based on confocal imaging, respectively, and calculating the affine transformation matrix by extracting the two-dimensional coordinates of the microsphere center, and applying this matrix to each two-dimensional slice of the three-dimensional image of the biological sample, precise spatial alignment with the ODT three-dimensional data can be achieved. With a single calibration, rapid and automated three-dimensional image alignment of all subsequent samples can be achieved, providing a reliable foundation for the accurate fusion and quantitative analysis of multimodal images.

[0068] Figure 7 This is a schematic diagram of the image registration process of the dual-modal microscopic imaging system provided in the embodiments of this application, as shown below. Figure 7 As shown, the 3D image registration uses an affine transformation algorithm based on feature points. The specific process is as follows: Step a, Initial Calibration and Affine Transformation (Coarse Registration): Prepare a standard sample containing fluorescent microspheres of known size (e.g., 1 μm). Acquire ODT phase images and confocal fluorescence images using an ODT system and a confocal system, respectively. Automatically identify multiple microsphere centers as feature point sets using image processing algorithms (e.g., Laplacian of Gaussian blob detection). Subsequently, solve a two-dimensional affine transformation matrix using the least squares method and store it in the system. This matrix is ​​used to initially correct translation, rotation, and scaling errors caused by differences in the optical path between the two modal images.

[0069] Building upon affine transformation, a non-rigid registration algorithm based on mutual information is employed for fine registration to further correct for potential local deformations and residual optical distortions caused by the sample itself. This algorithm does not rely on specific image features but optimizes the spatial transformation relationship by maximizing the statistical dependence between the overall grayscale distributions of the two images. Specifically, a B-spline free deformation model is used as the transformation model, normalized mutual information is used as the similarity measure, and optimization algorithms such as gradient descent are employed to find the optimal transformation parameters. This effectively compensates for minute local deformations, achieving higher-precision spatial alignment.

[0070] Step b, Application and Visualization: Perform live-cell dual-modal imaging, simultaneously acquiring label-free ODT image data and confocal fluorescence image data. Apply the composite transformation model (affine transformation + non-rigid transformation) obtained in the above steps to each two-dimensional slice of the confocal fluorescence image of all subsequent biological samples, achieving precise alignment of the fluorescence image and the ODT three-dimensional refractive index data in the XY plane. Axial (Z-direction) alignment is ensured by the physical focal plane of the shared objective lens. The final output is a registered dual-modal three-dimensional image. The registered data is fused and visualized in a fusion subunit. The ODT three-dimensional refractive index data is displayed in grayscale, while the multi-channel fluorescence signal from the confocal image is superimposed in pseudo-color, allowing users to analyze the distribution and dynamics of specific biomolecules against a precise three-dimensional spatial background.

[0071] The dual-modal microscopy imaging method provided in this application is described below. The dual-modal microscopy imaging method described below can be referred to in correspondence with the dual-modal microscopy imaging system described above. The parts that are the same as those in the previous embodiments and the beneficial effects will not be described in detail here.

[0072] Figure 8 This is a schematic flowchart of the dual-modal microscopic imaging method provided in the embodiments of this application, as shown below. Figure 8 As shown, the method includes at least the following steps: S801. Obtain a three-dimensional refractive index distribution image of the sample based on the first imaging module; S802. Acquire a fluorescence image of the sample based on the second imaging module; S803. Control the data acquisition timing of the first imaging module and the second imaging module based on the control module; S804. Based on the control module, the three-dimensional refractive index distribution image and the fluorescence image are processed to generate a fused image.

[0073] In some embodiments, S803 specifically includes: Control the first imaging module and the second imaging module to image synchronously; or, Control the first imaging module to perform full-time imaging and the second imaging module to perform equal-interval imaging.

[0074] In some embodiments, S804 specifically includes: Determining the affine transformation matrix based on standard microsphere sample images; Based on the affine transformation matrix, each layer of the fluorescence image of the acquired sample is spatially aligned with the corresponding layer of the three-dimensional refractive index distribution image. The three-dimensional refractive index distribution image acquired by the first imaging module and the fluorescence image acquired by the second imaging module are fused and visualized.

[0075] The technical solution provided in the embodiments of this application will be further illustrated by a specific example below.

[0076] Taking living HeLa cells as an example, this paper elaborates on the specific application process and unique advantages of the system of the present invention in studying the typical life activity of mitochondrial-organelle interaction.

[0077] 1. Sample preparation and system initialization Sample preparation: Live HeLa cells were cultured in dedicated confocal culture dishes. For specific imaging, mitochondria were fluorescently labeled using MitoTracker Red CMXRos. Simultaneously, to simulate cellular responses to drug intervention, an appropriate amount of an apoptosis inducer (such as Staurosporine) was added to the imaging medium.

[0078] System Initialization: Upon system startup, the control module first performs optical path calibration. This ensures precise alignment of the ODT scanning galvanometer zero point with the SDCM rotary scanning center on the sample surface. The culture dish is placed on the sample stage, and the control subsystem drives the sample stage for coarse positioning to locate the target cell population.

[0079] 2. Dual-modal temporal imaging workflow The core advantage of this system lies in its ability to acquire two types of information under the same platform, same field of view, and near-synchronous conditions. The first and second imaging modules are controlled to execute the following cyclic process according to a preset time sequence to achieve long-term observation: Step A: ODT label-free three-dimensional morphological imaging.

[0080] The control module shuts down the SDCM laser and turns on the 532nm laser of the ODT.

[0081] The dual-axis scanning galvanometer performs multi-angle circular scanning according to a preset program (for example, uniformly selecting a preset number of illumination angles within a 180-degree range).

[0082] At each illumination angle, the ODT camera acquires an off-axis hologram.

[0083] Data and Results: After data acquisition, the reconstruction unit uses an ODT reconstruction algorithm (such as the Rytov approximation or Fourier diffraction theorem) to quickly reconstruct the three-dimensional refractive index distribution of the cell at the current moment. This result directly reflects the overall three-dimensional morphology of the cell, its dry mass distribution, and the label-free structural information of internal organelles (such as the nucleus, nucleolus, and vacuoles). For example, the high refractive index region of the nucleus and the surrounding mitochondria distributed due to compression can be clearly seen.

[0084] Step B: SDCM fluorescence-specific imaging.

[0085] The control module automatically switches the optical path, shutting down the ODT laser and turning on the 561nm laser of the SDCM to excite the MitoTracker Red fluorescence.

[0086] The turntable rotates at high speed, and the SDCM camera exposes the sample and acquires a two-dimensional confocal image of the sample in the fluorescence channel, which typically takes milliseconds.

[0087] Data and Results: A highly specific, high signal-to-noise ratio two-dimensional fluorescence image was obtained, in which the bright dot-like or filamentous structures are mitochondria. The image clearly reveals which structures are specifically labeled mitochondria, but it lacks three-dimensional structural information and cannot show the spatial relationship between mitochondria and other unlabeled organelles.

[0088] 3. Data fusion and collaborative analysis The control module automatically registers and fuses the data acquired in steps A and B, a crucial step that cannot be achieved with single-modality imaging. Spatial registration: Since the ODT and SDCM share the same set of objectives and sample surfaces, they are naturally aligned in hardware. The software then uses a sub-pixel-level registration algorithm based on image features to ensure that the three-dimensional refractive index data of the ODT and the two-dimensional fluorescence data of the SDCM are completely and accurately superimposed in space.

[0089] Information fusion and quantitative analysis, precise localization and morphological correlation: In the fused image, each mitochondrial identified by the SDCM fluorescence signal can be found in the precise three-dimensional location and morphology of the ODT three-dimensional refractive index map. It is possible to visually observe whether the mitochondria are distributed around the cell nucleus or extend into the pseudopodia.

[0090] Functional and morphological linkage analysis: During drug-induced apoptosis, ODT images can observe label-free dynamic changes such as overall cell shrinkage, nuclear shrinkage, and the appearance of apoptotic bodies within the cell. Simultaneously, SDCM images can monitor a decrease in mitochondrial membrane potential (manifested as a reduction in fluorescence intensity) and fragmentation of the mitochondrial network (from filamentous to punctate).

[0091] Quantitative Research: By fusing data, precise quantitative analysis can be performed. For example, in the three-dimensional space reconstructed by ODT, the distance between a specific mitochondria (defined by fluorescence signals) and the cell nucleus surface can be precisely calculated. The correlation between the degree of mitochondrial fragmentation (from SDCM images) and the rate of overall cell dry mass reduction (from ODT images) during apoptosis can be analyzed. Using phase information from ODT, changes in the refractive index of the cytoplasm surrounding mitochondria can be non-invasively measured, thus indirectly reflecting the local metabolic state.

[0092] In summary, the technical solutions provided by the embodiments of this application are as follows: (1) It overcomes the shortcomings of ODT's lack of specificity: it can no longer rely solely on morphological recognition, but can now clearly identify specific three-dimensional morphologies (such as a high-refractive-index particle) as "mitochondria".

[0093] (2) It makes up for the shortcomings of insufficient axial resolution of SDCM: by using the precise three-dimensional spatial background provided by ODT, the two-dimensional SDCM fluorescence image is endowed with three-dimensional positioning capability, which can analyze the distribution of mitochondria in the depth direction of the cell.

[0094] (3) The synergistic effect of dual-modal imaging can better analyze and study living cells: Through the real-time correlation between label-free morphological imaging and specific fluorescence imaging, it is possible to conduct an unprecedented comprehensive, quantitative and dynamic analysis of life activities from both structural and functional dimensions in the natural state of living cells, providing a powerful new tool for cell biology research.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A dual-modal microscopic imaging system, characterized in that, include: The first imaging module is configured to acquire a three-dimensional refractive index distribution image of the sample based on the principle of optical diffraction tomography. The second imaging module is configured to acquire fluorescence images of the sample based on the principle of confocal microscopy. The control module includes a timing control unit and an image fusion unit; the timing control unit is configured to control the data acquisition timing of the first imaging module and the second imaging module; the image fusion unit is configured to process the three-dimensional refractive index distribution image and the fluorescence image to generate a fused image.

2. The dual-modal microscopic imaging system according to claim 1, characterized in that, The first imaging module includes: The scanning illumination unit is configured to illuminate the sample from multiple angles using a scanning coherent beam. An imaging detection unit includes a beam combiner and an image detector; the beam combiner is configured to split a coherent beam into an object beam and a reference beam, and combine the object beam and the reference beam after passing through the sample to form interference fringes; the image detector is configured to acquire the interference fringes and generate a holographic image. The reconstruction unit is configured to reconstruct the three-dimensional refractive index distribution of the sample based on the holographic image.

3. The dual-modal microscopic imaging system according to claim 2, characterized in that, The beam combining element includes a D-type mirror beam combining element or a cubic crystal beam combining element.

4. The dual-modal microscopic imaging system according to claim 1, characterized in that, The second imaging module includes a rotating confocal microscope (SDCM) or a laser scanning confocal microscope (LSCM).

5. The dual-modal microscopic imaging system according to claim 4, characterized in that, The rotating confocal microscope includes a single-rotor confocal microscope or a dual-rotor confocal microscope.

6. The dual-modal microscopic imaging system according to claim 1, characterized in that, The timing control unit includes: The first timing control subunit is configured to control the first imaging module and the second imaging module to image synchronously; or... The second timing control subunit is configured to control the first imaging module to perform full-time imaging and the second imaging module to perform equal-interval imaging.

7. The dual-modal microscopic imaging system according to claim 1, characterized in that, The image fusion unit includes: The registration subunit is configured to determine the affine transformation matrix based on the standard microsphere sample image; based on the affine transformation matrix, each layer of the acquired sample fluorescence image is spatially aligned with the corresponding layer of the three-dimensional refractive index distribution image; The fusion subunit is configured to fuse and visualize the three-dimensional refractive index distribution image acquired by the first imaging module and the fluorescence image acquired by the second imaging module.

8. A dual-modal microscopic imaging method, characterized in that, The method, applied to the dual-modal microscopic imaging system as described in any one of claims 1 to 7, comprises: The three-dimensional refractive index distribution image of the sample is obtained based on the first imaging module; The fluorescence image of the sample is acquired based on the second imaging module; The control module controls the data acquisition timing of the first imaging module and the second imaging module. The control module processes the three-dimensional refractive index distribution image and the fluorescence image to generate a fused image.

9. The dual-modal microscopic imaging method according to claim 8, characterized in that, The control module controls the data acquisition timing of the first imaging module and the second imaging module, including: Control the first imaging module and the second imaging module to image synchronously; or, Control the first imaging module to perform full-time imaging and the second imaging module to perform equal-interval imaging.

10. The dual-modal microscopic imaging method according to claim 8, characterized in that, The processing of the three-dimensional refractive index distribution image and the fluorescence image based on the control module includes: Determining the affine transformation matrix based on standard microsphere sample images; Based on the affine transformation matrix, each layer of the fluorescence image of the acquired sample is spatially aligned with the corresponding layer of the three-dimensional refractive index distribution image. The three-dimensional refractive index distribution image acquired by the first imaging module and the fluorescence image acquired by the second imaging module are fused and visualized.

Images