A label-free multimodal microscopy system for studying organelle interaction networks

Abstract

The present invention discloses a label-free multimodal microscopy system suitable for the study of organelle interaction networks, including: Raman channels, phase channels, and fluorescence channels; precision displacement stages; illumination light sources, and multiple lasers; wherein the Raman channel includes a Raman spectrometer, a 2D galvanometer, and a 1D galvanometer, for providing fast scanning Raman spectral signals; the phase channel is used to record the morphological information of label-free organelles and record phase images. By integrating high spatiotemporal resolution phase technology with fast Raman scanning technology, not only can the morphology of various organelles be rapidly detected, but also the moving organelles can be tracked in real time and their Raman spectral information can be collected, so as to analyze the material exchange between the cell structure and other cell structures. Compared with the existing fluorescence technology, the present invention can simultaneously observe the morphology of various organelles, detect the exchange of various substances in situ, and depict the dynamic change characteristics of the organelle network interaction with minimal interference.

Description

Technical Field

[0001] The present invention relates to biological microscopic imaging technology, and in particular to a label-free multimodal microscopic system suitable for studying organelle interaction networks. Background Art

[0002] Cells are the basic units of life activities, and organelles are the functional departments of cells, performing life activities closely related to cell fate, such as metabolism, stress, autophagy, etc. Cells contain more than ten kinds of organelles, which form an organelle interaction network through mutual cooperation and close contact to achieve material exchange, and also transmit signal factors to participate in regulation. Disorders in the organelle network function will cause a variety of metabolic diseases (obesity, diabetes, Alzheimer's disease, etc.), and even the occurrence and development of tumors. At present, research on the form, function and mechanism of organelle interaction is still in its infancy.

[0003] At present, the observation of the dynamic process of the interaction network mainly relies on fluorescence microscopy, but fluorescence microscopy has problems such as phototoxicity, short observation time, difficulty in labeling multiple objects, and the need for prior knowledge of fluorescence labeling. Therefore, there is an urgent need to develop measurement technologies that can simultaneously observe the morphology of multiple organelles, detect the exchange of multiple substances in situ, and have little effect on cell activity. Summary of the invention

[0004] To solve the above problems, the technical solution adopted by the present invention is: a label-free multimodal microscopy system suitable for studying organelle interaction networks, comprising: a Raman channel, a phase channel, and a fluorescence channel; a precision translation stage; an illumination light source, and multiple lasers;

[0005] Wherein, the Raman channel includes a Raman spectrometer, a 2D galvanometer, and a 1D galvanometer, which are used to provide a fast scanning Raman spectrum signal and record Raman spectrum information;

[0006] The phase channel includes a CMOS camera for recording morphological information of unlabeled organelles and recording phase images;

[0007] The fluorescence channel includes a first fluorescence channel and a second fluorescence channel, each of which includes a CMOS camera for detection; the fluorescence channel is used to generate label information required for network training and deep learning, as well as protein signals with regulatory functions; a two-way fluorescence channel structure is used to simultaneously obtain fluorescence signals of different labels;

[0008] Precision translation stage, used to place cells for observation.

[0009] Furthermore, unlabeled cell microscopic images are captured through the phase channel, and labeled images of specific organelles are obtained through the fluorescence channel. This paired data is provided to the CNN network for training to obtain the characteristics of such organelles.

[0010] Further, the light output by the first laser irradiates the first wavelength beam splitter, and the first wavelength beam splitter is connected to the second wavelength beam splitter, the third wavelength beam splitter, the fourth wavelength beam splitter, and the fifth wavelength beam splitter in sequence; the light output by the second laser irradiates the second wavelength beam splitter; the first fluorescence channel is connected to the first wavelength beam splitter; the second fluorescence channel is connected to the fourth wavelength beam splitter; the phase channel is connected to the third wavelength beam splitter, and the phase channel includes a CMOS detector and a spatial light modulator SLM;

[0011] The organelle is placed on a precision translation stage, and the illumination light source illuminates the organelle through a bandpass filter. The precision translation stage is connected to a fifth wavelength spectrometer; the fifth wavelength spectrometer is further connected to a 2D galvanometer and a seventh wavelength spectrometer, the seventh wavelength spectrometer is connected to a third laser, and is also connected to a Raman channel, which includes a 1D galvanometer and a Raman spectrometer.

[0012] Furthermore, phase images are used for online identification and tracking; the user first defines the ROI, exposure time, and interval; the field of view of the phase system is larger than a cell, and the IOU indicator is used to track organelles; the phase acquisition system collects cell images on the nanostage to determine whether they are focused. If not, a control command is sent to the console to fine-tune the position until the focus is accurate, and then online morphological analysis is performed to identify organelles and their positions; the updated position information of the organelles will be fed back to the Raman camera scanning system; at the same time, in dynamic experiments, the sample is controlled by a precision translation stage in its overall XY position and focused in the Z direction. The system will also overcome the drift caused by temperature or other environmental factors through automatic focusing to achieve continuous excitation of moving organelles.

[0013] Furthermore, the light source emitted by the third laser irradiates the 2D galvanometer, and according to the continuously updated organelle position information, the 2D galvanometer is used to keep the excitation spot used for Raman excitation on the organelle; the laser signal is time-multiplexed, and the fast scanning capability of the 2D galvanometer is utilized to scan multiple organelles on the sample surface, thereby realizing the rapid switching of the positions of these organelles; at the detection end, the 2D galvanometer is first used for descanning, and then the 1D galvanometer is used to disperse the Raman signals of these organelles to different positions of the slit of the spectrometer, corresponding to different rows of the camera, thereby obtaining the Raman spectrum of the lipid droplets.

[0014] Furthermore, during the data acquisition process, the organelles need to be identified and tracked online. After acquisition, the data of the two modalities need to be analyzed and correlated. The data processing of the system includes the following steps:

[0015] Step 1: Cell image information collected by phase channel;

[0016] Step 2: Use the deep learning network CNN to identify the organelles in the image;

[0017] Step 3: Perform morphological analysis on each type of organelle to obtain the morphological information and physical movement information of each type of organelle, including size / morphology, division / fusion, and organelle contact;

[0018] Step 4: Using the cell image information collected by the phase channel to track the organelles, the 2D galvanometer scanning system synchronously scans the spectrum of the organelles;

[0019] Step 5: Analyze the contact between different organelles according to their spatial positions; for Raman spectral information, use asymmetric least squares method or vertex component analysis to achieve spectral separation; for Raman dynamic information, use statistical analysis to extract chemical dynamic information of organelles over time;

[0020] Step 6: Since the information of the phase and Raman channels is collected synchronously, the morphological information and chemical information are correlated to study the corresponding chemical changes of organelles during morphological changes.

[0021] Furthermore, by pairing fluorescence and phase channels, labeling data, training data, and test data are provided to the deep learning network to achieve automatic recognition of fine organelles, including unlabeled organelles such as mitochondria and endoplasmic reticulum.

[0022] Furthermore, the acquisition frame rate of the phase channel is 200-250 Hz, and single or multiple organelles can be tracked online using images acquired by the phase channel and a trained neural network automatic recognition algorithm.

[0023] Furthermore, synchronized phase and Raman measurements are performed to measure the dynamic changes of morphology and chemistry on organelles in real time and in situ, and to analyze and study the functions of organelle interaction networks.

[0024] Furthermore, for Raman spectral information, spectral separation is achieved using an asymmetric least squares method or a vertex component analysis method; for long-term Raman dynamic information, a statistical analysis method is used to extract the chemical dynamic information of organelles over time, and the statistical analysis method includes vertex components or MCR-ALS (multivariate curve resolution with alternating least squares).

[0025] Furthermore, in the above-mentioned label-free multimodal microscopy system, in collecting Raman spectra of moving organelles, the position information obtained by the previous online tracking is used to continuously excite these organelles using the galvanometer of the Raman channel, thereby collecting high-quality Raman spectra.

[0026] Preferably, in the above-mentioned label-free multimodal microscopy system, based on the continuously updated organelle position information, a beam position adjustment device such as a two-dimensional galvanometer is used to keep the light beam used for Raman excitation on the organelle; in order to measure multiple organelles at the same time, a time multiplexing method can be adopted to utilize the rapid scanning capability of the galvanometer to achieve rapid switching of the positions of these organelles.

[0027] Furthermore, in the above-mentioned label-free multimodal microscopy system, at the detection end, a beam position adjustment device such as a galvanometer is used to disperse the Raman signals of these organelles to different rows of the camera. During the cyclic measurement of these target organelles, the Raman beam stays on a single organelle for a single time at the millisecond level, avoiding affecting the free movement of the organelle.

[0028] Preferably, in the above-mentioned label-free multimodal microscopy system, organelles need to be identified and tracked online during data acquisition, and the data of the two modalities need to be analyzed and correlated after acquisition.

[0029] Preferably, in the above-mentioned label-free multimodal microscopy system, for the information of the phase channel, after the organelles in the image are identified, morphological analysis is performed on each type of organelle to obtain the morphological information and physical movement information of each type of organelle; at the same time, the contact between different organelles will also be analyzed according to the spatial position of each organelle.

[0030] Compared with the fluorescence technology, the beneficial effects achieved by the present invention are:

[0031] The present invention provides a label-free multimodal microscopy system suitable for the study of organelle interaction networks, which can observe the dynamic changes of organelle morphology and chemistry without labeling, avoid phototoxicity, and help maintain the activity and normal physiological activities of cells; label-free observation will not have the problem of sample bleaching signal weakening, and the dynamic changes of living cells can be observed continuously for a long time; high-resolution phase microscopy technology and Raman technology collect endogenous signals of samples, and can observe multiple organelles at the same time, which is conducive to capturing the dynamic changes of interaction networks in a panoramic view; Raman, as a chemical quantification method, can track the material changes on organelles without prior knowledge, and more comprehensively reflect the chemical changes occurring in cells. Therefore, the present invention has obvious advantages, can simultaneously observe the morphology of multiple organelles, analyze material exchange in situ, and depict the dynamic change characteristics of organelle network interaction with minimal interference, which can be widely used in biology, immunology, pathology, pharmacology and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is the overall schematic diagram of the system hardware structure;

[0033] Figure 2 It is the U-Nnet network architecture for label-free cell recognition;

[0034] Figure 3 It is the system data acquisition control flow chart;

[0035] Figure 4 It is a schematic diagram of tracking and Raman measurement of multiple moving lipid droplets using time multiplexing;

[0036] Figure 5 It is a data processing flow chart of the present invention, wherein CNN: convolutional neural network; VCA: vertex component analysis; ASLS: asymmetric least squares method. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] The present invention discloses a label-free multimodal microscopy system suitable for studying organelle interaction networks. The system hardware structure is as follows Figure 1 As shown, it includes a Raman channel, a phase channel, and a fluorescence channel; a precision translation stage for placing cells for observation; an illumination light source, and multiple lasers;

[0039] The Raman channel includes a Raman spectrometer for recording high-quality Raman spectral information;

[0040] The phase channel includes a CMOS camera for recording morphological information of unlabeled organelles;

[0041] The fluorescence channel includes a first fluorescence channel and a second fluorescence channel, each of which includes a CMOS camera for detection; the fluorescence channel is used to generate the labeling information required for deep learning. The structure of two fluorescence channels can simultaneously obtain fluorescence signals with different labels, thereby improving the performance of deep learning. Unlabeled cell microscopic images are captured through a phase channel with high spatiotemporal resolution, and labeled images of specific organelles are obtained through the fluorescence channel. This paired data is provided to CNN training to obtain the characteristics of such organelles.

[0042] Specifically, Figure 1As shown, the light output by the first laser 3 irradiates the first wavelength beam splitter 16, and the first wavelength beam splitter 16 is connected to the second wavelength beam splitter 9, the third wavelength beam splitter 10, the fourth wavelength beam splitter 11, and the fifth wavelength beam splitter 12 in sequence; the light output by the second laser 4 irradiates the second wavelength beam splitter 9; the first fluorescent channel 1 is connected to the first wavelength beam splitter 16; the second fluorescent channel 2 is connected to the fourth wavelength beam splitter 11; the phase channel is connected to the third wavelength beam splitter 10, and the phase channel includes a CMOS detector and a spatial light modulator SLM;

[0043] The organelle is placed on a precision translation stage 7, and the illumination light source 6 illuminates the organelle through a bandpass filter. The precision translation stage 7 is connected to a fifth wavelength spectrometer 12; the fifth wavelength spectrometer 12 is further connected to a 2D galvanometer 14 and a seventh wavelength spectrometer 15, and the seventh wavelength spectrometer 1 is connected to a third laser 8 and is also connected to a Raman channel 17, wherein the Raman channel 17 includes a 1D galvanometer and a Raman spectrometer;

[0044] According to one embodiment of the present invention, taking mitochondria as an example, mitochondria can be stained by Mitotracker, and fluorescence images and phase images can be collected to construct the following: Figure 2 The U-Net network shown, where the loss function is MSE and the activation function is ReLU.

[0045] like Figure 3 As shown, phase images with high spatiotemporal resolution are used for online identification and tracking. The user first defines the ROI, exposure time, interval, etc.; the field of view of the phase system is larger than that of a cell, and the image acquisition speed is fast, the recognition speed is also fast, and the overlap of organelles between adjacent frames is large. Therefore, simple indicators with low computational overhead (such as IOU) can be used to track organelles. The phase acquisition system collects cell images on the nanostage and determines whether they are focused. If they are not focused, a control command is sent to the console to fine-tune the position until the focus is accurate, and then online morphological analysis is performed to identify organelles and their positions; the position update information of the organelles will be fed back to the Raman camera scanning system. At the same time, in dynamic experiments, the sample is controlled by a precision displacement stage (i.e., nanostage) to control its overall XY position and focus in the Z direction. The system will also overcome the drift caused by temperature or other environmental factors through automatic focusing to achieve continuous excitation of moving organelles.

[0046] like Figure 4As shown, the light source emitted by the third laser 8 irradiates the 2D galvanometer, and according to the continuously updated organelle position information, the 2D galvanometer is used to keep the excitation spot for Raman excitation on the organelle. In order to measure multiple organelles at the same time, the laser signal can be time-multiplexed, and the fast scanning capability of the 2D galvanometer can be used to scan multiple organelles on the sample surface to achieve rapid switching of the positions of these organelles. At the detection end, the 2D galvanometer is first used for descanning, and then the 1D galvanometer is used to disperse the Raman signals of these organelles to different positions of the slit of the spectrometer, corresponding to different rows of the camera, so as to obtain the Raman spectrum of the lipid droplets.

[0047] like Figure 4 As shown (taking lipid droplets as an example), there are multiple lipid droplets (lipid droplets 1 to 5) on the sample surface. When using Raman to study micron- and submicron-scale organelles such as lipid droplets, the focused excitation beam, especially the high-power laser, will clamp the organelles, restrict their free movement, and affect the spontaneous interaction between the organelles. Therefore, during the cyclic measurement of these target organelles, the Raman beam stays on a single organelle for a single time at the millisecond level, which increases the speed and range of the measurement and avoids affecting the free movement of the organelle. During the data acquisition process, the organelles need to be identified and tracked online, such as Figure 3 and 4 shown.

[0048] After acquisition, the data of the two modes need to be analyzed and correlated. The system data processing is as follows: Figure 5 As shown, the following steps are included:

[0049] Step 1: Cell image information collected by phase channel;

[0050] Step 2: Use the deep learning network CNN to identify the organelles in the image;

[0051] Step 3: Perform morphological analysis on each type of organelle to obtain the morphological information and physical movement information of each type of organelle. Taking mitochondria as an example, open source software such as MitoGraph will be used to analyze the size / morphology, fission / fusion, and organelle contact of each mitochondria.

[0052] Step 4: Using the cell image information collected by the phase channel to track the organelles, the 2D galvanometer scanning system synchronously scans the spectrum of the organelles;

[0053] Step 5: Analyze the contact between different organelles according to their spatial positions. For Raman spectral information, we will use asymmetric least squares method, vertex component analysis and other methods to achieve spectral separation. For long-term Raman dynamic information, we will use statistical analysis such as vertex component or MCR-ALS (multivariate curve resolution with alternating least squares) to extract the chemical dynamic information of organelles over time.

[0054] Step 6. Since the information of the phase and Raman channels is collected synchronously, the morphological information and chemical information can be correlated to study the corresponding chemical changes of organelles during morphological changes.

[0055] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to the embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A label-free multimodal microscopy system suitable for studying organelle interaction networks, characterized in that: include: Raman channel, phase channel, and fluorescence channel; precision translation stage; Illumination source, 2D galvanometer, and multiple lasers; Wherein, the Raman channel includes a Raman spectrometer and a 1D galvanometer, which are used to provide a fast scanning Raman spectrum signal and record Raman spectrum information; The phase channel includes a CMOS camera for recording morphological information of unlabeled organelles and recording phase images; The fluorescence channel includes a first fluorescence channel and a second fluorescence channel, each of which includes a CMOS camera for detection; the fluorescence channel is used to generate label information required for network training and deep learning, as well as protein signals with regulatory functions; a two-way fluorescence channel structure is used to simultaneously obtain fluorescence signals of different labels; Precision translation stage, used to place organelles for observation; The light output by the first laser irradiates the first wavelength beam splitter, and the first wavelength beam splitter is connected to the second wavelength beam splitter, the third wavelength beam splitter, the fourth wavelength beam splitter, and the fifth wavelength beam splitter in sequence; the light output by the second laser irradiates the second wavelength beam splitter; the first fluorescence channel is connected to the first wavelength beam splitter; the second fluorescence channel is connected to the fourth wavelength beam splitter; the phase channel is connected to the third wavelength beam splitter, and the phase channel includes a CMOS detector and a spatial light modulator SLM; The cell organelle is placed on a precision translation stage, and the illumination light source illuminates the cell organelle through a bandpass filter. The precision translation stage is connected to a fifth wavelength spectrometer; the fifth wavelength spectrometer is further connected to a 2D galvanometer and a seventh wavelength spectrometer, and the seventh wavelength spectrometer is connected to a third laser and is also connected to a Raman channel, wherein the Raman channel includes a 1D galvanometer and a Raman spectrometer; The light source emitted by the third laser irradiates the 2D galvanometer. According to the continuously updated organelle position information, the 2D galvanometer is used to keep the excitation spot used for Raman excitation on the organelle. The laser signal is time-multiplexed, and the fast scanning capability of the 2D galvanometer is used to scan multiple organelles on the sample surface, thereby realizing the rapid switching of the positions of these organelles. At the detection end, the 2D galvanometer is first used for descanning, and then the 1D galvanometer is used to disperse the Raman signals of these organelles to different positions of the slit of the spectrometer, corresponding to different rows of the camera, thereby obtaining the Raman spectrum of the lipid droplets.

2. A label-free multimodal microscopy system suitable for studying organelle interaction networks according to claim 1, characterized in that: Unlabeled cell microscopic images are captured through the phase channel, and labeled images of specific organelles are obtained through the fluorescence channel. This paired data is provided to the CNN network for training to obtain the characteristics of such organelles.

3. A label-free multimodal microscopy system suitable for studying organelle interaction networks according to claim 1, characterized in that: Use phase images for online identification and tracking; the user first defines ROI, exposure time, and interval; the field of view of the phase system is larger than a cell, and the IOU indicator is used to track organelles; the phase acquisition system collects cell images on the nanostage to determine whether they are focused. If not, a control command is sent to the console to fine-tune the position until the focus is accurate, and then online morphological analysis is performed to identify organelles and their positions; the position update information of the organelles will be fed back to the Raman camera scanning system; at the same time, in dynamic experiments, the sample is controlled by a precision translation stage in its overall XY position and focused in the Z direction. The system will also overcome the drift caused by temperature or other environmental factors through automatic focusing to achieve continuous excitation of moving organelles.

4. A label-free multimodal microscopy system suitable for studying organelle interaction networks according to claim 1, characterized in that: During the data acquisition process, the organelles need to be identified and tracked online. After acquisition, the data of the two modalities need to be analyzed and correlated. The data processing of the system includes the following steps: Step 1: Cell image information collected by phase channel; Step 2: Use the deep learning network CNN to identify the organelles in the image; Step 3: Perform morphological analysis on each type of organelle to obtain the morphological information and physical movement information of each type of organelle, including size / morphology, division / fusion, and organelle contact; Step 4: Using the cell image information collected by the phase channel to track the organelles, the 2D galvanometer scanning system synchronously scans the spectrum of the organelles; Step 5: Analyze the contact between different organelles according to their spatial positions; for Raman spectral information, use asymmetric least squares method or vertex component analysis to achieve spectral separation; for Raman dynamic information, use statistical analysis to extract chemical dynamic information of organelles over time; Step 6: Since the information of the phase and Raman channels is collected synchronously, the morphological information and chemical information are correlated to study the corresponding chemical changes of organelles during morphological changes.

5. A label-free multimodal microscopy system suitable for studying organelle interaction networks according to claim 1, characterized in that: By pairing fluorescence and phase channels, labeled data, training data, and test data are provided to the deep learning network to achieve automatic recognition of fine organelles, including unlabeled organelles such as mitochondria and endoplasmic reticulum.

6. A label-free multimodal microscopy system suitable for studying organelle interaction networks according to claim 1, characterized in that: The acquisition frame rate of the phase channel is 200-250 Hz. The images acquired by the phase channel and the trained neural network automatic recognition algorithm are used to track single or multiple organelles online.

7. A label-free multimodal microscopy system suitable for studying organelle interaction networks according to claim 1, characterized in that: Synchronous phase and Raman measurements are performed to measure dynamic changes in morphology and chemistry of organelles in real time and in situ, and to analyze the function of organelle interaction networks.

8. A label-free multimodal microscopy system suitable for studying organelle interaction networks according to claim 1, characterized in that: For Raman spectral information, the asymmetric least squares method or the vertex component analysis method is used to achieve spectral separation; for long-term Raman dynamic information, statistical analysis methods are used to extract the chemical dynamic information of organelles over time, and the statistical analysis methods include vertex components or MCR-ALS (multivariate curve resolution with alternating least squares).

Images