Endoscope system based on harmonic imaging and multiphoton fluorescence imaging

Abstract

The invention discloses an endoscope system based on harmonic imaging and multiphoton fluorescence imaging. The endoscope system comprises a first femtosecond pulse laser, a second femtosecond pulse laser, a prism, an X-Y scanning galvanometer, a first long-wave-pass dichroscope, a coupling objective lens, an optical fiber bundle, an endoscope and an imaging device. The first femtosecond pulse laser and the second femtosecond pulse laser respectively emit near-infrared femtosecond pulse laser, the near-infrared femtosecond pulse laser enters the X-Y scanning galvanometer after being converged by the prism, then penetrates through the first long-wave-pass dichroscope, is coupled into the optical fiber bundle through the coupling objective lens, and finally is focused on a measured tissue through the endoscope. The first long-wave-pass dichroscope excites the detected tissue and the fluorescent substance thereof to generate second harmonic, third harmonic, two-photon fluorescence and three-photon fluorescence, and harmonic and fluorescence signals return along the same path and enter the imaging device after being reflected by the first long-wave-pass dichroscope, so that a second harmonic image, a third harmonic image, a two-photon fluorescence image and a three-photon fluorescence image of the detected tissue are obtained. Harmonic waves and multi-photon fluorescence imaging are combined, and pathological tissues in the cavity can be detected.

Description

Technical Field

[0001] The invention relates to the field of endoscopes, and in particular to an endoscope system based on harmonic imaging and multi-photon fluorescence imaging. Background Art

[0002] With the advent of lasers, nonlinear effects such as harmonics and multiphoton absorption caused by strong light in the medium have been discovered one after another. Among them, second harmonic generation (SHG), third harmonic generation (THG), two-photon fluorescence (2PEF), and three-photon fluorescence (3PEF) images can be collected using the same system. Two-photon fluorescence is a third-order nonlinear effect in which a fluorescent molecule absorbs two photons with longer wavelengths at the same time under high photon density, transitions from the ground state to the excited state and then returns to the ground state, produces non-radiative energy loss, and releases a short-wavelength photon to emit a fluorescence signal. Similar to two-photon fluorescence, a fluorescent molecule will also release a short-wavelength photon when it absorbs three photons. This phenomenon is called three-photon fluorescence, which is a fifth-order nonlinear effect.

[0003] In the process of second harmonic generation, no photon absorption occurs, no energy level transition and energy loss occurs, that is, under the action of strong laser, the molecules inside the medium undergo second-order nonlinear polarization, thereby radiating frequency-doubled coherent light. Since no electron energy level transition occurs in the process of second harmonic generation, the sample is not affected by photobleaching. The third harmonic is the third-order nonlinear effect generated by the medium molecules after being irradiated by strong laser, and the frequency of the light wave released is three times that of the excitation light.

[0004] The light source of the second harmonic, third harmonic and multiphoton fluorescence system generally adopts femtosecond pulse lasers in the near infrared region I and II (NIR-I, NIR-II), which can reduce photon scattering and effectively penetrate the biological matrix, and can detect tissues at a certain depth. The intensity of multiphoton fluorescence, second harmonic and third harmonic is closely related to the density of excitation photons. Only molecules at the focus of the excitation light can be excited, and molecules in non-focal areas do not produce multiphoton fluorescence, second harmonic and third harmonic, which makes the received signal have a higher signal-to-noise ratio.

[0005] Common second harmonic / third harmonic / multiphoton fluorescence microscope systems have complex structures and large dimensions, and their scope of use is limited to the detection of body surface and ex vivo tissues. Miniaturizing the second harmonic / third harmonic / multiphoton fluorescence imaging system and applying it to the field of endoscopy enables this technology to observe diseased tissues in the body cavity. Among them, the second harmonic is usually used for imaging non-centrosymmetric molecules and ordered structures, such as collagen fibers, microtubules, myosin, skin tissue, and corneal stroma; the third harmonic is usually used for imaging substances or interfaces with high refractive index (compared to the surrounding water), such as organelles, red blood cells or white blood cells, lipid droplets, adipose tissue, axon myelin and bones; and multiphoton absorption can stimulate a variety of endogenous fluorophores, such as keratin, melanin, flavin, elastic fibers, collagen, etc. The structure, blood vessels, protein distribution and morphology of diseased tissues are quite different from those of normal groups. Therefore, the state of the tissue can be judged by observing the second harmonic / third harmonic / multiphoton fluorescence images, such as intracerebral vascular imaging, brain glioma diagnosis, early diagnosis of lung cancer, ovarian cancer, cervical cancer, bladder cancer and other lesions inside the body cavity. The use of endoscopes for detection avoids the harm of extracting tissue biopsies to patients.

[0006] In summary, ordinary single-photon fluorescence endoscopes usually use lasers with shorter wavelengths to excite tissue fluorescence, and light in this band is easily reflected and scattered by tissues. In addition, since the excitation light and fluorescence wavelengths are close, the excitation light cannot be completely filtered out, causing interference to the fluorescence image and a reduced signal-to-noise ratio. Harmonic and multiphoton fluorescence imaging uses femtosecond pulsed lasers in the first and second near-infrared regions (800-1700nm) as light sources, which are less affected by tissue reflection and scattering. Since second harmonic / third harmonic / multiphoton fluorescence can only be excited in the area near the focus of the illumination light path, and the wavelength difference between the excitation light and the second harmonic / third harmonic / multiphoton fluorescence is large, the excitation light is easily filtered out, so the image collected using the confocal system is not interfered by stray light and has a high signal-to-noise ratio. However, such systems are mostly implemented using confocal microscope systems, which have complex structures and large volumes. They are usually used for the detection of body surfaces and ex vivo tissues, and their scope of use is limited. Summary of the invention

[0007] The object of the present invention is to provide an endoscope system based on harmonic imaging and multiphoton fluorescence imaging, which miniaturizes the second harmonic / third harmonic / multiphoton fluorescence imaging system and applies it to the endoscope, and can simultaneously perform second harmonic, third harmonic, two-photon fluorescence, and three-photon fluorescence imaging; in addition, the illumination and imaging share the same optical path, and the scanning device of the harmonic and multiphoton fluorescence imaging system is designed to the outside of the endoscope, so that the endoscope system can observe lesions in smaller body cavities.

[0008] The technical solution adopted by the present invention is as follows:

[0009] In a first aspect, the present invention provides an endoscope system based on harmonic imaging and multiphoton fluorescence imaging, the endoscope system comprising a first femtosecond pulse laser, a second femtosecond pulse laser, a laser optical path, an endoscope, and an imaging device;

[0010] The laser optical path includes a prism, an XY scanning galvanometer, a first long-wave dichroic mirror, a coupling objective lens and an optical fiber bundle, which are arranged in sequence, and the end of the optical fiber bundle is connected to the endoscope; the first femtosecond pulse laser and the second femtosecond pulse laser respectively emit near-infrared femtosecond pulse lasers, which are combined by the prism and enter the XY scanning galvanometer, and then pass through the first long-wave dichroic mirror, and are coupled to the optical fiber bundle by the coupling objective lens, and finally are focused on the tissue to be tested through the endoscope;

[0011] Two near-infrared femtosecond pulse lasers scan the measured tissue under the action of the XY scanning galvanometer, and excite the measured tissue and its fluorescent substances to produce second harmonic, third harmonic, two-photon fluorescence and three-photon fluorescence. The second harmonic, third harmonic, two-photon fluorescence and three-photon fluorescence return along the original path, are reflected by the first long-wave pass dichroic mirror and enter the imaging device to obtain the second harmonic image, third harmonic image, two-photon fluorescence image and three-photon fluorescence image of the measured tissue.

[0012] In the above scheme, the wavelength range of the near-infrared femtosecond pulse laser emitted by the first femtosecond pulse laser is 850-1250nm, which is used to excite the measured tissue and its fluorescent substances to produce second harmonic and two-photon fluorescence; the wavelength range of the near-infrared femtosecond pulse laser emitted by the second femtosecond pulse laser is 1300-1350nm, which is used to excite the measured tissue and its fluorescent substances to produce third harmonic and three-photon fluorescence.

[0013] In the above scheme, the wavelength of the near-infrared femtosecond pulse laser emitted by the first femtosecond pulse laser is 920nm, and the wavelength of the near-infrared femtosecond pulse laser emitted by the second femtosecond pulse laser is 1300nm.

[0014] In the above scheme, the imaging device includes a second long-wave pass dichroic mirror, a third long-wave pass dichroic mirror, a fourth long-wave pass dichroic mirror, a third harmonic imaging sub-device, a second harmonic imaging sub-device, a two-photon fluorescence imaging sub-device and a three-photon fluorescence imaging sub-device;

[0015] A second long-wave dichroic mirror, a third long-wave dichroic mirror and a fourth long-wave dichroic mirror are arranged in sequence; the second harmonic, the third harmonic, the two-photon fluorescence and the three-photon fluorescence entering the imaging device pass through the second long-wave dichroic mirror, the third long-wave dichroic mirror and the fourth long-wave dichroic mirror in sequence;

[0016] Among them, the third harmonic is reflected by the second long-wave dichroic mirror and enters the third harmonic imaging sub-device to obtain a third harmonic image; then the second harmonic is reflected by the third long-wave dichroic mirror and enters the second harmonic imaging sub-device to obtain a second harmonic image; finally, the two-photon fluorescence is reflected by the fourth long-wave dichroic mirror and enters the two-photon fluorescence imaging sub-device to obtain a two-photon fluorescence image, and the three-photon fluorescence passes through the fourth long-wave dichroic mirror and enters the three-photon fluorescence imaging sub-device to obtain a three-photon fluorescence image.

[0017] In the above scheme, each imaging sub-device includes a narrow-band filter, a converging lens and a photomultiplier tube.

[0018] In the above scheme, the endoscope system also includes a control system, which is connected to the XY scanning galvanometer and the photomultiplier tubes of each imaging sub-device, and is used to control the movement of the XY scanning galvanometer and receive the signals of each photomultiplier tube, and combine the scanning trajectory of the XY scanning galvanometer with the signals of each photomultiplier tube to reconstruct each image to obtain a second harmonic image, a third harmonic image, a two-photon fluorescence image and a three-photon fluorescence image.

[0019] In the above solution, the control system is also connected to the first femtosecond pulse laser and the second femtosecond pulse laser, and is used to control the opening and closing of the first femtosecond pulse laser and the second femtosecond pulse laser.

[0020] In the above scheme, the endoscope includes an achromatic objective lens and a portion of the optical fiber bundle; the portion of the optical fiber bundle is connected to the end of the optical fiber bundle, and two near-infrared femtosecond pulse lasers are focused on the measured tissue through the achromatic objective lens in the endoscope.

[0021] In the above solution, the endoscope is equipped with a handle, and the outer diameter of the working insertion portion of the endoscope is less than 6 mm.

[0022] In a second aspect, the present invention provides an imaging method of an endoscope system based on harmonic imaging and multiphoton fluorescence imaging as described in any one of the above technical solutions, comprising the following steps:

[0023] The first femtosecond pulse laser and the second femtosecond pulse laser emit near-infrared femtosecond pulse lasers respectively; the two near-infrared femtosecond pulse lasers are combined through a prism and enter an XY scanning galvanometer, then pass through a first long-wave dichroic mirror, are coupled into an optical fiber bundle by a coupling objective lens, and finally are focused on the tissue to be tested through an endoscope;

[0024] Two near-infrared femtosecond pulse lasers excite the tissue under test and its fluorescent substances to generate second harmonics, third harmonics, two-photon fluorescence and three-photon fluorescence, which return along the original path and are reflected by the first long-wave dichroic mirror into the imaging device;

[0025] Two near-infrared femtosecond pulse lasers scan the measured tissue under the action of an XY scanning galvanometer, and an imaging device obtains a second harmonic image, a third harmonic image, a two-photon fluorescence image, and a three-photon fluorescence image of the measured tissue.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] The present invention adopts two femtosecond pulse lasers, which respectively emit near-infrared femtosecond pulse lasers. The two near-infrared femtosecond pulse lasers can excite the measured tissue and its fluorescent material to produce second harmonics, third harmonics, two-photon fluorescence and three-photon fluorescence. Finally, the second harmonic image, third harmonic image, two-photon fluorescence image and three-photon fluorescence image of the measured tissue can be obtained through an imaging device. The diseased tissue is combined with harmonics and multi-photon fluorescence to perform joint imaging diagnosis, thereby improving the accuracy of diagnosis. The endoscope system applies harmonics and multi-photon fluorescence effects to the endoscope system, so that it can perform diagnosis of internal lesions in the body cavity such as brain vascular imaging, brain glioma diagnosis, early diagnosis of lung cancer, etc., thereby avoiding the harm of extracting tissue biopsy to the patient. In addition, the illumination optical path and imaging optical path of the endoscope system share part of the optical path, namely, the optical path from the first long-wave pass dichroic mirror to the endoscope, and the scanning device of the harmonic and multiphoton fluorescence imaging system is designed to the outside of the endoscope, which greatly reduces the size of the endoscope and enables the technology to observe lesions in smaller body cavities.

[0028] In addition, the wavelength range of the near-infrared femtosecond pulse laser emitted by the first femtosecond pulse laser in the present invention is 850-1250nm, which is mainly used to excite the measured tissue and its fluorescent material to produce second harmonic and two-photon fluorescence, while the wavelength range of the near-infrared femtosecond pulse laser emitted by the second femtosecond pulse laser is 1300-1350nm, which is mainly used to excite the measured tissue and its fluorescent material to produce third harmonic and three-photon fluorescence. The femtosecond pulse lasers in two specific wavelength ranges can make the imaging device obtain the second harmonic image, third harmonic image, two-photon fluorescence image and three-photon fluorescence image of the measured tissue better; the achromatic objective lens designed in the present invention has a wide working band and can be compatible with excitation light, second harmonic, third harmonic and multi-photon fluorescence at the same time to obtain second harmonic / third harmonic / multi-photon fluorescence images, and the images can be analyzed or fused according to the use requirements to increase the accuracy of distinguishing the diseased tissue and the type of disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the structure of an endoscope system based on harmonic imaging and multiphoton fluorescence imaging;

[0030] Figure 2 The workflow diagram of the endoscope system based on harmonic imaging and multiphoton fluorescence imaging;

[0031] Figure 3 This is the optical path diagram of the achromatic objective.

[0032] In the figure: 1. a first femtosecond pulse laser; 2. a second femtosecond pulse laser; 3. a prism; 4. an XY scanning galvanometer; 5. a first long-wave dichroic mirror; 6. a coupled objective lens; 7. an optical fiber bundle; 8. an endoscope; 9. a tissue to be measured; 10. a second long-wave dichroic mirror; 11. a first narrow-band filter; 12. a first converging lens; 13. a first photomultiplier tube; 14. a third long-wave dichroic mirror; 15. a second narrow-band filter; 16. a second converging lens; 17. a second photomultiplier tube; 18. a fourth long-wave dichroic mirror; 19. a third narrow-band filter; 20. a third converging lens; 21. a third photomultiplier tube; 22. a fourth narrow-band filter; 23. a fourth converging lens; 24. a fourth photomultiplier tube; 25. a control system. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0034] The present invention provides an endoscope system based on harmonic imaging and multiphoton fluorescence imaging, which includes main components such as a femtosecond pulse laser, a photomultiplier tube, an XY scanning galvanometer, and an achromatic objective lens, and diagnoses the lesion based on the second harmonic, third harmonic, and multiphoton fluorescence images. The endoscope system combines harmonics and multiphoton fluorescence to perform joint imaging diagnosis of the lesion tissue, thereby improving the accuracy of the diagnosis; the harmonic and multiphoton fluorescence effects are applied to the endoscope system, so that it can perform brain vascular imaging, brain glioma diagnosis, early diagnosis of lung cancer, and other internal lesions of the body cavity, thereby avoiding the harm of extracting tissue biopsies to patients; the scanning device of the harmonic and multiphoton fluorescence imaging system is designed to the outside of the endoscope, greatly reducing the size of the endoscope, so that the technology can observe lesions in smaller body cavities; the achromatic objective lens designed in the present invention has a wide working band and can be compatible with excitation light, second harmonics, third harmonics, and multiphoton fluorescence at the same time.

[0035] The endoscope system based on harmonic imaging and multiphoton fluorescence imaging provided by the present invention miniaturizes the second harmonic / third harmonic / multiphoton fluorescence imaging system and applies it to the field of endoscopes. Among them, the light source adopts a femtosecond pulse laser. After passing through the XY scanning galvanometer, the femtosecond pulse laser enters the optical fiber bundle through the coupling objective lens. The laser emitted from the optical fiber bundle converges to the focus through the achromatic objective lens. The focus of the achromatic objective lens coincides with the surface of human tissue, and excites the non-centrosymmetric molecules and ordered structures in the measured tissue to produce second harmonics, such as structural proteins; substances or interfaces with high refractive index (compared with the surrounding water), such as organelles, red blood cells or white blood cells, produce third harmonics; various endogenous fluorophores produce multiphoton fluorescence. Afterwards, the second harmonic / third harmonic / multiphoton fluorescence returns to the original path, is reflected by the long-wave dichroic mirror behind the coupling objective lens, and is split, filtered, and focused on the light-sensitive surface of the photomultiplier tube. The control system combines the movement of the XY scanning galvanometer with the output of the photomultiplier tube to reconstruct the image and fuse and analyze the image. The blood vessels, tissue structure, and protein distribution in tumor tissues are quite different from those in normal tissues. Combining second harmonic / third harmonic images can be used to observe the distribution of blood vessels and blood cells in tumor tissues, and using multiphoton fluorescence to observe the distribution of various proteins and colloids. Therefore, combining second harmonic / third harmonic / multiphoton fluorescence images can achieve accurate identification of tumor tissues. The lighting and imaging systems share part of the optical path, making the lighting and imaging systems more consistent, and the design of the scanning system outside the endoscope greatly reduces the outer diameter of the endoscope, making it more suitable for non-invasive or minimally invasive surgery.

[0036] Figure 1 The schematic diagram of the structure of the endoscope system based on harmonic imaging and multiphoton fluorescence imaging is shown in Figure 2. Figure 2 The flowchart of the work flow of the endoscope system based on harmonic imaging and multiphoton fluorescence imaging is shown in the figure. As shown in the figure, the illumination optical path and the imaging optical path partially overlap, and the confocal optical path improves the accuracy of the collection of harmonic and multiphoton fluorescence signals, thereby improving the signal-to-noise ratio.

[0037] like Figure 1 and Figure 2As shown, the near-infrared laser beams emitted by two femtosecond pulse lasers are reflected twice by the XY scanning galvanometer 4, respectively controlling the position changes of the X-axis and the Y-axis, and completing the two-dimensional scanning of the measured surface; the first long-wave dichroic mirror 5 is used to separate the excitation light and the second harmonic / third harmonic / multiphoton fluorescence, and the near-infrared laser beam is coupled to the optical fiber bundle 7 by the coupling lens 6, and is focused on the measured tissue after passing through the achromatic objective lens; at the focus of the achromatic objective lens, the high-density photons cause the measured tissue 9 to produce nonlinear optical effects, that is, after being excited by the near-infrared femtosecond pulse laser, the second harmonic / third harmonic / multiphoton fluorescence is generated. Near-infrared light with a wavelength less than 1300nm mainly excites the second harmonic, and near-infrared light with a wavelength greater than 1300nm mainly excites the third harmonic. The wavelength of multiphoton fluorescence is related to the type of fluorescent dye / receptor protein. Among them, there is no energy loss in the generation process of second harmonic / third harmonic, so the wavelength of the second harmonic is exactly half of the excitation wavelength, and the wavelength of the third harmonic is one-third of the wavelength of the excitation light; the multi-photon absorption induced by near-infrared light is two-photon absorption and three-photon absorption, and non-radiative energy loss is generated in the process when the molecules in the ground state absorb two / three long-wave photons and release one short-wave photon.

[0038] Second harmonic generation is commonly used for imaging of non-centrosymmetric molecules and ordered structures, such as collagen fibers, microtubules, myosin, skin tissue, and corneal stroma; third harmonic generation is commonly used for imaging of substances or interfaces with a high refractive index (compared to the surrounding water), such as organelles, red blood cells or white blood cells, lipid droplets, adipose tissue, axon myelin, and bones; and multiphoton absorption can stimulate a variety of endogenous fluorophores, such as keratin, melanin, flavin, elastic fibers, collagen, etc.

[0039] The second harmonic / third harmonic / multi-photon fluorescence enters the achromatic objective lens, the optical fiber bundle 7, and the coupling objective lens 6 in sequence along the direction opposite to the excitation light, and is reflected by the first long-wave dichroic mirror 5 and separated into two paths by the second long-wave dichroic mirror 10, one path entering the third harmonic imaging optical path, and the other path is separated into two paths by the third long-wave dichroic mirror 14, one path entering the second harmonic imaging optical path, and the other path is further separated by the fourth long-wave dichroic mirror 18, one path entering the two-photon fluorescence imaging optical path, and the other path entering the three-photon fluorescence imaging optical path. After filtering, focusing, and amplification, the optical signals in the four imaging optical paths are converted into electrical signals by the photomultiplier tube and output to the control system 25. The control system 25 combines the scanning trajectory of the XY scanning galvanometer 4 with the signal output by the photomultiplier tube to reconstruct the image, and obtain the second harmonic / third harmonic / multi-photon fluorescence image. The image can be analyzed or fused according to the use requirements to increase the accuracy of the identification of the lesion tissue and the lesion type.

[0040] Specifically, Figure 1As shown, the endoscope system based on harmonic imaging and multiphoton fluorescence imaging of the embodiment of the present application includes: a first femtosecond pulse laser 1, a second femtosecond pulse laser 2, a laser optical path, an endoscope 8, a measured tissue 9, an imaging device and a control system 25. Among them, the laser optical path includes a prism 3, an XY scanning galvanometer 4, a first long-wave dichroic mirror 5, a coupling objective lens 6 and an optical fiber bundle 7, and the prism 3, the XY scanning galvanometer 4, the first long-wave dichroic mirror 5, the coupling objective lens 6 and the optical fiber bundle 7 are arranged in sequence. The imaging device includes a second long-wave pass dichroic mirror 10, a first narrow-band filter 11, a first converging lens 12, a first photomultiplier tube 13, a third long-wave pass dichroic mirror 14, a second narrow-band filter 15, a second converging lens 16, a second photomultiplier tube 17, a fourth long-wave pass dichroic mirror 18, a third narrow-band filter 19, a third converging lens 20, a third photomultiplier tube 21, a fourth narrow-band filter 22, a fourth converging lens 23 and a fourth photomultiplier tube 24.

[0041] The first femtosecond pulse laser 1 and the second femtosecond pulse laser 2 are femtosecond pulse lasers, which serve as the light sources in the system. The prism 3 allows the light emitted by the two lasers to enter the XY scanning galvanometer 4, and scans the entire endoscope field of view point by point under the control of the control system 25. The first long-wave dichroic mirror 5 allows the near-infrared femtosecond pulse laser to pass through, and the second harmonic, third harmonic and multiphoton fluorescence are reflected. The coupling objective lens 6 allows the near-infrared femtosecond pulse laser to enter the optical fiber bundle 7 or allows the second harmonic, third harmonic and multiphoton fluorescence in the optical fiber bundle 7 to enter the first long-wave dichroic mirror 5. The near-infrared femtosecond pulse laser beam scans the end face of the optical fiber bundle point by point under the action of the XY galvanometer system. The endoscope 8 is composed of an achromatic objective lens and a part of the optical fiber bundle, and is equipped with a handle for the user to grasp. The outer diameter of the working insertion part of the endoscope is less than 6 mm, which can realize the detection of diseased tissues in the cavity.

[0042] The laser is focused on the surface of the tissue 9 to be measured, and excites the tissue and its fluorescent material to generate second harmonics, third harmonics, two-photon fluorescence and three-photon fluorescence. The second long-wave dichroic mirror 10 reflects the third harmonic; the first narrow-band filter 11, the first converging lens 12 and the first photomultiplier tube 13 are the third harmonic channels, forming a third harmonic imaging sub-device, wherein the bandwidth of the first narrow-band filter 11 is 10nm, the first converging lens 12 focuses the third harmonic on the photosensitive surface of the first photomultiplier tube 13, and the first photomultiplier tube 13 amplifies the signal and outputs it to the control system 25. The third long-wave pass dichroic mirror 14 reflects the second harmonic; the second narrow-band filter 15, the second converging lens 16, and the second photomultiplier tube 17 are the second harmonic channel, constituting a second harmonic imaging sub-device, wherein the bandwidth of the second narrow-band filter 15 is 10 nm, and the second converging lens 16 focuses the second harmonic on the photosensitive surface of the second photomultiplier tube 17, and the second photomultiplier tube 17 amplifies the signal and outputs it to the control system 25. The fourth long-wave pass dichroic mirror 18 reflects the two-photon fluorescence and transmits the three-photon fluorescence; the third narrow-band filter 19, the third converging lens 20, and the third photomultiplier tube 21 are two-photon fluorescence channels, constituting a two-photon fluorescence imaging sub-device, wherein the bandwidth of the third narrow-band filter 19 is 10 nm, the third converging lens 20 focuses the two-photon fluorescence on the photosensitive surface of the third photomultiplier tube 21, and the third photomultiplier tube 21 amplifies the signal and outputs it to the control system 25; the fourth narrow-band filter 22, the fourth converging lens 23, and the fourth photomultiplier tube 24 are three-photon fluorescence channels, constituting a three-photon fluorescence imaging sub-device, wherein the bandwidth of the fourth narrow-band filter 22 is 10 nm, the fourth converging lens 23 focuses the two-photon fluorescence on the photosensitive surface of the fourth photomultiplier tube 24, and the fourth photomultiplier tube 24 amplifies the signal and outputs it to the control system 25.

[0043] The control system 25 is responsible for controlling the illumination of the femtosecond pulse laser, controlling the movement of the XY scanning galvanometer 4, receiving the signals of the four photomultiplier tubes, and performing the reconstruction, image processing and image fusion of the second harmonic, third harmonic and multiphoton fluorescence images. Based on harmonic imaging and multiphoton fluorescence imaging, the distribution of blood vessels, blood cells and proteins in the tissue is observed to determine whether the tissue has lesions and the extent of the lesions.

[0044] Among them, the wavelength range of the near-infrared femtosecond pulse laser emitted by the first femtosecond pulse laser is 850-1250nm, which is used to excite the measured tissue and its fluorescent substances to produce second harmonic and two-photon fluorescence; the wavelength range of the near-infrared femtosecond pulse laser emitted by the second femtosecond pulse laser is 1300-1350nm, which is used to excite the measured tissue and its fluorescent substances to produce third harmonic and three-photon fluorescence.

[0045] The embodiment of the present application also provides an imaging method of an endoscope system based on harmonic imaging and multiphoton fluorescence imaging as described in any one of the above technical solutions, comprising the following steps:

[0046] The first femtosecond pulse laser and the second femtosecond pulse laser emit near-infrared femtosecond pulse lasers respectively; the two near-infrared femtosecond pulse lasers are combined through a prism and enter an XY scanning galvanometer, then pass through a first long-wave dichroic mirror, are coupled into an optical fiber bundle by a coupling objective lens, and finally are focused on the tissue to be tested through an endoscope;

[0047] Two near-infrared femtosecond pulse lasers excite the tissue under test and its fluorescent substances to generate second harmonics, third harmonics, two-photon fluorescence and three-photon fluorescence, which return along the original path and are reflected by the first long-wave dichroic mirror into the imaging device;

[0048] Two near-infrared femtosecond pulse lasers scan the measured tissue under the action of an XY scanning galvanometer, and an imaging device obtains a second harmonic image, a third harmonic image, a two-photon fluorescence image, and a three-photon fluorescence image of the measured tissue.

[0049] Figure 3 This is the optical path diagram of the achromatic objective lens, which can be used with a fiber bundle with an end diameter less than or equal to 4 mm. The maximum scanning range on biological tissue is a circular area with a diameter of 1.27 mm. The parameters of the achromatic objective lens are shown in Table 1.

[0050] Table 1: Achromatic objective parameters

[0051]

[0052]

[0053] The present invention combines harmonic and multiphoton fluorescence imaging. The external scanning system makes the illumination optical path miniaturized and is suitable for the field of endoscopy. Therefore, this technology can observe the structural proteins, high-refractive index substances or interfaces, and the distribution and morphology of various endogenous fluorophores in small body cavities to determine the pathological conditions of tissues and avoid the pain of biopsy to patients. The combined use of harmonic and multiphoton fluorescence imaging technology increases the accuracy of disease or pathological tissue discrimination.

[0054] To help understand the present invention, the following takes 920nm and 1300nm femtosecond lasers as examples. The working principle of the system is as follows:

[0055] The 920nm and 1300nm femtosecond pulse lasers provide illumination for the illumination optical path. After passing through the XY scanning galvanometer 4, the near-infrared femtosecond pulse laser is transmitted by the first long-wave dichroic mirror 5, and is focused on the end face of the optical fiber bundle 7 by the coupling objective lens 6. The end face of the optical fiber bundle is scanned point by point under the action of the XY scanning galvanometer 4. The near-infrared femtosecond pulse laser is transmitted by the optical fiber bundle 7 to the achromatic objective lens of the endoscope 8. The achromatic objective lens focuses the laser on the surface of the tissue to be measured. At the focus of the achromatic objective lens, the high-density photons induce the ground state molecules in the tissue to be measured to produce second harmonics, third harmonics and multi-photon fluorescence.

[0056] Three-photon fluorescence imaging technology requires the use of fluorescent dyes. For example, the red fluorophore Texas Red is widely used to study neuronal morphology and as a cell type-selective fluorescent marker for astrocytes. When the excitation wavelength is lower than 1250nm, the fluorophore only excites two-photon signals; as the excitation wavelength increases, the excitation signal will mix two-photon and three-photon fluorescence; when the wavelength is greater than 1300nm, the excitation signal is mainly three-photon fluorescence, so a 1300nm laser is used to excite the three-photon fluorescence of fluorescent dyes and the third harmonic of materials or interfaces with high refractive index (compared to the surrounding water).

[0057] The excitation wavelength range used in two-photon imaging is between 700 and 1000 nm. The scattering coefficient of light in this band is small in tissue and has good penetration. However, near-infrared light of 700 nm to 850 nm will cause spontaneous fluorescent substances with shorter excitation wavelengths in tissues to produce fluorescence, interfering with the two-photon fluorescence imaging effect. Therefore, a 920 nm femtosecond pulse laser is used as the excitation light source for two-photon fluorescence. In addition, non-centrosymmetric molecules and ordered structures, such as collagen fibers, microtubules, myosin, skin tissue and corneal stroma, will also produce second harmonics under the action of femtosecond pulse lasers in this band.

[0058] Under the irradiation of the above-mentioned 920nm and 1300nm femtosecond pulse lasers, the molecules in the tissue under test emit the second harmonic of 460nm, the third harmonic of 433nm, the three-photon fluorescence of 615nm, and the two-photon fluorescence of 520nm. The second harmonic, the third harmonic, the three-photon fluorescence, and the two-photon fluorescence return through the achromatic objective, the optical fiber bundle, and the coupling objective. After being reflected by the first long-wave dichroic mirror, they enter the imaging optical path, and are respectively split, filtered, and focused on the light-sensitive surface of the photomultiplier tube. The photomultiplier tube converts the optical signal into an electrical signal and outputs it to the control system. The control system reconstructs the scanned signal to obtain the second harmonic image, the third harmonic image, the three-photon fluorescence image, and the two-photon fluorescence image. After image processing, the four images are fused and analyzed, and the state of the tissue under test is judged by comparing the distribution of blood vessels, blood cells, and proteins in the tissue, effectively realizing the detection of diseased tissue.

[0059] This embodiment combines harmonic and multi-photon fluorescence imaging technology and miniaturizes its structure, so that the technology can be applied to the endoscope system and can detect the diseased tissue in the cavity. In the embodiment, only 920nm and 1300nm femtosecond pulse lasers are used as examples to describe the working principle of the present invention. By changing the wavelength of the excitation light and replacing the filter, the system can be used to excite and receive the second harmonic, third harmonic, and multi-photon fluorescence of other wavelengths. Such changes still fall within the scope of the claims of the present invention. The light source can be a combination of femtosecond pulse lasers in the near-infrared zone I and II (NIR-I, NIR-II: 800-1700nm), and the light waves received by the imaging system can be a combination of one or more of the second harmonic, third harmonic, two-photon fluorescence, and three-photon fluorescence. In addition, in this embodiment, the image scanning system is externalized, so that the structure of the endoscope part in the imaging light path is simplified, and the illumination light path and the imaging light path share part of the light path, and are confocal on the surface of the tissue to be measured, thereby improving the accuracy of scanning imaging.

[0060] In summary, the present invention uses two femtosecond pulse lasers with different wavelengths as light sources to excite the measured tissue and fluorescent agent to produce second harmonic, third harmonic, three-photon fluorescence, and two-photon fluorescence. The image sensor can identify specific tissues and structures after fusing and analyzing the four received images, assisting doctors in accurately identifying lesions; the illumination light path and the imaging light path share the same light path, and are confocal on the surface of the measured tissue, that is, the focus of the excitation light is highly overlapped with the light emitting point of the second harmonic, third harmonic, and multi-photon fluorescence, providing accurate position information for subsequent image reconstruction; the illumination light path and the imaging light path share the same light path, which improves the space utilization of the scope The second harmonic, third harmonic and multiphoton fluorescence are only generated at the focus of the achromatic objective lens, and their light intensity is highly dependent on the photon density of the excitation light. This feature avoids the interference of stray light generated by the tissue around the focus on the system, greatly improving the signal-to-noise ratio. The scanning device of the endoscope system is placed outside the endoscope, so that the outer diameter of the working insertion part of the endoscope is less than 6mm, reducing the size and weight of the handle. At the same time, the size of the external scanning system is less restricted, and a more accurate galvanometer scanning system can be used to scan and image the measured tissue more accurately.

[0061] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0062] It should be pointed out that, according to the needs of implementation, the various steps / components described in this application can be split into more steps / components, and two or more steps / components or partial operations of steps / components can be combined into new steps / components to achieve the purpose of the present invention.

[0063] It will be easily understood by those skilled in the art that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An endoscope system based on harmonic imaging and multiphoton fluorescence imaging, characterized in that: The endoscope system includes a first femtosecond pulse laser, a second femtosecond pulse laser, a laser optical path, an endoscope and an imaging device; The laser optical path includes a prism, an XY scanning galvanometer, a first long-wave dichroic mirror, a coupling objective lens and an optical fiber bundle, which are arranged in sequence, and the end of the optical fiber bundle is connected to the endoscope; the first femtosecond pulse laser and the second femtosecond pulse laser respectively emit near-infrared femtosecond pulse lasers, which are combined by the prism and enter the XY scanning galvanometer, and then pass through the first long-wave dichroic mirror, and are coupled to the optical fiber bundle by the coupling objective lens, and finally are focused on the tissue to be tested through the endoscope; Two near-infrared femtosecond pulse lasers scan the measured tissue under the action of the XY scanning galvanometer, and excite the measured tissue and its fluorescent substances to produce second harmonic, third harmonic, two-photon fluorescence and three-photon fluorescence. The second harmonic, third harmonic, two-photon fluorescence and three-photon fluorescence return along the original path, are reflected by the first long-wave pass dichroic mirror and enter the imaging device to obtain the second harmonic image, third harmonic image, two-photon fluorescence image and three-photon fluorescence image of the measured tissue.

2. The endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to claim 1, characterized in that: The wavelength range of the near-infrared femtosecond pulse laser emitted by the first femtosecond pulse laser is 850-1250nm, which is used to excite the measured tissue and its fluorescent substances to produce second harmonic and two-photon fluorescence; the wavelength range of the near-infrared femtosecond pulse laser emitted by the second femtosecond pulse laser is 1300-1350nm, which is used to excite the measured tissue and its fluorescent substances to produce third harmonic and three-photon fluorescence.

3. The endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to claim 2, characterized in that: The wavelength of the near-infrared femtosecond pulse laser emitted by the first femtosecond pulse laser is 920 nm, and the wavelength of the near-infrared femtosecond pulse laser emitted by the second femtosecond pulse laser is 1300 nm.

4. The endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to claim 1, characterized in that: The imaging device includes a second long-wave pass dichroic mirror, a third long-wave pass dichroic mirror, a fourth long-wave pass dichroic mirror, a third harmonic imaging sub-device, a second harmonic imaging sub-device, a two-photon fluorescence imaging sub-device and a three-photon fluorescence imaging sub-device; A second long-wave dichroic mirror, a third long-wave dichroic mirror and a fourth long-wave dichroic mirror are arranged in sequence; the second harmonic, the third harmonic, the two-photon fluorescence and the three-photon fluorescence entering the imaging device pass through the second long-wave dichroic mirror, the third long-wave dichroic mirror and the fourth long-wave dichroic mirror in sequence; Among them, the third harmonic is reflected by the second long-wave dichroic mirror and enters the third harmonic imaging sub-device to obtain a third harmonic image; then the second harmonic is reflected by the third long-wave dichroic mirror and enters the second harmonic imaging sub-device to obtain a second harmonic image; finally, the two-photon fluorescence is reflected by the fourth long-wave dichroic mirror and enters the two-photon fluorescence imaging sub-device to obtain a two-photon fluorescence image, and the three-photon fluorescence passes through the fourth long-wave dichroic mirror and enters the three-photon fluorescence imaging sub-device to obtain a three-photon fluorescence image.

5. The endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to claim 4, characterized in that: Each imaging sub-device includes a narrow-band filter, a converging lens and a photomultiplier tube.

6. The endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to claim 5, characterized in that: The endoscope system also includes a control system, which is connected to the XY scanning galvanometer and the photomultiplier tubes of each imaging sub-device, and is used to control the movement of the XY scanning galvanometer and receive signals from each photomultiplier tube, and reconstruct each image by combining the scanning trajectory of the XY scanning galvanometer with the signals of each photomultiplier tube to obtain a second harmonic image, a third harmonic image, a two-photon fluorescence image, and a three-photon fluorescence image.

7. The endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to claim 6, characterized in that: The control system is also connected to the first femtosecond pulse laser and the second femtosecond pulse laser, and is used for controlling the opening and closing of the first femtosecond pulse laser and the second femtosecond pulse laser.

8. The endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to claim 1, characterized in that: The endoscope includes an achromatic objective lens and a part of an optical fiber bundle; the part of the optical fiber bundle is connected to the end of the optical fiber bundle, and two near-infrared femtosecond pulse lasers are focused on the measured tissue through the achromatic objective lens in the endoscope.

9. The endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to claim 1, characterized in that: The endoscope is equipped with a handle, and the outer diameter of the working insertion portion of the endoscope is less than 6 mm.

10. An imaging method of an endoscope system based on harmonic imaging and multiphoton fluorescence imaging according to any one of claims 1 to 9, characterized in that: The following steps are involved: The first femtosecond pulse laser and the second femtosecond pulse laser respectively emit near-infrared femtosecond pulse lasers; Two near-infrared femtosecond pulse lasers merge through a prism and enter the XY scanning galvanometer. They then pass through the first long-wavelength dichroic mirror, are coupled into the fiber bundle by the coupling objective lens, and are finally focused on the tissue being tested through an endoscope. Two near-infrared femtosecond pulse lasers excite the tissue under test and its fluorescent substances to generate second harmonics, third harmonics, two-photon fluorescence and three-photon fluorescence, which return along the original path and are reflected by the first long-wave dichroic mirror into the imaging device; Two near-infrared femtosecond pulse lasers scan the measured tissue under the action of an XY scanning galvanometer, and an imaging device obtains a second harmonic image, a third harmonic image, a two-photon fluorescence image, and a three-photon fluorescence image of the measured tissue.

Images