Confocal endoscope imaging system based on structured light illumination

Abstract

The invention discloses a confocal endoscope imaging system based on structured light illumination. The system comprises a laser light source used for generating exciting light; the half-wave plate is placed on a light path of exciting light emitted by the exciting light source; the electro-optical modulator is placed on a light path after the exciting light passes through the half-wave plate; the excitation light filter is placed on a light path of the excitation light modulated by the electro-optical modulator; the dichroic mirror is placed on a light path after the exciting light passes through the exciting light filter; the first objective lens is placed on a light path after the exciting light passes through the dichroic mirror; and the confocal pinhole is placed on a light path formed after the exciting light is converged by the first objective lens. According to the confocal endoscope imaging system based on structured light illumination, the defect that an existing confocal endoscope imaging system does not have the super-resolution imaging capability is overcome, the technical problem that the resolution ratio is insufficient during microscopic imaging is solved, and industrial application of the system is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of super-resolution imaging, and particularly to a confocal endoscope imaging system based on structured light illumination. Background Art

[0002] Confocal microscopy imaging technology is a technology that has developed rapidly in the past decade or so. Currently, it is widely used in disciplines such as cytology, microbiology, developmental biology, genetics, neurobiology, physiology, and pathology, and has become one of the essential tools in modern biology. However, due to the limited penetration depth of laser in tissue and the serious influence of tissue scattering, its application range is limited to the tissue surface. Even when using a two-photon microscope with a relatively long wavelength, the imaging depth is still limited to a few hundred micrometers on the surface, and it cannot image the internal tissues and organs of organisms. The imaging resolution is still greater than dozens of micrometers, making it difficult to provide a resolution capable of observing at the cellular level. Summary of the Invention

[0003] In view of the above technical problems, the present invention proposes a confocal endoscope imaging system based on structured light illumination.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] The present invention provides a confocal endoscope imaging system based on structured light illumination, and the system includes:

[0006] A laser light source for generating excitation light;

[0007] A half-wave plate placed on the optical path of the excitation light emitted by the excitation light source;

[0008] An electro-optic modulator placed on the optical path of the excitation light after passing through the half-wave plate;

[0009] An excitation filter placed on the optical path of the excitation light after being modulated by the electro-optic modulator;

[0010] A dichroic mirror placed on the optical path of the excitation light after passing through the excitation filter;

[0011] A first objective lens placed on the optical path of the excitation light after passing through the dichroic mirror;

[0012] A confocal pinhole placed on the optical path of the excitation light converged by the first objective lens;

[0013] A second objective lens placed on the optical path of the excitation light after passing through the confocal pinhole;

[0014] A two-dimensional scanner placed on the optical path of the excitation light collimated by the second objective lens, and the two-dimensional scanner is a two-dimensional galvanometer or an acousto-optic device;

[0015] A scanning lens placed on the optical path after the excitation light is two-dimensionally scanned by the two-dimensional scanner, and the two-dimensional scanner is placed on the focal plane of the scanning lens;

[0016] A tube lens placed on the optical path after the excitation light passes through the scanning lens, and the excitation light is collimated by the tube lens and then passes through the third objective lens;

[0017] An optical fiber bundle placed on the optical path after the excitation light is converged by the third objective lens;

[0018] A microscopic lens placed on the optical path after the excitation light excites the optical fiber bundle;

[0019] An emission filter placed on the optical path after the fluorescence obtained by the excitation light exciting the sample is reflected by the dichroic mirror;

[0020] A focusing lens placed on the optical path after the fluorescence is filtered by the emission filter;

[0021] A detector placed on the optical path after the fluorescence is focused by the focusing lens, the detector is placed on the rear focal plane of the focusing lens, and the detector is a photomultiplier tube.

[0022] Further, the electro-optic modulator is placed between the half-wave plate and the excitation filter, the first objective lens and the emission filter are respectively on the opposite sides of the dichroic mirror, the confocal pinhole is placed between the first objective lens and the second objective lens, the tube lens is placed between the scanning lens and the third objective lens, and the optical fiber bundle connects the microscopic lens and the third objective lens.

[0023] Further, the electro-optic modulator, the two-dimensional scanner, and the detector are all controlled, displayed, and stored by a mobile communication terminal.

[0024] The confocal endoscope imaging system based on structured light illumination of the present invention generates a structured illumination pattern by sinusoidally modulating the excitation light through an electro-optic modulator. The structured illumination pattern interacts with the sample structure to produce a Moiré fringe effect, shifting the originally undetectable high-frequency information into the microscope passband to achieve a higher resolution imaging level. The confocal endoscope imaging system based on structured light illumination of the present invention makes up for the lack of super-resolution imaging ability in the existing confocal endoscope imaging system, solves the technical problem of insufficient resolution in microscopic imaging, and is conducive to the industrial application of the system. Description of the Drawings

[0025] Figure 1 It is a structural diagram of the confocal endoscope imaging system based on structured light illumination of the present invention;

[0026] Figure 2 is Figure 1 a schematic diagram of the Moiré effect of the confocal endoscope imaging system based on structured light illumination shown in

[0027] Figure 3 is Figure 1 a schematic diagram of the ordinary wide - field spectrum of an embodiment of the confocal endoscope imaging system based on structured light illumination shown in

[0028] Figure 4 is Figure 1 the ordinary wide - field spectrum of another embodiment of the confocal endoscope imaging system based on structured light illumination shown in

[0029] Figure 5 is Figure 1 a schematic diagram of the extended spectrum of the single - frequency - direction structured light of the confocal endoscope imaging system based on structured light illumination shown in

[0030] Figure 6 is Figure 1 a schematic diagram of the extended spectrum of the multi - direction structured light of the confocal endoscope imaging system based on structured light illumination shown in

[0031] In the figure: the excitation light source 1, the half - wave plate 2, the electro - optic modulator 3, the excitation filter 4, the dichroic mirror 5, the first objective lens 6, the confocal pinhole 7, the second objective lens 8, the two - dimensional scanner 9, the scanning lens 10, the tube lens 11, the third objective lens 12, the fiber optic bundle 13, the microscopic lens 14, the sample 15, the emission filter 16, the focusing lens 17, the detector 18. Specific embodiments

[0032] Please refer to Figures 1-6 , the present invention provides a confocal endoscope imaging system based on structured light illumination, and the system includes:

[0033] a laser light source 1 for generating excitation light;

[0034] a half - wave plate 2 placed on the optical path of the excitation light emitted by the excitation light source;

[0035] an electro - optic modulator 3 placed on the optical path of the excitation light after passing through the half - wave plate;

[0036] an excitation filter 4 placed on the optical path of the excitation light after being modulated by the electro - modulator;

[0037] a dichroic mirror 5 placed on the optical path of the excitation light after passing through the excitation filter;

[0038] a first objective lens 6 placed on the optical path of the excitation light after passing through the dichroic mirror;

[0039] The confocal pinhole 7 placed on the optical path after the excitation light is converged by the first objective lens;

[0040] The second objective lens 8 placed on the optical path after the excitation light passes through the confocal pinhole;

[0041] The two-dimensional scanner 9 placed on the optical path after the excitation light is collimated by the second objective lens;

[0042] The scanning lens 10 placed on the optical path after the excitation light is two-dimensionally scanned by the two-dimensional scanner, and the two-dimensional scanner is placed on the focal plane of the scanning lens;

[0043] The tube lens 11 placed on the optical path after the excitation light passes through the scanning lens, and the excitation light passes through the third objective lens 12 after being collimated by the tube lens;

[0044] The fiber bundle 13 placed on the optical path after the excitation light is converged by the third objective lens;

[0045] The microscopic lens 14 placed on the optical path after the excitation light excites the fiber bundle;

[0046] The emission filter 16 placed on the optical path after the fluorescence obtained by the excitation light exciting the sample 15 is reflected by the dichroic mirror;

[0047] The focusing lens 17 placed on the optical path after the fluorescence is filtered by the emission filter;

[0048] The detector 18 placed on the optical path after the fluorescence is focused by the focusing lens, and the detector is placed on the rear focal plane of the focusing lens.

[0049] In one embodiment, the electro-optic modulator is placed between the half-wave plate and the excitation filter.

[0050] In one embodiment, the two opposite sides of the dichroic mirror are the first objective lens and the emission filter respectively.

[0051] In one embodiment, the confocal pinhole is placed between the first objective lens and the second objective lens, and the tube lens is placed between the scanning lens and the third objective lens.

[0052] In one embodiment, the fiber bundle connects the microscopic lens and the third objective lens.

[0053] In one embodiment, it is placed between the emission filter and the detector.

[0054] The working principle of the confocal endoscope imaging system based on structured light illumination provided by the present invention is:

[0055] The conventional microscope imaging process can be regarded as a linear shift-invariant system, and the imaging process can be expressed by the formula:

[0056]

[0057] where represents the light field intensity distribution on the image plane, characterizing the sample information obtained by the microscope system, represents the light field intensity distribution of the light emerging from the sample surface, and PSF represents the point spread function of the microscope system, represents the convolution operation. Performing Fourier transform on the above formula, the frequency domain form is obtained:

[0058]

[0059] where OTF is the optical transfer function obtained by the microscope system and is the frequency domain representation of PSF, represents the frequency domain information of the sample image obtained by the microscope, represents the frequency domain information of the light field emerging from the sample surface. For a conventional fluorescence microscope, OTF limits the amount of information passing through the microscope system, only allowing low-frequency information with frequencies lower than the cut-off frequency to pass through the system, filtering out the high-frequency information representing details, thus limiting the resolution of the system.

[0060] For a microscope system, the two-dimensional PSF can be characterized by Bessel function or Gaussian function. Therefore, the passband of OFT in the transverse plane can be approximately represented by a circle, and the radius of the circle is approximately equal to the reciprocal of the minimum distance between any two points that can be resolved in the sample, that is, the cut-off frequency, which characterizes the resolution of the microscope. For a fluorescence microscope, when the excitation light intensity is weak, the fluorescence light field emitted by the sample has a linear relationship with the intensity distribution of the illumination light field

[0061]

[0062] is the fluorescence molecule concentration (characterizing the structural information of the sample). For a conventional fluorescence microscope using uniform field illumination, is a constant Therefore, can be used to characterize the structural information of the sample. According to the microscope imaging formula, the size of OFT directly limits the amount of sample information passing through the microscope system. The key to breaking through the resolution limit of the structured light illumination confocal endoscopy imaging system lies not in directly changing the size of OTF, but in changing the pattern of the illumination light, that is, using illumination light with a specific structure to replace During the imaging process, a part of the high-frequency information located outside the OTF circle is transferred into the circle, and a specific algorithm is used to move the high-frequency information moved into the OTF circle back to its original position, thereby expanding the frequency-domain information of the sample image, which is equivalent to indirectly expanding the size of the OTF and enabling the resolution of the reconstructed image to break through the limitations of the original system.

[0063] Structured light illumination generally uses a cosine-form illumination light, as shown in the following formula:

[0064]

[0065] Where, I 0 and are the average intensity and initial phase of the cosine illumination light fringe respectively, and k 0 is the spatial frequency of the cosine illumination fringe, that is, the reciprocal of the fringe period. Converting to the frequency-domain form:

[0066]

[0067] As shown by the above formula, are δ functions at three positions, from which the following can be obtained:

[0068]

[0069] The above formula shows that the role of the cosine structured illumination fringe is to replicate the frequency-domain information of the sample into three parts, and the zero-frequency positions of two of them are located at -k x and +k x respectively. Therefore, contains sample information with three different zero-frequency positions. Since the OTF only depends on the microscope system, that is, the size and position of the circle in the figure remain unchanged, the sample image recorded by the detector contains three parts of information, and the entire imaging process is equivalent to moving the information originally outside the OTF into the OFT:

[0070]

[0071] Where:

[0072]

[0073] DN represents the low-frequency information of the obtained sample, which is equivalent to the frequency-domain information of the sample obtained by a conventional fluorescence microscope; DP and DS represent the high-frequency information obtained by the detector with zero-frequency positions at -k x and +k x respectively. A specific reconstruction algorithm can move the high-frequency information moved into the OTF back to its original position, expand the frequency-domain information of the sample obtained by the detector, and change k 0in a direction to expand the frequency-domain information in all directions, so as to uniformly expand the frequency-domain information, improve the resolution of the system, and break through the limitation of the system resolution.

[0074] For the acquisition of three-dimensional tomographic images, first, an excitation light is emitted by a laser light source 1. The excitation light can be continuous light or pulsed light, and its wavelength is located in the absorption region of a fluorescent dye or autofluorescent substance. Then, after passing through a half-wave plate 2, the excitation light is modulated by an electro-optic modulator 3, and the intensity of the modulated laser light changes rapidly with time in a cosine manner. Then, the excitation light filtered by an excitation filter 4 is transmitted through a dichroic mirror 5 to a first objective lens 6, and is converged by the first objective lens 6 to a confocal pinhole 7. The confocal pinhole 7 is a confocal pinhole with adjustable size, and can adjust parameters such as signal-to-noise ratio, contrast, and resolution to increase practicability. Then, the excitation light emitted from the confocal pinhole 7 is collected and collimated by a second objective lens 8 and enters a two-dimensional scanner 9. Then, the excitation light deflected by the two-dimensional scanner 9 enters a scanning lens 10 and then enters a tube lens 11 to form a parallel light beam. Then, the excitation light collimated by the tube lens 11 enters a third objective lens 12, and is converged and coupled into a certain optical fiber in an optical fiber bundle 13 by the third objective lens 12. Then, after being emitted from the corresponding optical fiber at the other end of the optical fiber bundle 13, it is focused on a sample 15 by a microscopic lens 14. Then, according to the principle of reversibility of the optical path, the fluorescence excited by the sample 15 by the excitation light is coupled into the corresponding optical fiber in the optical fiber bundle 13 by the microscopic lens 14, and then is transmitted through the third objective lens 12 to the tube lens 11. Then, it passes through the scanning lens 10, the two-dimensional scanner 9, and the second objective lens 8 in sequence and enters the confocal pinhole 7. The confocal pinhole 7 can block the fluorescence collected and transmitted by other optical fibers except the corresponding optical fiber that transmits the excitation light in the fluorescence beam. Then, the fluorescence filtered by the confocal pinhole 7 forms a collimated light by the first objective lens 6. This collimated light is then reflected by the dichroic mirror 5 to an emission filter 16, and the emission filter 16 blocks the remaining excitation light and filters out the fluorescence. Then, the filtered fluorescence is focused on a detector 18 by a focusing lens 17, and a confocal fluorescence super-resolution microscopic image is obtained by the detector 18.

[0075] In the present invention, in order to obtain a confocal fluorescence super-resolution microscopic image of the sample 15, each time the two-dimensional scanner 9 deflects the beam of the excitation light by one step, the optical fiber coupled into the fiber bundle 13 by focusing of the third objective lens 12 also changes accordingly, resulting in a corresponding change in the position of the excitation light spot in the sample 15, so as to achieve the purpose of scanning the sample. At the same time, with the modulation of the electro-optic modulator 3, the sinusoidal change of the light intensity with time is converted into a sinusoidal change of the light intensity with position on the sample 15, forming a sinusoidal illumination fringe. The fluorescence collected by other optical fibers in the fiber bundle 13 is blocked by the confocal pinhole 7. In this way, compared with the traditional method of only using the aperture of the optical fiber in the fiber bundle to realize the confocal optical path, this system introduces the confocal pinhole 7 to achieve true conjugate imaging, and can filter out the interference fluorescence signals collected by other adjacent optical fibers in the fiber bundle, while the traditional method using the aperture of the optical fiber itself cannot achieve this effect. In addition, the confocal pinhole 7 is in the self-scanning circuit, which not only realizes the effect of the excitation light spot light source, but also realizes the control of the point object light emission. The structure is simple and the performance is stable, which is beneficial to the industrialization of the system.

[0076] In one embodiment, the two-dimensional scanner 9 can be a two-dimensional galvanometer, an acousto-optic device or other scanners, and can realize raster scanning or random scanning.

[0077] In one embodiment, the dichroic mirror 5 has the characteristics of total reflection of the wide-field imaging laser and high transmission of the excitation light and fluorescence.

[0078] In one embodiment, the detector 18 can be a photomultiplier tube (PMT) or other single-point detector;

[0079] In one embodiment, the electro-optic modulator 3, the two-dimensional scanner 9, and the detector 18 are all controlled, displayed, and stored by a mobile communication terminal. Specifically, the mobile communication terminal can be a computer.

[0080] The confocal endoscope imaging system based on structured light illumination of the present invention introduces an electro-optic modulator to sinusoidally modulate the excitation light to generate a structured illumination pattern. The structured illumination pattern interacts with the sample structure to produce a Moiré fringe effect, and shifts the originally undetectable high-frequency information into the microscope passband to achieve a higher resolution imaging level. The confocal endoscope imaging system based on structured light illumination of the present invention makes up for the lack of super-resolution imaging ability of the existing confocal endoscope imaging system, solves the technical problem of insufficient resolution in microscopic imaging, and is beneficial to the industrial application of the system.

[0081] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not limited to the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A confocal endoscope imaging system based on structured light illumination, characterized in that: The system comprises: A laser light source for generating excitation light; A half-wave plate placed on the optical path of the excitation light emitted by the excitation light source; An electro-optic modulator placed on the optical path of the excitation light after passing through the half-wave plate; an excitation filter placed on the optical path of the excitation light after being modulated by the electronic modulator; a dichroic mirror placed on the optical path of the excitation light after it passes through the excitation filter; A first objective lens placed on the optical path of the excitation light after it passes through the dichroic mirror; a confocal pinhole placed on the optical path of the excitation light after it is converged by the first objective lens; A second objective lens placed on the optical path of the excitation light after passing through the confocal pinhole; A two-dimensional scanner placed on the optical path of the excitation light after being collimated by the second objective lens, wherein the two-dimensional scanner is a two-dimensional galvanometer or an acousto-optic device; A scanning lens is placed on the optical path of the excitation light after being two-dimensionally scanned by the two-dimensional scanner, and the two-dimensional scanner is placed on the focal plane of the scanning lens; a tube lens placed on the optical path of the excitation light after it passes through the scanning lens, and the excitation light passes through a third objective lens after being collimated by the tube lens; An optical fiber bundle placed on the optical path of the excitation light after it is converged by the third objective lens; a microlens placed on the optical path after the excitation light excites the optical fiber bundle; An emission filter placed on the optical path of the fluorescence obtained by the excitation light exciting the sample after being reflected by the dichroic mirror; A focusing lens placed on the optical path of the fluorescence after it is filtered by the emission filter; A detector is placed on the optical path of the fluorescence after it is focused by the focusing lens. The detector is placed on the rear focal plane of the focusing lens. The detector is a photomultiplier tube.

2. The confocal endoscopic imaging system based on structured light illumination according to claim 1, characterized in that: The electro-optic modulator is placed between the half-wave plate and the excitation filter, the first objective lens and the emission filter are located on opposite sides of the dichroic mirror, the confocal pinhole is placed between the first objective lens and the second objective lens, the tube lens is placed between the scanning lens and the third objective lens, and the optical fiber bundle connects the microlens and the third objective lens.

3. The confocal endoscopic imaging system based on structured light illumination according to claim 2, characterized in that: The electro-optic modulator, the two-dimensional scanner and the detector are all controlled, displayed and stored by the mobile communication terminal.

Images