Scanning structured light confocal super-resolution microscopic endoscopic imaging system and method

Abstract

The invention relates to the technical field of optical microscopic imaging, and discloses a scanning structured light confocal super-resolution microscopic endoscopic imaging system and method.The exciting light emitted by a laser light source sequentially passes through an electro-optical modulator, a first dichroic mirror, a pinhole light filter, a two-dimensional scanning galvanometer, a scanning lens, a tube lens, a second dichroic mirror, an objective lens and an optical fiber bundle and then irradiates a sample; the exciting light interacts with a fluorescent marker in the sample to excite the sample to generate fluorescence; fluorescent light generated by a sample is reversely transmitted to the first dichroic mirror according to a light path reversibility principle and then is collected by the PMT detector. According to the confocal microscopic endoscopic imaging system and method, the SIM technology and the confocal microscopic endoscopic imaging technology are combined, super-resolution images are obtained while in-vivo peeping is achieved, the imaging resolution is improved, more accurate and comprehensive iconography information can be provided for clinicians, and lesion and malignant tumor structures can be detected more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of optical microscopic imaging, and in particular to a scanning structured light confocal super-resolution microendoscopic imaging system and method. Background Art

[0002] The confocal microendoscopic imaging system is an emerging real-time, non-destructive in vivo detection technology at the cellular level, which has excellent performance in directly locating lesions and malignant tumors in organs. Compared with traditional optical endoscopes, the system uses a pair of confocal conjugate pinholes to filter out the light signal outside the excitation point and only extract the light signal at the focus. This technical principle gives the system the ability to perform non-destructive, real-time three-dimensional high-resolution detection of deep tissues. It has shown potential for a wide range of application scenarios in clinical settings, covering in vivo diagnosis of diseases such as esophageal squamous cell carcinoma, Barrett's esophagus, celiac disease, and colon polyps.

[0003] At present, the development of confocal microendoscopic imaging systems has provided advanced detection methods for diseases of human internal organs. However, due to the limitation of diffraction limit, the lateral resolution of general imaging is 200nm, which limits the observation and analysis of finer structural details. Therefore, the existing confocal microendoscopic imaging systems have the problem of failing to reach the super-resolution level during the imaging process, and thus cannot accurately detect the fine structure of lesions and malignant tumors.

[0004] In order to overcome the constraints of the diffraction limit on the confocal microendoscopic imaging system, a series of novel super-resolution imaging methods have emerged. The most representative of these is the scanning structured light super-resolution microscopy (SIM). SIM uses an electro-optical modulator to modulate the excitation light to form a cumulative sinusoidal or non-sinusoidal spatial excitation (illumination) stripe pattern on the sample, and improves the imaging resolution by superimposing multiple stripe patterns. This method can effectively overcome the diffraction limit and is conducive to super-resolution imaging. It is worth emphasizing that the SIM technology does not rely on special fluorescent dyes and the device is simple. This means that the dyes that are already routinely used in the confocal microendoscopic imaging system can be used directly without the introduction of additional, specially designed labeling substances. This not only helps to reduce experimental costs, but also maintains the consistency of the experiment, and is easy to integrate into the existing confocal microendoscopic imaging technology. Therefore, the introduction of SIM technology into confocal microendoscopic imaging technology is of great significance and has become an important research topic in this field. Summary of the invention

[0005] The purpose of the present invention is to provide a scanning structured light confocal super-resolution microendoscopy imaging system and method, which combines SIM technology with confocal microendoscopy imaging technology to achieve super-resolution images while peering into the body, thereby improving the resolution of imaging and providing clinicians with more accurate and comprehensive imaging information, which is conducive to more accurate detection of lesions and malignant tumor structures.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] On the one hand, the present invention provides a scanning structured light confocal super-resolution microendoscopic imaging system, which includes: a laser light source, an electro-optical modulator, a first dichroic mirror, a pinhole filter, a two-dimensional scanning galvanometer, a scanning lens, a tube lens, a second dichroic mirror, an objective lens, an optical fiber bundle, and a PMT detector on the optical path after the first dichroic mirror along the reverse transmission optical path of the fluorescence obtained by exciting the sample with the excitation light;

[0008] The laser light source is used to emit excitation light; the excitation light emitted by the laser light source is sequentially passed through an electro-optic modulator, a first dichroic mirror, a pinhole filter, a two-dimensional scanning galvanometer, a scanning lens, a tube lens, a second dichroic mirror, an objective lens, and an optical fiber bundle, and then illuminated on the sample, and the excitation light interacts with the fluorescent marker in the sample to excite the sample to generate fluorescence; wherein the electro-optic modulator is used to modulate the intensity of the excitation light in the form of a required intensity modulation function;

[0009] The fluorescence generated by the sample is transmitted back to the first dichroic mirror according to the reversible principle of the optical path and then collected by the PMT detector.

[0010] Furthermore, a half-wave plate is arranged between the laser light source and the electro-optic modulator.

[0011] Furthermore, a reflecting mirror is arranged between the electro-optic modulator and the first dichroic mirror.

[0012] Furthermore, the pinhole filter includes a first objective lens, a pinhole, and a second objective lens which are sequentially arranged along the excitation light transmission path.

[0013] Furthermore, a first filter is arranged between the tube lens and the second dichroic mirror.

[0014] Furthermore, a second filter and a converging lens are arranged between the first dichroic mirror and the PMT detector.

[0015] Furthermore, the system also includes a computer, which is electrically connected to the electro-optic modulator, the two-dimensional scanning galvanometer and the PMT detector respectively. The computer controls the electro-optic modulator to modulate the intensity of the excitation light in the form of a required intensity modulation function, modulates the excitation light into a time-varying light beam with non-sinusoidal intensity variation, and then controls the two-dimensional scanning galvanometer to sequentially scan the excitation light spots focused with different intensities point by point to the sample surface, and finally processes the fluorescence information according to the structured light super-resolution reconstruction algorithm to obtain a super-resolution image of the sample.

[0016] On the other hand, the present invention also provides a scanning structured light confocal super-resolution microendoscopic imaging method, which is applied to the above-mentioned scanning structured light confocal super-resolution microendoscopic imaging system, comprising the following steps:

[0017] The laser light source emits excitation light;

[0018] The excitation light is intensity modulated by an electro-optic modulator in the form of a desired intensity modulation function, and then filtered by a first dichroic mirror into a pinhole filter;

[0019] After the excitation light is confocalized by the pinhole filter, it is scanned point by point by a two-dimensional scanning galvanometer. After that, the excitation light passes through the scanning lens and the tube lens, and the stray light is filtered out by the first filter, and then the light is split by the second dichroic mirror.

[0020] Afterwards, the excitation light is focused by the objective lens and transmitted to the sample surface through the optical fiber bundle. Then, the excitation light interacts with the fluorescent markers in the sample, exciting the fluorescent markers to produce fluorescence.

[0021] The fluorescence generated by the sample is collected by the optical fiber bundle and transmitted back to the first dichroic mirror in front of the pinhole filter, filtered by the second filter, converged by the converging lens, and finally collected by the PMT detector;

[0022] The PMT detector transmits the fluorescence information to the computer, which processes the fluorescence information according to the structured light super-resolution reconstruction algorithm to obtain a super-resolution image of the sample.

[0023] Furthermore, the wavelength of the excitation light is determined according to the properties of the fluorescent marker to be excited.

[0024] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0025] The present invention combines the confocal microscopic endoscopy imaging technology with the scanning structured light super-resolution microscopic imaging technology to construct a scanning structured light confocal super-resolution microscopic endoscopy imaging system. The electro-optic modulator modulates the excitation light to generate a time-varying light beam with non-sinusoidal intensity changes. The modulated excitation light passes through the confocal imaging optical path (pinhole filter) and is scanned point by point in combination with a two-dimensional galvanometer. After being converged by the objective lens, it reaches the optical fiber bundle; finally, cumulative stripes are formed in the sample, so that the high-frequency information of the sample is mixed into the detectable passband of the microscope; the fluorescence excited by the sample returns along the original path to the PMT detector to collect the fluorescence signal; the detected image is subjected to spectrum analysis to obtain a super-resolution image. It can be seen that the present invention forms a cumulative non-sinusoidally distributed spatial excitation (illumination) stripe pattern on the sample by adding an electro-optical modulator to the optical path for non-sinusoidal modulation, and improves the imaging resolution by superimposing multiple stripe patterns.

[0026] This invention breaks through the constraints of existing technologies. By combining structured light scanning and confocal technology, it can obtain super-resolution images while peering into the body, so as to more accurately detect lesions and malignant tumor structures. The application prospects of this technology cover many aspects of the field of medical imaging, providing clinicians with more accurate and comprehensive imaging information, and providing patients with earlier and more accurate diagnosis and treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0028] Figure 1 A schematic diagram of the structure of the scanning structured light confocal super-resolution microendoscopy imaging system provided by the present invention;

[0029] Figure 2 A schematic diagram of the two-dimensional scanning galvanometer focusing excitation light of different intensities in the present invention;

[0030] Figure 3 The high-frequency information carried in different intensity modulation functions of the present invention, wherein (a) is a sinusoidal intensity modulation waveform, and (b)-(c) are intensity modulation waveforms when n=2, 3, and 6, respectively;

[0031] Explanation of the reference numerals: 1. laser light source; 2. half-wave plate; 3. electro-optic modulator; 4. reflector; 5. first dichroic mirror; 6. first objective lens; 7. pinhole; 8. second objective lens; 9. two-dimensional scanning galvanometer; 10. scanning lens; 11. tube lens; 12. first filter; 13. second dichroic mirror; 14. objective lens; 15. optical fiber bundle; 16. sample; 17. second filter; 18. converging lens; 19. PMT detector; 20. computer. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The scanning structured light confocal super-resolution microendoscopy imaging system and method provided by the present invention combine the confocal microendoscopy imaging technology with the scanning structured light super-resolution microendoscopy imaging technology to construct a complete optical path for scanning structured light confocal super-resolution microendoscopy imaging, and then process the imaging information according to the structured light super-resolution reconstruction algorithm, which is helpful to assist clinicians in providing patients with earlier and more accurate diagnosis and treatment plans.

[0034] The introduction of SIM technology in this invention is a powerful supplement to confocal microendoscopic imaging technology, and its main goal is to overcome traditional limitations and improve the resolution of imaging to more accurately detect lesions and malignant tumor structures. By adopting advanced technical means, these methods achieve higher lateral resolution at the nanometer level, allowing doctors and researchers to observe and analyze the microscopic structure of cells and tissues more clearly.

[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] like Figure 1 As shown, the scanning structured light confocal super-resolution microendoscopy imaging system provided by the present invention comprises: a laser light source 1, a half-wave plate 2, an electro-optical modulator 3, a reflector 4, a first dichroic mirror 5, a pinhole filter (including a first objective lens 6, a pinhole 7, and a second objective lens 8) arranged in sequence along the excitation light transmission optical path, a two-dimensional scanning galvanometer 9, a scanning lens 10, a tube lens 11, a first filter 12, a second dichroic mirror 13, an objective lens 14, an optical fiber bundle 15, and a second filter 17, a lens 18, and a PMT detector 19 on the optical path after the first dichroic mirror 5 along the fluorescence reverse transmission optical path obtained by exciting a sample 16 with the excitation light;

[0037] The laser light source 1 is used to emit excitation light; the excitation light emitted by the laser light source 1 passes through the half-wave plate 2, the electro-optic modulator 3, the reflector 4, the first dichroic mirror 5, the first objective lens 6, the pinhole 7, the second objective lens 8, the two-dimensional scanning galvanometer 9, the scanning lens 10, the tube lens 11, the first filter 12, the second dichroic mirror 13, the objective lens 14, and the optical fiber bundle 15 in sequence, and then is illuminated on the sample 16, and the excitation light interacts with the fluorescent marker in the sample 16 to excite the sample 16 to produce fluorescence; wherein the electro-optic modulator 3 is used to modulate the intensity of the excitation light in the form of a required intensity modulation function;

[0038] The fluorescence generated by the sample 16 is transmitted back to the first dichroic mirror 5 according to the reversible principle of the optical path, passes through the second filter 17 and the lens 18, and is collected by the PMT detector 19.

[0039] The system also includes a computer 20, which is electrically connected to the electro-optic modulator 3, the two-dimensional scanning galvanometer 9 and the PMT detector 19 respectively. The computer 20 controls the electro-optic modulator 3 to modulate the intensity of the excitation light in the form of a required intensity modulation function, modulates the excitation light into a time-varying light beam with non-sinusoidal intensity variation, and then controls the two-dimensional scanning galvanometer 9 to scan the excitation light spots focused with different intensities point by point to the surface of the sample 16 in sequence, and finally processes the fluorescence information according to the structured light super-resolution reconstruction algorithm to obtain a super-resolution image of the sample 16.

[0040] The required intensity modulation function forms include linear modulation mode operation and nonlinear modulation mode operation.

[0041] When the electro-optic modulator works in 3-phase linear modulation mode, the intensity of the excitation light changes with time in a sinusoidal function, and its expression is:

[0042]

[0043] Where I is the required excitation light intensity, k a is the modulation frequency of the structured light, which is usually consistent with the diffraction limit of the imaging system. Figure 3 (a) shows a sinusoidal intensity modulation waveform with only one peak in one cycle. After Fourier transform, in the frequency domain, in addition to the baseband information, it only contains a high-frequency information k a .

[0044] When the electro-optic modulator 3 works in the nonlinear modulation mode, the intensity of the excitation light changes with time in a non-sinusoidal form, and the expression is:

[0045]

[0046] Among them, n represents the number of terms of the cosine function, that is, the multiple of the highest frequency.

[0047] When n=2,3,6, the intensity modulation waveforms are shown in Figure 2. Figure 3 In (b)-(d), it can be seen from the figure that within one cycle, there are multiple peaks, and each additional peak in the frequency domain (Fourier transform) indicates the existence of one more frequency multiplication information. The present invention uses SIM technology to move the high-frequency information carried in the modulation function to the baseband and parse it out through a reconstruction algorithm. The more high-frequency information is carried, the higher the imaging resolution of the system.

[0048] The working principle of the scanning structured light confocal super-resolution microendoscopy imaging system provided by the present invention is as follows:

[0049] The laser light source 1 uses a semiconductor laser to generate excitation light, and the wavelength of the excitation light depends on the properties of the fluorescent marker to be excited. The excitation light is modulated by a half-wave plate 2 and an electro-optical modulator 3, wherein the half-wave plate 2 is used to adjust the polarization state of the excitation light, and the electro-optical modulator 3 is arranged opposite to the half-wave plate 2, and is used to modulate the excitation light into a time-varying light beam whose intensity changes in the form of a required modulation function. By adjusting the half-wave plate 2, the modulation efficiency of the electro-optical modulator 3 is ensured to be the highest, and when the excitation light passes through the electro-optical modulator 3, the laser intensity is modulated in the form of a required intensity modulation function. Subsequently, the excitation light is filtered by the first dichroic mirror 5, and then enters the pinhole filter composed of two identical objective lenses 6, 8 and a pinhole 7. Since the electro-optical modulator 3 has modulated the excitation light into the required intensity modulation form, it is necessary to use a two-dimensional scanning galvanometer 9 to scan the excitation light spots focused with different intensities point by point to the surface of the sample 16 in turn. In this way, structured illumination is formed on the surface of the sample, and the intensity of the excitation light changes with position, as shown in FIG. Figure 2 As shown in the figure, circles of different colors represent different excitation light intensities, and t represents the degree of structured illumination formed on the sample surface at each moment.

[0050] After being scanned by the two-dimensional scanning galvanometer 9, the excitation light passes through the scanning lens 10 and the tube lens 11, and the stray light is filtered out by the second filter 12, and then the light is split by the second dichroic mirror 13. After that, the excitation light is focused by the objective lens 14 and transmitted to the surface of the sample 16 through the optical fiber bundle 15. Then, the excitation light interacts with the fluorescent markers in the sample, and excites these markers to produce fluorescence. The fluorescence generated by the sample is collected and transmitted back to the optical system to the first dichroic mirror 5 in front of the pinhole filter, filtered by the second filter 17, converged by the converging lens 18, and finally collected by the PMT detector 19. Since the excitation light, the sample imaging surface and the pinhole are conjugated to each other, this means that only signals near the confocal point can be received, thereby improving the tomographic capability of imaging. Then, an intensity-type electro-optical modulator is used to modulate the laser into a time-varying beam with non-sinusoidal intensity changes, and then the imaging information is processed according to the structured light super-resolution algorithm to obtain a super-resolution image of the sample. This technology optimizes the existing confocal super-resolution microendoscopy system, giving it excellent super-resolution imaging capabilities.

[0051] The present invention provides a scanning structured light confocal super-resolution microendoscopy imaging method, comprising the following steps:

[0052] The laser light source 1 emits an excitation light of a specific wavelength; the wavelength of the excitation light is determined according to the properties of the fluorescent marker to be excited;

[0053] The excitation light passes through the half-wave plate 2 and the electro-optic modulator 3 to be intensity modulated in the form of the required intensity modulation function, and then is filtered by the reflector 4 and the first dichroic mirror 5 to enter the pinhole filter;

[0054] After the excitation light is confocalized by the pinhole filter, it is scanned point by point by the two-dimensional scanning galvanometer 9. After that, the excitation light passes through the scanning lens 10 and the tube lens 11, the stray light is filtered out by the first filter 12, and then the light is split by the second dichroic mirror 13;

[0055] Afterwards, the excitation light is focused by the objective lens 14 and transmitted to the surface of the sample 16 through the optical fiber bundle 15. Then, the excitation light interacts with the fluorescent marker in the sample 16 to excite the fluorescent marker to produce fluorescence.

[0056] The fluorescence generated by the sample 16 is collected by the optical fiber bundle 15 and transmitted back to the first dichroic mirror 5 in front of the pinhole filter, filtered by the second filter 17, converged by the lens 18, and finally collected by the PMT detector 19;

[0057] The PMT detector 19 transmits the fluorescence information to the computer 20 , and the computer 20 processes the fluorescence information according to a structured light super-resolution reconstruction algorithm to obtain a super-resolution image of the sample 16 .

[0058] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A scanning structured light confocal super-resolution microendoscopy imaging system, characterized in that: include: A laser light source (1), an electro-optic modulator (3), a first dichroic mirror (5), a pinhole filter, a two-dimensional scanning galvanometer (9), a scanning lens (10), a tube lens (11), a second dichroic mirror (13), an objective lens (14), an optical fiber bundle (15) are sequentially arranged along an excitation light transmission optical path, and a PMT detector (19) is arranged after the first dichroic mirror (5) along a reverse transmission optical path of fluorescence obtained by exciting a sample (16) with the excitation light; The laser light source (1) is used to emit excitation light; the excitation light emitted by the laser light source (1) is sequentially passed through an electro-optic modulator (3), a first dichroic mirror (5), a pinhole filter, a two-dimensional scanning galvanometer (9), a scanning lens (10), a tube lens (11), a second dichroic mirror (13), an objective lens (14), and an optical fiber bundle (15) and then illuminated onto a sample (16); the excitation light interacts with a fluorescent marker in the sample (16) to excite the sample (16) to generate fluorescence; wherein the electro-optic modulator (3) is used to modulate the intensity of the excitation light in the form of a desired intensity modulation function; The fluorescence generated by the sample (16) is transmitted back to the first dichroic mirror (5) according to the reversible principle of the optical path, and then collected by the PMT detector (19).

2. The scanning structured light confocal super-resolution microendoscopy imaging system according to claim 1, characterized in that: A half-wave plate (2) is also provided between the laser light source (1) and the electro-optic modulator (3).

3. The scanning structured light confocal super-resolution microendoscopy imaging system according to claim 1, characterized in that: A reflecting mirror (4) is also provided between the electro-optic modulator (3) and the first dichroic mirror (5).

4. The scanning structured light confocal super-resolution microendoscopy imaging system according to claim 1, characterized in that: The pinhole filter comprises a first objective lens (6), a pinhole (7), and a second objective lens (8) which are arranged in sequence along the excitation light transmission path.

5. The scanning structured light confocal super-resolution microendoscopy imaging system according to claim 1, characterized in that: A first filter (12) is also provided between the tube lens (11) and the second dichroic mirror (13).

6. The scanning structured light confocal super-resolution microendoscopy imaging system according to claim 1, characterized in that: A second filter (17) and a converging lens (18) are also provided between the first dichroic mirror (5) and the PMT detector (19).

7. The scanning structured light confocal super-resolution microendoscopy imaging system according to claim 1, characterized in that: The system further comprises a computer (20), wherein the computer (20) is electrically connected to the electro-optic modulator (3), the two-dimensional scanning galvanometer (9) and the PMT detector (19) respectively. The computer (20) controls the electro-optic modulator (3) to modulate the intensity of the excitation light in the form of a required intensity modulation function, modulating the excitation light into a time-varying light beam with a non-sinusoidal intensity variation, and then controls the two-dimensional scanning galvanometer (9) to sequentially scan the excitation light spots focused with different intensities point by point onto the surface of the sample (16), and finally processes the fluorescence information according to a structured light super-resolution reconstruction algorithm to obtain a super-resolution image of the sample (16).

8. A scanning structured light confocal super-resolution microendoscopy imaging method, characterized in that: The scanning structured light confocal super-resolution microendoscopy imaging system applied to any one of claims 1 to 7 comprises the following steps: The laser light source (1) emits excitation light; The excitation light is intensity modulated in the form of a desired intensity modulation function by an electro-optical modulator (3), and then filtered by a first dichroic mirror (5) to enter a pinhole filter; After the excitation light is confocalized by a pinhole filter, it is scanned point by point by a two-dimensional scanning galvanometer (9). After that, the excitation light passes through a scanning lens (10) and a tube lens (11), and then is filtered out of stray light by a first filter (12), and then is split by a second dichroic mirror (13); Afterwards, the excitation light is focused by the objective lens (14) and transmitted to the surface of the sample (16) through the optical fiber bundle (15). Then, the excitation light interacts with the fluorescent marker in the sample (16) to excite the fluorescent marker to generate fluorescence; The fluorescence generated by the sample (16) is collected by the optical fiber bundle (15) and transmitted back to the first dichroic mirror (5) in front of the pinhole filter, filtered by the second filter (17), converged by the converging lens (18), and finally collected by the PMT detector (19); The PMT detector (19) transmits the fluorescence information to the computer (20), and the computer (20) processes the fluorescence information according to the structured light super-resolution reconstruction algorithm to obtain a super-resolution image of the sample (16).

9. The scanning structured light confocal super-resolution microendoscopy imaging method according to claim 8, characterized in that: The wavelength of the excitation light is determined according to the properties of the fluorescent marker to be excited.

Images