A control method for multi-modal data acquisition for in situ pathology analysis of tumors

Abstract

The application discloses a control method for multi-modal data acquisition of tumor in-situ pathological analysis, which is executed by a main controller, integrates a system of SC laser, DUV laser, multiplexing beam combiner, endoscopic fiber bundle, FAST fiber bundle and corresponding spectrometer, and realizes automatic acquisition and fusion of three kinds of spectral data. The method comprises the following steps: S1, controlling the SC laser to irradiate a detection point, and controlling the SC spectrometer to receive a reflected light signal and generate a first spectral data set; S2, controlling the DUV laser to irradiate the detection point, and controlling the DUV spectrometer to receive excited Raman and fluorescence signals and generate a second spectral data set; S3, controlling the polarized excitation light to irradiate the detection point, and controlling the SC spectrometer to receive the fluorescence signal and generate a third spectral data set; and S4, based on the first three kinds of data sets, reconstructing single-column signals at a near end of the FAST fiber bundle according to a far end square array arrangement to generate hyperspectral image data of the detection point. The method realizes automatic acquisition of in-vivo, in-situ and multi-dimensional chemical information.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of photoelectric detection, and particularly relates to a control method for multi-modal data acquisition of in-situ pathological analysis of tumors. BACKGROUND

[0002] Currently, in the field of in-situ component analysis in a minimally invasive and deep environment, two types of technical paths are mainly relied on: one is morphological analysis based on endoscopic imaging, and the other is in-vitro laboratory analysis based on sampling. However, both types of technical paths have significant technical performance bottlenecks. Endoscopic imaging technology mainly provides morphological information of the target, and generally lacks the ability to perform real-time, in-situ and quantitative analysis of the biochemical components of the target substance, resulting in a single dimension of the output information and a fundamental limitation on the accuracy and reliability of the analysis. On the other hand, although in-vitro laboratory analysis is a benchmark method for component analysis, its process is complex and time-consuming, and there is a serious timeliness problem, which cannot realize real-time in-vivo analysis feedback and is difficult to meet the technical expectations of modern precision analysis systems for “detection-analysis-decision” integration.

[0003] The above technical bottlenecks are particularly prominent in specific application scenarios such as analysis of deep lung tissue. The bronchial structure of the lung is complex and finely graded. To effectively analyze early-stage small targets in-situ, the analysis device must have the technical capability to reach and act on the distal region of the 14th grade and above, such as the terminal bronchioles. However, most existing in-situ analysis devices are limited by their physical size and structural design, and their effective working range usually only reaches the 6th-8th grade bronchus, resulting in a huge “depth gap”, which fundamentally limits the direct analysis capability of deep targets.

[0004] Existing technical solutions expose obvious technical defects and operational risks when trying to solve the above depth problem. For example, ultra-fine endoscopic devices for G4-G8 grade bronchus, even combined with ultrasound or electromagnetic navigation technology, have a too narrow working channel and a severely limited optical field of view, resulting in unsatisfactory performance of the end analysis tool and success rate of sampling analysis. For more distal >G8 targets, existing technologies often rely on external means such as CT-guided percutaneous puncture, which not only exposes the system to a radiation environment, but also accompanies high-risk technical risks such as pneumothorax and bleeding, and is more invasive, which does not conform to the trend of minimally invasive analysis. The emergence of the bronchoscope robot NBSR platform provides a new technical basis for solving the “reach” problem from the perspective of precise mechanical transmission, and its hollow catheter with an inner diameter of about 1 mm creates an unprecedented physical channel for the introduction of minimally invasive in-situ analysis tools.

[0005] The NBSR robot platform provides the possibility to solve the deep access, but there is currently a lack of a special in-situ component analysis system capable of fully utilizing the advantages of the platform, and there is no system in the prior art that can meet the following key technical requirements: 1) miniaturization: capable of passing through the approximately 1mm inner diameter working channel of the NBSR; 2) multi-modal: capable of integrating and switching multiple complementary analysis modes (such as reflection spectrum, Raman spectrum, fluorescence spectrum, etc.) in a deep and narrow space to obtain multi-dimensional component information; 3) high precision and high signal-to-noise ratio: still capable of realizing high-sensitivity spectrum acquisition under minimally invasive conditions; 4) automation and integration: realizing automatic switching of multiple analysis modes, coordinated control of light sources and detectors, and rapid transmission and preprocessing of data.

[0006] Therefore, there is an urgent need in the art for a new multi-modal data acquisition method capable of deep adaptation with minimally invasive access platforms such as NBSR, in order to overcome the core technical difficulties of the prior art, such as inability to access deep, single information dimension, low system integration, complex operation, and high risk. SUMMARY

[0007] The purpose of the present application is to provide a multi-modal data acquisition control method for in-situ pathological analysis of tumors, which addresses the problems in the prior art.

[0008] To this end, the above-mentioned purpose of the present application is achieved by the following technical solutions:

[0009] A multi-modal data acquisition control method for in-situ pathological analysis of tumors, the method is executed by a main controller, the main controller is configured to control a multi-modal data acquisition system, the system includes an SC laser, a DUV laser, a multiplexing beam combiner, an endoscopic fiber bundle, a FAST fiber bundle, an SC spectrometer, and a DUV spectrometer,

[0010] The method includes the following steps: controlling the system to perform SC reflection spectrum acquisition, DUV joint spectrum acquisition, and polarization fluorescence spectrum acquisition by the main controller,

[0011] S1, SC reflection spectrum data is collected, the supercontinuum spectrum laser is controlled to irradiate the detection point, and the SC spectrometer is controlled to receive the reflected light signal from the detection point to generate a first spectrum data set;

[0012] S2, DUV joint spectrum data is collected, the deep ultraviolet laser is controlled to irradiate the detection point, and the DUV spectrometer is controlled to receive the light signal excited thereby to generate a second spectrum data set;

[0013] S3, collecting polarization fluorescence spectrum data, controlling the polarization excitation light to be coupled to the endoscope fiber bundle through the multiplexing beam combiner and the conjugate coupling module and transmitted to a detection point; at the same time, controlling the SC spectrometer to receive the fluorescence signal returned from the detection point to generate a third spectrum data set;

[0014] S4, hyperspectral data reconstruction, based on the first spectrum data set, the second spectrum data set and the third spectrum data set, the signals corresponding to the single-column arrangement of the proximal end of the FAST fiber bundle are image reconstructed according to the square fiber array arrangement of the distal end of the endoscope fiber bundle to generate hyperspectral image data of the detection point.

[0015] In addition to the above technical solutions, the application can also use or combine the following technical solutions:

[0016] As a preferred technical solution of the application: in step S1, the supercontinuum laser is coupled to the endoscope fiber bundle through the multiplexing beam combiner and the conjugate coupling module and transmitted to a detection point;

[0017] The SC spectrometer receives the light signal returned from the detection point through the endoscope fiber bundle and the conjugate coupling module to generate a first spectrum data set.

[0018] As a preferred technical solution of the application: in step S2, the deep ultraviolet laser is coupled to the endoscope fiber bundle through the multiplexing beam combiner and the conjugate coupling module and transmitted to the detection point;

[0019] The DUV spectrometer receives the light signal returned from the detection point through the endoscope fiber bundle and the conjugate coupling module to generate a second spectrum data set.

[0020] As a preferred technical solution of the application: in step S3, the polarization excitation light is coupled to the endoscope fiber bundle through the multiplexing beam combiner and the conjugate coupling module and transmitted to the detection point;

[0021] The SC spectrometer receives the fluorescence signal returned from the detection point through the endoscope fiber bundle and the conjugate coupling module to generate a third spectrum data set.

[0022] As a preferred technical solution of the application: the method further comprises: when switching to a different spectrum collection mode, moving the proximal end arrangement of the FAST fiber bundle to align with the entrance slit of the SC spectrometer or the DUV spectrometer corresponding to the current mode.

[0023] As a preferred technical scheme of the present application: before the step S3 of collecting polarization fluorescence spectrum data, an in-situ pretreatment step is further included: controlling the endoscopic optical fiber bundle to move out and be connected to a plurality of pretreatment function heads in sequence, and completing fluorescence labeling at the detection point.

[0024] Compared with the prior art, the control method for multi-modal data acquisition for in-situ pathological analysis of tumors has the following beneficial effects:

[0025] First, efficient complementary and integrated acquisition of multi-modal information is realized, and the technical bottleneck of single information dimension is solved: the present application controls the SC reflection spectrum, the DUV Raman / fluorescence combined spectrum and the polarization immunofluorescence three detection modes to time-division multiplex the same set of endoscopic probes and optical path channels through the main controller, and in one detection process, the wide-spectrum reflection information, specific molecular fingerprint spectrum information and targeted fluorescence information of the same detection point can be automatically and sequentially obtained. This controlled multi-modal data acquisition strategy fundamentally overcomes the limitation of traditional endoscopes that can only provide morphological information, and provides multi-dimensional and complementary quantitative chemical information for in-situ analysis;

[0026] Second, in-situ high-speed analysis of "image-spectrum integration" is realized, and the problem of separation of spatial morphology and chemical component information is solved: by controlling the FAST optical fiber bundle to convert the far-end square array signal into a linear arrangement at the near end and couple it with the spectrometer slit, combined with the corresponding control algorithm, the system can simultaneously acquire the spatial image and point-by-point spectrum of the detection point in one exposure. This "image-spectrum integration" control capability realizes seamless connection from macroscopic morphological observation to microscopic chemical component analysis, and can directly generate original hyperspectral data cubes for chemical component spatial distribution analysis without scanning, greatly improving the in-situ analysis efficiency and accuracy.

[0027] Third, the spatial consistency and reliability of multi-modal data acquisition are ensured, and the accuracy and comparability of data are improved; since all optical modes are controlled by the same main controller and share the same physical probe, it is ensured that the detection spots of the three different modes of SC, DUV RS / DUV LIF and mIHC accurately act on the same detection point. This control method avoids data misplacement and errors caused by changing probes or moving sites, and lays a solid foundation for precise registration and fusion analysis of multi-modal data;

[0028] Fourthly, the system integration and detection efficiency are improved by the optimized light source multiplexing and signal detection control mechanism: the main controller controls the multiplexing beam combiner to realize the time-sharing beam combining and introduction of the SC original laser, SC polarization excitation light and DUV laser; meanwhile, the DUV spectrometer is controlled to jointly collect the Raman and fluorescence signals in one measurement. This flexible control mechanism enables the automatic completion of complex multi-modal detection in a highly integrated and simplified system, which not only improves the reliability of the detection process, but also significantly improves the information flux of single detection.

[0029] The control method for multi-modal data acquisition for tumor in-situ pathological analysis of the application organically integrates various optical detection technologies, realizes the automatic acquisition of in-vivo, in-situ and multi-dimensional chemical information, provides strong data support for high-precision in-situ pathological analysis, and has high clinical application value. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The control method for multi-modal data acquisition for tumor in-situ pathological analysis of the application is used in a multi-modal data acquisition system structure diagram, Figure 2 The microarray lens head structure is shown in the figure;

[0031] In the figure, the hyperspectral image 1; the spectral dimension 2; the spatial dimension 3; the DUV laser 4; the main controller 5; the SC laser 6; the comb filter 7; the polarizer 8; the multiplexing beam combiner 9; the conjugate coupling module 10; the proximal and distal endoscope fiber arrangement 11; the FAST distal fiber arrangement 12; the hollow thin catheter 13; the NBSR 14; the lung bronchus 15; the microarray lens head 16; the FAST proximal fiber arrangement 17; the DUV spectrometer 18; the spectrometer detector 19; the point-by-point spectrum 20; the SC spectrometer 21; the square fiber array 22; the lung tissue 23; the main lens 24; the front lens array 25; the rear lens array 26; the main optical axis 27; the diagnostic point 28; the endoscope fiber bundle 29; the FAST fiber bundle 30; the proportional beam splitter 31; the two-dimensional area array 32. DETAILED DESCRIPTION

[0032] The application will be further described in detail with reference to the accompanying drawings and specific examples.

[0033] The control method for multi-modal data acquisition for tumor in-situ pathological analysis of the application is executed by a main controller, which is configured to control a multi-modal data acquisition system including an SC laser, a DUV laser, a multiplexing beam combiner, an endoscope fiber bundle, a FAST fiber bundle, an SC spectrometer and a DUV spectrometer.

[0034] The system comprises a main controller, a light source module composed of an SC laser and a DUV laser, a detection module composed of an SC spectrometer and a DUV spectrometer, a multiplexing combiner, a conjugate coupling module and an endoscopic fiber bundle connected in sequence, wherein a comb filter and a polarizer are arranged on the outgoing light path of the SC laser to generate polarized fluorescence excitation light; the distal end of the endoscopic fiber bundle is provided with a microarray lens head, and the optical fibers at the proximal end and the distal end are arranged in a square optical fiber array; the conjugate coupling module is further connected in optical path with a FAST fiber bundle, the proximal end of which is arranged in a single column optical fiber array, the main controller is electrically connected with the light source module and the detection module for coordinating the time-sharing work timing of each laser and spectrometer, and receiving and processing the collected spectral data, the system comprises the main controller, the SC laser, the DUV laser, the multiplexing combiner, the endoscopic fiber bundle, the FAST fiber bundle, the SC spectrometer and the DUV spectrometer; the method is executed by the main controller of a detection system for controlling multiple hardware modules of the system to work cooperatively according to a predetermined timing to collect multi-modal spectral data.

[0035] The control method comprises the following steps automatically executed by the main controller:

[0036] Mode switching and control instruction issuing step: according to a preset program, control instructions are sequentially issued to the system hardware to make the system enter the SC reflection spectrum acquisition mode, the DUV joint spectrum acquisition mode and the polarized fluorescence spectrum acquisition mode respectively;

[0037] SC reflection spectrum acquisition control step: when the system enters the SC reflection spectrum acquisition mode, the following is executed:

[0038] Start the SC laser and control its output of supercontinuum spectrum laser;

[0039] Control the light path so that the supercontinuum spectrum laser is coupled into the endoscopic fiber bundle through the multiplexing combiner and transmitted to the detection point;

[0040] At the same time, control the SC spectrometer to be turned on to receive the reflected light signal returned from the detection point through the endoscopic fiber bundle and generate a first spectral data set, i.e. an SC reflection spectrum data set;

[0041] DUV joint spectrum acquisition control step: when the system enters the DUV joint spectrum acquisition mode, the following is executed:

[0042] Start the DUV laser and control its output of deep ultraviolet laser;

[0043] Control the light path so that the deep ultraviolet laser is coupled into the endoscopic fiber bundle through the multiplexing combiner and transmitted to the detection point;

[0044] Meanwhile, the DUV spectrometer is controlled to be turned on to receive the optical signal returned from the detection point via the endoscope fiber bundle and generate a second optical spectrum data set, i.e., a DUV combined optical spectrum data set containing both deep ultraviolet Raman scattering and laser-induced fluorescence.

[0045] The polarization fluorescence spectrum acquisition control step: when the system enters the polarization fluorescence spectrum acquisition mode, the following is performed:

[0046] The SC laser is started, and the output laser thereof is controlled to pass through the comb filter and the polarizer to generate polarized excitation light.

[0047] The optical path is controlled so that the polarized excitation light is coupled into the endoscope fiber bundle via the multiplexing combiner and transmitted to the detection point.

[0048] Meanwhile, the SC spectrometer is controlled to be turned on to receive the fluorescence signal returned from the detection point via the endoscope fiber bundle and generate a third optical spectrum data set, i.e., a polarization fluorescence spectrum data set.

[0049] S4, high-spectrum data reconstruction, based on the first optical spectrum data set, the second optical spectrum data set and the third optical spectrum data set, the signal corresponding to the single-column arrangement of the proximal end of the FAST fiber bundle is image reconstructed according to the square fiber array arrangement of the distal end of the endoscope fiber bundle to generate the hyperspectral image data of the detection point.

[0050] The method further includes: when switching to a different optical spectrum acquisition mode, moving the proximal end arrangement of the FAST fiber bundle to align with the entrance slit of the SC spectrometer or the DUV spectrometer corresponding to the current mode.

[0051] Before the polarization fluorescence spectrum data acquisition step in step S3, the in-situ pretreatment step is further included: the endoscope fiber bundle is controlled to move out and sequentially connected to a plurality of pretreatment functional heads to complete fluorescence labeling at the detection point.

[0052] The purpose of the present application is to provide a control method for multi-modal data acquisition for tumor in-situ pathological analysis required by the tumor in-situ pathological detection based on an endoscopic multi-mode probe matched with NBSR, which comprehensively uses three optical detection methods of supercontinuum (SC) laser reflection (absorption), deep ultraviolet (DUV) laser scattering and polarization hyperspectral multiplex immunohistochemical (mIHC) to provide data support for lung in-situ early pathological diagnosis. The method includes the following steps:

[0053] Step 1, system preparation;

[0054] Step 2, SC spectrum rough measurement;

[0055] Step 3, DUV spectrum fine measurement;

[0056] Step 4, in-situ mIHC pretreatment;

[0057] Step 5, polarization mIHC detection;

[0058] Step 6, multi-mode spectrum analysis.

[0059] In the system preparation, the NBSR hollow thin catheter enters the patient's lung bronchus, and then reaches the lung tissue diagnosis point under the guidance of the endoscope of the NBSR hollow thin catheter. Then the endoscope fiber bundle is inserted into the NBSR hollow thin catheter, and synchronously reaches the lung tissue diagnosis point.

[0060] In the SC spectrum rough measurement, the FAST proximal fiber arrangement is fixed in parallel alignment with the entrance slit of the SC spectrometer. The main controller starts the SC laser, and the supercontinuum spectrum laser emitted by the SC laser is coupled into the endoscope fiber bundle through the multiplexing beam combiner and the conjugate coupling module, and is emitted by the square fiber array. After being focused point by point by the rear lens array one by one along the main optical axis, and then being diverged point by point by the front lens array, and finally being focused point by point by the main lens to the lung tissue diagnosis point, the in-situ target SC reflection signal at the diagnosis point is focused point by point by the main lens into the front lens array, and then is collected point by point by the rear lens array into the endoscope fiber bundle with square fiber array arrangement. The signal is reversely transmitted to the conjugate coupling module, reflected into the FAST fiber bundle by the center proportional beam splitter, received by the detector of the SC spectrometer, and output to the main controller. The system algorithm software is used for storage for subsequent processing and analysis.

[0061] In the DUV spectrum fine measurement, the FAST proximal fiber arrangement is fixed in parallel alignment with the entrance slit of the DUV spectrometer. The main controller starts the DUV laser, and the deep ultraviolet laser emitted by the DUV laser is coupled into the endoscope fiber bundle through the multiplexing beam combiner and the conjugate coupling module, and is emitted by the square fiber array. After being focused point by point by the rear lens array one by one along the main optical axis, and then being diverged point by point by the front lens array, and finally being focused point by point by the main lens to the lung tissue diagnosis point, the DUVRS and DUVLIF signals excited by the in-situ target at the diagnosis point are focused point by point by the main lens into the front lens array, and then are collected point by point by the rear lens array into the endoscope fiber bundle. The signal is reversely transmitted to the conjugate coupling module, reflected into the FAST fiber bundle by the center proportional beam splitter, received by the detector of the DUV spectrometer, and output to the main controller. The system algorithm software is used for storage for subsequent processing and analysis.

[0062] In-situ mIHC pretreatment, the hollow thin catheter of NBSR is pulled out of the endoscopic fiber bundle, and then the NBSR multiple sampling and staining head (for multiple simultaneous staining), the rapid incubation head (for rapid incubation), and the in-situ high-temperature head (for elution) are inserted in turn to complete the multiple immunofluorescence staining of the in-situ target mIHC diagnosis of the diagnostic point.

[0063] Polarized mIHC detection, the FAST proximal fiber array is fixed in parallel alignment with the entrance slit of the SC spectrometer. The endoscopic fiber bundle is inserted into the hollow thin catheter of NBSR again to reach the lung tissue diagnostic point. The main controller starts the SC laser, which emits supercontinuum spectrum laser filtered by a comb filter, and then obtains the laser required for mIHC multiple polarization fluorescence excitation through a polarizer. After passing through the multiplexing combiner and the conjugate coupling module, the laser is coupled into the endoscopic fiber bundle and emitted by the square fiber array. After being focused point by point by the rear lens array along the main optical axis, and then being diverged point by point by the front lens array, the laser is finally focused to the lung tissue diagnostic point by the main lens. The in-situ target mIHC multiple polarization fluorescence signal is focused point by point by the main lens into the front lens array, and then collected into the endoscopic fiber bundle by the rear lens array. The mIHC multiple polarization fluorescence signal is reversely transmitted to the conjugate coupling module, reflected by the central proportional beam splitter into the FAST fiber bundle, and then enters the SC spectrometer through the proximal end of the FAST fiber bundle. The signal is received by the detector of the SC spectrometer and output to the main controller, which is stored by the system algorithm software for subsequent processing and analysis.

[0064] Multi-mode spectral analysis, the diagnostic analysis algorithm model in the system algorithm software contains the FAST hyperspectral image spectrum integration function of SC, DUVRS, DUVLIF, and mIHC four kinds of spectral detection, which can perform single-point component quantitative calculation, pattern recognition, and Mapping analysis. First, according to the characteristics of the SC reflectance spectrum database, the in-situ target is coarsely detected and classified, including proteins, lipids, carbohydrates, and water content. Then, according to the DUVRS data, the in-situ molecules are finely detected, including nucleic acids (guanine, adenine, cytosine, thymine, uracil, DNA, RNA) and proteins (phenylalanine, tryptophan, tyrosine, glutamic acid, etc.), lipids, and some important groups such as CH2 and P=O. Then, according to the DUVLIF data, some molecules with cyclic structures such as some natural chromophore-containing molecules are identified. Then, according to the polarization mIHC data, the specific detection of tumor marker molecules such as PD-L1, PD1, CD68, CD3, Ki67, and panCK is performed, and the tumor markers and fibrosis proteins are distinguished.

[0065] The application has the following beneficial effects: the FAST hyperspectral image spectrum detection of SC, DUVRS, DUVLIF and mIHC can be realized, wherein the SC reflection spectrum is used for rough detection and rough classification of the in-situ target; the DUV spectrum is used for accurate detection of important molecules of the tissue; the polarization mIHC is used for targeted detection of tumor marker molecules and distinguishing the tumor markers from fibrosis proteins; and a new method is provided for in-situ pathological detection of the lung bronchus.

[0066] Embodiment 1

[0067] The control method for multi-modal data acquisition of in-situ tumor pathological analysis of the application provides data support for endoscopic multi-mode optical probe in-situ tumor pathological diagnosis, and is realized based on an endoscopic multi-mode probe in-situ tumor pathological detection system. The detection system is a multi-modal data acquisition system, which is composed of a main controller 5, an SC laser 6, a DUV laser 4, a comb filter 7, a polarizer 8, a microarray lens head 16, a multiplexing combiner 9, a conjugate coupling module 10, an endoscopic optical fiber bundle 29, a FAST optical fiber bundle 30, an SC spectrometer 21 and a DUV spectrometer 18.

[0068] The main controller 5 controls the SC laser 6, the DUV laser 4, the SC spectrometer 21 and the DUV spectrometer 18, sets and adjusts the working time sequence, receives spectral data and processes and analyzes the spectral data by system algorithm software.

[0069] The SC laser 6 (the wavelength range of which is 400-2500 nm in this embodiment) and the SC spectrometer 21 (the wavelength range of which is 400-2500 nm in this embodiment) are matched to realize supercontinuum spectrum detection. In addition, the supercontinuum spectrum laser emitted by the SC laser 6 is filtered by the comb filter 7, and then the multiple polarization fluorescence excitation source required by the mIHC is obtained through the polarizer 8. The multiple polarization fluorescence excitation source is matched with the SC spectrometer to realize mIHC detection. The SC spectrometer 21 covers the visible to infrared spectrum (400-2500 nm in this embodiment) to meet the requirements of wide spectrum and high resolution, and is composed of multiple modules (3 modules in this embodiment) to realize visible, near infrared and mid infrared spectrum detection.

[0070] The DUV laser 4 (the wavelength of which is 248.6 nm in this embodiment) and the DUV spectrometer 18 are matched to realize the joint detection of deep ultraviolet Raman and laser-induced fluorescence, i.e., DUVRS and DUVLIF. The DUV spectrometer 18 is an ultraviolet narrow spectrum high resolution high signal-to-noise ratio spectrometer. The Stokes wave number frequency shift of most molecules of deep ultraviolet Raman is still located in the medium and short wave ultraviolet segment, and the fluorescence spectrum excited at the same time is mostly located in the long wave ultraviolet segment and the visible spectrum segment. The excitation Raman and fluorescence spectrum can be separated to a certain extent, and the spectrum division multiplexing joint detection of DUVRS and DUVLIF is realized.

[0071] The multiplexing combiner 9 time-division combines the three kinds of laser source light, i.e., the laser emitted by the SC laser 6, the laser filtered by the comb filter 7, and the laser emitted by the DUV laser 4, to the conjugate coupling module 10, and then outputs the coupling into the endoscope fiber bundle 29. The proximal and distal endoscope fiber arrangement 11 of the endoscope fiber bundle 29 is a square fiber array 22. The microarray lens head 16 is attached to the distal end of the endoscope fiber bundle 29.

[0072] The microarray lens head 16 is composed of a main lens 24, a front lens array 25, and a rear lens array 26. The front lens array 25 and the rear lens array 26 have the same square array arrangement as the square fiber array 22.

[0073] The distal end of the FAST fiber bundle 30 is also a square fiber array 22 with a square array arrangement, which is conjugated with the distal end face of the endoscope fiber bundle 29. The proximal end of the FAST fiber bundle 30 rotates and splices the square array into a single column, i.e., the FAST proximal fiber arrangement 17, which is parallelly aligned and fixed with the entrance slit of the SC spectrometer 21 and the DUV spectrometer 18 (in this embodiment, time-division detection is performed, i.e., the corresponding spectrometer is connected when performing detection). All the spectrometer detectors 19 are two-dimensional area arrays 32, including a spatial dimension 3 and a spectral dimension 2. The spatial dimension 3 corresponds to each fiber unit of the FAST proximal fiber arrangement 17, and the spectral dimension 2 is the spectral composition of the signal transmitted by each fiber unit. When analyzing the signal, the spatial dimension 3 is rotated and rearranged every square fiber array 22 row of image elements, and the original square fiber array 22 arrangement is restored, and a hyperspectral image 1 can be obtained. At the same time, the target image element corresponding to each fiber of the FAST fiber bundle 30 can be individually analyzed, and a point-by-point spectrum 20 can be obtained.

[0074] The control method for multi-modal data acquisition for in-situ tumor pathological analysis of the application provides data support for endoscopic multi-modal light probe in-situ tumor pathological diagnosis, and specifically includes the following steps:

[0075] (1) System preparation

[0076] The NBSR 14 moves the hollow thin catheter 13 into the patient's bronchus 15, and then, under the guidance of the endoscope of the hollow thin catheter 13 of the NBSR 14, reaches the lung tissue 23 diagnosis point 28. Then, the endoscope fiber bundle 29 is inserted into the hollow thin catheter 13 of the NBSR 14, and synchronously reaches the lung tissue 23 diagnosis point 28.

[0077] (2) SC spectrum rough measurement

[0078] The FAST proximal fiber array 17 is fixed in parallel alignment with the entrance slit of the SC spectrometer 21. The main controller 5 starts the SC laser 6, and the supercontinuum laser emitted by the SC laser 6 is coupled into the endoscopic fiber bundle 29 through the multiplexing beam combiner 9 and the conjugate coupling module 10, and is emitted by the square fiber array 22. After being focused point by point by the main lens array 26 along the main optical axis 27 and then being diverged point by point by the front lens array 25, the supercontinuum laser is finally focused point by point by the main lens 24 to the diagnosis point 28 of the lung tissue 23. The in-situ target SC reflection signal of the diagnosis point 28 is focused point by point by the main lens 24 into the front lens array 25, and then is collected point by point by the rear lens array 26 into the endoscopic fiber bundle 29 with the arrangement of the square fiber array 22. The signal is reversely transmitted to the conjugate coupling module 10, reflected by the central proportional beam splitter 31 into the FAST fiber bundle 30, and then enters the SC spectrometer 21 through the proximal end of the FAST fiber bundle 30. The signal is received by the detector 19 of the SC spectrometer 21 and output to the main controller 5, and is stored by the system algorithm software for subsequent processing and analysis.

[0079] (3) DUV spectrum fine detection

[0080] The FAST proximal fiber array 17 is fixed in parallel alignment with the entrance slit of the DUV spectrometer 18. The main controller 5 starts the DUV laser 4, and the deep ultraviolet laser emitted by the DUV laser 4 is coupled into the endoscopic fiber bundle 29 through the multiplexing beam combiner 9 and the conjugate coupling module 10, and is emitted by the square fiber array 22. After being focused point by point by the main lens array 26 along the main optical axis 27 and then being diverged point by point by the front lens array 25, the deep ultraviolet laser is finally focused point by point by the main lens 24 to the diagnosis point 28 of the lung tissue 23. The DUVRS and DUVLIF signals excited by the in-situ target of the diagnosis point 28 are focused point by point by the main lens 24 into the front lens array 25, and then are collected point by point by the rear lens array 26 into the endoscopic fiber bundle 29. The signal is reversely transmitted to the conjugate coupling module 10, reflected by the central proportional beam splitter 31 into the FAST fiber bundle 30, and then enters the DUV spectrometer 18 through the proximal end of the FAST fiber bundle 30. The signal is received by the detector 19 of the DUV spectrometer 18 and output to the main controller 5, and is stored by the system algorithm software for subsequent processing and analysis.

[0081] (4) In-situ mIHC pretreatment

[0082] The endoscopic fiber bundle 29 is pulled out of the hollow thin catheter 13 of the NBSR 14, and then is sequentially inserted into the NBSR multiple sampling and staining head (for multiple simultaneous staining), the rapid incubation head (for rapid incubation), and the in-situ high-temperature head (for elution) respectively, so as to complete the multiple immunofluorescence staining of the in-situ target mIHC diagnosis of the diagnosis point 28.

[0083] (5) Polarized mIHC detection

[0084] The FAST proximal optical fiber array 17 is fixed in parallel alignment with the entrance slit of the SC spectrometer 21. The endoscopic optical fiber bundle 29 is inserted into the hollow thin catheter 13 of the NBSR 14 again, reaching the lung tissue 23 diagnosis point 28. The main controller 5 starts the SC laser 6, and the supercontinuum spectrum laser emitted by the SC laser 6 is filtered by the comb filter 7, and then the laser required for mIHC multiple polarization fluorescence excitation is obtained through the polarizer 8, and then the laser is coupled into the endoscopic optical fiber bundle 29 through the multiplexing combiner 9 and the conjugate coupling module 10, and then the laser is emitted from the square optical fiber array 22, and then the laser is focused point by point through the main optical axis 27 through the rear lens array 26, and then the laser is diverged point by point through the front lens array 25, and finally the laser is focused point by point to the lung tissue 23 diagnosis point 28 through the main lens 24. The in-situ target mIHC multiple polarization fluorescence signal of the diagnosis point 28 is focused point by point into the front lens array 25 through the main lens 24, and then the signal is collected point by point into the endoscopic optical fiber bundle 29 through the rear lens array 26. The mIHC multiple polarization fluorescence signal is reversely transmitted to the conjugate coupling module 10, reflected into the FAST optical fiber bundle 30 through the center proportional beam splitter 31, and then the signal is transmitted into the SC spectrometer 21 through the FAST optical fiber bundle 30 proximal end, and then the signal is received by the detector 19 of the SC spectrometer 21 and output to the main controller 5, and then the signal is stored for subsequent processing and analysis by the system algorithm software.

[0085] (6) Multi-mode spectrum analysis

[0086] In the hyperspectral data reconstruction of the present application, the FAST hyperspectral image spectrum integration function containing SC, DUVRS, DUVLIF and mIHC four kinds of spectral detection can be used for single-point component quantitative calculation, pattern recognition and Mapping analysis. First, according to the characteristics of the SC reflection spectrum database, the in-situ target is coarsely detected and coarsely classified, including protein, lipid, sugar, water content, etc.; then according to the DUVRS data, the in-situ molecule is finely detected, including nucleic acid (guanine, adenine, cytosine, thymine, uracil, DNA, RNA) and protein (phenylalanine, tryptophan, tyrosine, glutamic acid, etc.), lipid and some important groups such as CH2 and P=O, etc., and then according to the DUVLIF data, the molecule groups with ring structure such as some molecules containing natural chromophores are identified; then according to the polarization mIHC data, the specific detection of PD-L1, PD1, CD68, CD3, Ki67, panCK and other tumor marker molecules is carried out, and the tumor markers and fibrosis proteins are distinguished.

[0087] In the present application, FAST, fiber array spectral translation, optical fiber array spectrum conversion; DUV, deep ultra violet, deep ultraviolet; SC, Supercontinuum, supercontinuum; NBSR, Navigation bronchoscopy surgical robot, bronchoscopy surgical robot.

[0088] The control method for multi-modal data acquisition for in-situ pathological analysis of tumors of the present application uses the "information complementarity" of multi-modal optical technology: four spectral technologies (SC reflection spectrum, DUV Raman spectrum DUVRS, DUV laser-induced fluorescence DUVLIF, and polarization multiple immunofluorescence mIHC) are integrated in a single probe, the multi-modal optical technology is used to realize information complementarity, through a square fiber array and a conjugate optical path design, the spatial morphology image and the point-by-point chemical composition spectrum of the target are obtained simultaneously in one measurement, the "image-spectrum integration" capability of the FAST hyperspectral technology is used, the limitations of traditional endoscopy relying only on morphology for diagnosis are solved, quantitative information at the molecular level is provided, and molecular-level accurate detection is possible, the problem of "single information" is solved, the information limitations of single optical technology are overcome, FAST hyperspectral image-spectrum integration detection of SC, DUVRS, DUVLIF, and mIHC can be realized, and multi-modal hyperspectral fusion images reflecting the spatial distribution of the chemical composition of the diagnostic point tissue and corresponding spectral feature data are output, which provide real-time, accurate, and multi-modal data or information for doctors to diagnose.

[0089] In the present application, the same set of endoscopic probes is shared by the three modes of SC reflection spectrum, polarization mIHC, and DUV Raman / fluorescence (DUVRS / DUVLIF), which avoids the trouble of changing probes, ensures the positional consistency of multiple detections, obtains complementary information, and truly integrates multiple modalities.

[0090] The FAST fiber bundle converts the square fiber array at the distal end into a single linear array at the proximal end, which is coupled with the spectrometer slit, and high-spectral images (spatial information) and point-by-point spectra (chemical composition information) can be obtained simultaneously in one exposure, realizing seamless connection from "seeing" to "analysis" and providing a FAST hyperspectral technology of "image-spectrum integration".

[0091] In the present application, the square fiber array and the microarray lens (front and rear lens arrays) have a one-to-one correspondence in the conjugate relationship, which maximizes the fidelity and spatial resolution of signal transmission, provides a precise conjugate optical path design, and makes the finally reconstructed hyperspectral image clear and accurate, laying a foundation for accurate Mapping analysis.

[0092] In the application, the flexible laser multiplexing mechanism, the multiplexing combiner time-sharing combines the SC original laser, the SC filtered polarization laser and the DUV laser into the same channel, realizes the multi-mode detection under the single physical channel, simplifies the system structure and improves the reliability.

[0093] In the application, the DUV laser is used to stimulate the Raman and fluorescence signals simultaneously, the spectral separation degree is used to realize the spectrum division multiplexing, the Raman characteristics and the sky fluorescence characteristics are obtained in one measurement, the joint detection of the deep ultraviolet spectrum greatly improves the detection efficiency and the information amount.

[0094] The above specific embodiments are used to explain and illustrate the application, and are only preferred embodiments of the application, not to limit the application, any modification, equivalent replacement, improvement, etc. of the application within the spirit and protection scope of the claims of the application, falls into the protection scope of the application.

Claims

1. A control method for multimodal data acquisition in in situ pathological analysis of tumors, characterized in that, The method is executed by a main controller configured to control a multimodal data acquisition system, which includes an SC laser, a DUV laser, a multiplexer, an endoscopic fiber bundle, a FAST fiber bundle, an SC spectrometer, and a DUV spectrometer. The method includes the following steps: controlling the system to perform SC reflectance spectral acquisition, DUV combined spectral acquisition, and polarization fluorescence spectral acquisition via the main controller. S1, acquire SC reflectance spectral data, control the supercontinuum laser to irradiate the detection point, and control the SC spectrometer to receive the reflected light signal from the detection point to generate the first spectral dataset; S2, acquire DUV combined spectral data, control the deep ultraviolet laser to irradiate the detection point, and control the DUV spectrometer to receive the light signal excited therefrom, so as to generate a second spectral dataset, the second spectral dataset simultaneously containing the DUV combined spectral dataset of deep ultraviolet Raman scattering and laser-induced fluorescence; S3. Acquire polarized fluorescence spectral data, start the SC laser, and control its output laser to first pass through a comb filter and polarizer to generate polarized excitation light. Control the polarized excitation light to be coupled to the endoscopic fiber bundle through the multiplexer and conjugate coupling module and transmitted to the detection point. At the same time, control the SC spectrometer to receive the fluorescence signal returned from the detection point to generate a third spectral dataset. S4, Hyperspectral data reconstruction: Based on the first spectral dataset, the second spectral dataset, and the third spectral dataset, the signal corresponding to the single-column arrangement at the near end of the FAST fiber bundle is reconstructed according to the square fiber array arrangement at the far end of the endoscopic fiber bundle to generate hyperspectral image data of the detection point. When switching to different spectral acquisition modes, the near-end arrangement of the FAST fiber bundle is moved to align with the entrance slit of the SC spectrometer or DUV spectrometer corresponding to the current mode.

2. The control method for multimodal data acquisition for in situ pathological analysis of tumors as described in claim 1, characterized in that: In step S1, the supercontinuum laser is controlled to be coupled to the endoscopic fiber bundle via the multiplexer and the conjugate coupling module, and then transmitted to a detection point; The SC spectrometer is controlled to receive optical signals from the detection point and returned via the endoscopic fiber bundle and conjugate coupling module to generate a first spectral dataset.

3. The control method for multimodal data acquisition for in situ pathological analysis of tumors as described in claim 1, characterized in that: In step S2, the deep ultraviolet laser is controlled to be coupled to the endoscopic fiber bundle via the multiplexer and the conjugate coupling module, and then transmitted to the detection point; The DUV spectrometer is controlled to receive the optical signal from the detection point and returned via the endoscopic fiber bundle and conjugate coupling module to generate a second spectral dataset.

4. The control method for multimodal data acquisition for in situ pathological analysis of tumors as described in claim 1, characterized in that: In step S3, the polarized excitation light is controlled to be coupled to the endoscopic fiber bundle via the multiplexer and the conjugate coupling module, and then transmitted to the detection point; The SC spectrometer is controlled to receive fluorescence signals from the detection point and returned via the endoscopic fiber bundle and conjugate coupling module to generate a third spectral dataset.

5. The control method for multimodal data acquisition for in situ pathological analysis of tumors as described in claim 1, characterized in that: Before step S3, which involves acquiring polarization fluorescence spectral data, an in-situ preprocessing step is also included: controlling the endoscopic fiber bundle to move out and sequentially connect to multiple preprocessing functional heads to complete fluorescence marking at the detection point.

Images