A tissue property identification method and system based on fluorescence and raman dual-mode fusion

Abstract

The application relates to the technical field of intraoperative medical image navigation and molecular spectral diagnosis, and discloses a tissue property recognition method and system based on fluorescence and Raman double-mode fusion, which comprises the following steps: collecting a tissue fluorescence image excited by an exogenous contrast agent and performing dynamic threshold segmentation to obtain a spatial coordinate set of a high-fluorescence candidate area; driving a Raman probe to perform targeted Raman spectrum collection only in the coordinate set; inputting the original spectrum into an XGBoost classifier after fluorescence background deduction preprocessing to output a tissue category probability value of a sampling point; mapping the probability value to a fluorescence image based on a mapping relationship, generating a color-coded risk atlas through inverse distance weighted interpolation and image superposition fusion; and outputting a surgical decision text prompt according to the resection priority of different color areas in the atlas. The application constructs a fluorescence preliminary screening and Raman confirmation mechanism, considers high sensitivity, effectively reduces a non-specific false positive rate, and improves the efficiency and accuracy of intraoperative tissue recognition.

Description

Technical Field

[0001] This invention relates to the field of intraoperative medical image navigation and molecular spectroscopy diagnostic technology, specifically to a method and system for tissue property identification based on fluorescence Raman dual-modal fusion. Background Technology

[0002] In oncological surgery, accurate identification of tumor boundaries is a crucial prerequisite for achieving a balance between maximizing tumor resection and preserving the function of surrounding normal tissues. However, while near-infrared fluorescence imaging can achieve real-time visualization over a wide field of view, its reliance on non-specific contrast agent accumulation results in a false positive rate exceeding 30%, and it cannot provide information on tissue molecular characteristics. This makes it difficult for surgeons to accurately distinguish tumors from highly perfused tissues such as inflammatory tissues based solely on fluorescence images, easily leading to over-resection or ineffective biopsies. On the other hand, while Raman spectroscopy can achieve highly specific diagnosis through molecular fingerprinting, its slow single-point acquisition speed makes it difficult to cover the entire surgical area in a short time, failing to independently meet the time-sensitive requirements of intraoperative real-time navigation.

[0003] This demonstrates a fundamental contradiction in existing single-modal technologies: a trade-off between sensitivity and specificity. While current dual-modal approaches often employ parallel acquisition and data fusion, they fail to establish an effective synergistic mechanism at the functional division and temporal coordination levels, thus failing to develop an intraoperative diagnostic approach capable of both rapid, wide-area localization and precise molecular verification. Therefore, developing a dual-modal fusion method and system capable of coordinating fluorescence imaging and Raman spectroscopy to simultaneously meet the clinical needs for real-time intraoperative diagnosis, wide field-of-view coverage, and precise molecular-level diagnosis has become an urgent technical challenge.

[0004] Furthermore, existing technologies, such as US20070167836A1, acquire Raman and fluorescence spectra side by side and then input them into a classifier for data fusion. However, they do not utilize fluorescence images to spatially guide Raman sampling, resulting in Raman acquisition covering the entire field of view, which is time-consuming and cannot be applied in real time during surgery.

[0005] There are currently no effective solutions to the problems in the relevant technologies. Summary of the Invention

[0006] To address the problems in related technologies, this invention proposes a tissue property identification method and system based on fluorescence Raman dual-modal fusion, in order to overcome the aforementioned technical problems existing in the prior art.

[0007] Therefore, the specific technical solution adopted by the present invention is as follows: According to one aspect of the present invention, a method for tissue property identification based on fluorescence Raman dual-modal fusion is provided, comprising the following steps: S1. Apply near-infrared excitation light to the tissue region to be identified that has been treated with exogenous near-infrared fluorescent contrast agent, and acquire a fluorescence image of the tissue region to be identified; perform dynamic threshold segmentation on the fluorescence image to divide the tissue region to be identified into a high fluorescence candidate region and a low fluorescence background region, and obtain the spatial coordinate set of the high fluorescence candidate region; S2. Based on the set of spatial coordinates, drive the Raman spectroscopy probe to move only to each sampling point within the high fluorescence candidate region to acquire Raman spectra, and obtain the raw Raman spectral data of each sampling point with spatial coordinate labels; S3. Perform fluorescence background subtraction preprocessing on the original Raman spectral data of each sampling point to obtain the corresponding corrected spectrum; input each corrected spectrum into a pre-trained extreme gradient boosting classifier for classification, and output the tissue category probability value of each sampling point; S4. Establish a mapping relationship between the Raman spectroscopy probe and the fluorescence image, and map the tissue category probability value of each sampling point to the corresponding pixel position of the fluorescence image based on the mapping relationship; use an inverse distance weighted interpolation algorithm to interpolate the mapped pixels to generate a classification probability distribution map with the same resolution as the fluorescence image; superimpose and fuse the classification probability distribution map with the original fluorescence intensity map of the fluorescence image to generate a color-coded risk map. S5. Based on the tissue resection priority corresponding to different color-labeled areas in the risk map, output text prompts for each different color-labeled area as surgical reference information to assist decision-making.

[0008] Preferably, the step of performing dynamic threshold segmentation on the fluorescence image to divide the tissue region to be identified into high-fluorescence candidate regions and low-fluorescence background regions, and obtaining the spatial coordinate set of the high-fluorescence candidate regions, includes: Calculate the signal-to-noise ratio (SNR) of each pixel in the fluorescence image, wherein the SNR is the ratio of the fluorescence signal intensity to the standard deviation of the background noise; Using the mean global signal-to-noise ratio of the fluorescence image plus 1.5 times the standard deviation as the segmentation threshold, the tissue region to be identified is divided into high fluorescence candidate regions with a signal-to-noise ratio greater than the segmentation threshold and low fluorescence background regions with a signal-to-noise ratio less than or equal to the segmentation threshold. Record and store the spatial coordinates of the high fluorescence candidate regions to form a set of spatial coordinates of the high fluorescence candidate regions.

[0009] Preferably, the exogenous near-infrared fluorescent contrast agent is indocyanine green, and the application concentration of the exogenous near-infrared fluorescent contrast agent is [missing information]. to ; The method for determining the application concentration is to pre-set... to Multiple concentration gradients were used to acquire test fluorescence images and test Raman spectra at each concentration gradient. The results were selected based on a comprehensive evaluation criterion: the fluorescence signal intensity of the test fluorescence image reached the imaging threshold, and the signal-to-noise ratio of the characteristic peaks of the test Raman spectrum was greater than 3. to The concentration range.

[0010] Preferably, the step of performing fluorescence background subtraction preprocessing on the raw Raman spectral data of each sampling point to obtain the corresponding corrected spectrum includes: Wavenumber points in the original Raman spectral data that are not affected by Raman characteristic peaks are selected as baseline points. An initial baseline is constructed by linear interpolation. The initial baseline is subtracted from the original Raman spectral data to obtain the preliminary corrected spectrum. The enhanced partial least squares (E-PLS) algorithm is used to iteratively optimize the preliminary corrected spectrum with a preset number of principal components and iterations. In each iteration, the fluorescence background residual is fitted using partial least squares to gradually separate the fluorescence signal and the Raman signal, and finally the corrected spectrum after subtracting the fluorescence background is obtained.

[0011] Preferably, before inputting each of the corrected spectra into a pre-trained extreme gradient boosting classifier for classification and outputting the tissue category probability value for each of the sampling points, the method further includes a training step for the extreme gradient boosting XGBoost classifier: Several Raman spectral data points for each of the three categories of samples—tumor core area, infiltrative transition area, and normal tissue—were collected in advance from tumor-bearing animal models. The fluorescence background subtraction preprocessing is performed on each of the sample Raman spectral data, and the intensity values ​​in the full spectrum are extracted as feature vectors; The XGBoost algorithm is used to train the XGBoost classifier based on the feature vector, and the hyperparameters are optimized by five-fold cross-validation until training is complete.

[0012] Preferably, establishing the mapping relationship between the Raman spectroscopy probe and the fluorescence image includes: Using cross-shaped markers as calibration objects, four marker points with known coordinates are set within the field of view of the Raman spectroscopy probe; Acquire fluorescence images of the marked points and record the original paths; Move the Raman spectroscopy probe to the origin and control its movement sequentially along the original path to establish a coordinate mapping relationship between the Raman spectroscopy probe coordinates and the fluorescence image pixel coordinates.

[0013] Preferably, the risk map includes multiple preset color labels, and different color labels are used to characterize different tissue categories and corresponding tissue resection priorities; The text prompts for each different color label area include: matching and outputting guiding clinical decision text information corresponding to the tissue resection priority based on the color label corresponding to each area in the risk map.

[0014] Preferably, the tissue category includes a tumor core area, an infiltrative transition area, and a normal tissue area, wherein the tumor core area is coded with a first color label, the infiltrative transition area is coded with a second color label, and the normal tissue area is coded with a third color label. The suggestion for the first color-labeled area is to completely remove it; the suggestion for the second color-labeled area is to remove it conservatively; and the suggestion for the third color-labeled area is to preserve it.

[0015] According to another aspect of the present invention, a tissue property identification system based on fluorescence Raman dual-modal fusion is provided, comprising: A fluorescence imaging module is used to apply near-infrared excitation light to the tissue region to be identified, which has been treated with an exogenous near-infrared fluorescent contrast agent, and to acquire and output a fluorescence image of the tissue region to be identified. The central control module is electrically connected to the fluorescence imaging module and is used to receive the fluorescence image and perform dynamic threshold segmentation on the fluorescence image to divide the tissue region to be identified into a high fluorescence candidate region and a low fluorescence background region, and output the set of spatial coordinates of the high fluorescence candidate region. The three-dimensional electric displacement platform module is electrically connected to the central control module and is used to receive the set of spatial coordinates and perform three-dimensional displacement drive under the control and guidance of the set of spatial coordinates. A Raman spectroscopy acquisition module, mounted on the three-dimensional electric displacement platform module and electrically connected to the central control module, is used to move only to each sampling point in the high fluorescence candidate region under the drive of the three-dimensional electric displacement platform module to acquire Raman spectra, and to transmit the raw Raman spectral data with spatial coordinate labels of each sampling point back to the central control module. The display module is electrically connected to the central control module and is used to receive the risk map and the surgical reference information, and to overlay and visualize the output.

[0016] Preferably, the central control module specifically includes: The image processing unit is used to calculate the signal-to-noise ratio of pixels in the fluorescence image, extract the high-fluorescence candidate region using a dynamic threshold segmentation algorithm, and output the set of spatial coordinates. The spectral processing unit is used to iteratively optimize the original Raman spectral data by using the enhanced partial least squares (E-PLS) algorithm combined with linear interpolation to separate the fluorescence signal and the Raman signal, and output the corrected spectrum. The classifier running unit has a built-in XGBoost classifier, which is used to run the XGBoost classifier to make a probability determination on the corrected spectrum and output the probability value of the tissue category. The spatial registration unit is used to store the mapping relationship, and perform pixel mapping from the tissue category probability value to the fluorescence image, inverse distance weighted interpolation, and map overlay fusion to generate the risk map and match and output the surgical reference information.

[0017] The beneficial effects of this invention are as follows: 1. This invention uses fluorescence imaging to initially locate tissue regions, and then uses Raman spectroscopy to acquire high-fluorescence candidate regions. The molecular fingerprint characteristics of Raman spectroscopy are then used to classify tissues, realizing a dual identification mechanism of fluorescence screening combined with Raman confirmation. While maintaining the high sensitivity advantage of fluorescence imaging, it effectively reduces false positive results caused by non-specific aggregation of contrast agents, and improves the accuracy and specificity of tissue identification.

[0018] 2. This invention utilizes fluorescence images to identify high-fluorescence candidate regions and uses these regions as the Raman spectroscopy acquisition range. Raman detection is performed only on the target region, transforming Raman sampling from full-area scanning to targeted sampling. This reduces the number of sampling points and detection time, improves intraoperative tissue identification efficiency, and meets real-time detection requirements.

[0019] 3. This invention spatially registers Raman classification results with fluorescence images and generates risk maps through interpolation, which can convert point-like molecular diagnostic results into planar visual information. At the same time, it outputs corresponding surgical reference information based on different tissue categories, providing doctors with intuitive tissue boundary identification results and auxiliary decision-making basis, thereby improving the efficiency of intraoperative decision-making.

[0020] 4. This invention employs enhanced partial least squares (E-PLS) combined with linear interpolation to perform fluorescence background subtraction on the original Raman spectrum, effectively separating the fluorescence background signal from the Raman characteristic signal, improving the identification quality of Raman characteristic peaks, providing a reliable data foundation for subsequent tissue classification, and thus enhancing the accuracy and stability of tissue property identification results. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the overall process of the fluorescence-Raman dual-mode fusion method according to an embodiment of the present invention; Figure 2 This is one of the experimental results of spectral preprocessing and concentration optimization according to an embodiment of the present invention; Figure 3 This is the second figure showing the experimental results of spectral preprocessing and concentration optimization according to an embodiment of the present invention; Figure 4 This is one of the fluorescence image partitioning and Raman spectral analysis results of different regions according to an embodiment of the present invention; Figure 5 This is the second image of the fluorescence image partitioning and Raman spectral analysis results of different regions according to an embodiment of the present invention; Figure 6 This is the third diagram of the fluorescence image partitioning and Raman spectral analysis results of different regions according to an embodiment of the present invention; Figure 7 This is one of the spatial correspondence verification results of the fluorescence-Raman dual-modal fusion method according to an embodiment of the present invention; Figure 8 This is the second image showing the spatial correspondence verification results of the fluorescence-Raman dual-modal fusion method according to an embodiment of the present invention; Figure 9 This is the third figure showing the spatial correspondence verification results of the fluorescence-Raman dual-mode fusion method according to the embodiments of the present invention; Figure 10 This is Figure 4, which shows the spatial correspondence verification results of the fluorescence-Raman dual-modal fusion method according to an embodiment of the present invention. Detailed Implementation

[0023] To further illustrate the various embodiments, the present invention provides figures, which are part of the disclosure of the present invention. These figures are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are usually used to represent similar components.

[0024] According to embodiments of the present invention, a method and system for tissue property identification based on fluorescence Raman dual-modal fusion are provided.

[0025] The present invention will now be further described in conjunction with the figures and specific embodiments, such as... Figure 1 As shown, according to an embodiment of the present invention, a method for tissue property identification based on fluorescence Raman dual-modal fusion is provided, comprising the following steps: S1. Apply near-infrared excitation light to the tissue region to be identified that has been treated with exogenous near-infrared fluorescent contrast agent, and acquire a fluorescence image of the tissue region to be identified; perform dynamic threshold segmentation on the fluorescence image to divide the tissue region to be identified into a high fluorescence candidate region and a low fluorescence background region, and obtain the spatial coordinate set of the high fluorescence candidate region; In a preferred embodiment, the step of performing dynamic threshold segmentation on the fluorescence image to divide the tissue region to be identified into a high-fluorescence candidate region and a low-fluorescence background region, and obtaining the spatial coordinate set of the high-fluorescence candidate region, includes: Calculate the signal-to-noise ratio (SNR) of each pixel in the fluorescence image, wherein the SNR is the ratio of the fluorescence signal intensity to the standard deviation of the background noise; Using the mean global signal-to-noise ratio of the fluorescence image plus 1.5 times the standard deviation as the segmentation threshold, the tissue region to be identified is divided into high fluorescence candidate regions with a signal-to-noise ratio greater than the segmentation threshold and low fluorescence background regions with a signal-to-noise ratio less than or equal to the segmentation threshold. Record and store the spatial coordinates of the high fluorescence candidate regions to form a set of spatial coordinates of the high fluorescence candidate regions.

[0026] In a preferred embodiment, the exogenous near-infrared fluorescent contrast agent is indocyanine green (hereinafter referred to as ICG), and the application concentration of the exogenous near-infrared fluorescent contrast agent is [missing information]. to ; The method for determining the application concentration is as follows: the application concentration of ICG is predetermined through orthogonal experiments. to Pre-set to Multiple concentration gradients were used to acquire test fluorescence images and test Raman spectra at each concentration gradient. The results were selected based on a comprehensive evaluation criterion: the fluorescence signal intensity of the test fluorescence image reached the imaging threshold, and the signal-to-noise ratio of the characteristic peaks of the test Raman spectrum was greater than 3. to The concentration range.

[0027] It should be noted that, in practice, ICG can be injected intravenously at a dose of 0.5 mg / kg body weight. During the procedure, a near-infrared light source with a wavelength of 780 nm is used to irradiate the tissue, and a CCD camera is used to acquire fluorescence images. The acquired fluorescence image data is transmitted to the image processing unit. This unit first calculates the signal-to-noise ratio (SNR = fluorescence signal intensity / background noise standard deviation) of each pixel in the image, and then uses a dynamic threshold segmentation algorithm: using the global SNR mean plus 1.5 times the standard deviation as the segmentation threshold, the tissue area is divided into high-fluorescence candidate regions and low-fluorescence background regions. The high-fluorescence candidate regions are marked as the target regions for subsequent Raman sampling, and their spatial coordinates are recorded and stored. The output of this step is the set of spatial coordinates of the high-fluorescence candidate regions. S2. Based on the set of spatial coordinates, drive the Raman spectroscopy probe to move only to each sampling point within the high fluorescence candidate region to acquire Raman spectra, and obtain the raw Raman spectral data of each sampling point with spatial coordinate labels; It should be noted that, guided by the spatial coordinates of the high-fluorescence candidate region output in step S1, this step only performs Raman spectroscopy acquisition on this region. Specifically, the Raman spectroscopy probe (using a 785nm excitation wavelength and a 200μm diameter fiber optic probe) is mounted on a three-dimensional motorized displacement platform module. The control system of the displacement platform receives the coordinate information output in step S1 and drives the probe to sequentially move to each sampling point within the high-fluorescence candidate region. The integration time for each sampling point is set to 1 second, the laser power to 50mW, and the spectral acquisition range to 400cm². -1 Up to 1800cm -1 For low-fluorescence background areas, the system automatically skips them without performing any Raman acquisition. Compared to the full-field grid scanning used in existing technologies (e.g., scanning the entire 10mm × 10mm area with a 100μm step size requires approximately 10,000 points and takes several hours), this step reduces the number of sampling points to a limited number within the high-fluorescence candidate area (e.g., 100-200 points), keeping the total acquisition time within 2-3 minutes, meeting the real-time requirements of the procedure. The output of this step is the raw Raman spectral data for each sampling point, with each spectrum accompanied by corresponding spatial coordinate labels.

[0028] S3. Perform fluorescence background subtraction preprocessing on the original Raman spectral data of each sampling point to obtain the corresponding corrected spectrum; input each corrected spectrum into a pre-trained extreme gradient boosting classifier for classification, and output the tissue category probability value of each sampling point; In a preferred embodiment, the step of performing fluorescence background subtraction preprocessing on the raw Raman spectral data of each sampling point to obtain the corresponding corrected spectrum includes: Wavenumber points in the original Raman spectral data that are not affected by Raman characteristic peaks are selected as baseline points. An initial baseline is constructed by linear interpolation. The initial baseline is subtracted from the original Raman spectral data to obtain the preliminary corrected spectrum. The enhanced partial least squares (E-PLS) algorithm is used to iteratively optimize the preliminary corrected spectrum with a preset number of principal components and iterations. In each iteration, the fluorescence background residual is fitted using partial least squares to gradually separate the fluorescence signal and the Raman signal, and finally the corrected spectrum after subtracting the fluorescence background is obtained.

[0029] In a preferred embodiment, before classifying each of the corrected spectra into a pre-trained extreme gradient boosting classifier and outputting the tissue category probability value for each sampling point, a training step for the extreme gradient boosting XGBoost classifier is also included: Several Raman spectral data points for each of the three categories of samples—tumor core area, infiltrative transition area, and normal tissue—were collected in advance from tumor-bearing animal models. The fluorescence background subtraction preprocessing is performed on each of the sample Raman spectral data, and the intensity values ​​in the full spectrum are extracted as feature vectors; The XGBoost algorithm is used to train the XGBoost classifier based on the feature vector, and the hyperparameters are optimized by five-fold cross-validation until training is complete.

[0030] It should be noted that this step preprocesses and classifies the raw Raman spectra acquired in step S2. First, enhanced partial least squares (E-PLS) combined with linear interpolation is used to subtract the fluorescence background. The specific algorithm flow is as follows: Figure 2 and Figure 3 As shown: For each original Raman spectrum ( Figure 2 Part a of the middle Figure 2 (Part b) Select wavenumber points that are not affected by Raman characteristic peaks (e.g., 1800-2000 cm⁻¹). -1 Using the interval as the baseline point, an initial baseline is constructed through linear interpolation. This baseline is then subtracted from the original signal to obtain the preliminary corrected spectrum. The E-PLS algorithm is used for iterative optimization: the number of principal components is set to 5, and the number of iterations is 10. In each iteration, partial least squares is used to fit the fluorescence background residual, gradually separating the fluorescence signal from the Raman signal, ultimately obtaining the corrected spectrum after subtracting the fluorescence background. Figure 3 Part C Figure 2(Part d). The preprocessed spectra are input into a pre-trained Extreme Gradient Boosting (XGBoost) classifier. The training process of this classifier is as follows: 100 Raman spectra of each of the three categories—tumor core region, infiltrative transition region, and normal tissue—are collected from tumor-bearing animal models. After performing the same preprocessing as described above on each spectrum, the intensity values ​​across the entire spectrum are extracted as feature vectors (a total of 1345 feature dimensions). The XGBoost algorithm is used for training, with hyperparameters set to a maximum tree depth of 6, a learning rate of 0.1, and 100 iterations. The model is optimized using five-fold cross-validation. After training, the classifier can output the tissue category probability value for each sampling point and ultimately determine whether it is tumor core region, infiltrative transition region, or normal tissue. This step uses the molecular fingerprint characteristics of Raman spectra to perform secondary verification on the candidate regions defined by fluorescence localization in step S1: if the Raman spectrum of a high-fluorescence candidate region is determined by the classifier to be normal tissue, it indicates that the fluorescence signal in that region originates from non-specific dye accumulation (false positive) and is thus effectively filtered out.

[0031] It should also be noted that, Figure 2 and Figure 3 middle Figure 2 Part a and Figure 2 Part b in the middle is a comparison diagram of the original Raman spectrum, the baseline, and the corrected spectrum; Figure 3 Part C and Figure 2 The middle d section shows the spectral correction effect after linear interpolation and enhanced partial least squares processing; Figure 2 Part e and Figure 3 The middle f section is a comparison of fluorescence signal intensity and Raman spectral signal quality at different ICG concentrations, showing the optimal concentration range as follows: to ; like Figure 2 Part a of the middle Figure 2 Part B of the middle section Figure 3 Part C and Figure 2 As shown in section d, the fluorescence background intensity in the original Raman spectrum is much higher than that of the Raman signal (the fluorescence background can be more than 10 times the signal intensity). If it is not subtracted, the classifier will have difficulty extracting the characteristic peaks.

[0032] Preferably, these high-fluorescence regions are further validated through Raman molecular diagnostics in step S3, utilizing the molecular fingerprint characteristics of Raman spectroscopy to distinguish between tumor and non-tumor tissues. For example... Figure 4 , Figure 5 and Figure 6 In Figure 4 Part b and Figure 5As shown in section d, in the high fluorescence candidate region, the region that is identified as normal tissue by the Raman classifier (i.e., the false positive region) shows the Raman spectrum of normal tissue characteristic peaks, while the tumor core region shows the tumor characteristic peaks.

[0033] It should be noted that, Figure 4 Part a shows the fluorescence image and its partitioning diagram. Based on the fluorescence signal intensity, the tissue region is divided into high fluorescence candidate regions (region-1, region-2, region-3, region-4) and low fluorescence background regions. Different regions are labeled with different color levels. Figure 4 Part b shows a comparison of the Raman spectra of each fluorescence region, revealing significant differences in the Raman characteristic peak positions and intensities between the high-fluorescence candidate region and the low-fluorescence background region. Figure 4 Part c shows the correlation analysis between the Raman characteristic peak intensity and the fluorescence signal intensity; Figure 5 Part d in the middle Figure 6 Part e of the middle Figure 5 Part f in the middle Figure 6 Part G of China Figure 5 The middle h part and Figure 6 Part i in the diagram shows a comparison of Raman spectra after fluorescence background subtraction (each fluorescence region), verifying the effectiveness of the spectral preprocessing method of this invention in subtracting fluorescence interference and highlighting Raman characteristic signals; among which, Figure 5 and Figure 6 In the image, FP and HWN represent the Raman fingerprint region and the Raman high-wavelength region, respectively. These two bands are spectral bands in the Raman spectrum that are richer in biological information than other regions.

[0034] Using E-PLS combined with linear interpolation, the fluorescence background was gradually stripped away through 10 iterations, and the Raman characteristic peaks (such as 2875 cm⁻¹) in the final corrected spectrum were obtained. -1 The lipid peaks at the location are clearly visible. Figure 5 (Part d in the middle).

[0035] S4. Establish a mapping relationship between the Raman spectroscopy probe and the fluorescence image, and map the tissue category probability value of each sampling point to the corresponding pixel position of the fluorescence image based on the mapping relationship; use an inverse distance weighted interpolation algorithm to interpolate the mapped pixels to generate a classification probability distribution map with the same resolution as the fluorescence image; superimpose and fuse the classification probability distribution map with the original fluorescence intensity map of the fluorescence image to generate a color-coded risk map. In a preferred embodiment, establishing the mapping relationship between the Raman spectroscopy probe and the fluorescence image includes: Using cross-shaped markers as calibration objects, four marker points with known coordinates are set within the field of view of the Raman spectroscopy probe; Acquire fluorescence images of the marked points and record the original paths; Move the Raman spectroscopy probe to the origin and control its movement sequentially along the original path to establish a coordinate mapping relationship between the Raman spectroscopy probe coordinates and the fluorescence image pixel coordinates.

[0036] It should be noted that, as Figure 7 , Figure 8 , Figure 9 and Figure 10 As shown, this step performs spatial coordinate registration between the Raman classification result output in step S3 and the fluorescence image acquired in step S1. The specific implementation is as follows: First, establish the mapping relationship between the Raman probe coordinates and the fluorescence image pixel coordinates. Using a cross marker as a calibration object, four marker points with known coordinates are set within the Raman probe's field of view. After acquiring the fluorescence image, the original path is recorded. The Raman probe is moved to the origin, and Raman spectra are acquired sequentially along the original path. Through the spatial registration and interpolation algorithm in step S4, the Raman classification result is converted into a color risk map aligned with the fluorescence image at the pixel level. Figure 10 In the middle section (d), surgeons can directly see the distribution of red (tumor core), yellow (transition zone), and green (normal) areas on the monitor; Figure 10 In the middle d section, the boundaries between the T (tumor) and N (normal) regions marked by Raman classification are clear, providing doctors with millimeter-level resection reference lines.

[0037] The classification result (tissue category label) of each Raman sampling point in step S3 is mapped to the corresponding pixel position in the fluorescence image. Since the number of Raman sampling points is much smaller than the number of pixels in the fluorescence image, an inverse distance weighted interpolation algorithm is used to generate a classification probability distribution map with the same resolution as the fluorescence image: for each pixel in the fluorescence image, the classification probability value of that pixel is calculated by weighting the values ​​according to the inverse distance, using the three to five nearest Raman sampling points as references. Finally, the classification probability distribution map is overlaid and fused with the original fluorescence intensity map to generate a color-coded risk map (e.g., ...). Figure 10 (As shown in section d). In this risk map, the tumor core area is represented in red, the infiltrative transition area in yellow, and normal tissue in green. Different color levels correspond to different resection priorities.

[0038] It should be noted that, Figure 7 The middle image shows a comparison of the Raman spectra of normal brain tissue and glioma tissue, revealing significant differences in the characteristic peak positions and intensities between the two. The two line segments represent the intensity of normal brain tissue and glioma tissue, respectively. Figure 8The image shows a comparison of Raman spectra after fluorescence background subtraction, which further highlights the differences in molecular fingerprints between tumor tissue and normal tissue. The two line segments represent the intensity of normal brain tissue and glioma tissue, respectively. Figure 9 Raman characteristic peak intensity variation curve along the tumor boundary (wavenumber 2875cm²) -1 This shows that the Raman signal undergoes a significant transition at the tumor boundary, which can be used to accurately locate the tumor edge; Figure 10 Part d in the middle is a fusion comparison diagram of bright-field image, fluorescence image and Raman classification results. The fluorescence image shows the high-fluorescence candidate region, while the Raman classification results mark the tumor core area and normal tissue area with different color levels. The two have good spatial correspondence, which verifies the spatial registration accuracy and diagnostic accuracy of the dual-modal fusion method of the present invention.

[0039] S5. Based on the tissue resection priority corresponding to different color-labeled areas in the risk map, output text prompts for each different color-labeled area as surgical reference information to assist decision-making.

[0040] In a preferred embodiment, the tissue category includes a tumor core area, an infiltrative transition area, and a normal tissue area, wherein the tumor core area is coded with a first color label, the infiltrative transition area is coded with a second color label, and the normal tissue area is coded with a third color label. The suggestion for the first color-labeled area is to completely remove it; the suggestion for the second color-labeled area is to remove it conservatively; and the suggestion for the third color-labeled area is to preserve it.

[0041] It should be noted that this step, based on the risk map generated in step S4, outputs surgical reference information to assist surgeons in decision-making. Specific output formats include: displaying the risk map on a monitor in an overlay format (e.g.,...). Figure 10 (See the right image in section d). Textual suggestions are provided for different areas of the risk map: red areas (tumor core) suggest "complete resection is recommended," yellow areas (infiltrative transition zone) suggest "conservative resection is recommended," and green areas (normal tissue) suggest "preservation is recommended." This output information is for reference only and does not directly control surgical instruments; the final decision rests with the surgeon based on the overall clinical situation.

[0042] It should also be noted that this patent uses a fluorescence image signal-to-noise ratio threshold to segment and define high-fluorescence candidate regions. Alternative solutions include using an absolute fluorescence intensity threshold, a region growing algorithm, or a deep learning-based segmentation model to define candidate regions. Any strategy that uses fluorescence imaging results as spatial constraints to reduce the Raman acquisition range is considered an equivalent substitution. This patent uses ICG with a concentration range of [missing information]. to Alternative solutions include using other near-infrared fluorescent dyes (such as IRDye, Cy5.5, and targeted probes), the optimal concentration of which is determined using the same orthogonal experimental principles. Any exogenous fluorescent contrast agent used at a concentration that ensures balance between fluorescence and Raman signals is considered an alternative. This patent uses affine transformation and inverse distance weighted interpolation to generate a color risk map. Alternative solutions include using spatial mapping methods such as perspective transformation, bilinear interpolation, and Gaussian process regression; the output format can be replaced with grayscale heatmaps, contour maps, or augmented reality projections. Any solution that spatially maps Raman point classification results to fluorescence images and generates a visual reference is considered an equivalent alternative.

[0043] According to another embodiment of the present invention, a tissue property identification system based on fluorescence Raman dual-modal fusion is provided, comprising: A fluorescence imaging module is used to apply near-infrared excitation light to the tissue region to be identified, which has been treated with an exogenous near-infrared fluorescent contrast agent, and to acquire and output a fluorescence image of the tissue region to be identified. The central control module is electrically connected to the fluorescence imaging module and is used to receive the fluorescence image and perform dynamic threshold segmentation on the fluorescence image to divide the tissue region to be identified into a high fluorescence candidate region and a low fluorescence background region, and output the set of spatial coordinates of the high fluorescence candidate region. The three-dimensional electric displacement platform module is electrically connected to the central control module and is used to receive the set of spatial coordinates and perform three-dimensional displacement drive under the control and guidance of the set of spatial coordinates. A Raman spectroscopy acquisition module, mounted on the three-dimensional electric displacement platform module and electrically connected to the central control module, is used to move only to each sampling point in the high fluorescence candidate region under the drive of the three-dimensional electric displacement platform module to acquire Raman spectra, and to transmit the raw Raman spectral data with spatial coordinate labels of each sampling point back to the central control module. The display module is electrically connected to the central control module and is used to receive the risk map and the surgical reference information, and to overlay and visualize the output.

[0044] In a preferred embodiment, the central control module specifically includes: The image processing unit is used to calculate the signal-to-noise ratio of pixels in the fluorescence image, extract the high-fluorescence candidate region using a dynamic threshold segmentation algorithm, and output the set of spatial coordinates. The spectral processing unit is used to iteratively optimize the original Raman spectral data by using the enhanced partial least squares (E-PLS) algorithm combined with linear interpolation to separate the fluorescence signal and the Raman signal, and output the corrected spectrum. The classifier running unit has a built-in XGBoost classifier, which is used to run the XGBoost classifier to make a probability determination on the corrected spectrum and output the probability value of the tissue category. The spatial registration unit is used to store the mapping relationship, and perform pixel mapping from the tissue category probability value to the fluorescence image, inverse distance weighted interpolation, and map overlay fusion to generate the risk map and match and output the surgical reference information.

[0045] It should be noted that the hardware system for each of the above steps consists of the following: a fluorescence imaging module (including a near-infrared light source, filters, and a CCD camera), a Raman spectroscopy acquisition module (including a 785nm laser, a fiber optic probe, and a spectrometer), a three-dimensional electric displacement platform module, a central control module (including an image processing unit, a spectral processing unit, a classifier operation unit, and a spatial registration unit), and a display. The connections between these modules are as follows: the central control module is electrically connected to the fluorescence imaging module, the Raman spectroscopy acquisition module, the three-dimensional displacement platform, and the display. The fluorescence imaging module transmits image data to the central control module. The central control module sends control commands to the displacement platform based on the image processing results. The displacement platform moves the Raman probe, and the spectrometer transmits the acquired spectral data back to the central control module. Finally, the processed results are output to the display.

[0046] In summary, by utilizing the above-mentioned technical solution of this invention, preliminary localization of tissue regions is achieved through fluorescence imaging, followed by Raman spectroscopy acquisition based on high-fluorescence candidate regions. Tissue classification is then performed using the molecular fingerprint characteristics of Raman spectroscopy, realizing a dual identification mechanism combining fluorescence screening and Raman confirmation. While maintaining the high sensitivity advantage of fluorescence imaging, this effectively reduces false positive results caused by non-specific aggregation of contrast agents, improving the accuracy and specificity of tissue identification. Furthermore, by using fluorescence images to determine high-fluorescence candidate regions and using these regions as the Raman spectroscopy acquisition range, Raman detection is performed only on the target region, transforming Raman sampling from full-area scanning to targeted sampling. This significantly reduces the number of sampling points and detection time, improving efficiency and effectiveness. It achieves high efficiency in intraoperative tissue identification, meeting real-time detection requirements. By spatially registering Raman classification results with fluorescence images and combining interpolation to generate risk maps, it can convert point-like molecular diagnostic results into planar visual information. Simultaneously, it outputs corresponding surgical reference information for different tissue categories, providing doctors with intuitive tissue boundary identification results and auxiliary decision-making basis, thus improving intraoperative decision-making efficiency. Enhanced partial least squares (E-PLS) combined with linear interpolation is used to perform fluorescence background subtraction on the original Raman spectrum, effectively separating the fluorescence background signal from the Raman characteristic signal, improving the identification quality of Raman characteristic peaks, and providing a reliable data foundation for subsequent tissue classification, thereby enhancing the accuracy and stability of tissue property identification results.

[0047] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, etc.) containing computer-usable program code.

[0048] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A tissue property identification method based on fluorescence Raman dual-modal fusion, characterized in that, Includes the following steps: S1. Apply near-infrared excitation light to the tissue area to be identified that has been treated with exogenous near-infrared fluorescent contrast agent, and acquire a fluorescence image of the tissue area to be identified; Dynamic threshold segmentation is performed on the fluorescence image to divide the tissue region to be identified into a high fluorescence candidate region and a low fluorescence background region, thereby obtaining the spatial coordinate set of the high fluorescence candidate region. S2. Based on the set of spatial coordinates, drive the Raman spectroscopy probe to move only to each sampling point within the high fluorescence candidate region to acquire Raman spectra, and obtain the raw Raman spectral data of each sampling point with spatial coordinate labels; S3. Perform fluorescence background subtraction preprocessing on the original Raman spectral data to obtain the corresponding corrected spectrum; Each of the corrected spectra is input into a pre-trained extreme gradient boosting classifier for classification, and the tissue category probability value of each sampling point is output. S4. Establish a mapping relationship between the Raman spectroscopy probe and the fluorescence image, and map the tissue category probability value of each sampling point to the corresponding pixel position of the fluorescence image based on the mapping relationship; use an inverse distance weighted interpolation algorithm to interpolate the mapped pixels to generate a classification probability distribution map with the same resolution as the fluorescence image; superimpose and fuse the classification probability distribution map with the original fluorescence intensity map of the fluorescence image to generate a color-coded risk map. S5. Based on the tissue resection priority corresponding to different color-labeled areas in the risk map, output text prompts for each different color-labeled area as surgical reference information to assist decision-making.

2. The tissue property identification method based on fluorescence Raman dual-modal fusion according to claim 1, characterized in that, The step of performing dynamic threshold segmentation on the fluorescence image to divide the tissue region to be identified into high-fluorescence candidate regions and low-fluorescence background regions, and obtaining the spatial coordinate set of the high-fluorescence candidate regions, includes: Calculate the signal-to-noise ratio (SNR) of each pixel in the fluorescence image, wherein the SNR is the ratio of the fluorescence signal intensity to the standard deviation of the background noise; Using the mean global signal-to-noise ratio of the fluorescence image plus 1.5 times the standard deviation as the segmentation threshold, the tissue region to be identified is divided into high fluorescence candidate regions with a signal-to-noise ratio greater than the segmentation threshold and low fluorescence background regions with a signal-to-noise ratio less than or equal to the segmentation threshold. Record and store the spatial coordinates of the high fluorescence candidate regions to form a set of spatial coordinates of the high fluorescence candidate regions.

3. The tissue property identification method based on fluorescence Raman dual-modal fusion according to claim 1, characterized in that, The exogenous near-infrared fluorescent contrast agent is indocyanine green, and the application concentration of the exogenous near-infrared fluorescent contrast agent is 10. -10 M to 10 -9 M; The method for determining the application concentration is to pre-set 10 -11 M to 10 -6 Multiple concentration gradients were used to acquire test fluorescence images and test Raman spectra at each concentration gradient. A selection of 10 samples was made based on a comprehensive evaluation criterion: the fluorescence signal intensity of the test fluorescence image reached the imaging threshold, and the signal-to-noise ratio of the characteristic peaks of the test Raman spectrum was greater than 3. -10 M to 10 - 9 The concentration range of M.

4. The tissue property identification method based on fluorescence Raman dual-modal fusion according to claim 1, characterized in that, The original Raman spectral data is preprocessed by fluorescence background subtraction to obtain the corresponding corrected spectra, including: Wavenumber points in the original Raman spectral data that are not affected by Raman characteristic peaks are selected as baseline points. An initial baseline is constructed by linear interpolation. The initial baseline is subtracted from the original Raman spectral data to obtain the preliminary corrected spectrum. The enhanced partial least squares (E-PLS) algorithm is used to iteratively optimize the preliminary corrected spectrum with a preset number of principal components and iterations. In each iteration, the fluorescence background residual is fitted using partial least squares to gradually separate the fluorescence signal and the Raman signal, and finally the corrected spectrum after subtracting the fluorescence background is obtained.

5. The tissue property identification method based on fluorescence Raman dual-modal fusion according to claim 1, characterized in that, Before inputting the corrected spectra into a pre-trained extreme gradient boosting classifier for classification and outputting the tissue category probability value for each sampling point, the process further includes a training step for the extreme gradient boosting XGBoost classifier: Several Raman spectral data points for each of the three categories of samples—tumor core area, infiltrative transition area, and normal tissue—were collected in advance from tumor-bearing animal models. The fluorescence background subtraction preprocessing is performed on each sample Raman spectrum data, and the intensity value in the full spectrum is extracted as a feature vector; The XGBoost algorithm is used to train the XGBoost classifier based on the feature vector, and the hyperparameters are optimized by five-fold cross-validation until training is complete.

6. The tissue property identification method based on fluorescence Raman dual-modal fusion according to claim 1, characterized in that, The process of establishing the mapping relationship between the Raman spectroscopy probe and the fluorescence image includes: Using cross-shaped markers as calibration objects, four marker points with known coordinates are set within the field of view of the Raman spectroscopy probe; Acquire fluorescence images of the marked points and record the original paths; Move the Raman spectroscopy probe to the origin and control its movement sequentially along the original path to establish a coordinate mapping relationship between the Raman spectroscopy probe coordinates and the fluorescence image pixel coordinates.

7. The tissue property identification method based on fluorescence Raman dual-modal fusion according to claim 1, characterized in that, The risk map includes a variety of preset color labels, and different color labels are used to characterize different tissue categories and corresponding tissue resection priorities; The text prompts for each different color label area include: matching and outputting guiding clinical decision text information corresponding to the tissue resection priority based on the color label corresponding to each area in the risk map.

8. The tissue property identification method based on fluorescence Raman dual-modal fusion according to claim 7, characterized in that, The tissue categories include the tumor core area, the infiltrative transition area, and the normal tissue area, wherein the tumor core area is coded with a first color label, the infiltrative transition area is coded with a second color label, and the normal tissue area is coded with a third color label. The suggestion for the first color-labeled area is to completely remove it; the suggestion for the second color-labeled area is to remove it conservatively; and the suggestion for the third color-labeled area is to preserve it.

9. A tissue property identification system based on fluorescence Raman dual-modal fusion, used to implement any one of the tissue property identification methods based on fluorescence Raman dual-modal fusion as described in claims 1-8, characterized in that, include: A fluorescence imaging module is used to apply near-infrared excitation light to the tissue region to be identified, which has been treated with an exogenous near-infrared fluorescent contrast agent, and to acquire and output a fluorescence image of the tissue region to be identified. The central control module is electrically connected to the fluorescence imaging module and is used to receive the fluorescence image and perform dynamic threshold segmentation on the fluorescence image to divide the tissue region to be identified into a high fluorescence candidate region and a low fluorescence background region, and output the set of spatial coordinates of the high fluorescence candidate region. The three-dimensional electric displacement platform module is electrically connected to the central control module and is used to receive the set of spatial coordinates and perform three-dimensional displacement drive under the control and guidance of the set of spatial coordinates. A Raman spectroscopy acquisition module, mounted on the three-dimensional electric displacement platform module and electrically connected to the central control module, is used to move only to each sampling point in the high fluorescence candidate region under the drive of the three-dimensional electric displacement platform module to acquire Raman spectra, and to transmit the raw Raman spectral data with spatial coordinate labels of each sampling point back to the central control module. The display module is electrically connected to the central control module and is used to receive the risk map and the surgical reference information, and to overlay and visualize the output.

10. A tissue property identification system based on fluorescence Raman dual-modal fusion according to claim 9, characterized in that, The central control module specifically includes: The image processing unit is used to calculate the signal-to-noise ratio of pixels in the fluorescence image, extract the high-fluorescence candidate region using a dynamic threshold segmentation algorithm, and output the set of spatial coordinates. The spectral processing unit is used to iteratively optimize the original Raman spectral data by using the enhanced partial least squares (E-PLS) algorithm combined with linear interpolation to separate the fluorescence signal and the Raman signal, and output the corrected spectrum. The classifier running unit has a built-in XGBoost classifier, which is used to run the XGBoost classifier to make a probability determination on the corrected spectrum and output the probability value of the tissue category. The spatial registration unit is used to store the mapping relationship, and perform pixel mapping from the tissue category probability value to the fluorescence image, inverse distance weighted interpolation, and map overlay fusion to generate the risk map and match and output the surgical reference information.

Images