Tumor detection method, system and computer equipment

Abstract

The present invention relates to the field of tumor detection technology and discloses a tumor detection method, system, and computer equipment. The method comprises: obtaining spontaneous fluorescence attenuation data of a target biological tissue and extracting fluorescence intensity and fluorescence lifetime data therefrom; fusing the fluorescence lifetime data and fluorescence intensity data of the target biological tissue to obtain dual-channel fused data; inputting the dual-channel fused data into a tumor detection model pre-trained based on a deep learning network model, outputting the predicted probability of each pixel corresponding to different categories, using the category with the highest probability value as the tumor detection result for each pixel, and obtaining the distribution location of the tumor and blood vessels in the target biological tissue. The present invention fuses the fluorescence lifetime data and fluorescence intensity data and inputs them into the tumor detection model, allowing the model to analyze tissue characteristics from a richer and more comprehensive dimension, helping to more accurately locate the distribution location of tumors and blood vessels in the target biological tissue, thereby improving detection accuracy and recognition speed.

Description

Technical Field

[0001] The present invention relates to the field of tumor detection and processing, and in particular to a tumor detection method, system and computer equipment. Background Art

[0002] Accurate identification of tumor boundaries is crucial for tumor resection and prognosis. Histological diagnosis, the gold standard for distinguishing cancerous tissue from normal tissue, relies heavily on pathologists analyzing morphological features on hematoxylin and eosin (H&E)-stained tissue sections. However, staining methods are complex, time-consuming, and inefficient, making them unsuitable for intraoperative pathological diagnosis.

[0003] With the development of nonlinear optical microscopy and artificial intelligence, high-resolution, stain-free histopathology techniques have long been used to assist in tumor diagnosis. Multimodal imaging techniques, such as second harmonic generation (SHG), two-photon excited fluorescence (TPEF), and coherent anti-Stokes Raman scattering (CARS), have recently shown great potential for rapid tumor diagnosis. By selectively detecting proteins and lipids, SRS produces images comparable to H&E staining and automatically identifies cancerous regions using a convolutional neural network (CNN). However, the spatiotemporal overlap of the Stokes and pump beams is a prerequisite for obtaining high-fidelity pathological images using SRS microscopy, placing additional demands on optical pathology design and system stability. Multiphoton fluorescence microscopy is easier to perform than coherent Raman microscopy and can provide fluorescence intensity imaging of brain tumors and the central nervous system (CNS) at the subcellular level. By extracting and statistically analyzing structural information from multichannel fluorescence images, tumor and vascular regions can be identified. However, this multichannel detection increases the complexity of the imaging system. Summary of the Invention

[0004] In view of this, the present invention provides a tumor detection method, system and computer device to solve the problem of how to improve the accuracy of non-staining tumor detection.

[0005] In a first aspect, the present invention provides a tumor detection method, comprising:

[0006] Obtain the spontaneous fluorescence decay data of the target biological tissue and extract the fluorescence intensity data and fluorescence lifetime data from it;

[0007] fusing the fluorescence lifetime data and the fluorescence intensity data of the target biological tissue to obtain dual-channel fused data;

[0008] The dual-channel fusion data is input into a tumor detection and recognition model pre-trained based on a deep learning network model, and the predicted probability of each pixel corresponding to different categories is output. The category with the highest probability value is used as the tumor detection result for each pixel, and the distribution location of the tumor and blood vessels in the target biological tissue is obtained.

[0009] The tumor detection method provided in this embodiment acquires fluorescence lifetime data and fluorescence intensity data and fuses them to form dual-channel fused data, integrating two different types of fluorescence-related information. Fluorescence lifetime primarily reflects the microenvironment of the organism, while fluorescence intensity primarily reflects the morphological structure of the biological tissue. Both represent the state of the target biological tissue from different perspectives. Fusion of these data and input into the tumor detection model allows the model to analyze tissue characteristics from a richer and more comprehensive perspective, helping to more accurately locate the distribution of tumors and blood vessels in the target biological tissue, reducing misjudgments caused by a single data type, and improving detection accuracy and recognition speed.

[0010] In an optional embodiment, the method further includes: converting the tumor detection result into a detection result image in a preset format;

[0011] generating a fluorescence intensity image based on the fluorescence intensity data of the target biological tissue;

[0012] The detection result image is superimposed on the fluorescence intensity image, and the final detection result is displayed by controlling the transparency of the detection result image.

[0013] The embodiment of the present invention superimposes the test result image with a fluorescence intensity image generated based on the fluorescence intensity data, enabling simultaneous display of key information related to tumor detection and the fluorescence intensity distribution of the tissue itself. Fluorescence intensity demonstrates the structure of the tissue. By superimposing the prediction results and adjusting the transparency of the prediction results, the corresponding pathological areas can be directly distinguished on the intensity map, achieving intuitive visualization of the target area. The flexibly adjustable transparency eliminates the need for users to switch back and forth between different images for comparison, allowing them to quickly obtain key information within the same image. This reduces the time cost of information integration during the analysis process, thereby speeding up the analysis of test results and improving the efficiency of the entire pathology diagnosis process.

[0014] In an optional embodiment, the process of training the tumor detection model includes:

[0015] Obtaining autofluorescence decay data of a preset tumor tissue slice sample, and extracting fluorescence intensity data, fluorescence lifetime data, and intensity-weighted lifetime images therefrom;

[0016] The intensity-weighted fluorescence lifetime image is used as a mask to annotate the tumor, blood vessels, and background, generating a pixel-level labeled image.

[0017] The fluorescence lifetime data and the fluorescence intensity data are fused to obtain dual-channel fusion data;

[0018] The dual-channel fusion data and the label image are used to form a data pair, and the training set and the validation set are divided according to a preset ratio. The preset convolutional neural network is trained based on the training set, and the trained model is verified based on the validation set to obtain a model that meets the preset requirements as a tumor detection model.

[0019] The embodiment of the present invention obtains and fuses the spontaneous fluorescence lifetime data and fluorescence intensity data of the tissue to form dual-channel fusion data, taking into account two key information reflecting the characteristics of the tissue. The fused data covers richer tissue details, allowing the model to fully learn the comprehensive characteristics of different tissues in these two aspects during the training process, and improve the model's ability to distinguish various types of tissues such as tumors and blood vessels; the intensity-weighted lifetime image is used as a mask to mark tumors, blood vessels and background, and then generate a pixel-level label image. This labeling method is accurate to the pixel level and can carefully divide the boundaries and ranges of different tissues, avoiding the errors caused by rough labeling, so that the model can receive accurate supervision information during training, which is conducive to learning the characteristic differences of different tissues at the microscopic level, thereby improving the accuracy of the model in identifying target tissues such as tumors.

[0020] In an optional embodiment, obtaining fluorescence lifetime data of a preset tumor tissue slice sample includes:

[0021] The collected fluorescence lifetime data were fitted with a double exponential model by least squares fitting to calculate the fluorescence lifetime distribution, and the average fluorescence lifetime data were obtained based on the fluorescence lifetime distribution.

[0022] The embodiment of the present invention adopts a dual-exponential model that is more consistent with the two-component characteristics that may exist in the fluorescence decay process under actual conditions. Compared with simple single-exponential models, it can more finely capture the decay law of fluorescent substances under different states, thereby more accurately reflecting the fluorescence lifetime corresponding to each pixel, reducing the fitting error caused by improper model selection, and improving the accuracy of the obtained fluorescence lifetime data. After obtaining the fluorescence lifetime distribution, the average fluorescence lifetime data is further obtained based on it. This approach fully considers the overall distribution characteristics of fluorescence lifetime in the sample tissue, so that the final average fluorescence lifetime data can more scientifically and reasonably reflect the overall characteristics of the fluorescence lifetime of the entire tumor tissue section sample, enhance the reliability of the data, and provide a solid data foundation for subsequent tumor detection, analysis, and other work based on this data.

[0023] In an optional embodiment, the step of using the intensity-weighted lifespan image as a mask to label the tumor, blood vessels, and background to generate a pixel-level label image includes:

[0024] Obtain H&E images corresponding to preset tumor tissue slice samples;

[0025] comparing the features of the H&E image and the intensity-weighted lifetime image, and marking the tumor, blood vessels, and background with different colors as annotation data to obtain an initial pixel-level labeled image;

[0026] The labeled data is screened and optimized based on expert experience and knowledge to obtain the final pixel-level labeled image.

[0027] H&E images are classic staining images commonly used in pathological diagnosis, clearly demonstrating important information such as cell morphology and tissue structure. The present invention utilizes intensity-weighted fluorescence lifetime images to reveal tissue characteristics from the perspective of fluorescence lifetime and tissue structure. Combining these two methods allows for detailed tissue analysis from different dimensions, enabling annotators to more comprehensively and accurately grasp the specific location and extent of tumors, blood vessels, and background, avoiding the potential bias associated with labeling based solely on a single image. This improves labeling accuracy and provides detailed, pixel-by-pixel classification of tissues using color-coded labels. This method clearly demonstrates the microscopic boundaries and distribution of different tissues, facilitating more precise characterization of tissue differences during subsequent model training and other steps. This provides a robust data foundation for accurate tumor detection. By leveraging expert knowledge to refine and refine these images, the resulting pixel-level labeled images more accurately reflect actual tissue characteristics and pathological patterns, significantly enhancing the reliability and professionalism of the labeled images.

[0028] In an optional embodiment, the method of training a preset convolutional neural network based on a training set and validating the trained model based on a validation set to obtain a model that meets preset requirements as a tumor detection model includes:

[0029] Input the training set into the U-Net network and train it based on the preset training parameters;

[0030] After the training is completed, the dual-channel fusion data and the corresponding label images in the verification set will be input into the trained network model as verification data. The effect of the model training will be verified by calculating the preset performance indicators and comparing them with the preset verification targets. The model that achieves the preset verification targets will be used as the tumor detection model.

[0031] This embodiment of the present invention selects the U-Net network as the basic architecture for training. The U-Net network has excellent performance in fields such as medical image segmentation. When processing dual-channel fusion data corresponding to tumor tissue slice samples, it can better transmit and utilize feature information at different scales, avoiding the loss of important details during feature extraction and recovery. This is well aligned with the need to accurately distinguish subtle differences between different tissues in tumor detection, enhancing the model's applicability in this specific application scenario.

[0032] In a second aspect, the present invention provides a tumor detection system, comprising:

[0033] A fluorescence data acquisition module is used to obtain the spontaneous fluorescence decay data of the target biological tissue and extract fluorescence intensity data and fluorescence lifetime data therefrom;

[0034] a data fusion module, configured to fuse the fluorescence lifetime data and the fluorescence intensity data of the target biological tissue to obtain dual-channel fused data;

[0035] The tumor detection module is used to input the dual-channel fusion data into a tumor detection model pre-trained based on a deep learning network model, output the predicted probability of each pixel corresponding to different categories, take the category with the highest probability value as the tumor detection result for each pixel, and obtain the distribution location of tumors and blood vessels in the target biological tissue.

[0036] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the tumor detection method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0037] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the tumor detection method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 is a schematic flow chart of a tumor detection method in an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of another tumor detection method according to an embodiment of the present invention;

[0041] Figure 3 Schematic diagram of the main framework of the tumor detection model training process according to an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of the U-Net network structure according to an embodiment of the present invention;

[0043] Figure 5 This is a schematic diagram of the detection results of the fusion of fluorescence lifetime and fluorescence emphasis information according to an embodiment of the present invention;

[0044] Figure 6 Schematic diagrams of detection results obtained based on fluorescence lifetime or fluorescence emphasis information, respectively, according to an embodiment of the present invention;

[0045] Figure 7 is a structural block diagram of a tumor detection system according to an embodiment of the present invention;

[0046] Figure 8 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0048] Tumor detection methods in the prior art include:

[0049] First, marker-based tumor recognition methods.

[0050] Tissue staining: Through the steps of fixing, sectioning, staining, and mounting, samples are collected to obtain information on cell morphology and tissue structure to identify tumor areas. Although this method provides intuitive histological features, it is cumbersome, time-consuming, and labor-intensive. Furthermore, diagnostic criteria may vary between pathologists, affecting the consistency and accuracy of the results.

[0051] Fluorescent dye staining: Specimens are labeled with dyes and the intensity of fluorescence emission from the target object is used to differentiate pathological types. However, factors such as the lack of photostability of fluorescent dyes and intratumor heterogeneity can easily lead to false-negative or false-positive results, limiting diagnostic reliability.

[0052] Second, based on marker-free tumor recognition methods.

[0053] Fluorescence intensity image analysis: This method extracts local or global information from the fluorescence intensity image and combines it with traditional image processing techniques (such as threshold segmentation) to segment the tumor region. Although this method can reflect the tumor region to a certain extent, the uneven distribution of image intensity due to tumor heterogeneity and variations in excitation light intensity, coupled with significant variability between different samples, makes it difficult for traditional algorithms to accurately segment the tumor region.

[0054] Fluorescence lifetime parameter analysis: Tumor regions are identified based on differences in fluorescence lifetime parameters, but existing methods are mostly limited to statistical analysis, making it difficult to achieve accurate identification with subcellular resolution.

[0055] Third, tumor recognition method based on deep learning.

[0056] Deep learning-based methods leverage the powerful feature extraction capabilities of convolutional neural networks to analyze morphological and structural information in fluorescence microscopy images or chemical bond vibration information obtained from stimulated Raman scattering (SRS) images to identify tumor regions. However, these methods typically rely on the classification of image patches and can only output regional-level results, failing to directly achieve accurate tumor identification at the subcellular resolution level.

[0057] Based on this, an embodiment of the present invention provides a tumor detection method based on autofluorescence lifetime information and autofluorescence intensity information. Fluorescence lifetime is highly sensitive to the microenvironment and the interactions around the fluorophore. Compared with intensity-based imaging, fluorescence lifetime imaging microscopy (FLIM) is less susceptible to artifacts such as fluorophore concentration, photobleaching, variations in excitation intensity, and uneven sample illumination. FLIM typically measures endogenous fluorophores associated with redox reactions, such as nicotinamide adenine dinucleotide (NADH) or flavin adenine dinucleotide (FAD), to reveal metabolic heterogeneity within tissues. The embodiment of the present invention utilizes a deep learning network to analyze dual-channel data fused with fluorescence lifetime information and fluorescence intensity information, which can significantly improve the speed and accuracy of identifying tumor boundaries and blood vessels.

[0058] According to an embodiment of the present invention, an embodiment of a tumor detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0059] In this embodiment, a tumor detection method is provided, which can be used in a computer device terminal. Figure 1 FIG. 1 is a flow chart of a tumor detection method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0060] Step S101 : acquiring spontaneous fluorescence attenuation data of a target biological tissue, and extracting fluorescence intensity data and fluorescence lifetime data therefrom.

[0061] Specifically, a fluorescence lifetime microscopy system based on time-correlated single-photon counting (TCSPC) is used to acquire spontaneous fluorescence decay data from the target biological tissue to be examined. In one embodiment, time-correlated single-photon counting (TCSPC) technology is used to acquire fluorescence lifetime data. The principle is to use a pulsed laser beam to scan the sample to excite fluorescent molecules. TCSPC technology utilizes a high-time-resolution detector (such as a photomultiplier tube) to record the arrival time of each fluorescent photon. The system synchronizes with the pulse signal of the excitation light to precisely time the arrival time of each fluorescent photon. By recording the temporal distribution of a large number of fluorescent photons, the corresponding fluorescence decay data can be obtained. Fluorescence lifetime data and an intensity-weighted fluorescence lifetime image are calculated by fitting and analyzing the fluorescence decay curve of each pixel. The corresponding fluorescence intensity image is obtained by summing the fluorescence decay curves of each pixel.

[0062] Step S102 : fusing the fluorescence lifetime data and the fluorescence intensity data of the target biological tissue to obtain dual-channel fused data.

[0063] Specifically, in one embodiment, the data fusion process treats the fluorescence lifetime data and fluorescence intensity data as two independent channels and splices them in a certain order across the data dimensions. In the case of image data, if the fluorescence lifetime image and the fluorescence intensity image have the same number of pixels, the fluorescence lifetime value and fluorescence intensity value corresponding to each pixel can be arranged in sequence to form a new dual-channel data structure. In another embodiment, considering that the fluorescence lifetime data and fluorescence intensity data may have different importance or contributions to the final result, they can be assigned different weights and then fused using a weighted average. The weights can be determined based on experience, prior knowledge, or experimental evaluation of the importance of the two data types for tumor detection. For example, if it is known that fluorescence lifetime data is more important in distinguishing tumor tissue from normal tissue, its corresponding weight can be set larger. The above data fusion process is provided as an example only and is not intended to be limiting.

[0064] In step S103, the dual-channel fusion data is input into a tumor detection model pre-trained based on a deep learning network model, and the predicted probability of each pixel corresponding to different categories is output. The category with the highest probability value is used as the tumor detection result for each pixel, and the distribution location of the tumor and blood vessels in the target biological tissue is obtained.

[0065] This embodiment of the present invention adjusts the format of dual-channel fused data based on the input requirements of a deep learning network model and inputs it into the model. The deep learning network model uses a convolutional neural network (CNN), using a U-Net architecture as an example. After the fluorescence lifetime and fluorescence intensity data of the sample tissue are fused, the data first enters the encoder portion of the model. Through a series of convolutional and pooling layers, features are gradually extracted from the data. During this process, the convolutional layers convolve the input data using convolution kernels to extract local features at different scales. The pooling layers reduce the data dimensionality, reducing computational effort and increasing accuracy. After feature extraction by the encoder, the data enters the decoder portion, where upsampling and convolutional layers gradually restore the data's resolution and fuse and refine the extracted features. Ultimately, the model outputs two main parts: one is the predicted probability of each pixel corresponding to different categories, which is obtained through a classifier (such as a fully connected layer followed by a softmax function). The softmax function can convert the numerical value output by the classifier into a probability distribution so that the sum of the probabilities of each category corresponding to each pixel is 1; the other is the distribution location information of tumors and blood vessels in the target biological tissue, which is usually represented by generating a segmented image of the same size as the input data (or after a certain scaling), in which different pixel values ​​(or colors) represent different tissue categories (such as tumors, blood vessels, background, etc.).

[0066] For each pixel, the model outputs the predicted probabilities for the different categories, selecting the category with the highest probability as the pixel's classification result. The classification results for all pixels are combined to obtain the tumor detection result. For example, if the model outputs the categories tumor, blood vessel, and background, and the predicted probabilities for a particular pixel are 0.7, 0.2, and 0.1, respectively, then the pixel is classified as a tumor. In this way, the tissue classification of different regions within the entire target biological tissue can be determined, thereby determining the location and extent of the tumor.

[0067] In this embodiment of the present invention, the model outputs the predicted probability of each pixel corresponding to different categories, providing a means of quantifying the uncertainty of the detection results. In practical applications, the complexity of biological tissue and individual differences may lead to a certain degree of uncertainty in the detection results. Through the probability output, users can understand the reliability of each pixel classification. For example, if a pixel's predicted probability for the tumor category is 0.55, the predicted probability for the blood vessel category is 0.4, and the predicted probability for the background category is 0.1, although the pixel is ultimately classified as a tumor, the low probability indicates a certain risk of misclassification, requiring further examination or confirmation with other diagnostic methods. In scenarios where detection accuracy is extremely important (such as early cancer screening), a higher probability threshold can be set to only classify pixels with higher probabilities as tumors to reduce false positives. In scenarios where a comprehensive understanding of the tissue context is required (such as tumor boundary analysis), the threshold can be appropriately lowered to capture a wider area of ​​possible tumors, which can then be further verified using other methods. This flexibility enables the model to better adapt to diverse clinical and research needs.

[0068] The tumor detection method provided in this embodiment obtains fluorescence lifetime data and fluorescence intensity data and fuses them to form dual-channel fusion data, which is then input into a tumor detection model. This allows the model to analyze tissue characteristics from a richer and more comprehensive dimension, helping to more accurately locate the distribution of tumors and blood vessels in target biological tissues, reduce misjudgments caused by a single data type, and improve detection accuracy and recognition speed.

[0069] In one embodiment, the tumor detection method provided by the embodiment of the present invention is as follows: Figure 2 As shown, after step S103, the following steps are also included:

[0070] Step S104: converting the tumor detection result into a detection result image in a preset format.

[0071] Specifically, tumor detection results are converted into images in a pre-set format (e.g., stored as PNG images), resulting in a standardized presentation of the results. This facilitates different users, such as doctors and researchers, to quickly view and interpret the test results according to the established format, improving the efficiency of information transmission.

[0072] Step S105 : generating a fluorescence intensity image based on the fluorescence intensity data of the target biological tissue.

[0073] Step S106 , superimposing the detection result image and the fluorescence intensity image, and displaying the final detection result by controlling the transparency of the detection result image.

[0074] This embodiment of the present invention overlays the detection result image with a fluorescence intensity image generated based on the fluorescence intensity data, simultaneously displaying key information related to tumor detection and the fluorescence intensity distribution of the tissue itself. Fluorescence intensity displays the tissue structure, and the predicted results are superimposed. By adjusting the transparency of the predicted results, the corresponding areas can be directly viewed on the intensity map. This allows for direct visualization of target areas on the intensity map, aiding more accurate analysis and judgment.

[0075] By controlling the transparency of the test result image to display the final test results, this method increases the flexibility of display. Users can adjust the transparency according to their actual needs. For example, during the initial review, the transparency can be appropriately lowered to more comprehensively observe the overall fluorescence intensity background of the tissue and the approximate location of the tumor; when focusing on the details of the tumor, the transparency can be appropriately increased to highlight the specific morphology, boundaries and other features of the tumor, meeting the needs of detailed observation and analysis of test results in different scenarios.

[0076] The comprehensive presentation of the test result image and fluorescence intensity image by the embodiment of the present invention can provide doctors and other professionals with richer information for diagnosis. The changes in fluorescence intensity can be compared to further confirm the rationality of the tumor detection results. The characteristics of the two images can be combined to determine whether there are misjudgments, etc., which helps to improve the accuracy of diagnosis and provide a reliable basis for the subsequent formulation of accurate treatment plans. At the same time, the user does not need to switch back and forth between different images for comparison, and can quickly obtain key information in the same screen, reducing the time cost of information integration during the analysis process, thereby speeding up the analysis of the test results and improving the efficiency of the entire diagnostic process.

[0077] In one embodiment, the main framework of the tumor detection model training process is as follows: Figure 3 As shown in the figure, using TCSPC-FLIM (time-correlated single photon counting-fluorescence lifetime imaging microscopy) technology on mouse brain slices, three different image modalities were obtained: fluorescence intensity, mean fluorescence lifetime, and intensity-weighted lifetime images. The fluorescence intensity and mean fluorescence lifetime data were fused to generate dual-channel data. The intensity-weighted lifetime images were then annotated to identify regions of interest or objects, generating labeled images. The dual-channel data and the corresponding labeled images were then used to train a U-Net network for image segmentation. The trained multi-view classifier was then used to classify the fused images. The classification results were then visualized for easy observation and analysis.

[0078] In one embodiment, the tumor detection model training process specifically includes the following steps:

[0079] Step S1 : obtaining autofluorescence attenuation data of a preset tumor tissue slice sample, and extracting fluorescence intensity data, fluorescence lifetime data, and an intensity-weighted lifetime image therefrom.

[0080] Specifically, tumor tissue samples were obtained from mouse brain tumors. Mice were sacrificed by cervical dislocation 15 days after tumor inoculation. The brain tissue was then rapidly removed, bisected at the tumor injection site, fixed in formalin, and embedded in paraffin. Serial sections (5 µm thickness) were cut from different brain regions using a freezing microtome. One section was used for H&E staining for histological comparison, while the remaining sections were deparaffinized and used for fluorescence lifetime imaging.

[0081] Furthermore, fluorescence lifetime imaging was performed on deparaffinized mouse brain tumor tissue sections using a fluorescence lifetime microscopy system based on time-correlated single photon counting. Fluorescence decay data for the endogenous fluorescent substance FAD were collected and saved as .sdt files. To ensure identical experimental conditions, all raw data were acquired at the same time. Fluorescence lifetime calculation and fitting were performed using SPCImage 8.9 software (Becker & Hickl GmbH). The fluorescence decay data were mapped to the fluorescence decay histogram of each pixel in the fluorescence lifetime image, and a biexponential model was used to perform a least-squares fit to calculate the fluorescence lifetime distribution. Based on the fluorescence lifetime distribution, the average fluorescence lifetime data were obtained. Finally, the average fluorescence lifetime data (.asc file), fluorescence intensity data (.asc file), and intensity-weighted lifetime images (.tif image) were exported for subsequent analysis.

[0082] The biexponential model employed in the present invention better captures the multi-component characteristics of fluorescence decay in real-world situations. Compared to simpler single-exponential models, it captures the decay patterns of fluorescent substances under different conditions in greater detail, thereby more accurately reflecting the fluorescence lifetime of each pixel. This reduces fitting errors caused by improper model selection, improves the accuracy of the acquired fluorescence lifetime data, and further, based on it, obtains average fluorescence lifetime data. This approach fully considers the overall distribution of fluorescence lifetime within the sample tissue. Rather than simply representing it with a single numerical value, it integrates the different fluorescence lifetimes of each pixel. This allows the resulting average fluorescence lifetime data to more scientifically and rationally reflect the overall fluorescence lifetime characteristics of the entire tumor tissue sample, enhancing data reliability and providing a solid foundation for subsequent tumor detection and analysis based on this data.

[0083] Step S2: Use the intensity-weighted lifetime image as a mask to label the tumor, blood vessels, and background to generate a pixel-level label image.

[0084] In this embodiment of the present invention, the Image Labeler tool in MATLAB R2019B was used to annotate intensity-weighted lifetime images. Using these images as masks, the image not only provides information about the tissue's morphological structure but also contains rich information about the microenvironmental characteristics. During the annotation process, H&E images were used as a reference. These are classic staining images commonly used in pathological diagnosis and clearly present important information such as cell morphology and tissue structure. Three predefined semantic categories—tumor, vessel, and background—were used to compare the detailed features of the two images, enabling exploration of tissue details from different dimensions and annotation of tumor regions and vascular distribution. To better distinguish these categories, each label was assigned a different color. The polygon and brush tools were used to precisely annotate the target areas. By meticulously segmenting tissues at the pixel level using color-coded categories, the boundaries and distribution of different tissues at the microscopic level are clearly visualized, helping subsequent model training and other steps to more accurately learn the characteristic differences between tissue types, laying a solid data foundation for accurate tumor detection.

[0085] After the labeling is completed, the label data is exported as a pixel-level label map. Each pixel value of these label maps represents the corresponding category, and the resolution of the label map is consistent with the original image. In addition, the embodiment of the present invention is based on a number of experts with extensive experience in tissue preparation and imaging. Based on the expert experience and knowledge, the labeled data has been strictly screened and optimized for multiple rounds, so that the pixel-level label image finally generated is more consistent with the actual tissue characteristics and pathological laws, greatly enhancing the reliability and professionalism of the label image. It not only improves the accuracy and scientific value of the data, but also lays a solid foundation for the training of tumor and blood vessel recognition models based on deep learning.

[0086] Step S3: fusing the fluorescence intensity data and the fluorescence lifetime data to obtain dual-channel fused data.

[0087] Specifically, software uses weighted fusion or splicing methods to combine fluorescence intensity and mean fluorescence lifetime data into dual-channel data, where the first channel represents mean fluorescence lifetime and the second channel represents fluorescence intensity. This dual-channel data preserves the mean fluorescence lifetime information of the tissue while simultaneously displaying the fluorescence intensity distribution, facilitating subsequent data analysis. Finally, the synthesized dual-channel data is saved as a .mat file for subsequent analysis.

[0088] In step S4, the dual-channel fusion data and the label image are combined into a data pair, and the training set and the validation set are divided according to a preset ratio. The preset convolutional neural network is trained based on the training set, and the trained model is verified based on the validation set to obtain a model that meets the preset requirements as a tumor detection model.

[0089] Specifically, the present invention is based on the U-Net multi-category semantic segmentation model for training, and the network structure is as follows Figure 4 As shown in Figure 1, it consists of an encoder and a decoder. The encoder primarily extracts image feature information while gradually reducing the spatial resolution. It follows the typical architecture of a convolutional network, consisting of repeated application of two 3×3 convolutional layers (conv), each followed by a rectified linear unit (ReLu) activation layer, and a 2×2 max pooling layer with 2 strides (maxpool) for downsampling. Each downsampling process doubles the number of feature channels, halving the image resolution. The decoder primarily restores the image's spatial resolution, producing an output of the same size as the input image. Each decoder unit performs a 2×2 upsampling (up-conv) of the feature map, halving the number of feature channels. It then makes a skip connection (copy) to the encoder's feature map of the same resolution to maintain precise localization of the extracted data patterns. Two 3×3 convolutional layers (conv) are then applied, each followed by a ReLu activation layer. In the final layer, a 1×1 convolutional layer (conv) maps each 64-dimensional feature vector to the desired number of categories. Zero padding is applied to each convolutional layer, maintaining consistent resolution at edges.

[0090] Furthermore, data pairs are constructed using the dual-channel data (.mat files) and the corresponding labeled images. These pairs are stored as the training set and validation set in an 8:2 ratio. During training, the training files (.mat files) and the training targets (label images, three-category semantic segmentation) are converted into tensor data as training data. This training data is then fed into the fully convolutional neural network constructed in Step 5 for training. The training parameters are: the number of epochs ranges from 100 to 500, the Adam optimization algorithm is used as the optimizer, the classification loss is calculated using the binary cross entropy loss function, the learning rate lr ranges from 0.001 to 0.0001, the exponential decay rate beta1 of the first-order moment estimate is 0.9, the exponential decay rate beta2 of the second-order moment estimate is 0.999, and the weight decay rate is 0. After training is complete, the validation files and corresponding labeled images in the validation set are converted into tensor data and fed into the trained network model as validation data. The effectiveness of model training is verified by comparing it with the validation target and calculating performance indicators such as IOU (Intersection over Union) and Dice coefficient. The model that achieves the preset validation target is used as the tumor detection model. The data used for the above training parameters are only examples and are not limited to this.

[0091] This embodiment of the present invention selects the U-Net network model as the basic architecture for training. The U-Net network has excellent performance in fields such as medical image segmentation. When processing dual-channel fusion data corresponding to tumor tissue slice samples, it can better transmit and utilize feature information at different scales, avoiding the loss of important details during feature extraction and recovery. This is well aligned with the need to accurately distinguish subtle differences between different tissues in tumor detection, enhancing the model's applicability in this specific application scenario.

[0092] In a specific embodiment, if Figure 5 As shown, each set of images contains four parts, from left to right: τm (fluorescence lifetime), Intensity (fluorescence intensity), H&E (hematoxylin-eosin staining) and Output - overlay (output overlay).

[0093] 1. The τm (fluorescence lifetime) image shows different color distributions, indicating the changes in fluorescence lifetime in the sample. Tumor tissue, blood vessels, and normal tissue may have differences in fluorescence lifetime. These images can be used to observe the distribution and pattern of these differences.

[0094] 2. The Intensity image displays the distribution of fluorescence intensity within the sample. Fluorescence intensity is related to factors such as the concentration of the fluorescent substance. Areas of strong fluorescence intensity may correspond to specific tissue components or cellular structures. The contrast between light and dark in the image reflects the fluctuations in fluorescence intensity, helping to identify the characteristics of different areas within the sample.

[0095] 3. H&E (hematoxylin-eosin staining) is used to show the structure and cell morphology of tissues. It can clearly see the morphology of cells, the size and shape of cell nuclei, and the overall structure of tissues, providing important basis for pathological diagnosis.

[0096] 4. FLIM-based detection (output overlay in this embodiment of the present invention) fuses the τm (fluorescence lifetime) and intensity (fluorescence intensity) image information and overlays the resulting fluorescence intensity image. Different tissue types are labeled with different colors in the image, visually demonstrating the distribution and location of tumors, blood vessels, and normal tissue within the sample, as well as their combined characteristics in terms of fluorescence lifetime, fluorescence intensity, and tissue structure, resulting in superior recognition results.

[0097] like Figure 6As shown, five sets of different images are provided, each containing three rows, each displaying different sample conditions. From left to right, the images are: τm (fluorescence lifetime), Intensity (fluorescence intensity), H&E (hematoxylin and eosin staining), FLIM-based (output overlay of an embodiment of the present invention), and Intensity-based (output overlay based on fluorescence intensity images). Compared to detection based solely on fluorescence intensity images, the detection results achieved by the embodiment of the present invention, which integrates both fluorescence intensity and lifetime information, are clearer and more accurate.

[0098] This embodiment also provides a tumor detection system for implementing the aforementioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. While the systems described in the following embodiments are preferably implemented using software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0099] This embodiment provides a tumor detection system. Figure 7 Shown, including:

[0100] Fluorescence data acquisition module 701, used to acquire the autofluorescence attenuation data of the target biological tissue and extract fluorescence intensity data and fluorescence lifetime data therefrom;

[0101] A data fusion module 702 is used to fuse the fluorescence lifetime data and the fluorescence intensity data of the target biological tissue to obtain dual-channel fused data;

[0102] Tumor detection module 703 is used to input the dual-channel fusion data into a tumor detection model pre-trained based on a deep learning network model, output the predicted probability of each pixel corresponding to different categories, use the category with the highest probability value as the tumor detection result for each pixel, and obtain the distribution location of tumors and blood vessels in the target biological tissue.

[0103] In an optional embodiment, the above-mentioned detection system further includes: a visualization display module for converting the tumor detection result into a detection result image in a preset format; generating a fluorescence intensity image based on the fluorescence intensity data of the target biological tissue; superimposing the detection result image with the fluorescence intensity image, and displaying the final detection result by controlling the transparency of the detection result image.

[0104] In an optional embodiment, the process of training the tumor detection model includes:

[0105] Obtaining autofluorescence decay data of a preset tumor tissue slice sample, and extracting fluorescence intensity data, fluorescence lifetime data, and intensity-weighted lifetime images therefrom;

[0106] The intensity-weighted lifetime image is used as a mask to annotate the tumor, blood vessels, and background, generating a pixel-level labeled image.

[0107] The fluorescence intensity data and the fluorescence lifetime data are fused to obtain dual-channel fusion data;

[0108] The dual-channel fusion data and the label image are used to form a data pair, and the training set and the validation set are divided according to a preset ratio. The preset convolutional neural network is trained based on the training set, and the trained model is verified based on the validation set to obtain a model that meets the preset requirements as a tumor detection model.

[0109] In an optional embodiment, obtaining fluorescence lifetime data of a preset tumor tissue slice sample includes:

[0110] The fluorescence lifetime data is mapped to the fluorescence decay histogram of each pixel in the fluorescence lifetime image, and a double exponential model is used for least squares fitting to calculate the fluorescence lifetime distribution, and the average fluorescence lifetime data is obtained based on the fluorescence lifetime distribution.

[0111] In an optional embodiment, the intensity-weighted lifespan image is used as a mask to label the tumor, blood vessels, and background to generate a pixel-level label image, including:

[0112] Obtain H&E images corresponding to preset tumor tissue slice samples;

[0113] comparing the features of the H&E image and the intensity-weighted lifetime image, and marking the tumor, blood vessels, and background with different colors as annotation data to obtain an initial pixel-level labeled image;

[0114] The labeled data is screened and optimized based on expert experience and knowledge to obtain the final pixel-level labeled image.

[0115] In an optional embodiment, a preset convolutional neural network is trained based on a training set, and the trained model is verified based on a validation set to obtain a model that meets preset requirements as a tumor detection model, including:

[0116] Input the training set into the U-Net network and train it based on the preset training parameters;

[0117] After the training is completed, the dual-channel fusion data and the corresponding label images in the verification set will be input into the trained network model as verification data. The effect of the model training will be verified by calculating the preset performance indicators and comparing them with the preset verification targets. The model that achieves the preset verification targets will be used as the tumor detection model.

[0118] The tumor detection system in this embodiment is presented in the form of functional units, where the units refer to ASIC circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0119] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0120] The embodiment of the present invention also provides a computer device having the above Figure 7 The tumor detection system shown.

[0121] See also Figure 8 , Figure 8 Schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0122] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0123] The memory 20 stores instructions that can be executed by at least one processor 10, so as to enable at least one processor 10 to execute the method shown in the above embodiment.

[0124] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0125] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0126] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0127] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0128] A portion of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0129] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A tumor detection method, characterized in that: The method comprises: Obtain the time-dependent decay data of the autofluorescence of the target biological tissue and extract the fluorescence intensity data and fluorescence lifetime data from it; fusing the fluorescence lifetime data and the fluorescence intensity data of the target biological tissue to obtain dual-channel fused data; Inputting the dual-channel fusion data into a tumor detection model pre-trained based on a deep learning network model, outputting the predicted probability of each pixel corresponding to different categories, taking the category with the highest probability value as the tumor detection result for each pixel, and obtaining the distribution location of the tumor and blood vessels in the target biological tissue; The process of training the tumor detection model includes: Obtaining autofluorescence decay data of a preset tumor tissue slice sample, and extracting fluorescence intensity data, fluorescence lifetime data, and intensity-weighted lifetime images therefrom; Each pixel is pseudo-colored according to its corresponding fluorescence lifetime and intensity to generate an intensity-weighted lifetime image, which is then used as a mask to annotate the tumor, blood vessels, and background to generate a pixel-level label image. The fluorescence intensity data and the fluorescence lifetime data are fused to obtain dual-channel fusion data; The dual-channel fusion data and the labeled image constitute a data pair, and the training set and the validation set are divided according to a preset ratio. The preset convolutional neural network is trained based on the training set, and the trained model is validated based on the validation set to obtain a model that meets the preset requirements as a tumor detection model; Acquire fluorescence lifetime data of pre-set tumor tissue slice samples, including: The fluorescence lifetime data is mapped to the fluorescence decay histogram of each pixel in the fluorescence lifetime image, and a double exponential model is used for least squares fitting to calculate the fluorescence lifetime distribution, and the average fluorescence lifetime data is obtained based on the fluorescence lifetime distribution; The intensity-weighted lifetime image is used as a mask to label the tumor, blood vessels, and background to generate a pixel-level label image, including: Obtain H&E images corresponding to preset tumor tissue slice samples; comparing the features of the H&E image and the intensity-weighted lifetime image, and marking the tumor, blood vessels, and background with different colors as annotation data to obtain an initial pixel-level labeled image; The labeled data is screened and optimized based on expert experience and knowledge to obtain the final pixel-level labeled image.

2. The method according to claim 1, characterized in that Also includes: Converting the tumor detection result into a detection result image in a preset format; generating a fluorescence intensity image based on the fluorescence intensity data of the target biological tissue; The detection result image is superimposed on the fluorescence intensity image, and the final detection result is displayed by controlling the transparency of the detection result image.

3. The method according to claim 1, characterized in that The method of training a preset convolutional neural network based on the training set and validating the trained model based on the validation set to obtain a model that meets the preset requirements as a tumor detection model includes: Input the training set into the U-Net network and train it based on the preset training parameters; After the training is completed, the dual-channel fusion data and the corresponding label images in the verification set will be input into the trained network model as verification data. The effect of the model training will be verified by calculating the preset performance indicators and comparing them with the preset verification targets. The model that achieves the preset verification targets will be used as the tumor detection model.

4. A tumor detection system, characterized in that: Based on the method of claim 1, the system comprises: A fluorescence data acquisition module is used to obtain the spontaneous fluorescence decay data of the target biological tissue and extract fluorescence intensity data and fluorescence lifetime data therefrom; a data fusion module, configured to fuse the fluorescence lifetime data and the fluorescence intensity data of the target biological tissue to obtain dual-channel fused data; The tumor detection module is configured to input the dual-channel fusion data into a tumor detection model pre-trained based on a deep learning network model, output the predicted probability of each pixel corresponding to different categories, use the category with the highest probability value as the tumor detection result for each pixel, and obtain the distribution location of the tumor and blood vessels in the target biological tissue. The tumor detection model training process includes: Obtaining autofluorescence decay data of a preset tumor tissue slice sample, and extracting fluorescence intensity data, fluorescence lifetime data, and intensity-weighted lifetime images therefrom; Each pixel is pseudo-colored according to its corresponding fluorescence lifetime and intensity to generate an intensity-weighted lifetime image, which is then used as a mask to annotate the tumor, blood vessels, and background to generate a pixel-level label image. The fluorescence intensity data and the fluorescence lifetime data are fused to obtain dual-channel fusion data; The dual-channel fusion data and the label image are used to form a data pair, and the training set and the validation set are divided according to a preset ratio. The preset convolutional neural network is trained based on the training set, and the trained model is verified based on the validation set to obtain a model that meets the preset requirements as a tumor detection model.

5. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the tumor detection method according to any one of claims 1 to 3 by executing the computer instructions.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the tumor detection method according to any one of claims 1 to 3.

7. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the tumor detection method according to any one of claims 1 to 3.

Images