Colorectal cancer preoperative lymph node metastasis prediction method based on deep learning and pathological HE section

By combining deep learning and pathological HE slice technology, using multi-instance learning Transformer network and staining normalization technology, the problem of accuracy and subjective factors in preoperative lymph node metastasis prediction for colorectal cancer patients has been solved, and efficient and accurate prediction has been achieved. Lymph node metastasis prediction.

CN119339944APending Publication Date: 2025-01-21NANCHANG UNIV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411384825.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-30
Publication Date
2025-01-21

AI Technical Summary

Technical Problem

In the preoperative lymph node metastasis prediction of colorectal cancer patients, it is difficult to accurately diagnose metastases in the lymph nodes without significant enlargement, and are greatly affected by the doctor's personal experience and subjective factors. It is easy to ignore small metastases, resulting in misdiagnosis and missed diagnosis.

Method used

A prediction method based on deep learning and pathological HE slices is adopted. By obtaining WSI images of HE slices of colorectal cancer patients, the Transformer network of multi-instance learning is used for feature extraction, and staining normalization technology and color deconvolution technology are introduced, respectively. Image features of blue, red and brown channels are extracted to predict lymph node metastasis.

Benefits of technology

Significantly improves the accuracy and efficiency of lymph node metastasis prediction, reduces interference caused by human factors, can generate prediction results within seconds, improves the timeliness of clinical decision-making, and maintains consistency in the face of multiple clinical environments and data sources. Stable performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119339944A_ABST
    Figure CN119339944A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of pathological diagnosis, and provides a colorectal cancer preoperative lymph node metastasis prediction method based on deep learning and a pathological HE slice, and the method comprises the steps: firstly obtaining an HE slice WS I image of preoperative intestinal endoscopic biopsy of a colorectal cancer patient; performing feature extraction on the WSI image by using a Transform network based on multi-instance learning; a dyeing normalization technology is introduced, and color standardization processing is carried out on the WSI image; according to the colorectal cancer lymph node metastasis prediction method, deep learning and pathological HE slices are combined, the digital level of colorectal cancer lymph node metastasis prediction is improved, by processing and analyzing the WSI images of the HE slices, the model can accurately extract pathological features and generate prediction results within seconds, and the timeliness of clinical decisions is remarkably improved; meanwhile, the model adopts a dyeing normalization technology, and image features are extracted in different channels; the model can keep stable performance in various clinical environments and data sources, and accurate issuing of treatment decisions is promoted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of pathological diagnosis, and specifically is a method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections. Background Art

[0002] Colorectal cancer ranks third in incidence and second in mortality among all cancers, and the incidence population is tending to be younger. The main treatment for colorectal cancer is complete surgical removal of the lesion. If there are tiny metastases, adjuvant chemotherapy may be needed. Lymph node metastasis is an important metastasis mode of colorectal cancer, so the lymph node metastasis is closely related to the patient's choice of treatment plan. Currently, doctors mainly use enhanced CT, MRI and other imaging data combined with clinical experience to determine whether a patient has lymph node metastasis before surgery. This method can only accurately diagnose when the lymph nodes are significantly enlarged, and even experienced doctors may ignore tiny metastases. Due to the inability to provide timely and accurate treatment, the five-year survival rate of patients with lymph node metastasis is less than 20%.

[0003] Deep learning technology is a cutting-edge technology in the field of machine learning. It automatically extracts features from big data and learns through multi-level nonlinear transformations, thereby realizing intelligent processing of tasks. The core of deep learning technology is to automatically extract features through multi-layer neural networks. In this process, the input data undergoes nonlinear transformations in multiple hidden layers, enabling the model to abstract complex features layer by layer. The model constructed by deep learning technology can process large amounts of medical image data, automatically identify basic features such as edges, shapes, and textures in images, and extract key indicators. This automated feature extraction capability can greatly improve diagnostic efficiency and reduce the risk of missed diagnosis and misdiagnosis. The real-time feedback and auxiliary diagnosis functions of deep learning technology can provide decision support for doctors, improve medical efficiency, and promote the formulation of personalized medical plans.

[0004] However, currently, the main method used to predict preoperative lymph node metastasis in patients with colorectal cancer is for clinicians to evaluate patients based on imaging data; however, this method is difficult to accurately diagnose when the lymph nodes are not significantly enlarged, and is greatly influenced by the doctor's personal experience and subjective factors, and it is easy to ignore tiny metastatic lesions, resulting in misdiagnosis and missed diagnosis.

[0005] To this end, technicians in this field have proposed a method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections to solve the problems raised by the background technology. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections, so as to solve the problem that in the prior art, for predicting preoperative lymph node metastasis of colorectal cancer patients, the main method adopted is that clinicians evaluate patients based on imaging data; however, this method is difficult to accurately diagnose when the lymph nodes are not significantly enlarged, and is greatly affected by the doctor's personal experience and subjective factors, and is prone to ignore tiny metastatic lesions, resulting in misdiagnosis and missed diagnosis.

[0007] A method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections, comprising the following steps:

[0008] S1. Obtain WSI images of HE slices of intestinal endoscopic biopsies of patients with colorectal cancer before surgery;

[0009] S2, using a Transformer network based on multiple instance learning to extract features from the WSI image;

[0010] S3, introducing a color normalization technique to perform color normalization processing on the WSI image to balance the color differences between different images;

[0011] S4, converting the RGB color space of the WSI image into the HED color space, and extracting images of the blue (Hematoxylin), red (Eosin) and brown (DAB) channels respectively;

[0012] S5, extracting features from the images of each channel, and predicting lymph node metastasis based on the extracted features;

[0013] S6. Output the prediction result.

[0014] Preferably, step S1 is the starting point of the entire prediction method, which involves collecting high-magnification (Whole Slide Imaging, WSI) HE (Hematoxylin and Eosin) slice images from pathological tissue samples obtained through intestinal endoscopic biopsy before colorectal cancer surgery; HE slices are a commonly used staining method in pathological diagnosis, which can clearly display the morphology of tissue structure and cells, and are crucial for the identification and analysis of pathological features; after obtaining the WSI images, they will be used for subsequent deep learning model processing and analysis to predict whether the patient has lymph node metastasis.

[0015] Preferably, in step S2, the Transformer network of multi-instance learning models the long-distance feature associations in the image through a self-attention mechanism, and combines data enhancement and adaptive learning strategies to improve the robustness of the model, which is specifically described as follows: the model first inputs the small blocks divided by multi-instance learning into the Transformer; the self-attention mechanism allows the model to adaptively assign weights to different instances, so that it can focus on analyzing important pathological areas; the model also combines data enhancement and adaptive learning strategies, and by introducing different image transformations (including random inversion, random contrast, brightness enhancement, color jitter) during the training process, the model can maintain robust performance in different clinical scenarios; at the same time, the self-attention mechanism of the model can provide better interpretability, and doctors can understand how the model makes predictions by paying attention to the output results of the mechanism, thereby enhancing doctors' trust in the model results.

[0016] One of the core innovations of the present invention is the use of a multiple instance learning (MIL)-based method to construct a deep learning model, and combined with a Transformer network to predict lymph node metastasis; in traditional pathological image analysis, the image size is huge and the amount of information is huge. Generally speaking, the size of a pathological slice is , and directly processing the entire image often leads to a waste of computing resources and is not conducive to the model extracting key lesion areas from complex images; therefore, the present invention adopts a multiple instance learning framework to divide the pathological image into multiple small blocks (patches); each small block is regarded as an instance, and the entire image is regarded as a sample; the model does not need to label each small block separately, and only needs to identify which small blocks are important for the determination of the disease through global labeling.

[0017] Preferably, in step S3, the staining normalization technology adopts the Vahadane method. The core idea of ​​the Vahadane method is to decompose the HE staining image into a dye basis matrix (representing the spectral characteristics of the stain) and a sparse matrix (representing the concentration distribution of each stain in the image); this decomposition form allows the color information of the image to be separated from the structural information, and then the color is standardized while keeping the structural information unchanged. The Vahadane method standardizes the color of the HE staining image based on non-negative matrix decomposition.

[0018] Staining normalization: Due to differences in equipment, operating methods, and staining processes in different pathology laboratories and hospitals, the HE slice images generated may have large inconsistencies in color. This color difference will have a negative impact on the generalization ability of the model. The model may only perform well on data from a specific source, but perform poorly on data from other sources. In order to solve this problem, the present invention introduces staining normalization technology, which aims to perform color standardization on HE slices from different sources to ensure the consistency of input data.

[0019] Preferably, in step S4, the conversion from RGB to HED color space is achieved by color deconvolution technology, and the blue and red components in the RGB image are respectively mapped to the blue and red channels in the HED space to analyze the pathological characteristics of the cell nucleus and cytoplasm. The core of this technology is to find a suitable deconvolution matrix so that the RGB color components can be accurately mapped to the spectral absorption values ​​of Hematoxylin, Eosin and HED dyes;

[0020] Color deconvolution is an image processing technology based on light absorption characteristics, which is used to separate the mixed contributions of different dyes in the RGB color space. The basic principle of this method is based on the absorption characteristics of different dyes to the spectrum. Assuming that the light absorption characteristics of each dye are known, the contributions of different dyes can be separated by matrix transformation. For HE-stained sections, the spectral absorption characteristics of hematoxylin and eosin can be obtained by experimental measurement, and the absorption characteristics determine the matrix used in the deconvolution process.

[0021] Preferably, the construction of the deconvolution matrix: In image processing libraries such as `scikit-image`, a predefined deconvolution matrix is ​​usually used; the matrix is ​​based on a large amount of experimental data to determine the spectral absorption characteristics of Hematoxylin, Eosin and DAB; for example, a commonly used deconvolution matrix may be:

[0022]

[0023] Among them, each row corresponds to the absorption coefficient of a dye, and the columns represent the RGB channels;

[0024] HED space: In HE staining, the main function of Hematoxylin and Eosin dyes is to stain the cell nucleus and cytoplasm, while DAB dye is often used for immunohistochemical staining; in HED space, different channels represent the concentration of each dye, the Hematoxylin channel mainly reflects the blue staining of the cell nucleus, the Eosin channel reflects the red staining of the cytoplasm, and the DAB channel corresponds to the brown color of immunostaining.

[0025] By separating the image into hematoxylin, eosin and DAB channels through deconvolution, the pathological features in the image can be analyzed more clearly. Specifically, the conversion from RGB to DAB separates the acid-base staining areas in the HE slice image, and maps the blue and red components to red (corresponding to hematoxylin) and blue (corresponding to eosin) in the HED space respectively. This helps to improve the accuracy of the model in capturing pathological features such as cell nuclei and cytoplasm. Especially when identifying early lesions or tiny lymph node metastases, the HED space can provide a clearer and more intuitive image expression, allowing the model to capture subtle changes in metastasis at the microscopic level.

[0026] In addition, through this color space conversion, the noise and unnecessary color interference in the image can be effectively reduced, significantly improving the model's feature extraction ability and prediction accuracy; at the same time, when Grad-CAM is used to generate heat maps in the future, the performance characteristics of lymph node metastasis on pathological sections can be qualitatively judged by combining the heat maps of acid-base channels, thereby enhancing the interpretability of the model.

[0027] This method plans to use the vahadane method for color normalization, which is a color standardization method based on non-negative matrix factorization (NMF) that can standardize different colored images to a unified color space.

[0028] The transformation in absorbance space is one of the key steps of the whole method, which can accurately capture the spectral characteristics of the dye and enable the NMF-based decomposition to effectively extract the dye contribution.

[0029] Non-negative matrix factorization (NMF) is the core algorithm for color decomposition; the Vahadane method separates color distribution from structural distribution through NMF, and ensures that the structural information of the original image will not be changed during color standardization by replacing the basis matrix; the optimization problem of the NMF algorithm itself is usually solved by alternating least squares (ALS), which can obtain stable decomposition results while ensuring non-negative constraints.

[0030] The replacement of the base matrix ensures that the staining characteristics of all images are consistent with the standard slices. This step can effectively remove color deviation problems caused by staining conditions, equipment differences, etc.

[0031] Preferably, the step S5 is specifically as follows: this step is performed after converting the RGB color space of the WSI image into the HED color space and extracting the images of the blue (Hematoxylin), red (Eosin) and brown (DAB) channels respectively; for the image of each channel, a Transformer network based on multi-instance learning is used to extract features; the features include texture, shape, color distribution, etc. in the image, and then based on the features extracted from the images of each channel, the model will perform a comprehensive analysis to predict whether there is lymph node metastasis in patients with colorectal cancer, and finally the model will output the prediction results.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The present invention combines deep learning with pathological HE sections to improve the digital level of colorectal cancer lymph node metastasis prediction; by processing and analyzing the WSI images of HE sections, the model can accurately extract pathological features and generate prediction results within seconds, significantly improving the timeliness of clinical decision-making;

[0034] 2. The model of the present invention adopts dye normalization technology and extracts image features by channel. These two technologies effectively eliminate the color differences of slices from different sources, enhance the ability to identify key pathological features, and enable the model to maintain stable performance in the face of multiple clinical environments and data sources. This method not only provides a more reliable scientific basis for prediction, but also makes the results more consistent with the cognition of clinicians, and promotes the accurate issuance of treatment decisions.

[0035] 3. The present invention is based on the Transformer architecture to improve the overall performance and data adaptability of the model. The Transformer structure is selected to enable the model to have stronger capabilities in processing complex pathological images and adapt to different clinical scenarios and data. The flexibility and adaptability of this deep learning model ensure that it can provide timely and accurate pathological diagnosis support for colorectal cancer patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of the method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections of the present invention. DETAILED DESCRIPTION

[0037] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0038] like Figure 1 As shown:

[0039] Embodiment: The present invention provides a method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections, comprising the following steps:

[0040] S1. Obtain WSI images of HE slices of intestinal endoscopic biopsies of patients with colorectal cancer before surgery;

[0041] S2, using a Transformer network based on multiple instance learning to extract features from the WSI image;

[0042] S3, introducing a color normalization technique to perform color normalization processing on the WSI image to balance the color differences between different images;

[0043] S4, converting the RGB color space of the WSI image into the HED color space, and extracting images of the blue, red and brown channels respectively;

[0044] S5, extracting features from the images of each channel, and predicting lymph node metastasis based on the extracted features;

[0045] S6. Output the prediction result.

[0046] From the above, it can be seen that the present invention realizes the combination of deep learning and pathological HE sections, improves the digital level of colorectal cancer lymph node metastasis prediction, and by processing and analyzing the WSI images of HE sections, the model can accurately extract pathological features and generate prediction results within seconds, which significantly improves the timeliness of clinical decision-making; at the same time, the model of the present invention adopts staining normalization technology to extract image features by channel; these two technologies effectively eliminate the color differences of sections from different sources, enhance the ability to recognize key pathological features, and enable the model to maintain stable performance in the face of various clinical environments and data sources; this method not only provides a more credible scientific basis for prediction, but also makes the results more in line with the cognition of clinicians, and promotes the accurate issuance of treatment decisions.

[0047] Specifically, in step S2, the Transformer network of multi-instance learning models the long-distance feature associations in the image through the self-attention mechanism, and combines data enhancement and adaptive learning strategies to improve the robustness of the model, which is specifically described as follows: the model first inputs the small blocks divided by multi-instance learning into the Transformer; the self-attention mechanism allows the model to adaptively assign weights to different instances, so that it can focus on analyzing important pathological areas; the model also combines data enhancement and adaptive learning strategies, and by introducing different image transformations during the training process, the model can maintain robust performance in different clinical scenarios; at the same time, the self-attention mechanism of the model can provide better interpretability, and doctors can understand how the model makes predictions by paying attention to the output results of the mechanism, thereby enhancing doctors' trust in the model results.

[0048] From the above, it can be seen that the present invention is based on the Transformer architecture to improve the overall performance and data adaptability of the model; the selection of the Transformer structure enables the model to have stronger capabilities in processing complex pathological images and adapt to different clinical scenarios and data; the flexibility and adaptability of this deep learning model ensure that it can provide timely and accurate pathological diagnosis support for patients with colorectal cancer.

[0049] Specifically, in step S3, the dye normalization technology adopts the vahadane method, which performs color normalization on the HE dyed image based on non-negative matrix decomposition, and its specific operation steps are as follows:

[0050] S31. Absorbance space conversion of the image: First, the RGB image of the HE-stained slice is converted to the absorbance (Optical Density, OD) space; the purpose of this step is to convert the color information of the image from the RGB space to a linear space that is more suitable for decomposition operations. The calculation formula of the absorbance is:

[0051]

[0052] This step maps the color information in the image to a space that is more consistent with the physical properties of light absorption, so that the subsequent matrix decomposition can accurately represent the absorption characteristics of the dye;

[0053] S32. Non-negative matrix factorization (NMF): In the absorbance space, the Vahadane method decomposes the absorbance matrix of the image into two sub-matrices through non-negative matrix factorization:

[0054] OD=W×H

[0055] in:

[0056] W is the dye basis matrix, which contains the spectral information of Hematoxylin and Eosin dyes in the image, and each column represents the light absorption spectrum of a dye;

[0057] H is a sparse coefficient matrix, which represents the concentration information of each dye in each pixel;

[0058] The advantage of non-negative matrix factorization is that it can ensure that the elements of the decomposed matrix are non-negative, which is consistent with physical reality because the concentration and absorbance of the dye cannot be negative;

[0059] S33, base matrix extraction of standard slice: Select a high-quality HE slice as the standard slice, and extract its base matrix W from it ref ;This base matrix represents the staining characteristics of the standard slices and is used as the target for color normalization;

[0060] S34, base matrix replacement of target image: For the target image to be standardized, firstly obtain its own base matrix W by non-negative matrix decomposition. target and the sparse coefficient matrix H target ; Next, the basis matrix W of the target image target Replaced with the basis matrix W of the standard slice ref , maintain the sparse coefficient matrix H target This operation ensures that the structural information of the image remains unchanged, while adjusting its color information to be consistent with the standard slice;

[0061] S35, Image reconstruction: Use the replaced basis matrix W ref and the original sparse coefficient matrix H target , reconstruct the absorbance matrix of the target image:

[0062] OD norm =w ref ×H target

[0063] S36. Finally, the standardized absorbance matrix is ​​converted back to RGB space to obtain a standardized image:

[0064] RGB norm =255×exp(-OD norm ) 1 / 2.2 .

[0065] From the above, we can see that the Vahadane method separates the color distribution from the structural distribution through NMF, and ensures that the structural information of the original image will not be changed during the color standardization process by replacing the basis matrix; the optimization problem of the NMF algorithm itself is usually solved by the alternating least squares (ALS) method, which can obtain stable decomposition results under the condition of ensuring non-negative constraints.

[0066] Specifically, in step S4, the conversion from RGB to HED color space is achieved by color deconvolution technology, and the blue and red components in the RGB image are respectively mapped to the blue and red channels in the HED space to analyze the pathological characteristics of the cell nucleus and cytoplasm. The core is to find a suitable deconvolution matrix so that the RGB color components can be accurately mapped to the spectral absorption values ​​of Hematoxylin, Eosin and HED dyes. The specific steps are:

[0067] S41. First, the pixel values ​​of the RGB image are standardized and converted into the optical density (OD) space; the OD value reflects the intensity of light absorbed by the dye, and the calculation formula is:

[0068]

[0069] S42. Use a known deconvolution matrix to decompose the RGB values ​​in the OD space into the contribution values ​​of each dye; this matrix is ​​usually obtained empirically and reflects the absorption coefficient of each dye at different wavelengths (red, green, and blue); the conversion formula is:

[0070] [H,E,DAB] T =M×[OD R ,OD G ,OD B ] T ;

[0071] Where M is the deconvolution matrix, which maps the absorbance of RGB to the concentration values ​​of Hematoxylin, Eosin, and DAB.

[0072] As can be seen from the above, the conversion from RGB to DAB separates the acid-base staining areas in the HE slice image, and maps the blue and red components to red (corresponding to hematoxylin) and blue (corresponding to Eosin) in the HED space respectively; it helps to improve the accuracy of the model in capturing pathological features such as the nucleus and cytoplasm; especially when identifying early lesions or tiny lymph node metastases, the HED space can provide a clearer and more intuitive image expression, allowing the model to capture subtle changes in metastasis at the microscopic level. At the same time, through this color space conversion, the noise and unnecessary color interference in the image can be effectively reduced, significantly improving the feature extraction ability and prediction accuracy of the model; at the same time, when Grad-CAM is used for subsequent heat map generation, the heat map of the acid-base channel can be combined to qualitatively judge the performance characteristics of lymph node metastasis on the pathological section, thereby enhancing the interpretability of the model.

[0073] The existing method (i.e., doctors predict whether colorectal cancer patients have lymph node metastasis by combining imaging data with clinical experience) is compared with the colorectal cancer preoperative lymph node metastasis prediction method based on deep learning and pathological HE sections provided in this embodiment. The comparison results are shown in Table 1:

[0074] Table 1- Performance index comparison results

[0075]

[0076]

[0077] It can be seen from the above comparative analysis table that the embodiments of the present invention are significantly superior to the existing prediction technologies in the background technology in many aspects, especially in terms of accuracy, prediction accuracy, timeliness, subjectivity, model robustness, interpretability and clinical application value. The method improves the accuracy and efficiency of the prediction, and reduces the interference caused by human factors, providing a more reliable and scientific solution for the preoperative prediction of lymph node metastasis in patients with colorectal cancer.

[0078] In summary: The present invention intends to construct a method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections, which can achieve the following objectives:

[0079] 1. The model will input the HE slice WSI (Whole Slide Imaging) images obtained by the patient's preoperative intestinal endoscopic biopsy, and quickly output the prediction results of lymph node metastasis; the required data are all from the routine examinations that colorectal cancer patients must undergo during the treatment process to avoid additional treatment costs; the model makes predictions based on the patient's preoperative data, providing clinicians with valuable treatment decision support, thereby buying valuable treatment time for patients;

[0080] 2. This model uses dye normalization technology to balance the color differences between different WSI images, thereby improving the generalization ability of the model; by eliminating the dye differences caused by different production process operations, the model can be applied to data from various sources and maintain high accuracy;

[0081] 3. The model splits the image into three channels: blue (hematoxylin), red (Eosin), and brown (DAB) according to the colors of HE and DAB slices; each channel is then input into the model for feature extraction; the model built based on the actual staining results can better capture the subtle changes in pathological slices, simulate the pathologist's reading method, and thus more accurately identify the pathological characteristics of the lesion; finally, by visualizing the HE channel, the pathological characteristics of the lesion area are displayed, making the output results easier for clinicians to understand and accept;

[0082] 4. The model is based on the transformer architecture. Compared with traditional machine learning methods, it has the ability to efficiently extract features, perform multi-scale analysis, and process complex pathological images. It combines data enhancement and adaptive learning to improve robustness, and uses the attention mechanism to enhance model interpretability. It has real-time application potential and good scalability, thus providing accurate and reliable auxiliary judgment tools for clinicians.

[0083] The embodiments of the present invention are provided for the purpose of illustration and description. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present invention. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections, characterized by: The following steps are involved: S1. Obtain WSI images of HE slices of intestinal endoscopic biopsies of patients with colorectal cancer before surgery; S2, using a Transformer network based on multiple instance learning to extract features from the WSI image; S3, introducing a dye normalization technique to perform color standardization processing on the WSI image; S4, converting the RGB color space of the WSI image into the HED color space, and extracting images of the blue, red and brown channels respectively; S5, extracting features from the images of each channel, and predicting lymph node metastasis based on the extracted features; S6. Output the prediction results.

2. A method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections as claimed in claim 1, characterized in that: In step S2, the Transformer network of multi-instance learning models the long-distance feature associations in the image through the self-attention mechanism, and combines data enhancement and adaptive learning strategies to improve the robustness of the model, which is specifically described as follows: the model first inputs the small blocks divided by multi-instance learning into the Transformer; the self-attention mechanism allows the model to adaptively assign weights to different instances, so that it can focus on important pathological areas for analysis; the model also combines data enhancement and adaptive learning strategies, and by introducing image transformations during the training process, the model can maintain robust performance in clinical scenarios; at the same time, the self-attention mechanism of the model can provide better interpretability, and doctors can understand how the model makes predictions by paying attention to the output results of the mechanism.

3. The method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections as claimed in claim 1, characterized in that: In step S3, the dye normalization technology adopts the vahadane method, which standardizes the color of the HE dyed image based on non-negative matrix decomposition.

4. The method for predicting preoperative lymph node metastasis of colorectal cancer based on deep learning and pathological HE sections as claimed in claim 1, characterized in that: In step S4, the conversion from RGB to HED color space is achieved by color deconvolution technology, and the blue and red components in the RGB image are mapped to the blue and red channels in the HED space respectively to analyze the pathological characteristics of the cell nucleus and cytoplasm. The core is to find the deconvolution matrix so that the RGB color components can be accurately mapped to the spectral absorption values ​​of Hematoxylin, Eosin and HED dyes.

Citation Information

Cited By

  • Pathological analysis method and device for colorectal cancer lymph node metastasis

    CN121148649A