Esophageal squamous carcinoma consensus molecular subtype classification system based on histopathologic image

Abstract

The invention belongs to the technical field of medical image processing and computer vision, and provides an esophageal squamous carcinoma consensus molecular subtype classification system based on histopathologic images. The number of the molecular subtype classification systems is four: ECMS1, ECMS2, ECMS3 and ECMS4; the method is characterized in that: ECMS 1: a metabolic pathway is abnormal, NFE2L2 is activated, and a selected drug is an NFE2L2 inhibitor; eCMS 2: tumor classical signal pathway is up-regulated, and methylation degree is low; eCMS 3 has the advantages of few copy number variation events, low tumor mutation load and high PD-1 expression, and is beneficial to immunosuppressor treatment; and ECMS 4: activating an epithelial-mesenchymal transition pathway. And extracting spatial organization features from the automatic sketching result of the histological image by using a spatial algorithm, and constructing a machine learning model for carrying out subtype classification on the ESCC based on the features. And accurate molecular subtype classification of ESCC can be realized, and an important basis is provided for individualized treatment.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing and computer vision technology, and specifically relates to a consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images. Background Art

[0002] Esophageal squamous cell carcinoma (ESCC) is a common but highly heterogeneous malignancy, and its treatment response and prognosis vary among patients. Currently, a variety of ESCC classification systems have been proposed in the clinic to try to resolve its heterogeneity and guide the formulation of treatment strategies. Traditionally, the classification of ESCC mainly relies on histomorphological characteristics, such as cell morphology and tissue structure, which is often subjective and limited.

[0003] In recent years, with the development of computer vision and deep learning technologies, computer-aided diagnosis (CAD) systems based on histological images have gradually become a research hotspot. These systems can automatically extract features from tissue section images and use machine learning algorithms to classify and analyze tumors, thereby improving the accuracy and efficiency of diagnosis. However, the current ESCC subtype classification system based on histological images still faces challenges. First, existing systems often use only a small number of features for classification and cannot fully characterize the heterogeneity of tumors. Second, these systems lack unified standards and reliable verification methods, resulting in instability and low reliability of classification results. Therefore, a more comprehensive and reliable ESCC classification system based on histological images is needed to better guide clinical treatment and prognosis assessment. Summary of the invention

[0004] The present invention provides a consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images. The classification system is imECMS, which first uses deep learning to distinguish eight tissue types from H&E stained histological images, including background (BACK), connective tissue (CON), squamous epithelium (EPI), gland (GLA), lymphocyte (LYM), smooth muscle (MUS), cancer-associated stroma (STR) and tumor (TUM), and then extracts multiple spatial tissue features (SOFs) based on the eight tissue distributions of histopathological images, and uses these features to train a random forest classifier for ESCC subtype classification, and verifies its accuracy on a large-scale independent cohort.

[0005] The present invention uses deep learning to quantify spatial tissue features from automatically calibrated H&E stained slice images. Through multiple independent cohorts, the powerful potential of using imECMS for ESCC subtyping was demonstrated. The subtyping is based on histopathological images, which is a common data type in clinical practice and can be easily obtained from living tissue samples. In contrast, multi-omics data requires additional sample processing and experimental steps, and may be affected by technical variability, experimental conditions and other factors. The image typing method can be directly applied to conventional histopathology diagnostic processes, with better universality and reliability.

[0006] The present invention is achieved by the following technical solutions: a consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images, the molecular subtype classification system has 4 systems, namely: ECMS1, ECMS2, ECMS3, ECMS4; the characteristics are: ECMS1: abnormal metabolic pathway, NFE2L2 activation, and the selected drug is an NFE2L2 inhibitor; ECMS 2: upregulation of classical tumor signaling pathways and low methylation level; ECMS 3: few copy number variation events, low tumor mutation load, high expression of PD-1, and benefit from immunosuppressant treatment; ECMS 4: activation of the epithelial-mesenchymal transition pathway.

[0007] Furthermore, the classification system is obtained by: Spatial tissue feature extraction: Using deep learning, based on the results of automatic delineation of esophageal squamous cell carcinoma histological images, eight tissue types were distinguished from H&E stained histological images, including background BACK, connective tissue CON, squamous epithelium EPI, glandular GLA, lymphocyte LYM, smooth muscle MUS, cancer-associated stroma STR and tumor TUM. Then, according to the eight tissue distributions of histopathological images, multiple spatial tissue features SOFs were extracted, including multi-scale spatial tissue features including first-order features, second-order features and high-order features. The random forest algorithm was combined with the extracted spatial tissue features to form a random forest classifier for ESCC subtype classification, and its accuracy was verified on a large-scale independent cohort.

[0008] The specific method of obtaining the classification system is as follows: (1) Spatial tissue feature extraction: The histopathological images of the samples to be classified were collected and automatically delineated: Eight tissue types were distinguished from H&E stained histological images using deep learning. Two cohorts, SXM-I and SXM-II, were selected. Each sample in the SXM-I cohort included the automatic delineation results of tissue types of multiple annotated histopathological sections and the molecular subtyping results of multi-omics data ECMS1-4, which were used to build and train the molecular subtyping classification model; each sample in the SXM-II cohort included the automatic delineation results of tissue types of multiple annotated histopathological sections and the corresponding clinical survival information for model verification; a variety of spatial tissue features including first-order features, second-order features and high-order features were extracted, as follows: A. First-order characteristics representing the distribution of the eight tissue types: quantified by the proportion of the eight tissue types to the total tissue; B. Overall SOFs: calculated by calculating the relative proportion of each tissue type in the tumor interior, tumor proximal, tumor distal area and the entire slice image; firstly, the distance of each tissue distribution from the tumor area centroid was fitted based on the Gaussian mixture model, and the entire histopathological image was divided into three distributions: tumor interior, tumor proximal and tumor distal area; the SOFs of the four regions were then quantified by the ratio of the proportion of tissue blocks classified into the corresponding category to the total number of tissue blocks in the entire histopathological image; C. The intra-organizational dispersion index (WCDI) is a measure of regional dispersion within a certain range and is defined as follows: ; Where A represents a set of specific types of organizational blocks; d(i,j) Represents an organization block i and j The Euclidean distance between i In the neighborhood of D. Second-order features representing interactions between different types of tissues: tumor-stroma, tumor-lymphocyte interactions: For each tumor tissue block, pairwise tissue interactions between tumor and non-tumor tissue were calculated, and these interactions were defined as the relative proportions of non-tumor tissue blocks assigned to different tissue types in each ROI; The intergroup dispersion index (BCDI) was calculated to measure the overall distance between two different types of organizations and was defined as follows: ;in, A and B Two different types of organizations are represented; nA and nB are the number of tissue blocks for these two tissues, respectively; d(i, j) Indicates that based on the organization A No. i Organization blocks and organizations B No.j The Euclidean distance calculated from the coordinates of each tissue block; E. High-order features representing regional / global diversity: The spatial diversity index (SDI) is an ecological statistic used to quantify the degree of spatial heterogeneity of various cell / tissue types and is calculated as follows: , where m is the number of tissue types and pi is the proportion of image patches belonging to the i-th tissue type; Kullback-Leibler divergence (KLD) was used to quantify intranuclear heterogeneity due to differences from the overall single-cell phenotypic distribution of the patient; intranuclear heterogeneity within a ROI was approximated by calculating the KLD from the tissue type distribution of the ROI, i.e., the proportion of each tissue type, to the average tissue type distribution of all ROIs in the WSI; The Morisita-Horn similarity index (MH index) was used to quantify the ecological measure of the degree of co-occurrence of tumors and other types of tissues in WSIs; spatial correlation was calculated by using Pearson correlation and Morisita-Horn's similarity index, where the number of tumors and other types of tissues in each ROI, and Respectively represent the proportions of other types of tissues and tumors in polygon i: ; Texture features include gray-level co-occurrence matrix GLCM, gray-level run length matrix GLRLM, and gray-level size zone matrix GLSZM; GLCM is a statistical method used to analyze the grayscale spatial relationship of image texture. The calculation steps are as follows: Select a grayscale （L） , and determine the distance between two pixels （d） and direction (θ) ; For a given d and θ , count all pixel pairs that meet the conditions in the image (i, j) Grayscale difference (Δg) and gray level covariance （C） ; Construct the GLCM matrix, whose elements P(i, j; d, θ) Indicates the gray level at a given distance and direction i and j The frequency of occurrence of pixel pairs; P (i, j; d, θ) = (1 / N) * Σ[Σ[g(i) - g(j)]^2 = Δg^2]* [g(i) and g(j) are at distance d and angle θ] ;in, N is the total number of pixel pairs that meet the conditions; GLRLM is the length of consecutive gray levels in an image; its calculation steps are as follows: Select a gray level threshold k; Count the length (l) of all continuous pixel sequences with gray levels greater than or equal to k in the image; Construct GLRLM A matrix whose elements P (l; k) Indicates that the gray level is greater than or equal to k The frequency of a continuous pixel sequence with a length of l; P(l; k) = (1 / N) * Σ [number of runs of length l with gray level>= k] ;in, N is the total number of consecutive sequences that meet the condition; GLSZM analyzes the grayscale distribution of different sized regions in an image; its calculation steps are as follows: Select a grayscale threshold k ; Count the areas of all regions in the image whose gray level is greater than or equal to k A ; Construct a GLSZM matrix whose elements P(A; k) Indicates that the gray level is greater than or equal to k The area of ​​the region is the frequency of A; P(A; k) = (1 / N) * Σ[number of regions with area A and gray level>= k] ;in, N is the total number of regions that meet the conditions; Finally, the tumor area was scanned with windows of different sizes to obtain regions of interest (ROIs); ROIs were identified by local tissue regions, which contained at least 25% tumor (TUM) blocks and no more than 50% background (BACK) blocks; the overall SOFs, WCDI, pairwise tissue interactions, SDI, KLD, Morisita-Horn index, gray-level co-occurrence matrix GLCM, gray-level run length matrix GLRLM, and gray-level size zone matrix GLSZM of all identified ROIs were calculated and averaged as the overall estimate for the entire tissue section and patient; (2) Subtype classification based on spatial organization feature SOF: The image classifier was trained and internally validated using H&E sections of esophageal squamous cell carcinoma and multi-omics molecular typing to ensure its accuracy and reliability; The associations of clinicopathological features observed in the discovery cohort with multi-omic subtypes were validated using an external independent cohort that included only H&E sections and clinical information; SOFs were z-score standardized to exclude features with low inter-sample variability and eliminate scale differences between different features; The SOFs were clustered unsupervisedly using the similarity network fused with SNF, and the median of the sample profile was selected as the cutoff value to identify highly representative samples; 90% of the samples were randomly sampled using the Extremely Random Trees classifier, and the remaining 10% of the samples were predicted. The slices with more than 90% misclassification times were considered outliers and excluded. A 10-fold cross validation was performed on the remaining samples to evaluate the performance of the prediction model. To train a subtype classifier, imECMS, based on selected core H&E sections and to assess the applicability of this classifier to other independent cohorts; An overall imECMS prediction for each individual with multiple slices is determined. Specifically, each overall imECMS prediction for the multiple slices is determined as the category corresponding to the ECMS subtype that most frequently occurs in the multiple slices.

[0009] Those with ECMS4 in multiple sections were classified as ECMS4 subtype.

[0010] The random forest-based ESCC molecular subtype classification model was trained and validated, and an image-based classifier for predicting the four ECMS subtypes was established using an extreme random tree based on SOFs: the extreme random tree had the following parameter settings: number of trees ntree: 100; number of features considered at each node mtry: 1; number of random cuts for each randomly selected feature numrandomcuts: 1; minimum number of samples for a leaf node nodesize: 2; based on 10-fold cross-validation repeated 100 times, the performance of the prediction model reached >0.80 AUC for each subtype.

[0011] A computer-readable storage medium stores a computer program, which, when executed by a computer, causes the computer to implement the consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images as described above.

[0012] In the present invention: The spatial tissue feature extraction module includes advanced statistical methods for quantifying spatial relationships between tissue blocks, and feature engineering strategies for identifying interactions and spatial relationships between different tissue types. The random forest algorithm used in the subtype classification module combines 310 spatial tissue features that have been feature selected, and the accuracy and good generalization ability of the classification model are verified by cross-validation and independent data set testing. The sample screening process implemented in the random forest algorithm modeling process used in the subtype classification module is to distinguish the core samples in multi-site pathological tissue sections.

[0013] The present invention finally uses windows of different sizes to scan the tumor area to obtain regions of interest (ROIs). ROIs are identified by local tissue regions, which contain at least 25% tumor (TUM) blocks and no more than 50% background (BACK) blocks. The overall SOFs, WCDI, pairwise tissue interactions, SDI, KLD, Morisita-Horn index, gray-level co-occurrence matrix (GLCM), gray-level run length matrix (GLRLM) and gray-level size zone matrix (GLSZM) of all identified ROIs are calculated and averaged as the overall estimate for the entire tissue section and patient. Specifically, we used ROI-based quantification methods at different magnifications (10x, 20x, 40x). A total of 317 SOFs were identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of the steps for obtaining the consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images according to the present invention; Figure 2 A flowchart of obtaining the core sample set for final modeling through SNF clustering and repeated modeling; Figure 3 To validate the survival curves of the predicted imECMS subtype overall survival and disease-free survival in the SXM-I cohort; Figure 4 Sankey diagram to verify the correspondence between the predicted imECMS subtypes and the actual ECMS subtypes in the SXM-I cohort; Figure 5 Survival curves to verify the predicted overall survival of imECMS subtypes in the SXM-II cohort; DETAILED DESCRIPTION In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0015] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, and the disclosure and materials cited therein are hereby incorporated by reference.

[0016] Technical equivalents to the specific embodiments described that are apparent to those skilled in the art using no more than routine experimentation are intended to be encompassed by this application.

[0017] The experimental methods in the following examples are all conventional methods unless otherwise specified. The instruments and equipment used in the following examples are all conventional laboratory instruments and equipment unless otherwise specified; the experimental materials used in the following examples are all purchased from conventional stores unless otherwise specified.

[0018] Example 1: Figure 1 As shown, a flowchart of the steps for obtaining a consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images includes the following steps: Step 101, collecting and automatically outlining pathological samples for patients to be classified: In order to train the molecular subtype classification model of ESCC, two cohorts SXM-I and SXM-II are selected from the hospital, wherein each patient sample in the SXM-I cohort includes multiple annotated tissue type automatic outlining results of histopathological sections and molecular subtyping results ECMS1-4 of multi-omics data for the construction and training of the molecular subtype classification model, and each patient sample in the SXM-II cohort includes multiple annotated tissue type automatic outlining results of histopathological sections and corresponding clinical survival information for model verification.

[0019] Step 102, automatically extracting spatial features from the automatic delineation results: In order to construct the features of each image, we extract a variety of spatial organization features including first-order features, second-order features, and high-order features, as follows: First-order features representing the distribution of the eight tissue types: quantified by the proportion of the eight tissue types to the total tissue.

[0020] Overall SOFs: The relative proportions of each tissue type in the tumor interior, tumor proximal, tumor distal, and entire slice image were calculated. First, the distances of each tissue distribution from the tumor region centroid were fitted based on the Gaussian mixture model, and the entire histopathology image was divided into three distributions: tumor interior, tumor proximal, and tumor distal. The SOFs of the four regions were then quantified by the ratio of the proportion of tissue blocks classified into the corresponding categories to the total number of tissue blocks in the entire histopathology image.

[0021] The World Content Dispersion Index (WCDI) is a measure of regional dispersion within a certain range and is defined as follows: ; Where A represents a set of specific types of organizational blocks; d(i,j) Represents an organization block i and j The Euclidean distance between i within the neighborhood.

[0022] Second-order features representing interactions between different types of tissues (e.g., tumor-stroma, tumor-lymphocyte interactions): For each tumor tissue block, pairwise tissue interactions between tumor and non-tumor tissue were calculated. These interactions were defined as the relative proportion of non-tumor tissue blocks assigned to different tissue types in each ROI.

[0023] The BCDI was calculated to measure the overall distance between two different types of organizations and was defined as follows: ;in, A and B Two different types of organizations are represented; nA and nB are the number of tissue blocks for these two tissues, respectively; d(i, j) Indicates that based on the organization A No. i Organization blocks and organizations B No. j The Euclidean distance is calculated from the coordinates of the tissue blocks.

[0024] High-order features representing regional / global diversity: The spatial diversity index (SDI) is an ecological statistic used to quantify the degree of spatial heterogeneity of various cell / tissue types and is calculated as follows: ; where m is the number of tissue types and pi is the proportion of image patches belonging to the i-th tissue type.

[0025] The Kullback-Leibler divergence (KLD) was used to quantify intranuclear heterogeneity determined due to differences in the overall single-cell phenotype distribution from the patient. Intranuclear heterogeneity within a ROI was approximated by calculating the KLD from the tissue type distribution (proportion of each tissue type) of the ROI to the mean tissue type distribution of all ROIs in the WSI.

[0026] The Morisita-Horn similarity index (MH index) is an ecological measure used to quantify the degree of co-occurrence of tumors with other types of tissue in a WSI. Spatial correlation was calculated by using Pearson correlation and Morisita-Horn's similarity index, where the number of tumors and other types of tissue in each ROI was calculated. and Respectively represent the proportions of other types of tissues and tumors in polygon i: .

[0027] The texture features include gray-level co-occurrence matrix (GLCM), gray-level run length matrix (GLRLM) and gray-level size zone matrix (GLSZM).

[0028] GLCM is a statistical method used to analyze the spatial relationship of grayscale levels in image texture. The calculation steps are as follows: 1: Select a grayscale level （L）, and determine the distance between two pixels （d） and direction (θ) . 2: For a given d and θ , count all pixel pairs that meet the conditions in the image (i, j) Grayscale difference (Δg) and gray level covariance （C） . 3: Construct the GLCM matrix, whose elements P(i, j; d, θ) Indicates the gray level at a given distance and direction i and j The frequency of occurrence of pixel pairs. P(i, j; d, θ) = (1 / N) * Σ[Σ[g(i) - g(j)]^2 = Δg^2]* [g(i) and g(j) are at distance d and angle θ]; in, N is the total number of pixel pairs that meet the condition.

[0029] GLRLM focuses on the length of consecutive gray levels in an image. The calculation steps are as follows: 1: Select a gray level threshold （k） . 2: Count the length (l) of all continuous pixel sequences with gray levels greater than or equal to k in the image. 3: Construct GLRLM A matrix whose elements P(l; k) Indicates that the gray level is greater than or equal to k The frequency of a continuous pixel sequence of length l. P(l; k) = (1 / N) * Σ[number of runs of length l with gray level>= k]; in, N is the total number of consecutive sequences that meet the condition.

[0030] GLSZM is used to analyze the grayscale distribution of different size regions in an image. The calculation steps are as follows: Select a grayscale threshold （k） . Count the areas of all regions in the image whose grayscale level is greater than or equal to k （A） . Construct a GLSZM matrix whose elements P(A; k) Indicates that the gray level is greater than or equal to k The area of ​​the region is the frequency of A. P(A; k) = (1 / N) * Σ[number of regions with area A and gray level>= k]; in, N is the total number of regions that meet the condition.

[0031] Finally, we scanned the tumor area with windows of different sizes to obtain regions of interest (ROIs). ROIs were identified by local tissue regions that contained at least 25% tumor (TUM) blocks and no more than 50% background (BACK) blocks. The overall SOFs, WCDI, pairwise tissue interactions, SDI, KLD, Morisita-Horn index, gray-level co-occurrence matrix (GLCM), gray-level run length matrix (GLRLM), and gray-level size zone matrix (GLSZM) of all identified ROIs were calculated and averaged as the overall estimate for the entire tissue section and patient. Specifically, we used ROI-based quantification at different magnifications (10x, 20x, 40x), and a total of 317 SOFs were identified.

[0032] Step 103, through SNF clustering and repeated modeling methods, a core sample set is obtained for final modeling. Specifically, in order to overcome intratumor heterogeneity, we designed the following Figure 2 process.

[0033] Specifically, a total of 152 patients with multi-omics subtypes were included in the study. To eliminate the effect of order of magnitude, we performed z-score normalization and filtered out 7 features with low inter-sample variation (median absolute deviation, MAD = 0). Since the subtyping of the SXM-I cohort was obtained by unsupervised classification with similarity network fusion (SNF), the mean of the silhouette coefficient of all patients was used as a measure of how tightly all data were grouped in the cluster. To determine a highly representative sample of our unsupervised multi-omics subtypes, we selected the median silhouette coefficient of the SXM-I sample as the cutoff value. 161 H&E sections from 37 patients were excluded, leaving 465 H&E sections from 115 patients for subsequent analysis. Considering the heterogeneity of multiple histological sections within each individual tumor, the molecular subtype label was set for each section according to the overall multi-omics subtyping results of the corresponding tumor sample, and then a random sampling method was used to identify outliers. Specifically, we constructed an extreme randomized tree classifier on 80% of the randomly selected sections and repeated the prediction 100 times on the other 20% of the sections. Sections that were misclassified more than 90% of the time were excluded as outliers. After excluding 125 sections, we performed 100 10-fold cross validations using the retained sample (106 patients, 340 H&E sections) to evaluate the performance of the prediction model. We then trained an extreme randomized tree-based subtype classifier (imECMS) using all 340 H&E sections and evaluated the generalizability of this classifier to an additional independent cohort (SXM-II). For each patient with multiple sections, their overall imECMS prediction result was determined as the ECMS subtype that appeared most frequently in multiple sections. Specifically, any patient with ECMS4 in multiple sections would be classified as ECMS4 subtype, as we observed a significant prognostic impact of ECMS4.

[0034] Step 104 , training and validating the random forest-based ESCC molecular subtype classification model, specifically, predicting the performance of the model: As described above, we used extreme random trees based on SOFs to build an image-based classifier for predicting four ECMS subtypes.

[0035] Our extreme randomized trees have the following parameter settings: ntree (number of trees): 100; mtry (number of features considered at each node): 1; numrandomcuts (number of random cuts for each randomly selected feature): 1; nodesize (minimum number of samples for a leaf node): 2.

[0036] These parameters are used to build an extreme random tree model, where each tree is trained using a randomly selected subset of features, and the split features and split points are randomly selected at each node. The minimum number of samples for a leaf node is 2, meaning that if a node has less than 2 samples, further splitting will stop.

[0037] We repeated 100 times based on 10-fold cross validation and the performance of the prediction model reached >0.80 AUC for each subtype ( Figure 3 ).

[0038] Step 105, input the automatic delineation result into the target system to obtain the automatic molecular subtype classification result.

[0039] We finally evaluated the model performance on all samples in the SXM-I cohort (152 patients, 626 pathology slides) and SXM-II cohort (358 patients, 1262 pathology slides), and we removed individuals with an even number of pathology slides and no significant dominant subtype in the classification results. Stratification of patients using SOFs-based classification labels showed significant correlation with OS and DFS in all SXM-I samples (all P < 0.05, Figure 3 The imECMS classification results are highly consistent with ECMS ( Figure 4 ), and was significantly associated with OS in the independent SXM-II cohort (P = 0.006, Figure 5 ).

[0040] Embodiment 2: A storage medium comprises a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute the screening method described in any of the aforementioned embodiments.

[0041] In this embodiment, the storage medium is a computer-readable storage medium. If the esophageal squamous cell carcinoma pathological image classification system of the gastroscopy quality assessment device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can be executed by the processor. The steps of the above-mentioned method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images, characterized by: There are four molecular subtype classification systems: ECMS1, ECMS2, ECMS3, and ECMS4; feature The results are as follows: ECMS 1: abnormal metabolic pathways, NFE2L2 activation, and the drug selected is NFE2L2 inhibitor; ECMS 2: upregulation of tumor classical signaling pathways, low methylation level; ECMS 3: Few copy number variation events, low tumor mutation burden, high PD-1 expression, and benefit from immunosuppressive therapy; ECMS4: epithelial-mesenchymal transition pathway activation.

2. The consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images according to claim 1, characterized in that: The classification system is obtained as follows: Spatial tissue feature extraction: Using deep learning, based on the results of automatic delineation of esophageal squamous cell carcinoma histological images, eight tissue types were distinguished from H&E stained histological images, including background BACK, connective tissue CON, squamous epithelium EPI, glandular GLA, lymphocyte LYM, smooth muscle MUS, cancer-associated stroma STR and tumor TUM. Then, according to the eight tissue distributions of histopathological images, multiple spatial tissue features SOFs were extracted, including multi-scale spatial tissue features including first-order features, second-order features and high-order features. The random forest algorithm was combined with the extracted spatial tissue features to form a random forest classifier for ESCC subtype classification, and its accuracy was verified on a large-scale independent cohort.

3. The consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images according to claim 2, characterized in that: The specific method of obtaining the classification system is as follows: (1) Spatial tissue feature extraction: The histopathological images of the samples to be classified were collected and automatically delineated: Eight tissue types were distinguished from H&E stained histological images using deep learning. Two cohorts, SXM-I and SXM-II, were selected. Each sample in the SXM-I cohort included the automatic delineation results of tissue types of multiple annotated histopathological sections and the molecular subtyping results of multi-omics data ECMS1-4, which were used to build and train the molecular subtyping classification model; each sample in the SXM-II cohort included the automatic delineation results of tissue types of multiple annotated histopathological sections and the corresponding clinical survival information for model verification; a variety of spatial tissue features including first-order features, second-order features and high-order features were extracted, as follows: A. First-order characteristics representing the distribution of the eight tissue types: quantified by the proportion of the eight tissue types to the total tissue; B. Overall SOFs: calculated by calculating the relative proportion of each tissue type in the tumor interior, tumor proximal, tumor distal area and the entire slice image; firstly, the distance of each tissue distribution from the tumor area centroid was fitted based on the Gaussian mixture model, and the entire histopathological image was divided into three distributions: tumor interior, tumor proximal and tumor distal area; the SOFs of the four regions were then quantified by the ratio of the proportion of tissue blocks classified into the corresponding category to the total number of tissue blocks in the entire histopathological image; C. The intra-organizational dispersion index (WCDI) is a measure of regional dispersion within a certain range and is defined as follows: ; Where A represents a set of specific types of organizational blocks; d(i,j) Represents an organization block i and j The Euclidean distance between i In the neighborhood of D. Second-order features representing interactions between different types of tissues: tumor-stroma, tumor-lymphocyte interactions: For each tumor tissue block, pairwise tissue interactions between tumor and non-tumor tissue were calculated, and these interactions were defined as the relative proportions of non-tumor tissue blocks assigned to different tissue types in each ROI; The intergroup dispersion index (BCDI) was calculated to measure the overall distance between two different types of organizations and was defined as follows: ;in, A and B Two different types of organizations are represented; nA and nB are the number of tissue blocks for these two tissues, respectively; d(i, j) Indicates that based on the organization A No. i Organization blocks and organizations B No. j The Euclidean distance calculated from the coordinates of each tissue block; E. High-order features representing regional / global diversity: The spatial diversity index (SDI) is an ecological statistic used to quantify the degree of spatial heterogeneity of various cell / tissue types and is calculated as follows: , where m is the number of tissue types and pi is the proportion of image patches belonging to the i-th tissue type; Kullback-Leibler divergence (KLD) was used to quantify intranuclear heterogeneity due to differences from the overall single-cell phenotypic distribution of the patient; intranuclear heterogeneity within a ROI was approximated by calculating the KLD from the tissue type distribution of the ROI, i.e., the proportion of each tissue type, to the average tissue type distribution of all ROIs in the WSI; The Morisita-Horn similarity index (MH index) was used to quantify the ecological measure of the degree of co-occurrence of tumors and other types of tissues in WSIs; spatial correlation was calculated by using Pearson correlation and Morisita-Horn's similarity index, where the number of tumors and other types of tissues in each ROI, and Respectively represent the proportions of other types of tissues and tumors in polygon i: ; Texture features include gray-level co-occurrence matrix GLCM, gray-level run length matrix GLRLM, and gray-level size zone matrix GLSZM; GLCM is a statistical method used to analyze the grayscale spatial relationship of image texture. The calculation steps are as follows: Select a grayscale （L） , and determine the distance between two pixels （d） and direction (θ) ; For a given d and θ , count all pixel pairs that meet the conditions in the image (i, j) Grayscale difference (Δg) and gray level covariance （C） ; Construct the GLCM matrix, whose elements P(i, j; d, θ) Indicates the gray level at a given distance and direction i and j The frequency of occurrence of pixel pairs; P(i, j; d, θ) = (1 / N) * Σ[Σ[g(i) - g(j)]^2 = Δg^2] * [g(i) and g(j) are at distance d and angle θ] ;in, N is the total number of pixel pairs that meet the conditions; GLRLM is the length of consecutive gray levels in an image; its calculation steps are as follows: Select a gray level threshold k ; Count the length (l) of all continuous pixel sequences with gray levels greater than or equal to k in the image; Construct GLRLM A matrix whose elements P(l; k) Indicates that the gray level is greater than or equal to k The frequency of a continuous pixel sequence with a length of l; P(l; k) = (1 / N) * Σ [number of runs of length l with gray level >= k] ;in, N is the total number of consecutive sequences that meet the condition; GLSZM analyzes the grayscale distribution of different sized regions in an image; its calculation steps are as follows: Select a grayscale threshold k ; Count the areas of all regions in the image whose gray level is greater than or equal to k A ; Construct a GLSZM matrix whose elements P(A; k) Indicates that the gray level is greater than or equal to k The area of ​​the region is the frequency of A; P(A; k) = (1 / N) * Σ[number of regions with area A and gray level >= k] ;in, N is the total number of regions that meet the conditions; Finally, the tumor area was scanned with windows of different sizes to obtain regions of interest (ROIs); ROIs were identified by local tissue regions, which contained at least 25% tumor (TUM) blocks and no more than 50% background (BACK) blocks; the overall SOFs, WCDI, pairwise tissue interactions, SDI, KLD, Morisita-Horn index, gray-level co-occurrence matrix GLCM, gray-level run length matrix GLRLM, and gray-level size zone matrix GLSZM of all identified ROIs were calculated and averaged as the overall estimate for the entire tissue section and patient; (2) Subtype classification based on spatial organization feature SOF: The image classifier was trained and internally validated using H&E sections of esophageal squamous cell carcinoma and multi-omics molecular typing to ensure its accuracy and reliability; The associations of clinicopathological features observed in the discovery cohort with multi-omic subtypes were validated using an external independent cohort that included only H&E sections and clinical information; SOFs were z-score standardized to exclude features with low inter-sample variability and eliminate scale differences between different features; The SOFs were clustered unsupervisedly using the similarity network fused with SNF, and the median of the sample profile was selected as the cutoff value to identify highly representative samples; 90% of the samples were randomly sampled using the Extremely Random Trees classifier, and the remaining 10% of the samples were predicted. The slices with more than 90% misclassification times were considered outliers and excluded. A 10-fold cross validation was performed on the remaining samples to evaluate the performance of the prediction model. To train a subtype classifier, imECMS, based on selected core H&E sections and to assess the applicability of this classifier to other independent cohorts; Determine the overall imECMS prediction for each individual with multiple slices. Specifically, each overall imECMS prediction with multiple slices is determined as the category corresponding to the most frequently occurring ECMS subtype in the multiple slices. The overall imECMS prediction for which the most frequently occurring ECMS subtype cannot be determined is determined to be uncertain.

4. The consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images according to claim 3 is characterized by: Those with ECMS4 in multiple sections were classified as ECMS4 subtype.

5. The consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images according to claim 2 is characterized by: The random forest-based ESCC molecular subtype classification model was trained and validated, and an image-based classifier for predicting the four ECMS subtypes was established using an extreme random tree based on SOFs: the extreme random tree had the following parameter settings: number of trees ntree: 100; number of features considered at each node mtry: 1; number of random cuts for each randomly selected feature numrandomcuts: 1; minimum number of samples for a leaf node nodesize: 2; based on 10-fold cross-validation repeated 100 times, the performance of the prediction model reached >0.80 AUC for each subtype.

6. A computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to implement the consensus molecular subtype classification system for esophageal squamous cell carcinoma based on histopathological images as claimed in claim 1.

Images