A method for predicting survival risk of gastric cancer patients based on histopathological images and gene expression data

Abstract

The present invention relates to a method for predicting the survival risk of gastric cancer patients based on histopathological images and gene expression data. The method performs cell nucleus segmentation on the histopathological images, selects the image blocks with the largest number of cell nuclei as the input of the neural network, and integrates the histopathological images with structured demographic and gene expression data to construct a multimodal fusion model. Compared with the prior art, the present invention has the advantages of improving prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of cancer risk prediction model construction, and in particular to a method for predicting the survival risk of gastric cancer patients based on histopathological images and gene expression data. Background Art

[0002] Histopathology images serve as the clinical gold standard for tumor diagnosis and prognosis, guiding clinicians to make more accurate treatment decisions. Pathologists grade tumor cells by evaluating their morphology and other characteristics under a microscope. However, manual evaluation of histopathology images is very time-consuming, subjective, and non-reproducible, especially for pathologists working in remote areas. Therefore, fully automatic models that predict the survival risk of tumor patients directly from histopathology images have attracted great attention. This computer-assisted tool can be used to improve the efficiency and accuracy of pathologists and ultimately provide better treatment for patients.

[0003] Accurately predicting a patient's survival time or progression will help clinicians make early treatment decisions, which is crucial to the patient's health. For example, low-risk patients can be advised to choose a relatively conservative treatment strategy. The methods for predicting patient survival risk are mainly divided into three categories: models based on image structured features, models based on deep convolutional networks, and multimodal fusion models.

[0004] Models based on image structured features usually extract structured image features, including cell shape, size, and texture, from unstructured histopathology images. However, the ability of structured image features to represent image information is limited. In recent years, researchers have attempted to use deep learning to learn survival-related information directly from histopathology images.

[0005] Models based on convolutional neural networks (CNNs) usually use CNNs for survival analysis, replacing the linear hazard function of the traditional Cox proportional hazard model with a nonlinear deep fully connected network. Depending on whether regions of interest (ROIs) need to be manually marked, CNN-based survival models can be divided into two categories: ROIs-based methods and whole slide image (WSI)-based methods. ROIs-based methods require pathologists to mark ROIs on WSIs, and are not end-to-end fully automatic analysis. WSI-based methods directly use WSIs to predict patient prognosis. As far as the investigation knows, there is currently no model that uses gastric cancer histopathology images to predict the survival risk of gastric cancer patients.

[0006] Multimodal fusion models integrate histopathological images and other omics data to improve the accuracy of prognosis. For example, the GSCNN method integrates genomic biomarkers and pathological images of glioblastoma patients into a unified prediction framework to predict the survival of tumor patients. However, GSCNN ignores the clinical information of patients. As far as the investigation knows, there is currently no gastric cancer survival prediction model that integrates histopathological images, clinical data, and gene expression data. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a method for predicting the survival risk of gastric cancer patients based on histopathological images and gene expression data.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] The method for predicting the survival risk of gastric cancer patients based on histopathological images and gene expression data includes the following steps:

[0010] Step 1: Data collection: Obtain histopathological images, clinical data, and gene expression data of gastric cancer patients.

[0011] Step 2: Establish a survival risk prediction model for gastric cancer patients based on histopathological images. The histopathological images were preprocessed (cut into small image blocks, color normalization, and cell nucleus segmentation), and the image blocks with the largest number of cell nuclei in the histopathological images were selected. A survival prediction model was constructed based on a convolutional neural network. The model output was the DeepCox-SC risk score, which represents the estimated patient mortality risk.

[0012] Step 3: Establish a multimodal fusion prediction model. The DeepCox-SC multimodal fusion model was constructed based on the DeepCox-SC risk score, demographics, and gene expression data.

[0013] The step 2 of establishing a survival risk prediction model for gastric cancer patients based on histopathological images specifically includes:

[0014] Step 2.1: Cropping the whole slide image (WSI) into image patches. The 20x magnified WSI was cropped into image patches of 1536×1536 pixels, and image patches with blank background exceeding 30% were filtered out.

[0015] Step 2.2: Color normalization: The existing Macenko method was used to normalize the image colors to the standard stained histopathology images.

[0016] Step 2.3: Cell nucleus segmentation: The cell nuclei of each image patch are segmented using a hierarchical multilevel thresholding method, and then the image patch with the largest number of cell nuclei is selected.

[0017] Step 2.4: The image patch with the largest number of cell nuclei is used as the input of the neural network, and the predicted value of the patient's risk of death (DeepCox-SC risk score) is finally output. The neural network extracts the spatial features of the histopathological image through the convolutional layer and the pooling layer, and then connects these features to the fully connected layer. The output of the convolutional neural network is a single node, which represents the estimated risk of death of the patient, that is, the DeepCox-SC risk score.

[0018] The step 3 of establishing a multimodal fusion prediction model specifically includes:

[0019] Step 3.1: Variable selection of high-dimensional gene data. The high-dimensional gene expression data were subjected to variable selection by the determined independent screening method and the Cox model with LASSO penalty term, and ten genes were obtained, including CHAF1A, REPIN1, SERPINE1, HTRA3, PWP2, GPR173, NCLN, NT5E, MYL4, and YWHABP2.

[0020] Step 3.2: Based on the DeepCox-SC risk score, demographics (age), and the 10 genes screened in step 3.1, a DeepCox-SC multimodal fusion model was constructed to predict the survival risk of gastric cancer patients.

[0021] The method for predicting the survival risk of gastric cancer patients based on histopathological images and gene expression data provided by the present invention has at least the following beneficial effects compared with the prior art:

[0022] 1) Compared with randomly selecting image blocks, the present invention selects image blocks with the largest number of cell nuclei after image preprocessing (cutting into image blocks, color standardization, and cell nucleus segmentation), proving that these image blocks contain more survival-related information, thereby improving the ability to predict the survival risk of gastric cancer patients.

[0023] 2) Traditional methods extract features of histopathological images (converting unstructured images into structured image features), and the ability of structured features to represent images is very limited. The present invention directly uses histopathological images as input of a deep convolutional neural network to construct a survival risk prediction model for gastric cancer patients.

[0024] 3) Compared with modeling based on a single model data, the present invention uses histopathological images, clinical information and gene expression data to build a multimodal fusion model to predict the survival risk of gastric cancer patients. Compared with the single-modality model, the multimodal fusion model improves the model prediction accuracy.

[0025] 4) The method of the present invention integrates histopathological images, clinical data and gene expression data, and uses a deep convolutional neural network to construct a multimodal fusion prediction model. Compared with the prediction model based only on single-modal data, the prediction accuracy has been significantly improved. The present invention selects the image patch with the largest number of cell nuclei to construct the prediction model, which improves the model prediction effect and robustness compared to the method of randomly selecting image patches. The model accuracy exceeds the accuracy of manual annotation by pathologists, and can be used as an auxiliary tool to improve the efficiency and accuracy of pathologists. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of the DeepCox-SC model architecture in the embodiment, wherein Part A is a schematic diagram of the DeepCoxSC preprocessing process, and Part B is a schematic diagram of the principle of the DeepCox-SC model taking WSI as input and outputting an image patch with the largest number of cell nuclei of the patient and a predicted risk score.

[0027] Figure 2 The forest plot in the embodiment shows the results of univariate and multivariate Cox regression analysis, wherein part A is a univariate Cox regression analysis, and the variables are coded in a categorical manner; part B is a univariate Cox regression analysis, and the variables are coded in a continuous manner; and part C is a multivariate Cox regression analysis.

[0028] Figure 3 : The C-index and time-dependent AUC values ​​of different models in the embodiment, wherein Part A compares the C-index of different models using ten-fold cross validation, the solid gray violin plot represents the model based only on the image: DeepCox-SC model, the random selection of small image blocks model, and the model for extracting WSI structured features; the dotted gray violin plot represents the model integrating histopathological images and demographic data (age); the solid white violin plot is a multimodal fusion model integrating histopathological images, demographic data (age) and gene expression data; the C-index of the DeepCox-SC multimodal fusion model is basically the same as that of the multimodal fusion benchmark model (including grade, stage, age and gene); Part B is the 1-year AUC value of different models.

[0029] Figure 4The figure shows the survival difference between the high-risk group and the low-risk group in the embodiment, wherein Part A is a Kaplan-Meier plot of the DeepCox-SC risk score for patients at grade II, Part B is a Kaplan-Meier plot for patients at grade III, Part C is a Kaplan-Meier plot for patients at stage II, and Part D is a Kaplan-Meier plot for patients at stage III. DETAILED DESCRIPTION

[0030] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should belong to the scope of protection of the present invention.

[0031] Example

[0032] The present invention relates to a method for predicting the survival risk of gastric cancer patients based on histopathological images and gene expression data. The method aims to provide a fully automated deep convolutional neural network model to directly predict the patient's survival risk from gastric cancer histopathological images (DeepCox-SC model). Using gastric cancer data in TCGA (The cancer genome atlas, cancer genome dataset), it was found that the prognostic accuracy of DeepCox-SC exceeded the accuracy of manual annotation by pathologists. By further integrating histopathological images, clinical data and high-dimensional gene expression data to construct a DeepCox-SC multimodal fusion model, the prediction accuracy is further improved. The implementation scheme of the method of the present invention specifically includes the following contents:

[0033] 1. Dataset preparation

[0034] The histopathological images, clinical data, and gene expression data of gastric cancer patients were collected. The histopathological images were formalin-fixed paraffin-embedded (FFPE) tissues, and FFPE sections are the gold standard for medical diagnosis.

[0035] 2. WSI preprocessing

[0036] Step 1: Crop the WSI into image patches. WSI is usually larger than 100000×100000 pixels. It is unrealistic to use WSI directly as the input of CNN. If the input image is too large, it will increase the number of parameters and computing power that need to be estimated. Taking these factors into consideration, the WSI of the present invention is cropped into image patches of 1536×1536 pixels. Since the edge of the WSI contains a white background, when the WSI is processed into image patches, some image patches contain a white background. Image patches with a blank background of more than 30% are filtered out.

[0037] Step 2: Color normalization. Many factors (such as pathology image staining and WSI digitization process) can lead to differences in the appearance of two histopathology images. This difference significantly affects the performance of the CNN model. Therefore, the image color is normalized to the standard stained histopathology image using the existing Macenko method. The standard stained histopathology image was downloaded from HistomicsTK.

[0038] Step 3: Nucleus segmentation. After filtering the image blocks, an average of 2937 image blocks (1536×1536 pixels) remain for each sample. All image blocks of each sample cannot be directly used in the CNN model. Cellular characteristics such as the number, size, and structure of nuclei play an important role in the diagnosis and grading of tumors. The nuclei of each image block are segmented by a hierarchical multilevel thresholding method, and then the image block with the largest number of nuclei is selected. The selected image block (1536×1536 pixels) is composed of 9 blocks of 512×512 pixels. The present invention uses the Cellprofiler public software to extract 1100 features of the 512×512 pixel block. The univariate Cox regression results show that among the 1100 features extracted by the Cellprofiler public software, the nuclear area feature (Median_PrimaryObject_AreaShape_Zernike_7_7) extracted based on the Zernike moment has the smallest P value and the largest absolute value of the coefficient. The present invention selects the image patch with the smallest feature (Median_PrimaryObject_AreaShape_Zernike_7_7) from 9 patches of 512×512 pixels, because the coefficient of the feature is negative. The selected image patch (512×512 pixels) containing more survival information is used to train the CNN model.

[0039] 3. Preprocessing of high-dimensional genetic data

[0040] First, the Sure Independent Screening (SIS) variable selection method was used to reduce 60,483 genes to a medium dimension (n / log(n), where n is the number of patients). Then, variable selection was performed by adding a LASSO penalty term to the Cox proportional hazard model. Finally, the genes with non-zero coefficients in the ten-fold cross validation were intersected. A total of 10 genes were obtained: CHAF1A, REPIN1, SERPINE1, HTRA3, PWP2, GPR173, NCLN, NT5E, MYL4, and YWHABP2.

[0041] 4. DeepCox-SC Model Architecture

[0042] Figure 1 is the DeepCox-SC model architecture. After WSI preprocessing (cropping WSI into small blocks, filtering and color normalization), each patient has approximately thousands of image blocks. Many previous studies randomly selected one or more small blocks as the input of the CNN model. The present invention speculates that the small block with the largest number of cell nuclei contains more information related to survival. Therefore, the present invention adopts a hierarchical multi-level threshold method to segment the cell nuclei and selects the small block with the largest number of cell nuclei. The input of the DeepCox-SC model is the image block with the largest number of cell nuclei, and finally outputs a prediction of the patient's risk of death (DeepCox-SC risk score). The DeepCox-SC model uses the InceptionResNetV2 convolutional layer pre-trained on ImageNet to extract the spatial features of histopathological images, and then connects these features to the fully connected layer. The last layer is a single node, representing the estimated risk function The Cox model likelihood function (Formula (1)) was optimized using patient survival time and events (death).

[0043]

[0044] In the formula, θ represents the convolutional neural network parameters; there are n samples in total, represents the set of all individuals whose survival time is greater than that of the i-th sample; δ i is an indicative function of whether the i-th sample is censored, 1 indicates that the i-th sample is dead, and 0 indicates that the i-th sample is censored.

[0045] The DeepCox-SC multimodal fusion model integrates the DeepCox-SC risk score obtained based on histopathological images, clinical data (age), and 10 gene expression data obtained after variable screening to improve the survival prediction of gastric cancer patients.

[0046] 5. DeepCox-SC model training

[0047] Ten-fold cross validation: The model was trained using ten-fold cross validation. Stratified random sampling was used to divide the data into ten folds, and the deletion rate and time distribution in each fold were consistent. 36 patients had two WSIs. If a patient was assigned to the training set, all WSIs of the patient were assigned to the training set. This method ensured that the histopathological image of a sample existed in both the training set and the validation set, effectively avoiding information leakage.

[0048] Data enhancement: Through data enhancement, we can prevent overfitting caused by insufficient training data and improve generalization performance. The present invention randomly rotates the training data by 40 degrees and randomly flips it horizontally and vertically. The pixel values ​​of the training set and validation set images are normalized to 0-1.

[0049] Hyperparameters: The Adam optimizer was used to minimize the loss function (Equation (1)), with an initial learning rate of 0.002 and a decay of 0.05. The model was trained with a mini-batch size of 32 and a training epoch of 100. To avoid overfitting, the training process was stopped early based on the performance of the validation set. When the loss function on the validation set did not decrease after 10 epochs, the training process was interrupted.

[0050] Hardware and software: The training was performed using the TensorFlow-v2.2.0 framework on the Pi 2.0 high-performance supercomputer at Shanghai Jiao Tong University, using NVIDIA Tesla V100 with 32GB memory per GPU. Basic histopathology image analysis was performed using HistomicsTK (https: / / digitalslidearchive.github.io / HistomicsTK / ).

[0051] 6. Model Evaluation

[0052] The prediction accuracy was evaluated by Harrell's C-index and time-dependent AUC values ​​on the validation set. The C-index was used to measure the consistency between the predicted risk of the validation set and the actual survival. The mean and standard deviation (SD) of the C-index and AUC values ​​on the 10 validation sets were calculated. The Wilcoxon signed-rank test was used to compare the performance differences between different models. Univariate and multivariate Cox regression analyses were performed, and the P value of the multivariate Cox model was calculated using the likelihood ratio test. Kaplan-Meier survival curves were plotted, and the log-rank test was used to compare the survival differences between the groups. All statistical analyses were performed in R version 3.6.1.

[0053] In order to verify the accuracy of the present invention, the example verification content of this embodiment is as follows:

[0054] 1) Dataset

[0055] The histopathological images, clinical data, and gene expression data of gastric cancer are derived from the cancer GenomeAtlas (https: / / portal.gdc.cancer.gov / projects / TCGA-STAD) dataset (TCGA). First, this example downloaded 442 histopathological sections (whole-slide images, WSIs) of 416 patients, of which 36 patients contained two WSIs. 41 patients without 20-fold magnified WSIs and 18 patients with missing survival data were further excluded. Finally, 382 WSIs of 357 patients were analyzed. Table 1 shows the clinical characteristics of 357 gastric cancer patients (234 males and 123 females). The age of 3 patients was missing, and the median age of 354 patients at diagnosis was 66.5 years old. The censoring rate was 0.6 (142 deaths). The median survival time of surviving patients was 357 days, and the median survival time of deceased patients was 616 days.

[0056] The clinical stage is based on the American Joint Committee on Cancer (AJCC) TNM staging system for gastric cancer. The clinical stage indicates the size of the primary tumor and the extent to which cancer cells have spread in the body. The clinical stage is closely related to the prognosis of cancer, and generally, a low stage has a better prognosis. The pathological grade depends on the appearance of cancer cells under a microscope and is classified according to the WHO standard.

[0057] Table 1 Clinical characteristics of patients with gastric cancer

[0058]

[0059] 2) Univariate and multivariate Cox regression

[0060] This example uses 10-fold cross validation to train and validate the prognostic model (DeepCox-SC model). The model output is the risk score predicted by the DeepCox-SC model, i.e., the DeepCox-SC risk score, which represents the patient's risk of death. The DeepCox-SC risk scores of the patients in the validation set are integrated. According to the median of the DeepCox-SC risk score, the patients are divided into high-risk group and low-risk group.

[0061] This example uses univariate and multivariate Cox regression models to evaluate the prognostic ability of the DeepCox-SC risk score. In the univariate analysis, the DeepCox-SC risk score and clinical data (age, sex, race, stage, and grade) were coded as continuous and categorical variables ( Figure 2Part A and Figure 2 Part B). Clinical stage and grade are important factors affecting prognosis. The higher the stage and grade, the higher the risk of death. This can also be seen from the hazard ratio (HR) of stage and grade. The higher the stage and grade, the higher the hazard ratio (HR). Figure 2 Part A in the figure). The hazard ratios for stage II, stage III, and stage IV were 1.936, 3.015, and 5.856, respectively (stage I was used as a reference). The hazard ratios for grade G2, grade G3, and grade GX were 1.254, 1.719, and 2.287, respectively (G1 was used as a reference). The grade was significantly correlated with the risk of survival only when the clinical grade was coded as a continuous variable. Whether the stage was a categorical variable or a continuous variable, it could significantly predict survival. However, the P value for the continuous variable was more significant (likelihood ratio test, 2e-07vs5e-06). In order to take this correlation into account, the present embodiment coded the stage and grade as continuous variables in the multivariate analysis ( Figure 2 (Part C).

[0062] In order to evaluate the independent prognostic ability of DeepCox-SC risk score, this embodiment performed a multivariate analysis. The multivariate analysis included DeepCox-SC risk score, age, gender, stage and grade. After adjusting for age, gender, stage and grade, DeepCox-SC risk score was still an important predictor of survival (HR: 1.528, 95% CI: 1.305-1.78, P value = 1.445E-7). Age (HR: 1.026, 95% CI: 1.009-1.044, P value = 2.18E-03), stage (HR: 1.784, 95% CI: 1.429-2.227, P value = 3.26E-07) and grade (HR: 1.388, 95% CI: 1.021-1.888, P value = 3.65E-02) were also significant in the multivariate Cox regression model.

[0063] 3) Evaluate the prediction accuracy of the DeepCox-SC model

[0064] This example uses ten-fold cross validation to evaluate the accuracy of the prediction. The accuracy of the prediction is measured by the C-index and the time-dependent AUC value. The C-index represents the consistency between the predicted risk and the actual survival. The C-index ranges from 0 to 1, and the higher the C-index score, the more accurate the prediction. For the image-based method alone, DeepCox-SC is compared with two other models: a model that randomly selects small patches of the image and a model based on structured features ( Figure 3 solid grey violin plot in ).

[0065] Model that randomly selects image patches: This model randomly selects image patches as input to the CNN model. The DeepCox-SC model selects the image patch with the largest number of cell nuclei, further indicating that this image patch contains more information related to survival.

[0066] Model based on structured features: This model extracts structured features of image blocks. This embodiment uses Cellprofiler public software to extract features from each image block. The staining separation module (UnmixColour) in the Cellprofiler public software is used to separate the pathological image into a hematoxylin-stained grayscale image and an eosin-stained grayscale image. For each image block, 1100 features are obtained using the "texture measurement", "intensity measurement", "size and shape measurement", "granularity measurement" and "object intensity distribution measurement" modules in the Cellprofiler public software. The mean and standard deviation of each feature of each image block are calculated. First, the 1100 features are reduced to a medium dimension (n / log(n), where n is the number of patients) using the Sure Independent Screening (SIS) variable selection method. Then, variable selection is performed by adding the LASSO penalty term to the Cox proportional hazard model. Finally, the ten features with non-zero compromise coefficients are taken as a union (35 features) for Cox regression analysis. Since the intersection of the features is zero, the union is calculated.

[0067] For gastric cancer patients, the C-index and 1-year AUC of the DeepCox-SC model for predicting survival were 0.660±0.057 and 0.701±0.146 (mean±SD), respectively ( Figure 3 Part A, Figure 3 Figure 2 (B). The DeepCox-SC model outperformed the model with randomly selected patches (C-index 0.609±0.078). This suggests that patches with the largest number of nuclei are better predictors of survival. Patches with the largest number of nuclei improved performance by 8.37%. Models based on structured features performed worse (C-index 0.601±0.065).

[0068] The results of univariate and multivariate Cox regression models showed that age was an important factor affecting survival. Figure 3 As shown in the dashed gray violin plot, this embodiment further integrates the DeepCox-SC risk score and age (image and age-based model). For comparison, this embodiment establishes a baseline model based on clinical grade, stage, and age. As mentioned above, considering the correlation between clinical grade, stage, and patient survival risk, grade and stage are coded as continuous variables.

[0069] The benchmark model included clinical grade, stage, and age, and the C-index of the benchmark model was 0.649±0.099 ( Figure 3 Part A, Figure 3 The dashed grey violin plot in part B of Figure 3). The results of integrating DeepCox-SC risk score and demographics (age) were slightly better than the baseline model (C-index was 0.667±0.050). In summary, the fully automated DeepCox-SC model can directly predict patient survival risk using pathological images and clinical data, and the accuracy of the prediction is comparable to that of pathologists.

[0070] 4) Integrate gene expression data to improve prognostic accuracy

[0071] The multimodal fusion model integrates histopathological images, clinical data (age), and gene expression data ( Figure 3 The preprocessing of high-dimensional gene expression data is similar to the preprocessing of histopathological image features, that is, SIS is used for preliminary screening, and then variable selection is performed based on the LASSO penalty term. Finally, the intersection of ten genes with non-zero compromise coefficients is obtained. A total of 10 genes are obtained: CHAF1A, REPIN1, SERPINE1, HTRA3, PWP2, GPR173, NCLN, NT5E, MYL4, and YWHABP2.

[0072] After integrating gene expression data, the C-index of DeepCox-SC increased from 0.660±0.057 to 0.744±0.070 (an average increase of 13%). The DeepCox-SC multimodal fusion model (including histopathological images, age, and gene expression) performed significantly better than the DeepCox-SC model (P value of the Wilcoxon signed rank test was 0.053). The C-index of the multimodal fusion benchmark model (clinical grade, stage, age, gene expression) was 0.754±0.065, slightly higher than the DeepCox-SC multimodal fusion model (0.744±0.070). The 1-year and 2-year AUCs of the DeepCox-SC multimodal fusion model (0.800±0.091, 0.833±0.055) were both greater than those of the multimodal fusion benchmark model (0.781±0.096, 0.820±0.084). The DeepCox-SC multimodal fusion model also outperformed previous studies using CT images to predict gastric cancer survival (C-index 0.724). In summary, the DeepCox-SC multimodal fusion model can accurately predict the survival of gastric cancer patients.

[0073] Six of the 10 genes used in the multimodal fusion model were associated with survival. The histone chaperone CHAF1A promotes cell invasion and inhibits apoptosis in many cancers. The replication initiator REPIN1 is closely associated with survival outcomes. SERPINE1 is a tumor promoter in gastric cancer. HTRA3 promotes tumor metastasis. PWP2 is a prognostic gene for gastric cancer. NT5E is associated with survival in patients with multiple cancers. The other three genes (NCLN, MYL4, and GPR173) are abnormally expressed in tumors. YWHABP2 is a tyrosine hydroxylase pseudogene, and tyrosine hydroxylase is essential for animal development and survival.

[0074] 5) Subgroup analysis

[0075] There were 113, 163, 125, and 216 patients in stage II, stage III, G2, and G3, respectively. Taking the 113 stage II patients as an example, although they belonged to the same clinical stage, their survival risks were different. It is meaningful to divide these patients into subgroups with significant survival differences. According to the median DeepCox-SC risk score, the 113 stage II patients were divided into high-risk and low-risk subgroups, and there was a statistically significant survival difference between the two groups ( Figure 4 The log-rank P values ​​of the survival differences between the two groups for G2, G3, stage II, and stage III were 5.66e-4, 1.97e-4, 1.29e-3, and 9.22e-3, respectively ( Figure 4 Part A, Figure 4 Part B, Figure 4 Part C, Figure 4 (section D).

[0076] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for predicting survival risk of gastric cancer patients based on histopathological images and gene expression data, characterized in that: The following steps are involved: 1) Obtain histopathological images, clinical data, and gene expression data of gastric cancer patients based on the TCGA database; 2) Preprocessing of histopathological images of gastric cancer patients; 3) In the preprocessed images, the image patches with the largest number of cell nuclei are selected as the input of the neural network to predict the survival risk of gastric cancer patients; 4) Perform variable selection on high-dimensional gene expression data of gastric cancer patients; 5) Integrate histopathological images, demographic information, and genetic data after variable selection to construct a multimodal fusion model to predict the survival risk of gastric cancer patients; In step 2), preprocessing the histopathological images of gastric cancer patients includes cutting, color standardization and cell nucleus segmentation of the histopathological images in sequence; In step 3), the spatial features of the histopathological image are extracted using convolution and pooling operations in a convolutional neural network, and then these features are connected to a fully connected layer. The loss function of the convolutional neural network model is the loss function of the semi-parametric Cox model, and the output of the convolutional neural network is a single node, representing the estimated risk function, i.e., the DeepCox-SC risk score; The high-dimensional gene expression data were subjected to variable selection using a determined independent screening method and a Cox model with a LASSO penalty term, and ten genes were obtained, including CHAF1A, REPIN1, SERPINE1, HTRA3, PWP2, GPR173, NCLN, NT5E, MYL4, and YWHABP2.

2. The method for predicting survival risk of gastric cancer patients based on histopathological images and gene expression data according to claim 1, characterized in that: The specific contents of cutting the tissue pathology images in sequence are as follows: The histopathology images with a pixel size of 100000×100000 were cropped into image patches with a size of 1536×1536 pixels, and image patches with a blank background of more than 30% were filtered out.

3. The method for predicting survival risk of gastric cancer patients based on histopathological images and gene expression data according to claim 1, characterized in that: The colors of all sample image patches were normalized to the standard stained histopathological images using the Macenko method.

4. The method for predicting survival risk of gastric cancer patients based on histopathological images and gene expression data according to claim 1, characterized in that: The specific contents of the cell nucleus segmentation process are as follows: The cell nuclei of each image block were segmented using the hierarchical multi-level threshold method, and then the image block with the largest number of cell nuclei was selected; the selected image block was 1536×1536 pixels, which was composed of 9 blocks of 512×512 pixels; 1100 features of the 512×512 pixel block were extracted using Cellprofiler software, and the image block with the smallest feature Median_PrimaryObject_AreaShape_Zernike_7_7 was selected among the 9 blocks of 512×512 pixels.

5. The method for predicting survival risk of gastric cancer patients based on histopathological images and gene expression data according to claim 1, characterized in that: In step 5), the DeepCox-SC risk score and age output from step 3) and the ten genes obtained from step 4) are integrated through a semi-parametric Cox model to obtain a multimodal fusion model for predicting the survival risk of gastric cancer patients.

Images