Deep learning-based pathomorphology and KRAS gene and SMAD4 protein correlation analysis method

Abstract

The invention belongs to the technical field of cancer survival risk prediction, and particularly relates to a deep learning-based pathomorphology and KRAS gene and SMAD4 protein correlation analysis method. According to the method, deep learning is carried out on the collected pathological scanning sections by utilizing the ResNet50 network, classification of 10 tissue types is realized, and classification results are counted. According to the KRAS gene mutation condition and the SMAD4 protein expression condition, the prognosis of the patient is predicted, and the risk stratification is performed on the patient. According to the method for quantitatively analyzing the pathological section by using the artificial intelligence technology, gene expression and protein expression can be associated, so that a pathologist is helped to realize high-precision and high-efficiency diagnosis on the pathological section, and associated information in more dimensions is provided. Through comprehensive analysis of the factors, doctors can predict disease progression more accurately and make more effective treatment strategies, and the survival rate and life quality of patients are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of cancer survival risk prediction, and specifically relates to a method for analyzing the correlation between pathological morphology and KRAS gene and SMAD4 protein. Background Art

[0002] Cancer is a major global public health challenge and a leading cause of death for many people around the world. It is estimated that there will be approximately 4,824,700 new cancer cases and 2,574,200 new cancer deaths in China in 2022. Among them, lung cancer, colorectal cancer, thyroid cancer, liver cancer and gastric cancer are the five most common types of cancer, accounting for 57.42% of new cases. The tumor microenvironment plays an important role in the development of tumors. It not only contains tumor cells that interact with surrounding cells through the circulation and lymphatic system, but also many non-malignant cell types, such as immune cells, stromal cells and endothelial cells. These cells can have a profound impact on the development and progression of tumors by secreting factors, changing stromal tissue or directly interacting with tumor cells. In addition, immune cells and stromal cells in the tumor microenvironment can also affect the tumor's response to treatment, thereby affecting the effectiveness of treatment and the patient's survival rate.

[0003] To date, morphological assessment by hematoxylin and eosin (H&E) staining remains the gold standard for cancer diagnosis. Since the assessment is performed by directly observing the morphology of tissue sections, it may be qualitative and subjective in nature, and in some cases may even lead to confusion or difficulty in diagnosis / prognosis. Therefore, there is an urgent need for a semi-quantitative, accurate, time-saving and labor-saving method to assist pathologists in the diagnosis and treatment of cancer and prognosis. In recent years, with the rapid development of machine learning algorithms, it has become possible for us to discover the temporal, spatial growth and differentiation characteristics of tumors based on characteristic analysis based on disease pathological morphological analysis.

[0004] The KRAS gene is an important oncogene that encodes the KRAS protein, which is a small molecule GTPase. In cells, the KRAS protein plays a key role in signal transduction, especially in regulating cell proliferation, differentiation and survival. The KRAS protein regulates signal transduction by binding to GTP (guanosine triphosphate) in cells. When KRAS binds to GTP, it is in an active state and can activate a series of downstream signaling pathways, including the RAF / MEK / ERK pathway (MAPK pathway) and the PI3K / AKT pathway. These signaling pathways are involved in regulating cell growth, proliferation and survival. In cancer, the KRAS gene often mutates, the most common being point mutations at positions 12, 13 or 61. These mutations cause the KRAS protein to remain in an active state, and it cannot be turned off even if GTP is hydrolyzed to GDP (guanosine diphosphate), thereby continuously activating downstream signaling pathways and promoting abnormal cell proliferation and tumor formation. KRAS mutations are driving factors for many types of cancer, including pancreatic cancer, colorectal cancer and lung cancer. In pancreatic cancer, almost all patients with pancreatic ductal adenocarcinoma (PDAC) have KRAS mutations, making KRAS an important target for pancreatic cancer. Although KRAS mutations play an important role in cancer, treatment targeting KRAS is difficult in clinical practice due to the structural characteristics of the KRAS protein. However, in recent years, some new drugs and therapies have been developed for KRAS mutations and have shown certain efficacy in clinical trials. The detection of KRAS gene mutations is of great significance for the diagnosis, prognosis assessment and formulation of treatment strategies for cancer. Modern technologies such as gene sequencing and PCR can detect KRAS mutations, thereby helping to determine individualized treatment plans.

[0005] SMAD4 (also known as DPC4) is an important signal transduction protein that plays a central role in the TGF-β (transforming growth factor-β) signaling pathway. SMAD4 plays an important role in regulating cell growth, differentiation, apoptosis and immune response, and is a key factor in inhibiting tumorigenesis. As a key regulator of the TGF-β signaling pathway, SMAD4 can inhibit cell proliferation and induce apoptosis. Its deletion or mutation can lead to inactivation of the TGF-β pathway and promote tumor development. Mutations or deletions in the SMAD4 gene are particularly common in a variety of cancers, especially pancreatic cancer and colorectal cancer. Mutations or deletions in SMAD4 are found in approximately 50% of patients with pancreatic ductal adenocarcinoma, and this mutation is often associated with cancer aggressiveness and poor prognosis. SMAD4 mutations lead to inactivation of the TGF-β signaling pathway, loss of normal cell growth control, and ultimately tumor formation. In colorectal cancer, mutations or deletions in SMAD4 are associated with cancer progression, increased metastasis, and poor prognosis.

[0006] Alterations in KRAS and SMAD4 are often associated with more aggressive tumor behavior, higher risk of metastasis, poor response to treatment, and overall poor prognosis in a variety of cancers. Specifically, KRAS mutation is not only a core driver of pancreatic cancer, but also plays an important role in the diagnosis, prognosis assessment, and treatment research of pancreatic cancer; while the loss of SMAD4 is a sign of poor prognosis, especially in pancreatic cancer and colorectal cancer; evaluating the status of these genes can help predict the prognosis of patients and guide the development of personalized treatment strategies. Summary of the invention

[0007] The purpose of the present invention is to propose a method for analyzing the correlation between pathological morphology and KRAS gene and SMAD4 protein based on deep learning to solve the problems existing in the above-mentioned prior art.

[0008] The method for analyzing the correlation between pathological morphology and KRAS gene and SMAD4 protein based on deep learning proposed in the present invention includes:

[0009] (1) Image data acquisition and processing: Specifically, the whole-slice digital pathology HE (Hematoxylin and Eosin) scans (WSI) of patients diagnosed with malignant tumors after surgery and the corresponding KRAS gene mutation and SMAD4 protein expression were collected, and the pathology scan slice images were divided into different classification data sets according to different tissue types, and the classification with unbalanced data sets was expanded;

[0010] (2) Training of deep learning network: The deep learning network uses ResNet50 network, which is trained with the above dataset to build a pathological section recognition model;

[0011] (3) Testing of the pathology slice recognition model and correlation analysis of pathology information; specifically, the constructed pathology slice recognition model is tested using a test set, and quantitative analysis information of the tested pathology scan slices is collected for correlation research and risk prediction to improve the accuracy and efficiency of disease diagnosis and treatment.

[0012] In the present invention, the malignant tumors include pancreatic ductal adenocarcinoma, gastric adenocarcinoma, intestinal adenocarcinoma, breast invasive ductal carcinoma, prostate acinar adenocarcinoma, cholangiocarcinoma, and hepatocellular carcinoma;

[0013] The present invention focuses on KRAS gene mutation and SMAD4 protein expression.

[0014] In step (1), the image data processing includes:

[0015] a. Exclusion of unqualified data; unqualified data include: whole slide images with uneven staining or uneven slice thickness, unable to match corresponding clinical data, pathological diagnosis of special subtypes such as adenosquamous carcinoma or IPMN malignant transformation, puncture or frozen specimens, TCGA unable to obtain WSI data, etc.;

[0016] b. Data labeling: The pathologist labels each tissue classification of the collected pathological section data, and two pathologists review it;

[0017] c. Image segmentation and classification: The pathological slice images were segmented into 100 μm × 100 μm non-overlapping image blocks, and included in the classification of different tissue types according to the provided annotation information, as the training data set for the ResNet50 network;

[0018] d. Dataset division: divide the dataset into training dataset, validation dataset and test dataset in a ratio of 7:2:1;

[0019] e. Data enhancement: enhancement methods include: rotation angle, color enhancement, brightness enhancement and contrast enhancement. Through data enhancement, the training data sets of different categories are expanded to achieve a balance in the amount of data for each category.

[0020] In step (2), the ResNet50 network is trained using the data set, and the model parameters are adjusted through training, and an excellent pathological section recognition model is selected according to the accuracy of the training set.

[0021] The constructed excellent pathological section recognition model was evaluated using the internal test set and TCGA data. The evaluation methods included: accuracy, loss function, ROC curve (Receiver Operating Characteristic Curve), AUC (Area Under the Curve) value and confusion matrix.

[0022] In step (3), the tested pathological slice recognition model is used to output quantitative information of the pathological slice, including: distinguishing the classification and resolution accuracy of 100μm×100μm image blocks; counting the number of image blocks of 10 categories in each pathological digital slice; assigning different colors to the image blocks according to different categories, and piecing together the image back to the original pathological digital slice size.

[0023] Batch processing of the above recognition models can automatically complete operations such as cutting a large number of pathological digital slices, tissue classification and recognition, specifying colors, and piecing together the images to the original size in a short period of time.

[0024] In the present invention, the obtained quantitative pathological digital slice microenvironment parameter information and the collected KRAS gene mutation and SMAD4 protein expression are subjected to Normality and Lognormality Tests, and then the normality result is judged according to the Normal QQ plot, and then the Mann-Whitney test and the Kruskal-Wallis test are selected to obtain the test results.

[0025] The present invention also includes a deep learning-based tumor and KRAS gene and SMAD4 protein correlation analysis system, which specifically includes an image data acquisition and processing module, a deep learning network training module, a pathological section recognition model test and a pathological information correlation analysis module; the three modules respectively perform the operations of the three steps of the above-mentioned correlation analysis method.

[0026] Compared with the prior art, the technical features and functional advantages of the present invention are mainly as follows:

[0027] (1) The present invention processes whole slide images with different survival distributions in batches, performs multi-classification according to tissue type, and performs quantitative statistics, which can effectively assist pathologists in manual image reading.

[0028] (2) While the present invention internally verifies the model, it also uses the TCGA dataset to obtain pathological images of multiple cancer patients for external verification, thereby ensuring better stability of the model and reliability of the method.

[0029] (3) Based on the high accuracy of the model, the present invention obtains correlation analysis results according to the comprehensive recognition and analysis results of the whole slide image by the constructed model and the collected KRAS gene mutation and SMAD4 protein expression.

[0030] (4) According to the obtained correlation analysis results, the present invention can enable pathologists and clinicians to predict the prognosis of patients and provide personalized treatment recommendations or medical decision support. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of a method for correlation analysis of pathological morphology and tumor microenvironment related parameters with KRAS gene mutation and SMAD4 protein expression based on deep learning in an embodiment of the present invention.

[0032] Figure 2 Schematic diagram of the ResNet50 model architecture used in the embodiment of the present invention.

[0033] Figure 3It is a curve chart of the accuracy during the training of the ResNet50 model in an embodiment of the present invention.

[0034] Figure 4 It is the ROC curve obtained according to the validation set in the embodiment of the present invention.

[0035] Figure 5 is the confusion matrix obtained according to the validation set in the embodiment of the present invention.

[0036] Figure 6 This is a schematic diagram of pan-cancer tumor multi-classification identification.

[0037] Figure 7 This is the positive result graph of the normal test NormalQQ plot and Mann-Whitney test statistics p < 0.05 between KRAS gene mutation and pathological morphology and tumor microenvironment related parameters.

[0038] Figure 8 This is the positive result graph of the normal QQ plot and Kruskal-Wallis test statistics p < 0.05 between SMAD4 protein expression and pathological morphology and tumor microenvironment related parameters. DETAILED DESCRIPTION

[0039] The present invention is further described below by way of embodiments in conjunction with the accompanying drawings.

[0040] The present embodiment is based on a method for correlation analysis of pathological morphology and tumor microenvironment related parameters with KRAS gene mutation and SMAD4 protein expression based on deep learning, such as Figure 1 As shown, including:

[0041] Data preparation

[0042] Obtain whole-slice digital pathology HE scans (WSIs) from patients diagnosed with malignant tumors after surgery at Huashan Hospital Affiliated to Fudan University and from the TCGA database, as well as the corresponding KRAS gene mutations and SMAD4 protein expression. (For this invention, only pancreatic ductal adenocarcinoma tissues were included as samples);

[0043] Whole slide images with uneven staining or uneven slice thickness, images that could not be matched to corresponding clinical data, pathological diagnosis of special subtypes such as adenosquamous carcinoma or IPMN malignant transformation, puncture or frozen specimens, and TCGA unable to obtain WSI data were excluded;

[0044] Data processing

[0045] The pathologists annotated the various tissue classifications of the collected pathological section data and reviewed them by two pathologists. The pathological section images were further divided into 100μm×100μm non-overlapping image blocks, and included in the classification of different tissue types according to the provided annotation information as a multi-classification dataset of the ResNet50 network. The dataset was divided into a training set, a validation set, and an internal test set in a ratio of 7:2:1, and the TCGA dataset was used as an external test set;

[0046] Among them, the multi-classification categories of pathological tissues include: tumor cells, blank areas, chronic pancreatitis, normal pancreatic acinar, islets, lymphocytes, fibers, necrosis, intestinal mucosa, and fat;

[0047] The data enhancement methods include: rotation angle, color enhancement, brightness enhancement and contrast enhancement. The enhancement method is used to expand the training data sets of different categories to achieve the balance of the data volume of each category. The present invention adopts methods such as image rotation, horizontal flipping and color jittering to selectively expand the data sets of different categories, thereby balancing the number of classified data (expanding the categories with less data volume to ensure the balance of the number of classified data). Specifically, the image rotation angle is set to 30°, 45°, 60°, 90°, 120°, 135°, 180°, 225°, 270° and 315°, and the color jittering includes three methods: brightness enhancement, color enhancement and contrast enhancement. The brightness enhancement is set to 1.2 and 1.5, the color enhancement is set to 1.3, 1.5 and 2, and the contrast enhancement is set to 1.3, 1.5 and 2.

[0048] Model Building

[0049] In order to achieve the best classification task, the present invention constructs a ResNet50 network model. Figure 2 As shown in the figure, the network model uses the Bottleneck Block structure as its basic residual module, which consists of five stages. The first stage uses a 7×7 convolution kernel, the number of convolution kernels is 64, and the step size is 2. Then it passes through the Batch Normalization (BN) layer and the Relu activation function, and then passes through the maximum pooling layer. The size of the pooling layer is 3×3 and the step size is 2. Through batch normalization, the activation of each neuron becomes a Gaussian distribution. After each convolution, the BN layer and the Relu layer are activated. The cross entropy loss function is used to calculate the loss value. The stochastic gradient descent (SGD) is selected as the optimization algorithm, and the parameter lr=1.18×10 -2 , batchsize=16; where lr and batchsize represent the learning rate and batch size respectively.

[0050] Model Evaluation

[0051] The present invention uses data that has never been recognized by the constructed model as the test set, of which 10% of the WSI of Huashan Hospital Affiliated to Fudan University is used as the internal test set, and the WSI in the TCGA database is used as the external test set. The classification performance of the model is comprehensively evaluated by ROC curve, AUC value and confusion matrix, and the ResNet50 model is verified as follows Figure 3-5 shown.

[0052] like Figure 3 As shown, the training model is at lr = 1.18 × 10 -2 , batchsize=16 parameters, and iterated 200 times, and the highest value of verification accuracy was reached in the 93rd iteration, at which time the training accuracy was 99.99% and the verification accuracy was 99.62%. After obtaining the best pathological section recognition model, the model was tested using the above internal test set and external test set. The test results are as follows: Figure 4-5 shown. Figure 4 The ROC curve and AUC value of the internal test and the external test are shown. The AUC value of all categories in the internal test is 1, and 0 in the external test represents necrosis, with an AUC value of 1; 1 represents chronic pancreatitis, with an AUC value of 0.94; 2 represents normal pancreatic acinar, with an AUC value of 0.98; 3 represents lymphocytes, with an AUC value of 1; 4 represents blank tissue, with an AUC value of 1; 5 represents fiber, with an AUC value of 1; 6 represents intestinal mucosa, with an AUC value of 1; 7 represents tumor tissue, with an AUC value of 0.98; 8 represents pancreatic islets, with an AUC value of 0.8; 9 represents fat, with an AUC value of 1; the average AUC value of all categories is 0.97. Figure 5 The confusion matrix of internal testing and external testing is shown. The confusion matrix of internal testing has a true positive rate of 98.57% for category 0, 97.02% for category 1, 99.53% for category 2, 99.58% for category 3, 99.9% for category 4, 99.68% for category 5, 99.76% for category 6, 98.93% for category 7, 81.82% for category 8, and 100% for category 9. The confusion matrix of the external test has a true positive rate of 95.07% for class 0, 47.62% for class 1, 86.62% for class 2, 96.57% for class 3, 100% for class 4, 90.74% for class 5, 98.78% for class 6, 86.97% for class 7, 5.26% for class 8, and 94.03% for class 9.

[0053] Model explanation

[0054] In order to better explain the model of the present invention, the pathological section recognition and classification model constructed in the present invention will output the classification results of each 100μm×100μm image block, and assign different colors according to the different tissue types of the non-overlapping image blocks. Among them, black represents blank tissue, red represents tumor tissue, yellow represents fiber, light blue represents chronic pancreatitis, orange represents normal pancreatic acinar, pink represents pancreatic islets, purple represents lymphocytes, dark blue represents necrosis, blue-purple represents intestinal mucosa, green represents fat, and outputs the classification accuracy corresponding to each image block.

[0055] Then, the WSI that needs to be counted is restored. That is, different categories of image blocks are represented by different colors, and different shades of color represent different judgment accuracy rates, and the image is restored to a large color image of the original pathological slice size. This makes it possible to segment areas of different tissue types using color. And the number of 100μm×100μm image blocks of each tissue type is weighted and counted, that is, when the judgment accuracy rate is less than 50%, it is not included in the statistics, and when the judgment accuracy rate is within the range of 50%-80%, it is counted as 1 / 2 image block. Figure 6 As shown in the figure, two WSIs that did not participate in data training are randomly selected as test sets. You can see that different colors are used to divide the WSI into regions, and the pathological data results are statistically calculated. In the left figure, the necrotic image block is 376, the chronic pancreatitis image block is 1067, the normal pancreatic acinar image block is 38, the lymphocyte image block is 1376, the blank area image block is 10388, the fiber image block is 12847, the intestinal mucosa image block is 255, the tumor tissue image block is 6434, the pancreatic islet image block is 6, and the fat image block is 4449. In the left figure, the necrotic image block is 530, the chronic pancreatitis image block is 2982, the normal pancreatic acinar image block is 592, the lymphocyte image block is 413, the blank area image block is 13733, the fiber image block is 14352, the intestinal mucosa image block is 139, the tumor tissue image block is 5245, the pancreatic islet image block is 7, and the fat image block is 1169.

[0056] The above quantitative identification information was further analyzed for correlation with KRAS gene mutation and SMAD4 protein expression. The obtained weighted quantitative pathological digital slice microenvironment parameter information was tested for differences with the collected KRAS gene mutation and SMAD4 protein expression. Through the collection of clinical pathological information of patients, KRAS gene mutation included mutant and wild type, and SMAD4 expression was divided into negative (expression intensity of 0), positive (SMAD4 protein strong expression 2+~3+, positive range>50%), and partial weak positive expression (expression intensity weaker than 2+, positive range less than 50%) by immunohistochemical interpretation. The number of all quantified histological components (tumor cells, chronic pancreatitis, normal pancreatic acini, islets, lymphocytes, fibers, necrosis, intestinal mucosa and fat) contained in a HE slice after deep learning interpretation was statistically analyzed with the expression of KRAS gene and SMAD4. First, a normality test was performed, using PRISM10 software to select Normality and Lognormality Tests, and then the normality results were judged based on the Normal QQ plot. If the normal distribution was met, the KRAS expression was tested by t test, and the SMAD4 expression was tested by one-way ANOVA. If the normal distribution was not met, the KRAS expression was tested by Mann-Whitney test, and the SMAD4 expression was tested by Kruskal-Wallis test, and finally P < 0.05 was obtained. The results are as follows Figure 7-8 shown. Figure 7 The Normal QQ plot analysis results in the left figure show that the KRAS distribution does not conform to the normality test, so the Mann-Whitney test is performed. Figure 7 The right panel shows that KRAS mutation is associated with greater tumor burden, P = 0.0205. Figure 8 The left figure shows that the expression of SMAD4 protein does not meet the normality test, so the Kruskal-Wallis test was performed. Figure 8 The statistical results in the right figure show that strong expression of SMAD4 is associated with less chronic pancreatitis, P = 0.011.

[0057] In summary, we can more accurately evaluate the prognosis of patients by combining the pathological morphology and microenvironment characteristic-related parameters with the KRAS gene mutation and SMAD4 protein expression.

Claims

1. A method for analyzing the correlation between pathological morphology and KRAS gene and SMAD4 protein based on deep learning, characterized in that: The specific steps are: (1) Image data collection and processing: specifically, collecting full-slice digital pathology HE scans of patients diagnosed with malignant tumors after surgery and the corresponding KRAS gene mutation and SMAD4 protein expression, dividing the pathology scan slice images into different classification data sets according to different tissue types, and expanding the classifications with unbalanced data sets; (2) Training of deep learning network: The deep learning network uses ResNet50 network, which is trained with the above dataset to build a pathological section recognition model; (3) Testing of the pathology slice recognition model and correlation analysis of pathology information; specifically, the constructed pathology slice recognition model is tested using a test set, and quantitative analysis information of the tested pathology scan slices is collected for correlation research and risk prediction to improve the accuracy and efficiency of disease diagnosis and treatment.

2. The method according to claim 1, characterized in that The malignant tumor is pancreatic ductal adenocarcinoma, gastric adenocarcinoma, intestinal adenocarcinoma, breast invasive ductal carcinoma, prostate acinar adenocarcinoma, cholangiocarcinoma or hepatocellular carcinoma.

3. The method according to claim 1, characterized in that The image data processing in step (1) comprises: a. Exclusion of unqualified data; unqualified data include: whole slide images with uneven staining or uneven slice thickness, unable to match corresponding clinical data, pathological diagnosis of special subtypes such as adenosquamous carcinoma or IPMN malignant transformation, puncture or frozen specimens, and TCGA unable to obtain WSI data; b. Data labeling: The pathologist labels each tissue classification of the collected pathological section data, and two pathologists review it; c. Image segmentation and classification: The pathological slice images were segmented into 100 μm × 100 μm non-overlapping image blocks, and included in the classification of different tissue types according to the provided annotation information, as the training data set for the ResNet50 network; d. Dataset division: divide the dataset into training dataset and validation dataset in a ratio of 8:2; e. Data enhancement: enhancement methods include: rotation angle, color enhancement, brightness enhancement and contrast enhancement. Through data enhancement, the training data sets of different categories are expanded to achieve a balance in the amount of data for each category.

4. The method according to claim 1, characterized in that: In step (2), the ResNet50 network is trained using the data set, and the model parameters are adjusted through training, and an excellent pathological section recognition model is selected according to the accuracy of the training set; The test set was used to evaluate the constructed excellent pathological section recognition model. The evaluation methods included: accuracy, loss function, ROC curve, AUC value and confusion matrix.

5. The method according to claim 1, characterized in that: In step (3), the tested pathological slice recognition model is used to output quantitative information of the pathological slice, including: the classification and resolution accuracy of the 100 μm×100 μm image block; the number of image blocks of 10 categories in each pathological digital slice; the image blocks are assigned different colors according to different categories, and the images are pieced together to restore the original pathological digital slice size; Batch processing of the above recognition models, including automated completion of a large number of pathology digital slice cutting, tissue classification and recognition, color assignment, and piecing back to the original image size in a short period of time; Normality and Lognormality Tests were performed on the obtained quantitative pathological digital slice microenvironment parameter information and the collected KRAS gene mutation and SMAD4 protein expression. The normality results were then judged based on the Normal QQ plot, and then the Mann-Whitney test and Kruskal-Wallis test were selected to obtain the result of P < 0.

05.

6. A pathological morphology and KRAS gene and SMAD4 protein correlation analysis system based on the method according to any one of claims 1 to 5, characterized in that: Specifically, it includes an image data acquisition and processing module, a deep learning network training module, a pathology section recognition model test and a pathology information correlation analysis module; the three modules respectively perform the three steps of the correlation analysis method.

Images