A method to improve the accuracy of gastric cancer classification and prognosis prediction based on gradient boosting deep feature selection algorithm
Through unsupervised clustering and gradient enhancement deep feature selection algorithm combined with clinical data, the problem of low classification accuracy in gastric cancer classification prediction was solved, the optimal subtype number was determined, the accuracy of prognosis prediction was improved, and it was verified by deep neural network. The effectiveness of the method.
Patent Information
- Application Number
- CN202310404611.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-17
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2043-04-17
AI Technical Summary
The existing feature selection methods have low classification accuracy in gastric cancer typing prediction, and there is a lack of research on subtype classification. It is difficult for traditional clustering methods to determine the optimal number of clusters, which affects the accuracy of prognosis prediction.
Unsupervised clustering method is used to combine PAC and clinical data to determine the optimal subtype number, and feature selection is used to select the improved gradient-enhanced deep feature selection algorithm, and classification is carried out through deep neural networks to improve the accuracy of prognosis prediction.
It effectively improves the accuracy of gastric cancer typing prognosis prediction, achieves a prediction accuracy of 99.115%, and enhances the practicality of the method and the identification of biomarkers through bioinformatics analysis.
Smart Images

Figure CN116417070B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for improving the accuracy of gastric cancer typing prognosis prediction based on a gradient boosting deep feature selection algorithm, and belongs to the technical field of biomedicine. Background Art
[0002] Gastric cancer is one of the most common cancers, with extremely high morbidity and mortality rates. Currently, in the medical community, different treatment options are selected for cancer treatment based on the type of cancer and the progression of the cancer. Cancer is a heterogeneous disease that includes multiple subtypes, and the same type of cancer can be subdivided into different subtypes based on different pathogenesis mechanisms. In traditional clinical practice, cancer subtypes are usually determined based on clinical characteristics such as cancer staging and grading. With the continuous advancement of high-throughput experimental technology, further classification of cancer subtypes based on molecular omics information of cancer samples has become a research hotspot in biomedicine.
[0003] Due to the high-dimensional and small sample size characteristics of genomic data, feature selection methods should be used for dimensionality reduction in research. Through feature selection methods, features that are weakly correlated with the objective function are removed. At the same time, noise and redundant attributes are removed to increase the calculation speed of subsequent related algorithms. This will not only retain the biological significance of the original features, but also improve the prediction accuracy.
[0004] In order to prove that feature selection provides higher accuracy, Rustam Z et al. compared the classification accuracy between datasets without and with feature selection. For details, see: Rustam Z, Maghfirah N. Correlatedbased SVM-RFE as feature selection for cancer classification using microarraydatabases[C] / / AIP Conference Proceedings. AIP Publishing LLC, 2018, 2023(1): 020235. A large number of scholars have studied how to perform efficient feature selection. Pang S et al. proposed a feature selection method based on the heterogeneity score of a single gene to identify cancer subtypes: HSSG. For details, see: Pang S, Wu W, Zhang Y, et al. HSSG: Identification of Cancer Subtypes Based on Heterogeneity Score of ASingle Gene[J]. Cells, 2022, 11(15): 2456. After feature selection, correctly classifying the sample is also a key step. Hui Jiang et al. used SVM to classify gastric cancer patients. For details, see: Jiang H, Gu J, Du J, etal. A21-gene Support Vector Machine classifier and a 10-gene risk score system constructed for patients with gastric cancer [J]. Molecular medicine reports, 2020, 21(1): 347-359.
[0005] Although relevant research has achieved considerable results, there are still many shortcomings. For example, compared with cancer classification, there are fewer studies on cancer subtype classification, and the important genes selected by existing feature selection methods generally have low classification accuracy in gastric cancer classification (under the same classification network). Summary of the invention
[0006] In order to address the shortcomings of the prior art, the present invention provides a method for improving the accuracy of gastric cancer classification prognosis prediction based on a gradient boosting deep feature selection algorithm. It adopts an unsupervised clustering method without the need to specify clusters in advance, and proposes the use of PAC and clinical data to assist in deciding the optimal number of subtypes, thereby effectively improving the accuracy of prognosis prediction.
[0007] The technical solution of the present invention is:
[0008] A method for improving the accuracy of gastric cancer classification prognosis prediction based on a gradient boosting deep feature selection algorithm comprises the following steps:
[0009] (1) Download transcriptomic datasets and clinical data of gastric cancer patients from TCGA and perform data preprocessing;
[0010] (2) Using the consensus clustering method combined with PAC and clinical data to determine the optimal number of subtypes for distinguishing different subtypes;
[0011] (3) Using an improved gradient boosting deep feature selection algorithm for feature selection;
[0012] (4) Use deep neural network for classification.
[0013] Preferably, in step (1), data preprocessing refers to removing zero variance features in the transcriptomics data set and performing differential analysis, specifically:
[0014] The data downloaded from TCGA are in two forms: raw counts and standardized FPKM. The edgeR package was used to perform differential analysis on the raw data. The mean was taken to retain one of the same gene names in the raw counts matrix. The variance formula is:
[0015] Variance formula:
[0016] Among them, S 2 is the variance, n is the total number of samples, It represents the average of this set of data;
[0017] The threshold is selected as 0, and gene features with variance greater than 0 are selected;
[0018] The edgeR package was used to perform differential analysis on the variance-filtered dataset, and some features were selected before input into the model, and data with pvalue < 0.05 and log2Foldchange > 1.5 were selected;
[0019] Among them, pvalue represents the statistical parameter of significant difference, and Foldchange represents the difference fold of sample quality inspection expression.
[0020] Preferably, for the operation of typing, most of the existing studies are unsupervised clustering. Traditional clustering methods usually require the number of clusters to be specified, but in practical applications, it is often difficult to determine the optimal number of clusters. Consistency clustering is a method based on k-means clustering, and the stability and reliability of clustering results can be evaluated by the consistency coefficient to determine the optimal number of clusters. Compared with traditional clustering methods, consistency clustering is more objective and effective in selecting the optimal number of clusters. The ratio of fuzzy clustering pairs (PAC) can evaluate the stability and consistency of clustering results, and has good robustness and interpretability. Therefore, the present invention uses the PAC indicator to more reliably infer the optimal number of subtypes.
[0021] The consistency clustering method in step (2) adopts the k-means clustering method, and the selected max k is 5, that is, a maximum value is set to continuously iterate and try, the number of iterations is 1000, and 80% of the sample subset is selected in each iteration. The k-means clustering method is run on the sample subset to obtain the cumulative density function CDF and consistency matrix heat map corresponding to different k values;
[0022] Select the curve with the smallest descending slope of the cumulative density function CDF and the k value corresponding to the matrix with the best distribution in the consistency matrix heat map as the preferred k value;
[0023] The proportion of fuzzy cluster pairs, PAC, is an indicator used to determine the optimal number of clusters. The optimal number of clusters k is determined by evaluating the consistency of multiple clustering results. Specifically, samples and gene features are randomly extracted from the FPKM dataset multiple times, and clustered using the selected k-means clustering method to obtain multiple clustering results; then, the consensus matrix between multiple clustering results is calculated to measure the degree of consistency between them; finally, the optimal number of clusters k is determined by calculating the area on the consensus matrix within a certain threshold range.
[0024] Preferably, the clinically relevant data include survival curves of gastric cancer patients. Large differences in the survival curves verify that the unsupervised clustering of the subtype number is reasonable.
[0025] Preferably, in step (3), gradient boosting, also known as gradient enhancement, is an improved gradient boosting algorithm that is an integration of base learners (decision trees). There is a sequence between the base learners, and the negative gradient is used as a measure of the error made by the previous round of base learners, wherein the negative gradient is also called a pseudo residual. During the training process, each round of base learners pays more attention to the samples with errors in the previous round of learning, and updates the sample weights after training. On this basis, the next base learner is trained, and finally all the base learners are weighted combined.
[0026] Preferably, the process of the improved gradient boosting algorithm is:
[0027] Input is the training set is the transcriptomics FPKM data after differential analysis, x represents the sample, y represents the subtype label, and n represents the total number of samples, that is, the total number of gastric cancer patients; the differentiable loss function is L(y, F(x)), F(x) is the predicted value, and the number of iterations is M. The specific process is as follows:
[0028] Initialize the model with a constant value
[0029]
[0030] γ is the predicted value of the constant initialization model;
[0031] For m from 1 to M:
[0032] Calculate the negative gradient (pseudo residual) using the negative gradient of the loss function in the current model F m-1 The value on (x) approximates the residual;
[0033]
[0034] The base learner h m (x) is fitted to the pseudo residual, h m (x) represents the mth base learner, that is, using the training set Train them;
[0035] Calculate r by the following optimization problem m :
[0036]
[0037] Update model: The next round of learners learns the negative gradient of the previous round of learners, thereby correcting
[0038] F m (x) = F m-1 (x)+γ m h m (x)
[0039] Output F M (x), which is the model of the weighted combination of all base learners, γ m represents the multiplier of the mth base learner, F m (x) is an imperfect model in the middle, and the predicted value is not so accurate. M (x) is the model of the last round;
[0040] Preferably, the improved gradient boosting uses a more advanced gradient boosting tree XGBoost model and adopts an early stopping mechanism to avoid overfitting and speed up the processing speed; the Optuna framework is used in conjunction with 10-fold cross validation to tune hyperparameters such as the maximum depth of the tree (max_depth), the number of trees (n_estimators), the learning rate (learning_rate), and the tree method (tree_method), so as to obtain important genes that play a role in prognostic typing and obtain a more accurate ranking of important features.
[0041] Preferably, in step (4), the deep neural network (DNN) is also called multi-layer perceptron (MLP), which is a neural network with many hidden layers;
[0042] The deep neural network consists of an input layer, two hidden layers and an output layer. The output of the hidden layer is assumed to be f(w1X+b1), where w1 is the weight, b1 is the bias, function f is the activation function, and X is the input.
[0043] The activation function of the hidden layer is ReLU, and the output layer selects a neuron whose activation function is the sigmoid function;
[0044] The loss function is used to calculate the degree of deviation between the true value and the predicted value of the target value in the test set. The Adam optimizer is used to minimize the loss function by training the optimization parameters.
[0045] Preferably, step (4) further includes:
[0046] (5) Support vector machine (SVM) is used to verify the robustness of the improved gradient boosting deep feature selection algorithm.
[0047] Preferably, the support vector machine is to solve the separation hyperplane that can correctly divide the training data set and has the largest geometric interval, that is, to find a hyperplane so that the point closest to the hyperplane can be the farthest. There are infinitely many hyperplanes that can separate two types of data, but the separation hyperplane with the largest geometric interval is unique.
[0048] Omics data has high dimensionality, and existing feature selection algorithms mostly use statistical methods or bagging techniques, and existing classifiers rarely consider adding neural networks, resulting in generally low classification accuracy. Therefore, this paper proposes a deep feature selection algorithm based on gradient boosting, and improves the classification prediction accuracy of gastric cancer subtypes by designing a deep neural network. Finally, the present invention identifies a biomarker, namely 24 selected marker genes, and performs bioinformatics analysis such as gene enrichment ontology (GO) terms and biological pathways to improve the practicality of the present invention.
[0049] For any details not provided in the present invention, please refer to the prior art.
[0050] The beneficial effects of the present invention are:
[0051] The present invention adopts an unsupervised clustering method without specifying clusters in advance, and proposes the use of PAC and clinical data to assist in deciding the optimal number of subtypes, thereby effectively improving the accuracy of prognosis prediction.
[0052] For genes with strong differences, the present invention proposes a gradient boosting deep feature selection algorithm that can learn from the previous round of erroneous samples to automatically update the weights, and compares it with eight other mainstream feature selection algorithms. The results show that the prognosis prediction accuracy of the present invention is the highest, which is 99.115%. The classification algorithm selected when compared with other feature selection algorithms is a self-constructed deep neural network structure. In order to verify the robustness, the biomarkers obtained by the gradient boosting algorithm were input into the SVM for classification, and a good classification accuracy was also obtained. Finally, the biomarkers selected by this feature selection algorithm were subjected to bioinformatics analysis such as survival analysis, gene enrichment ontology (GO) terms and biological pathways, which improved the practicality of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.
[0054] Figure 1 A schematic diagram of a method for improving the accuracy of gastric cancer classification prognosis prediction based on a gradient boosting deep feature selection algorithm of the present invention;
[0055] Figure 2 is the cumulative density function (CDF) of the consensus clustering;
[0056] Figure 3 Flowchart of the improved gradient boosting algorithm
[0057] Figure 4 The basic structure diagram of deep neural network (DNN)
[0058] Figure 5 Comparison results between the present invention and existing feature selection algorithms;
[0059] Figure 6 is the marker gene heat map;
[0060] Figure 7 is the survival curve of some marker genes;
[0061] Figure 8 This is a KEGG enrichment pathway analysis diagram. DETAILED DESCRIPTION
[0062] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the drawings in the implementation of this specification, but are not limited to this. Anything not fully described in the present invention shall be based on the conventional technology in the art.
[0063] The overall research ideas of the present invention are as follows:
[0064] First, we downloaded gastric cancer-related transcriptomics data and clinical data from GDC TCGA, processed them into readable transcriptomics data using R code, and then removed zero-variance features to achieve preliminary filtering of the data, because the gene features that are expressed identically in all gastric cancer patients are definitely not biomarkers (i.e., genes that can correctly distinguish different gastric cancer subtypes). In order to distinguish different subtypes, we used a consistent clustering method based on the resampling method, and used PAC and clinically related gastric cancer patient survival and mortality information to assist in deciding the optimal number of subtypes. At this point, there are still a large number of gene features, so we chose to use the edgeR package for differential analysis to select genes with large differences that are helpful for gastric cancer classification.
[0065] Then, the obtained differential genes are respectively input into the deep feature selection algorithm of the present invention to obtain the same number of gene feature subsets. Among them, the gradient boosting deep feature selection algorithm proposed in the present invention has the following improvements:
[0066] ① To improve the performance of the model, a more advanced gradient boosting tree, XGBoost, is selected. Compared with the traditional gradient boosting algorithm, this model is based on a regularization method based on feature scaling to avoid overfitting; it is more efficient in handling missing values; the addition of an approximate algorithm can increase the training speed; it supports parallel training and improves operating efficiency; it can handle nonlinear relationships and has stronger representation capabilities. Enable the early stopping mechanism, the "early_stopping_rounds" parameter defaults to None, and setting it to 8 means that if there is no performance improvement in 8 consecutive rounds of training, the training will be stopped to avoid overfitting and improve the generalization ability of the model.
[0067] ② In order to optimize the performance of the gradient boosting deep feature selection algorithm model, the Optuna hyperparameter tuning framework is added to XGBoost. Combined with 10-fold cross validation, the optimal hyperparameter combination can be quickly found, thereby improving the performance and generalization ability of the model.
[0068] The present invention uses a deep neural network for classification, and performs classification according to the feature subsets selected by different feature selection algorithms. It can be concluded from the accuracy of the final classification that the gradient boosting feature selection algorithm proposed in the present invention has better performance. The SVM algorithm is used to verify the robustness of the improved gradient boosting deep feature selection algorithm. Facts have proved that the SVM classification network can also have extremely high accuracy, that is, it has good robustness.
[0069] Finally, the present invention conducted a series of bioinformatics analyses and combined with the survival curves of clinical data to conclude that subtypes with high and low expression of characteristic genes have different survival curves (for example, high expression of a certain gene leads to long survival time), which is helpful for medical research and proves to have great practical significance and research value.
[0070] Example 1
[0071] A method for improving the accuracy of gastric cancer classification prognosis prediction based on a gradient boosting deep feature selection algorithm comprises the following steps:
[0072] (1) Download transcriptomic datasets and clinical data of gastric cancer patients from TCGA and perform data preprocessing;
[0073] (2) Using the consensus clustering method combined with PAC and clinical data to determine the optimal number of subtypes for distinguishing different subtypes;
[0074] (3) Using an improved gradient boosting deep feature selection algorithm for feature selection;
[0075] (4) Use deep neural network for classification.
[0076] Example 2
[0077] A method for improving the accuracy of gastric cancer classification prognosis prediction based on a gradient boosting deep feature selection algorithm, as described in Example 1, except that in step (1), data preprocessing refers to removing zero variance features in the transcriptomics data set and performing differential analysis, specifically:
[0078] The data downloaded from The Cancer Genome Atlas (TCGA) are in two forms: raw counts and standardized FPKM (fragments per kilobase per million mapped reads). The main purpose of standardized FPKM is to remove technical deviations from sequencing data: sequencing depth and gene length. Under the same conditions, the deeper the sequencing depth, the more reads of gene expression; similarly, different gene lengths produce unequal reads. The longer the gene, the higher the read of the gene; the edgeR package is used to perform differential analysis on the raw data, and one gene name in the rawcounts matrix is retained by taking the mean. After completing the above work, 407 samples and 59,427 characteristic genes of the gastric cancer gene expression matrix can be obtained. However, considering that the research content is the classification of gastric cancer subtypes, this embodiment only selects 375 samples from gastric cancer patients, and no normal samples are required.
[0079] If the variance of a feature itself is very small, it means that there is basically no difference in this feature among the samples. It is possible that most of the values in the feature are the same, or even the values of the entire feature are the same, then this feature has no effect on distinguishing samples. Taking the above problems into consideration, variance filtering should be performed before clustering. If the threshold of variance filtering is not set well, important features may be missed. For example, the income range is 0 to 100 million, and the age range is 0 to 150. Relatively speaking, the variance of age is much smaller than that of income, but it is not necessarily an important feature. Therefore, this embodiment chooses to remove features with 0 variance. The expression values of these features are the same in all gastric cancer patient samples, and they are definitely not differential genes. This operation changes the number of features from 59427 to 57106.
[0080] The variance formula is:
[0081] Variance formula:
[0082] Among them, S 2 is the variance, n is the total number of samples, It represents the average of this set of data;
[0083] The threshold is selected as 0, and gene features with variance greater than 0 are selected;
[0084] The edgeR package was used to perform differential analysis on the variance-filtered dataset, and some features were selected before input into the model, and data with pvalue < 0.05 and log2Foldchange > 1.5 were selected;
[0085] Among them, pvalue represents the statistical parameter of significant difference, Foldchange represents the difference multiple of the expression level of sample quality control. pvalue < 0.05 and log2Foldchange > 1.5 can make the obtained differential genes meaningful. Among them, log2Foldchange can narrow the gap between particularly large and relatively small differential values.
[0086] Pvalue: H0 null hypothesis: There is no difference in the expression of A and B. Based on this hypothesis, calculate the probability of observing the occurrence of A and B, and then obtain the Pvalue. If Pvalue < 0.05, it means that a small-probability event has occurred, and the null hypothesis should be rejected, that is, the expression of A and B is different, that is, there is a significant difference. The P value of the left-sided test is the probability that the test statistic X is less than the sample statistic value C, that is: P = P{X < C}; the P value of the right-sided test is the probability that the test statistic X is greater than the sample statistic value C: P = P{X > C}. Hypothesis testing based on statistics.
[0087] Calculate log2foldchange:
[0088] It can actually be calculated by the edgeR R package for differential analysis.
[0089] Perform differential analysis on the raw counts data of RNA-Seq using the edgeR package according to pvalue < 0.05 and log2Foldchange > 1.5, and screen out 5098 genes with strong differences in the RNA-Seq FPKM data according to this rule.
[0090] Example 3
[0091] A method for improving the prognosis prediction accuracy of gastric cancer typing based on the gradient boosting deep feature selection algorithm, as described in Example 2. The difference is that for the operation of typing, most existing studies are unsupervised clustering. Traditional clustering methods usually require specifying the number of clusters, but in actual applications, it is often difficult to determine the optimal number of clusters. Consensus clustering is a method based on k-means clustering, which can evaluate the stability and reliability of the clustering results through the consensus coefficient, so as to determine the optimal number of clusters. Compared with traditional clustering methods, consensus clustering is more objective and effective in selecting the optimal number of clusters. The proportion of fuzzy clustering pairs (PAC) can evaluate the stability and consistency of the clustering results, and has good robustness and interpretability. Therefore, the present invention selects the PAC index to more reliably infer the optimal number of subtypes.
[0092] Clustering is an unsupervised learning task. It uses an unlabeled gene expression dataset to train a machine learning algorithm. The consistency clustering method in step (2) uses the k-means clustering method. Based on the prior knowledge of the medical community on gastric cancer subtype research, the max k is selected as 5, that is, a maximum value is set to iterate and try continuously. The number of iterations is 1000. In each iteration, 80% of the sample subset is selected, and the k-means clustering method is run on the sample subset to obtain the cumulative density function CDF and consistency matrix heat map corresponding to different k values;
[0093] The next step is how to choose the most appropriate number of clusters, such as Figure 2 As shown, the curve with the smaller descending slope of the cumulative density function CDF and the k value corresponding to the matrix with the best distribution in the consistency matrix heat map are selected as the preferred k value;
[0094] from Figure 2 It can be seen that the slope is smaller when k = 2 or 4. It is precisely because the evaluation index of cluster number is fuzzy, so the subsequent PAC and clinical data are introduced (in the final survival time analysis);
[0095] The consistency matrix heat map can be used to visualize and compare data by mapping each value in the data matrix to a color display according to a certain rule. In the consistency matrix heat map, if the clustering effect is good, the colors corresponding to 0 and 1 in the matrix will be clearly divided and look relatively clean. The values of the consistency matrix heat map matrix represent the possibility of belonging to the same cluster. The value ranges from 0 to 1, and the color ranges from white to colored. Try not to choose a k value that is mixed with color.
[0096] Since the above process of determining the k value requires intuitive observation of the smallest slope and the “best” heat map effect, it is not rigorous enough to determine the optimal number of clusters, so further verification in combination with PAC has better results.
[0097] The proportion of fuzzy cluster pairs, PAC, is an indicator used to determine the optimal number of clusters. The optimal number of clusters k is determined by evaluating the consistency of multiple clustering results. Specifically, samples and gene features are randomly extracted from the FPKM dataset multiple times, and clustered using the selected k-means clustering method to obtain multiple clustering results; then, the consensus matrix between multiple clustering results is calculated to measure the degree of consistency between them; finally, the optimal number of clusters k is determined by calculating the area on the consensus matrix within a certain threshold range.
[0098] The code for determining the optimal number of clusters using the PAC method is as follows:
[0099]
[0100]
[0101] In this embodiment, the optimal cluster number is finally determined to be 2. After determining the optimal cluster number k, 375 gastric cancer patient samples are labeled: there are 256 people of type I subtype and 119 people of type II subtype.
[0102] Example 4
[0103] A method for improving the accuracy of gastric cancer classification prognosis prediction based on a gradient boosting deep feature selection algorithm is as described in Example 3, except that the clinically relevant data include survival curves of gastric cancer patients, and the survival curves are very different, which verifies that the unsupervised clustering of the subtype number is reasonable.
[0104] Example 5
[0105] A method for improving the accuracy of gastric cancer classification prognosis prediction based on a gradient boosting deep feature selection algorithm, as described in Example 4, except that in step (3), gradient boosting, also known as gradient enhancement, is an improved gradient boosting algorithm that is an integration of base learners (decision trees), and there is a sequence between the base learners. The negative gradient is used as a measure of the error made by the base learner in the previous round, wherein the negative gradient is also called a pseudo residual. During the training process, each base learner pays more attention to the samples with errors in the previous round of learning, and updates the sample weights after training. On this basis, the next base learner is trained, and finally all the base learners are weightedly combined (each round of iteration will learn the erroneous samples of the previous round, the base learner fits the residual value, and the base learner model is optimized and improved until this process reaches the Mth base learner. Finally, all the base learners are summed according to the weights).
[0106] The specific process of the gradient boosting algorithm is as follows Figure 3 .
[0107] The process of the improved gradient boosting algorithm is:
[0108] Input is the training set is the transcriptomics FPKM data after differential analysis, x represents the sample, y represents the subtype label, and n represents the total number of samples, that is, the total number of gastric cancer patients; the differentiable loss function is L(y, F(x)), F(x) is the predicted value, and the number of iterations is M. The specific process is as follows:
[0109] Initialize the model with a constant value
[0110]
[0111] γ is the predicted value of the constant initialization model;
[0112] For m from 1 to M:
[0113] Calculate the negative gradient (pseudo residual) using the negative gradient of the loss function in the current model F m-1 The value on (x) approximates the residual;
[0114]
[0115] The base learner h m (x) is fitted to the pseudo residual, h m (x) represents the mth base learner, that is, using the training set Train them;
[0116] Calculate r by the following optimization problem m :
[0117]
[0118] Update model: The next round of learners learns the negative gradient of the previous round of learners, thereby correcting
[0119] F m (x) = F m-1 (x)+γ m h m (x)
[0120] Output F M (x), which is the model of the weighted combination of all base learners, γ m represents the multiplier of the mth base learner, F m (x) is an imperfect model in the middle, and the predicted value is not so accurate. M (x) is the model of the last round;
[0121] Preferably, the improved gradient boosting uses a more advanced gradient boosting tree XGBoost model and adopts an early stopping mechanism to avoid overfitting and speed up the processing speed; the Optuna framework is used in conjunction with 10-fold cross validation to tune hyperparameters such as the maximum depth of the tree (max_depth), the number of trees (n_estimators), the learning rate (learning_rate), and the tree method (tree_method), so as to obtain important genes that play a role in prognostic typing and obtain a more accurate ranking of important features.
[0122] This paper first uses gradient boosting for gastric cancer typing feature selection. In order to prevent overfitting, a regularization term (penalty term) is added to the loss function to limit the number of decision tree leaf nodes and leaf node values, reduce model variance, and prevent overfitting. A more advanced XGBoost model is selected, and an early stopping mechanism is set; the Optuna hyperparameter tuning framework is added to the XGBoost model, and 10-fold cross validation is used to determine the best hyperparameter combination, where the hyperparameters include the maximum depth of the tree, the number of trees, the learning rate, the tree method, etc.
[0123] Example 6
[0124] A method for improving the accuracy of prognosis prediction of gastric cancer classification based on gradient boosting deep feature selection algorithm, as described in Example 5, except that, in the gastric cancer subtype classification of this study, the optimal number of clusters obtained by matrix heat map analysis and PAC method is 2. Therefore, the subtype prediction classification problem is actually a binary classification problem. Robustness referred to in biology refers to the ability of systems to maintain their characteristic behaviors under perturbations or uncertainties. In order to better verify the robustness of the gradient boosting deep feature selection algorithm (GBDFS), this paper chooses to use support vector machines (SVM) and deep neural networks (DNN) to predict the best low-dimensional feature subsets selected by the algorithm, respectively.
[0125] In step (4), Deep Neural Networks (DNN) is also called Multi-Layer Perceptron (MLP), which is a neural network with many hidden layers;
[0126] like Figure 4 As shown, the deep neural network includes an input layer, two hidden layers and an output layer. Assume that the output of the hidden layer is f(w1X+b1), where w1 is the weight, b1 is the bias, function f is the activation function, and X is the input layer;
[0127] Neural networks have three basic elements: weights, biases, and activation functions. The strength of the connection between neurons is represented by weights, and the size of the weights represents the size of the possibility. The bias is set to correctly classify samples, and it is an important parameter in the model. Using activation functions can introduce nonlinear factors to neurons, which can make the neural network arbitrarily approximate any nonlinear function. If the activation function is not used, no matter how many layers the neural network has, the output is a linear combination of the input;
[0128] The activation function of the hidden layer uses ReLU, which usually refers to the ramp function in mathematics, as shown in the formula:
[0129]
[0130] There are several reasons for using ReLU as the activation function:
[0131] (1) When using activation functions such as sigmoid, exponential operations are involved; when backpropagating to find the error gradient, the derivative involves division, which results in a large amount of calculation. However, using the ReLU activation function saves a lot of calculation in the entire process.
[0132] (2) For deep networks, when the sigmoid function is back-propagated, the gradient vanishes easily (when the sigmoid function is close to the saturation region, the transformation is too slow and the derivative approaches 0, which causes information loss), making it impossible to complete the training of the deep network. The gradient of ReLU is a constant in the non-negative interval, so there is no gradient vanishing problem, which keeps the convergence speed of the model at a stable state.
[0133] (3) ReLu will make the output of some neurons 0, which will cause the sparsity of the network. It reduces the interdependence of parameters and alleviates the occurrence of overfitting problems.
[0134] The output layer selects a neuron whose activation function is a sigmoid function, as shown in the following formula. It converts a scalar number between 0 and 1. If it is greater than a probability threshold (set to 0.5 in this embodiment), it is considered to belong to a certain category, otherwise it does not belong to a certain category.
[0135]
[0136] The cross entropy loss function uses binary_crossentropy, and its loss function formula is as follows:
[0137]
[0138] in, is the true value y i The predicted value of , n is the total number of samples.
[0139] For the optimization algorithm for updating the deep learning grid parameters, the Adam optimizer is selected to minimize the loss function by training the optimization parameters. The loss function is used to calculate the degree of deviation between the true value and the predicted value of the target value in the test set.
[0140] The Adam algorithm is different from the traditional stochastic gradient descent. Stochastic gradient descent maintains a single learning rate (ie, alpha) to update all weights, and the learning rate does not change during the training process. Adam designs independent adaptive learning rates for different parameters by calculating the first-order moment estimate and the second-order moment estimate of the gradient.
[0141] For hyperparameter tuning, commonly used methods include grid search, random search, and Bayesian optimization. Optuna is the most common model parameter tuning tool in the 2021 Kaggle competition. It was originally published in 2019. This study uses Optuna as a hyperparameter tuning tool.
[0142] Example 7
[0143] A method for improving the accuracy of gastric cancer classification prognosis prediction based on a gradient boosting deep feature selection algorithm, as described in Example 6, except that step (4) further includes:
[0144] (5) Support vector machine (SVM) is used to verify the robustness of the improved gradient boosting deep feature selection algorithm.
[0145] The support vector machine is to solve the separation hyperplane that can correctly divide the training data set and has the largest geometric interval, that is, to find a hyperplane so that the point closest to the hyperplane can be the farthest. There are infinitely many hyperplanes that can separate two types of data, but the separation hyperplane with the largest geometric interval is unique.
[0146] Omics data has high dimensionality, and existing feature selection algorithms mostly use statistical methods or bagging techniques, and existing classifiers rarely consider adding neural networks, resulting in generally low classification accuracy. Therefore, this paper proposes a deep feature selection algorithm based on gradient boosting, and improves the classification prediction accuracy of gastric cancer subtypes by designing a deep neural network. Finally, the present invention identifies a biomarker, namely 24 selected marker genes, and performs bioinformatics analysis such as gene enrichment ontology (GO) terms and biological pathways to improve the practicality of the present invention.
[0147] Performance Metrics
[0148] In order to evaluate the performance of multiple feature selection algorithms, this paper uses deep neural networks as classifiers to compare the classification effects of different features obtained by different algorithms under the same number of feature subsets. The evaluation indicators selected are classification accuracy, area under the curve (AUC), sensitivity (SN) and specificity (SP). For the binary classification model, the combination of the predicted situation and the actual situation will result in 4 combinations, forming a confusion matrix, as shown in Table 1:
[0149] Table 1: Confusion Matrix
[0150] Predict positive examples Prediction counterexample Actual positive example True Positive(TP) False Negative(FN) Actual counterexample False Positive (FP) True Negative(TN)
[0151] In order to evaluate the performance of multiple feature selection algorithms, the classification effects of different features obtained by different algorithms under the same number of feature subsets were compared. The evaluation indicators selected in this study are accuracy, area under the curve (AUC), sensitivity (SN) and specificity (SP). The results are shown in Table 2:
[0152] Table 2: SVM classification results of biomarkers
[0153] Evaluation Metrics Numerical value Accuracy 0.991150 AUC 0.993506 SN 0.987013 SP 1.0
[0154] TP represents the positive samples predicted as positive by the model;
[0155] FP represents negative samples predicted as positive by the model;
[0156] FN represents positive samples predicted as negative by the model;
[0157] TN represents negative samples predicted by the model as negative;
[0158] The true positive rate (TPR) indicates the probability of predicting a positive sample among samples that are actually positive.
[0159] The true negative rate (TNR) indicates the probability of predicting a negative sample when it is actually negative.
[0160] The false positive rate (FPR) indicates the probability of predicting a positive sample when it is actually negative.
[0161] The false negative rate (FNR) indicates the probability of predicting a negative value among samples that are actually positive.
[0162] The specific formula is as follows:
[0163]
[0164]
[0165]
[0166]
[0167] The accuracy calculation formula is as follows:
[0168]
[0169] Figure 5 The comparison results of the present invention and the existing feature selection algorithm are shown in Table 3:
[0170] Table 3: Results of each feature selection algorithm when the number of feature subsets is 24
[0171] Method Accuracy AUC SN SP Chi-square 0.946903 0.931457 0.974026 0.888889 F-test 0.955752 0.952742 0.961039 0.944444 MI 0.938053 0.924964 0.961039 0.888889 RFE 0.964602 0.966631 0.961039 0.972222 ET 0.955752 0.9453446 0.974026 0.916667 RDFS 0.964602 0.959235 0.974026 0.944444 fDNN 0.964602 0.974026 0948052 1.0 forgeNet 0.973451 0.965729 0.987013 0.944444 GBDFS (this embodiment) 0.991150 0.993506 0.987013 1.0
[0172] In this example, other mainstream feature selection algorithms are compared, such as chi-square test, variance analysis, mutual information algorithm, recursive feature elimination, extreme forest algorithm and random forest algorithm, and marker genes selected by these algorithms are compared using the same classification model. The results show that the deep feature selection algorithm proposed in the present invention has a higher prognosis classification accuracy.
[0173] Among them, filtering feature selection algorithms: chi-square test, variance analysis, mutual information algorithm;
[0174] Wrap-around feature selection algorithm: recursive feature elimination algorithm;
[0175] Embedded feature selection algorithms: extreme random trees, graph embedded deep feedforward network (GEDFN, reference: Kong Y, Yu T. A graph-embedded deep feedforward network for disease outcome classification and feature selection using gene expression data [J]. Bioinformatics, 2018, 34 (21): 3727-3737.), deep feedforward network embedded in forest graph (forgeNet, reference: Kong Y, Yu T. forgeNet: a graph deep neural network model using tree-based ensemble classifiers for feature graph construction [J]. Bioinformatics, 2020, 36 (11): 3507-3515.), random forest deep feature selection algorithm (RDFS, reference: Hu Y, Zhao L, Li Z, et al. Classifying the multi-omics data of gastric cancer using a deep feature selection method [J]. Expert Systems with Applications, 2022, 200: 116813.).
[0176] The best low-dimensional feature subset obtained by the gradient boosting deep feature selection algorithm was used as the biomarker to obtain its survival analysis (such as Figure 7 ), GO and KEGG enrichment analysis (such as Figure 8) and other bioinformatics analysis results. The Metascape gene list analysis website was used for KEGG enrichment analysis. Metascape is a web-based portal. It aims to provide comprehensive gene list annotation and analysis resources for relevant researchers. By generating interpretable outputs through a one-click rapid analysis interface, Metascape provides a significantly simplified user experience. It is an effective tool for relevant researchers to comprehensively analyze and interpret omics-based research in the big data era.
[0177] Figure 6 It can be seen from the heat map of marker genes that the selected optimal low-dimensional feature subset has different expression conditions (high and low expression) in the two subtypes, which can distinguish the two subtypes well. Figure 7 Gene survival curve, the horizontal axis is time and the vertical axis is survival probability.
[0178] Figure 8 It is a KEGG enrichment pathway analysis diagram. Pathway-related databases are databases based on human pathways, and KEGG is one of the pathway-related databases. The database contains gene-related functional information, and enrichment analysis is an algorithm that integrates and calculates these functions.
[0179] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method to improve the accuracy of gastric cancer classification and prognosis prediction based on gradient boosting deep feature selection algorithm. It is characterized in that The following steps are involved: (1) Download transcriptomic datasets and clinical data of gastric cancer patients from TCGA and perform data preprocessing; (2) Using the consensus clustering method combined with PAC and clinical data to determine the optimal number of subtypes for distinguishing different subtypes; (3) Using an improved gradient boosting deep feature selection algorithm for feature selection; (4) using a deep neural network to classify the feature subset after feature selection in step (3); In step (1), data preprocessing refers to removing zero-variance features in the transcriptomics dataset and performing differential analysis. Specifically: The data downloaded from TCGA are in two forms: raw counts and standardized FPKM. The edgeR package was used to perform differential analysis on the raw data. One gene name in the raw counts matrix was retained by taking the mean. The variance formula is: Variance formula: Among them, S 2 is the variance, n is the total number of samples, It represents the average of this set of data; The threshold is selected as 0, and gene features with variance greater than 0 are selected; The edgeR package was used to perform differential analysis on the variance-filtered dataset, and some features were selected before input into the model, and data with pvalue < 0.05 and log2Foldchange > 1.5 were selected; Among them, pvalue represents the statistical parameter of significant difference, and Foldchange represents the difference fold of sample quality inspection expression; The consistency clustering method in step (2) adopts the k-means clustering method, and the selected max k is 5, that is, a maximum value is set to continuously iterate and try, the number of iterations is 1000, and 80% of the sample subset is selected in each iteration. The k-means clustering method is run on the sample subset to obtain the cumulative density function CDF and consistency matrix heat map corresponding to different k values; Select the curve with the smallest descending slope of the cumulative density function CDF and the k value corresponding to the matrix with the best distribution in the consistency matrix heat map as the preferred k value; The proportion of fuzzy cluster pairs, PAC, is an indicator used to determine the optimal number of clusters. The optimal number of clusters k is determined by evaluating the consistency of multiple clustering results. Specifically, samples and gene features are randomly extracted from the FPKM dataset multiple times, and clustered using the selected k-means clustering method to obtain multiple clustering results; then, the consensus matrix between multiple clustering results is calculated to measure the degree of consistency between them; finally, the optimal number of clusters k is determined by calculating the area size on the consensus matrix within a certain threshold range; Clinically relevant data include the survival curves of gastric cancer patients. The large differences in the survival curves verify that the unsupervised clustering of the subtype number is reasonable; In step (3), gradient boosting, also known as gradient enhancement, is an improved gradient boosting algorithm that is an integration of base learners. There is a sequence between base learners. The negative gradient is used as a measure of the error made by the previous round of base learners. The negative gradient is also called a pseudo residual. During the training process, each round of base learners pays more attention to the samples with errors in the previous round of learning. After training, the sample weights are updated, and the next base learner is trained on this basis. Finally, all base learners are weighted combined. The process of the improved gradient boosting algorithm is: Input is the training set is the transcriptomics FPKM data after differential analysis, x represents the sample, y represents the subtype label, and n represents the total number of samples, that is, the total number of gastric cancer patients; the differentiable loss function is L(y, F(x)), F(x) is the predicted value, and the number of iterations is M. The specific process is as follows: Initialize the model with a constant value γ is the predicted value of the constant initialization model; For m from 1 to M: Calculate the negative gradient using the negative gradient of the loss function in the current model F m-1 The value on (x) approximates the residual; The base learner h m (x) is fitted to the pseudo residual, h m (x) represents the mth base learner, that is, using the training set Train them; Calculate r by the following optimization problem m : Update model: The next round of learners learns the negative gradient of the previous round of learners, thereby correcting F m (x)=F m-1 (x)+γ m h m (x) Output F M (x), which is the model of the weighted combination of all base learners, γ m represents the multiplier of the mth basis learner; The improved gradient boosting uses a more advanced gradient boosting tree XGBoost model and adopts an early stopping mechanism to avoid overfitting and speed up processing. The Optuna framework is used in conjunction with 10-fold cross-validation to tune the maximum depth of the tree, the number of trees, the learning rate, and the tree method hyperparameters to obtain important genes that play a role in prognostic typing and obtain a more accurate ranking of important features.
2. The method for improving the accuracy of gastric cancer classification prognosis prediction based on the gradient boosting deep feature selection algorithm according to claim 1, It is characterized in that In step (4), the deep neural network is also called a multi-layer perceptron, which is a neural network with many hidden layers; The deep neural network consists of an input layer, two hidden layers and an output layer. Let the output of the hidden layer be f(w 1 X+b 1 ), where w 1 is the weight, b 1 is the bias, function f is the activation function, and X is the input; The activation function of the hidden layer is ReLU, and the output layer selects a neuron whose activation function is the sigmoid function; The loss function is used to calculate the degree of deviation between the true value and the predicted value of the target value in the test set. The Adam optimizer is used to minimize the loss function by training the optimization parameters.
3. The method for improving the accuracy of gastric cancer classification prognosis prediction based on the gradient boosting deep feature selection algorithm according to claim 2, It is characterized in that After step (4), the method further includes: (5) Support vector machine is used to verify the robustness of the improved gradient boosting deep feature selection algorithm.
4. The method for improving the accuracy of gastric cancer classification prognosis prediction based on the gradient boosting deep feature selection algorithm according to claim 3, It is characterized in that The support vector machine solves the separation hyperplane that can correctly divide the training data set and has the largest geometric interval, that is, finds a hyperplane so that the point closest to the hyperplane can be the farthest.
