Heart disease key factor prediction method based on random forest

Abstract

The invention relates to the technical field of medical data analysis, in particular to a heart disease key factor prediction method based on random forest, which comprises the following steps: 1, data preprocessing: preprocessing input data, separating independent variables from dependent variables, and dividing into a training set and a test set; 2, model construction and training: constructing a random forest model, generating a plurality of decision trees through a Bagging algorithm, and performing node splitting by adopting a CART algorithm; 3, performing model evaluation, and adjusting model parameters (such as the number of trees and a node splitting threshold value) to optimize prediction performance; 4, evaluating the generalization ability of the model by using an out-of-bag sample (OOB), and outputting a feature importance analysis result; and 5, model inspection: predicting a new sample based on the trained model, and generating a reliability calibration curve. By optimizing the data preprocessing process, the parameter debugging strategy and the model evaluation system, the accuracy and reliability of heart disease prediction are improved, and a scientific basis is provided for clinical diagnosis.

Description

Technical Field

[0001] This invention relates to the field of medical data analysis technology, and in particular to a method for predicting key factors of heart disease based on random forest. Background Technology

[0002] In the field of healthcare, heart disease is one of the most serious threats to human life and health. Early and accurate prediction of heart disease risk is crucial for disease prevention and treatment. Traditional classification models, such as logistic regression and support vector machines, are prone to insufficient prediction accuracy or overfitting when processing multi-dimensional physiological data, making them difficult to meet actual clinical needs.

[0003] Random forests, as an ensemble learning method, can effectively improve prediction accuracy and model robustness by constructing multiple decision trees and summarizing the decision results. However, existing random forest-based heart disease prediction models still have room for improvement in terms of data preprocessing strategies, parameter tuning methods, and model evaluation systems. A more accurate and reliable random forest-based method for predicting key factors in heart disease is urgently needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for predicting key factors of heart disease based on random forests. By optimizing the data preprocessing process, parameter tuning strategy, and model evaluation system, this method improves the accuracy and reliability of heart disease prediction and provides a scientific basis for clinical diagnosis.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for predicting key factors of heart disease based on random forest includes the following steps:

[0007] Step 1: Data preprocessing. Preprocess the input data, separate the independent and dependent variables, and divide the data into training and test sets.

[0008] Step 2: Model building and training. Construct a random forest model, generate multiple decision trees using the Bagging algorithm, and perform node splitting using the CART algorithm.

[0009] Step 3: Model evaluation, adjusting model parameters (such as the number of trees, node splitting threshold) to optimize prediction performance;

[0010] Step 4: Evaluate the model's generalization ability using out-of-bag (OOB) samples and output the feature importance analysis results;

[0011] Step 5: Model validation. Based on the trained model, predict new samples and generate a reliability calibration curve.

[0012] Preferably, step 1 specifically includes:

[0013] Load the cleaned CSV dataset, separate the independent and dependent variables to obtain the independent variable feature set x and the dependent variable series y; where each row of x is a set of feature values ​​for a sample, and each column is a feature; y is a binary series indicating whether a sample has heart disease.

[0014] All data samples were randomly divided into a test set and a training set in a ratio of 3:7 (test:train). The training set was used for model training, and the test set was used for model evaluation.

[0015] Preferably, step 2 specifically includes:

[0016] Based on Python's scikit-learn library, a random forest classifier model is built using the RandomForestClassifier class; and the model parameters are tuned to appropriate values ​​to make the model fit the heart disease prediction problem.

[0017] In the first stage, the Bagging (Bootstrap Aggregating) algorithm is used. First, the Bootstrap method is used to resample the samples for the original sample set:

[0018]

[0019] Each row represents a single sample, where x i1 …x in Let y be the n independent variables of the i-th sample. n Let D represent the dependent variable of the sample; then, random resampling (i.e., sampling the sample and independent variable with replacement) will extract B subsets D1, D2, ..., D from D. B They respectively contain p1, p2, ..., p B samples (p) i (i = 1, 2, ..., B) ≤ m) and q1, q2, ..., q B 1 independent variable (q) i (i=1,2,…,B)≤n), since random sampling with replacement is performed, the sampled samples and the combination of independent variables may be repeated but also different from each other.

[0020] Subsequently, the CART (Classification And Regression Tree) algorithm was used to process each subset D. b Construct decision trees for each of the following (b = 1, 2, ..., B): Each decision node makes a split decision based on the Gini function, which is expressed as:

[0021]

[0022] Where J represents the number of categories, p i This represents the proportion of samples belonging to the i-th class in the current node T;

[0023] Therefore, according to the following formula:

[0024]

[0025] The process involves dividing the current node T into two child nodes T1 and T2, and then recursively dividing it into these two child nodes until all decision nodes have been expanded to indivisible leaf nodes.

[0026] Preferably, in the second stage, the prediction results of each decision tree constructed in the previous stage are summarized to obtain the final evaluation. For binary classification problems, this is done by voting, with the conclusion supported by the majority vote becoming the final conclusion. That is, the process of random forest is as follows: Figure 1 .

[0027] Random Forest (RF) is a type of ensemble learning model. It typically constructs multiple decision trees, each learning independently and making its own decisions. The final prediction is then obtained by aggregating all decisions. The Random Forest model mainly consists of two steps: constructing the decision forest and aggregating the decisions. In the first stage, multiple different trees are trained, each receiving a random subset of data samples, resulting in different perspectives and independent judgments based on their understanding. In the second stage, fair methods are used to integrate these judgments. Therefore, compared to traditional classification models, the Random Forest model, using multiple trees to make decisions collaboratively, achieves higher prediction accuracy and is less prone to overfitting. Furthermore, the Random Forest model also exhibits high adaptability to outliers in the data source.

[0028] Preferably, in step 3, adjusting the model parameters specifically includes:

[0029] To enhance ensemble performance, the number of decision trees (n_estimators) was increased from the initial value of 100 to 500. The minimum number of samples for node splits (min_samples_split) was increased from 2 to 200, and the minimum number of samples for leaf nodes (min_samples_leaf) was increased from 1 to 150 to control overfitting. Multi-task mode (n_jobs = -1) was enabled to accelerate training, and out-of-bag evaluation (oob_score = True) was enabled.

[0030] Preferably, in step 5, a confusion matrix is ​​constructed to characterize the model's prediction performance from four dimensions: true prediction, false positive prediction, true negative prediction, and false negative prediction. The confusion matrix is ​​then visually displayed using a heatmap.

[0031] Plot calibration curves to evaluate the consistency between the model's predicted probabilities and the actual probabilities of occurrence, and analyze the calibration effect of the model in different predicted probability intervals.

[0032] Preferably, the calibration curve analysis divides the predicted probability into multiple intervals and calculates the consistency between the average predicted probability of each interval and the actual disease probability. This addresses the overconfidence in the predicted probability of high-risk samples and supports subsequent calibration optimization by introducing Platt scaling or quantile regression.

[0033] By adopting the above technical solution and the above method to construct a prediction system, it can receive physiological indicators such as age, systolic blood pressure, diastolic blood pressure, cholesterol, pulse pressure and other physiological indicators as input, output the disease probability prediction results, and provide a multi-dimensional evaluation report including accuracy, AUC value, OOB score, importance of independent variables, confusion matrix and calibration curve.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. This invention uses the random forest algorithm to construct a prediction model. By integrating the decisions of multiple decision trees, it effectively improves the accuracy of heart disease prediction and reduces the risk of overfitting.

[0036] 2. This invention optimizes the data preprocessing process and rationally divides the test set and training set to ensure the scientific nature of model training and evaluation.

[0037] 3. This invention improves the reliability and robustness of the model by system debugging of model parameters, combined with OOB evaluation and confusion matrix analysis.

[0038] 4. This invention, through the importance analysis of independent variables, clarifies the impact of key physiological indicators such as systolic blood pressure, diastolic blood pressure, pulse pressure, age, and cholesterol on the incidence of heart disease, providing clear indicator references for clinical diagnosis. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the random forest process in this invention;

[0040] Figure 2 This is an importance analysis diagram of the random forest model in this invention;

[0041] Figure 3 This is a heatmap of the confusion matrix of the random forest model in this invention;

[0042] Figure 4This is the reliability curve (calibration curve) of the random forest model in this invention;

[0043] Figure 5 This is the overall flowchart of the present invention. Detailed Implementation

[0044] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, so that those skilled in the art can better understand the advantages and features of the present invention, thereby making a clearer definition of the scope of protection of the present invention. The embodiments described in this invention are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0045] Example:

[0046] Step 1: Data Preprocessing

[0047] The program loads the cleaned CSV dataset and first separates the independent and dependent variables, resulting in a feature set x and a series y for the dependent variables. Each row of x represents a set of feature values ​​for a sample, and each column represents a feature. y is a series of target values ​​corresponding to each sample in x, which is a binary series indicating whether each sample has heart disease.

[0048] Next, all data samples are randomly divided into two groups, test set and training set, according to the ratio of test:train = 3:7. The samples in the training set will be used as the original input of the random forest model to train the random forest model and obtain parameters, while the data in the test set will be used to test the prediction accuracy of the constructed model and to evaluate the various indicators of the model when predicting whether or not the disease exists.

[0049] Step 2: Model Building and Training

[0050] For the training set data samples, the model is built using the sklearn library in Python. sklearn is an open-source Python library focused on machine learning and statistical modeling, providing tools to support a range of tasks from data preprocessing and feature engineering to model selection and ensemble learning. The random forest model built here will use the RandomForestClassifier class to implement a binary classification problem such as multi-factor heart disease prediction.

[0051] By adjusting the input parameters, the model building process can be more finely controlled. The parameters involved in debugging and their explanations are as follows:

[0052] Table 1 Random Forest Debugging Parameters

[0053]

[0054]

[0055] By adjusting the above parameters, the model can be tuned to better fit the current problem.

[0056] Step 3: Model Evaluation

[0057] After fine-tuning the parameters, a debugged random forest model was obtained. Then, the test set data was input into the model for testing, and the model achieved an accuracy of 73.1% and an AUC of 0.799.

[0058] Because of the unique nature of random forest models, which use out-of-bag (OOB) samples for evaluation, an OOB (Out-of-Bag) error rate is provided to characterize the performance of the trained model. Specifically, in random forests, each tree is constructed using partial sampling, resulting in an unsampled portion, known as OOB samples. During the construction of the random forest model, due to the randomness of sampling, some samples are repeatedly drawn, but a certain number of samples are never drawn. Therefore, by collecting these unsampled samples and feeding them into the random forest model for prediction, we can examine the accuracy of the prediction under this sample input scenario. The model's OOB score shows an excellent performance of 73.4%.

[0059] The trained random forest model is output as the weights (i.e., importance) of each independent variable. Figure 2 As shown.

[0060] Depend on Figure 2 As can be seen, the random forest model considers systolic blood pressure (ap_hi) to have the greatest impact on heart disease, followed by diastolic blood pressure (ap_lo) and pulse pressure (plus_pressure), all of which are indicators of blood pressure, and then age and cholesterol.

[0061] Step 4: Model Validation

[0062] To more intuitively characterize the model's predictive accuracy, a confusion matrix will be introduced below for visual representation. The confusion matrix characterizes the model's predictions across four dimensions, categorizing a single data prediction into four types: true prediction, false positive prediction, true negative prediction, and false negative prediction. These represent: correctly identifying a positive term as positive in a positive context, incorrectly identifying a positive term as positive in a negative context, incorrectly identifying a negative term as positive in a positive context, and correctly identifying a negative term as negative in a negative context, respectively. A good model should make predictions that fall more frequently into the true and false negative categories (i.e., the first and fourth categories). Furthermore, to visually represent the confusion matrix, a similar concept will be introduced... Figure 3 A heatmap is used to visually represent the numbers of the four items, which are correspondingly described in the four quadrants.

[0063] Depend on Figure 3 As can be seen, most predictions fall into the second and fourth quadrants, and the model's prediction performance is generally good.

[0064] like Figure 4 The reliability of the random forest model in predicting heart disease was evaluated using a calibration curve, which reflects the consistency between the predicted probability and the actual probability of occurrence. Analysis of the overall trend of the curve shows that the true probability distribution in the low to medium predicted probability range is close to the predicted value, indicating that the model calibrates well within this range. However, as the predicted probability increases, the rate of increase in the true probability slows down, and the two gradually deviate from the diagonal, reflecting an overconfidence in the model's classification of high-risk samples. This phenomenon may be related to the majority voting mechanism of the random forest or sample imbalance, and needs to be mitigated by introducing cost-sensitive learning or adjusting the classification threshold.

[0065] Therefore, in subsequent studies, a calibration module is integrated, such as post-processing using scaling or quantile regression, and high-risk sample data is expanded to optimize the model's generalization ability, thereby improving the accuracy of the predicted probability while ensuring the discriminative power.

[0066] In addition, the basic symbols in the embodiments of the present invention are explained as follows:

[0067]

[0068] In summary, the random forest-based method for predicting key factors of heart disease constructed through the above steps can effectively utilize multi-dimensional physiological indicator data to achieve accurate prediction of heart disease, and has good clinical application value.

[0069] The descriptions and practices disclosed in this invention are readily apparent and understandable to those skilled in the art, and various modifications and refinements can be made without departing from the principles of this invention. Therefore, any modifications or improvements made without departing from the spirit of this invention should also be considered within the scope of protection of this invention.

Claims

1. A heart disease key factor prediction method based on random forest, characterized in that, Comprising the following steps: Step 1: data preprocessing, preprocessing the input data, separating independent variables and dependent variables, and dividing the training set and test set; Step 2: model building and training, building a random forest model, generating multiple decision trees through Bagging algorithm, and using CART algorithm for node splitting; Step 3: model evaluation, adjusting model parameters to optimize prediction performance; Step 4: use out-of-bag samples OOB to evaluate the generalization ability of the model, and output the feature importance analysis results; Step 5: model verification, based on the trained model, predict new samples and generate reliability calibration curve. 2.The cardiac key factor prediction method based on random forest according to claim 1, wherein, In step 1, specifically including: Load the cleaned CSV data set, separate the independent variables and dependent variables, get the independent variable feature set x and the dependent variable series y; Where, each row of x is a feature value set of a sample, and each column is a feature; Y is a binary series, indicating whether the sample has heart disease; All data samples are randomly divided into test set and training set according to the ratio of test: train = 3:7, the training set is used for model training, and the test set is used for model evaluation. 3.The cardiac key factor prediction method based on random forest according to claim 1, characterized in that, In step 2, specifically including: Based on Python's scikit-learn library, use RandomForestClassifier class to build random forest classifier model; And debug the model parameters to the appropriate value, so that the model fits the heart disease prediction problem; In the first stage, using Bagging algorithm, first using Bootstrap method to resample the samples, for the original sample set: where each row represents a single sample, where x i1 …x in are n independent variables for the ith sample, and y n represents the dependent variable for that sample; then randomly resample, taking B subsets D1, D2, …, D B from D, which contain p1, p2, …, p B samples (p i (i = 1, 2, …, B) ≤ m) and q1, q2, …, q B independent variables (q i (i = 1, 2, …, B) ≤ n). Subsequently, the CART algorithm is used to each sub-sample set D b (b = 1, 2, …, B) are constructed respectively, and each decision node is split according to a Gini function, which is expressed as: where J denotes the number of classes, p i represents the proportion of samples in the current node T that belong to the i-th class. Then according to the following formula: Where, the current node T is divided into two sub nodes T1, T2, and thus recursively until all decision nodes are expanded to non-divisible leaf nodes. 4.The cardiac key factor prediction method based on random forest according to claim 3, characterized in that, In the second stage, the prediction results of each decision tree constructed in the previous stage are summarized to obtain the final evaluation. For binary classification problems, the results are summarized by voting, and the majority supported conclusion is the final conclusion. 5.The method of claim 1, wherein, In step 3, adjusting the model parameters specifically includes: Adjust the number of decision trees (n_estimators) from the initial value 100 to 500 to enhance the integration effect, increase the minimum sample size for node splitting (min_samples_split) from 2 to 200, increase the minimum sample size for leaf nodes (min_samples_leaf) from 1 to 150 to control overfitting, enable multi-task mode (n_jobs =-1) to speed up training, and enable out-of-bag sample evaluation (oob_score = True). 6.The method of claim 1, wherein, In step 5, the confusion matrix is constructed to describe the prediction of the model from four dimensions: true positive prediction, false positive prediction, true negative prediction, and false negative prediction. The confusion matrix is visually presented through a heat map; Draw the calibration curve to evaluate the consistency of the model prediction probability and the actual occurrence probability, and analyze the calibration effect of the model in different prediction probability intervals.

7. The method of claim 6, wherein the method is based on a random forest. The calibration curve analysis divides the prediction probability into multiple intervals, calculates the consistency of the average prediction probability and the actual prevalence probability in each interval, and supports the subsequent introduction of Platt scaling or quantile regression for calibration optimization in view of the overconfidence problem of the prediction probability of the high-risk samples.

Images