Heart disease prediction method based on feature combination and ensemble learning

Abstract

The invention provides a heart disease prediction method based on feature combination and ensemble learning. The heart disease prediction method comprises four stages of data set preprocessing, feature combination, basic classifier training and ensemble prediction. In the data preprocessing stage, dimension unification and missing value processing are performed on a public heart disease data set, feature combination is performed on the processed data set by using a designed feature combination algorithm to generate a new feature matrix, and a diversity enhancement algorithm is adopted to process the new feature matrix to obtain a new feature matrix; finally, different sub-feature matrixes are generated to serve as input of a plurality of basic classifiers for training, prediction results of all the basic classifiers are integrated, and final classification prediction is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical data analysis, and in particular relates to a heart disease prediction method based on feature combination and ensemble learning. Background Art

[0002] According to the summary of the 2020 China Cardiovascular Health and Disease Report, in 2018, cardiovascular disease was the leading cause of death among the total causes of death of urban and rural residents in my country. The proportion of deaths caused by cardiovascular disease to the total number of deaths of urban and rural residents is 46.66% in rural areas and 43.81% in cities, which is much higher than other diseases. Therefore, its prognosis problems are becoming more and more prominent. We can build a heart disease prediction model based on existing medical data to help prevent and treat heart disease in advance and make correct examinations and treatments for potential patients. Common prediction algorithms include naive Bayes algorithm, decision tree algorithm and random forest algorithm. These algorithm models have excellent prediction functions, but they also have their own limitations. The Naive Bayes algorithm requires that the features in the data set take discrete values ​​and are independent of each other, which makes it only suitable for scenarios with simple business backgrounds and low correlation between different features. The decision tree algorithm can generate a clear tree structure and clarify the impact of each feature on the prediction results, but this also causes its instability. When the feature data changes slightly, the resulting decision tree may be completely different from the previous one. Therefore, it is more often used as the cornerstone of other algorithms. The random forest algorithm is not good at high-dimensional sparse data and may perform poorly on high-dimensional sparse data.

[0003] The above problems can be solved by using ensemble learning to predict heart disease. Ensemble learning completes learning tasks by constructing and combining multiple learners. This algorithm can integrate the advantages of a single machine learning algorithm and improve classification accuracy and generalization ability. In order to obtain an ensemble with strong generalization performance, the individual learners in the ensemble should be as independent as possible; although "independence" cannot be achieved in real tasks, it is possible to make the base learners as different as possible. In addition, existing heart disease prediction models trained based on public heart disease datasets often fail to take into account the correlation between medical features. For example, age and blood pressure are not independent of each other. Older people often have higher blood pressure than younger people. There is also a correlation between age and personal signs such as heart rate. Therefore, when predicting heart disease, it is necessary to design a feature combination algorithm to visualize this association by considering the correlation between heart disease features, which is very helpful for improving the training effect of heart disease prediction models. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a method for predicting heart disease based on feature combination and ensemble learning, which can fully utilize the correlation of medical features and improve the accuracy of prediction and the generalization of the model.

[0005] The present invention solves the technical problem by adopting the following technical solution: designing a heart disease prediction method based on feature combination and ensemble learning, characterized in that the method comprises the following specific steps:

[0006] The first step is to preprocess the open source heart disease dataset to obtain a sample dataset D with n features.

[0007] The second step is to use the feature combination algorithm based on the Spearman correlation coefficient to expand D, gradually traverse each feature in D, and associate and combine the two features based on the Spearman correlation coefficient to obtain an n×n×3-dimensional feature matrix X′.

[0008] The third step is to use the perturbation-based difference enhancement algorithm to process the new feature matrix and generate different sub-feature matrices; this algorithm is divided into two stages. First, based on data sample perturbation, samples are extracted from the original data set with replacement based on the Bootstrap sampling method to generate three training subsets, and then based on the input attribute perturbation, the most appropriate sampling ratio is selected using the cross-validation method, and new attribute subsets are generated for these three different subsets based on the sub-attributes with the same ratio of feature sampling.

[0009] Step 4: Build the scikit-learn machine learning framework on Windows, establish basic classifier models of support vector machine, decision tree, and naive Bayes, and use the attribute subsets generated in the third step for training and testing respectively.

[0010] Step 5: The prediction results of each basic classifier are integrated based on the voting method to achieve the final classification prediction.

[0011] Step 6: Verify the generalization and accuracy of the model.

[0012] The beneficial effects of the present invention are as follows: Compared with the prior art, the present invention has the following outstanding properties and characteristics:

[0013] (1) The present invention fully explores the correlation between the characteristics of heart disease, combines the original characteristics based on the spearman correlation coefficient, and generates new characteristics. This feature combination method is different from the common method of only performing mathematical calculations on the two characteristics. Instead, it further introduces the correlation coefficient, and then fully considers the correlation between the characteristics. These new features contain richer information and more inherent laws of the data. This helps to improve the model's ability to express data, thereby improving the model's predictive performance. (2) The present invention designs a perturbation-based difference enhancement algorithm in the model integration stage, which is different from using multiple simple classifiers for integration and majority voting, such as researchers have used

[0014] M.Shouman(M.Shouman,T.Turner,R.Stocker,Using data

[0015] mining techniques in heart disease diagnosisand

[0016] treatment[C].Japan-Egypt Conference on

[0017] Electronics, Communications and Computers. 2012, 173-177.) combined decision tree, Bayesian classifier and support vector machine algorithm to train a new classifier based on belonging voting, but did not obtain better generalization. The present invention reduces the sensitivity to noise and outliers by enhancing the differences between basic classifiers, thereby improving the stability of the model. At the same time, when the differences between basic classifiers increase, since diversity helps to reduce the model's over-learning of local patterns in specific noise or training data, the integrated model proposed by the present invention is less likely to overfit the training data, thereby improving the generalization ability of new data.

[0018] (3) The system architecture of the present invention is clear, simple and easy to implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the present invention.

[0020] Figure 2 This is the dataset used in the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings. Figure 1 The present invention provides a method for predicting heart disease based on feature combination and ensemble learning, using a dataset refer to Figure 2, the detailed steps are as follows:

[0022] Step 1: Unify the dimensions and preprocess the missing values ​​of the public heart disease dataset and divide the dataset into a training set and a test set, with 80% of the data used for training and 20% for testing;

[0023] Step 2: Combine the features of the processed data set using a feature combination algorithm to generate a new feature matrix;

[0024] Step 3: Use the perturbation-based difference enhancement algorithm to process the new feature matrix and generate different training subsets;

[0025] Step 4: Use different training subsets as inputs for multiple basic classifiers for training;

[0026] Step 5: Integrate the prediction results of each basic classifier to achieve the final classification prediction;

[0027] Step 6: Verify model generalization and accuracy.

[0028] The process of preprocessing the open-source heart disease dataset in the first step is as follows: For the dataset used in the present invention, there are 303 rows and 14 columns, of which the number of major blood vessels detected by fluorescent staining (ca column) and thalassemia (thal column) have 4 and 2 missing values, respectively. Since the sample size is not particularly large, directly deleting the sample may lead to erroneous conclusions. Although ca and thal are numerical variables, they are regarded as categorical variables, so the mode of the values ​​before and after the missing values ​​is used to fill them.

[0029] Since each feature attribute has different dimensions, it is necessary to perform numerical normalization on the continuously changing numerical data to eliminate the influence of different dimensions on the comprehensive performance of each attribute. The present invention adopts maximum and minimum value normalization to change the numerical range of each numerical feature to between 0 and 1. For a certain dimension of data, normalization can be performed according to the formula:

[0030]

[0031] Among them, x represents the data before the maximum and minimum values ​​are normalized, and y represents the data after the maximum and minimum values ​​are normalized. min Indicates the minimum value of the dimension data, x max Indicates the maximum value of the dimension data.

[0032] The feature combination algorithm of step 2 is as follows: each feature in the initial data set sample containing n columns of features is traversed step by step, and the features are associated and combined based on the Spearman correlation coefficient, so as to obtain an n×n×3-dimensional feature matrix X′. In X′, not only the original features are retained, but also the features after the features are combined by feature correlation between the two features; the specific algorithm flow is as follows:

[0033] Input: original sample data X, number of features T max

[0034] Output: Feature combination, dimension-enhanced data X'

[0035] 1) Initialize i, j = 1

[0036] 2) While i <T max

[0037] 3) Take the i-th feature x from X i

[0038] 4) Whilej <T max do

[0039] 5) Take the jth feature x from X j

[0040] 6)X i ′ ,j,0 =x i

[0041] 7)X i ′ ,j,1 =x j

[0042] 8)

[0043] 9) end while

[0044] 10) end while

[0045] 11) Output the combined feature X′

[0046] Among them: X represents the original low-dimensional heart disease sample data, including age, gender, blood pressure, etc.; X′ represents the combined high-dimensional data; x i and x j denote the eigenvalues ​​of the i-th and j-th columns respectively; ρ i,j represents the feature correlation value between the feature in the i-th column and the feature in the j-th column, such as the feature correlation value between age and blood pressure, and the value range is [-1, 1]; X i ′,j,0 represents the first channel of the combined feature, X i ′ ,j,1 represents the second channel of the combined feature, X i ′ ,j,2 Represents the third channel of the combined features. The above Spearman correlation coefficient is:

[0047]

[0048] In the above formula (2), d i It represents the difference between the positions of two variables after they are sorted in the same order, and N represents the number of samples.

[0049] Step 3 uses data sample perturbation and input attribute perturbation to enhance the difference; it is divided into two steps:

[0050] (1) First, based on data sample perturbation, an original data set D containing N samples and M features is randomly sampled with replacement: N samples are randomly sampled with replacement from the original data set, which means that some samples may be selected multiple times in the sampling, while other samples may not be selected once. This sampling forms a new subset, recorded as sample subset D1. The random sampling of the present invention uses the random.choice function of the NumPy library in Python, where replace = True indicates sampling with replacement.

[0051] (2) Based on the input attribute perturbation, the sample subset D1 is sampled based on the features, and M features are randomly sampled with replacement to generate a new subset D2 for the final basic classifier training.

[0052] (3) Repeat (1) and step (2) multiple times to generate multiple different training subsets for training different basic classifiers.

[0053] Note: The random sampling method with replacement involved in steps (1) and (2) uses the random.choice function of the NumPy library in Python, where the parameter replace is set to True, indicating sampling with replacement.

[0054] The specific operations of step four are as follows: build the scikit-learn machine learning framework on Windows, use multiple different prediction algorithms such as support vector machine algorithm, logistic regression algorithm, decision tree algorithm, etc. for testing, and find a suitable basic classifier based on prediction accuracy; after testing, it is finally determined to establish heart disease prediction models based on support vector machine algorithm, naive Bayes algorithm, and decision tree algorithm, and use multiple data subsets generated in the fourth step for training, parameter adjustment and optimization.

[0055] Step five is the integration stage of the basic classifier, which is specifically implemented as follows: calling the VotingClassifier integration method library in the scikit-learn machine learning framework, and using the "voting method" for the integration of the basic classifier. Each classifier will vote for a category label, and the final prediction result is determined by majority voting. The final output category label is the label that obtains more than half of the votes.

[0056] Step 6: Use the common quantitative evaluation indicators of the prediction model performance in medical diagnosis, namely accuracy A (Accuracy), recall R (Recall), and precision P (Precision), to verify the generalization and accuracy of the model.

[0057] The above method was simulated and compared with common heart disease prediction methods. The results are as follows:

[0058]

[0059] Description: DT (Decision Tree), SVM (Support Vector Machine), NB (Naive Bayes), DE-EL (Differences Enhanced-Ensemble Learning), an ensemble learning model based on difference enhancement, i.e., the innovative method proposed in the present invention.

[0060] Analysis of the experimental results shows that the accuracy, precision and recall of the method proposed in the present invention are greatly improved compared with the traditional method.

[0061] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A heart disease prediction method based on feature combination and ensemble learning, characterized in that: The following steps are involved: Step 1: Unify the dimensions and process missing values ​​of the public heart disease dataset; Step 2: Combine the features of the processed data set using a feature combination algorithm to generate a new feature matrix; Step 3: Use the perturbation-based difference enhancement algorithm to process the new feature matrix to generate different sub-feature matrices; Step 4: Use different sub-feature matrices as inputs for training three basic classifiers; Step 5: The prediction results of each basic classifier are integrated to achieve the final classification prediction; Step 6: Verify model generalization and accuracy.

2. The method for predicting heart disease based on feature combination and ensemble learning according to claim 1, characterized in that: In step 1, the public heart disease dataset is dimensionally unified and missing values ​​are processed. Specifically, the missing data are filled using the mode of the values ​​before and after the missing value. The maximum and minimum values ​​are normalized to change the value range of each numerical feature to between 0 and 1.

3. The method for predicting heart disease based on feature combination and ensemble learning according to claim 1, characterized in that: In step 2, the feature combination algorithm is as follows: each feature in the initial data set sample containing n columns of features is traversed step by step, and the features are associated and combined with each other based on the Spearman correlation coefficient, so as to obtain an n×n×3-dimensional feature matrix X′; in X′, not only the original features are retained, but also the features after the features are combined with each other through feature correlation; The Spearman correlation coefficient above is: Among them, d i It represents the difference between the positions of two variables after they are sorted in the same order, and N represents the number of samples.

4. The method for predicting heart disease based on feature combination and ensemble learning according to claim 1, characterized in that: Steps three and four are as follows: difference enhancement is performed using data sample perturbation and input attribute perturbation. First, based on data sample perturbation, the training samples are sampled based on the number of samples to generate different subsets D1. Then, based on input attribute perturbation, subset D1 is sampled based on features, and sub-attributes of the same proportion are extracted to generate a new attribute subset D2. Three subsets are generated in this way, and finally a basic classifier is trained based on each subset.

5. The method for predicting heart disease based on feature combination and ensemble learning according to claim 1, characterized in that: In step 5, the "voting method" is used for the integration of basic classifiers. Each basic classifier will vote for a category label, and the final output category label is the label that obtains more than half of the votes.

6. The method for predicting heart disease based on feature combination and ensemble learning according to claim 1, characterized in that: In step six, indicators for quantitatively evaluating the performance of prediction models in medical diagnosis are used to verify the generalization and accuracy of the model, including accuracy A, recall R, and precision P.

Images