A hypertension nephropathy prediction method based on multi-model fusion for feature screening

Abstract

The present invention provides a method for predicting hypertensive nephropathy by screening features based on multi-model fusion. An important feature set affecting the occurrence of hypertensive nephropathy is screened out by a multi-model fusion method, and a multi-model fusion prediction model for hypertensive nephropathy is established using this feature set. Early screening work for hypertensive patients is realized, so as to detect the risk of hypertensive nephropathy as early as possible, provide timely treatment, and improve the prognosis of patients.

Description

Technical Field

[0001] The present invention relates to the technical field of hypertension nephropathy prediction, and particularly relates to a hypertension nephropathy prediction method based on multi-model fusion for feature screening. Background Art

[0002] Hypertension is a syndrome manifested by elevated systemic arterial blood pressure, and its occurrence is related to lifestyle, eating habits and living environment. Long-term hypertension can cause arterial lesions and damage target organs, among which hypertension nephropathy is relatively common. The prevalence of hypertension nephropathy shows an increasing trend year by year. Therefore, hypertension nephropathy is one of the main causes of ESRD. In the initial stage of hypertension nephropathy, there may be no symptoms, and during the disease process, there may be manifestations such as microalbuminuria and increased serum creatinine, which can increase the risks of end-stage renal disease, cardiovascular adverse events and sudden death, posing a serious threat to the life safety of patients. Therefore, it is crucial to conduct early screening, diagnosis and treatment for hypertensive patients.

[0003] In the prior art, on the one hand, when using clinical relevant information to predict the occurrence of hypertension nephropathy, directly using medical features for hypertension nephropathy prediction, or using medical statistical methods and single machine learning algorithms to select risk factors affecting the occurrence of hypertension nephropathy. However, the influencing factors are multi-dimensional, and some influencing factors have little impact on the occurrence of hypertension nephropathy. Therefore, when predicting based on all medical features, there will be problems of large computational volume, and there will also be difficulties in data collection during screening applications. And using a single method to select important features is one-sided. On the other hand, for predicting hypertension nephropathy in hypertensive populations, most current studies use traditional statistical methods and traditional machine learning such as support vector machine (SVM), random forest (RF), etc. Single prediction models have problems such as low prediction accuracy and weak generalization ability. Therefore, it is necessary to improve and fuse existing prediction models to improve the prediction accuracy and generalization ability of hypertension nephropathy. Summary of the Invention

[0004] The purpose of the present invention is to solve the disadvantages existing in the prior art, and to propose a hypertension nephropathy prediction method based on multi-model fusion for feature screening.

[0005] To achieve the above purpose, the present invention adopts the following technical scheme: A hypertension nephropathy prediction method based on multi-model fusion for feature screening, the specific steps of the prediction method are as follows:

[0006] S1: Collect relevant information of hypertensive populations, including hypertensive nephropathy patients and non-hypertensive nephropathy patients, and preprocess the relevant information to obtain initial data;

[0007] The relevant information covers demographic characteristics, physical measurement characteristics, examination and inspection characteristics, symptom characteristics, family history, and past history;

[0008] Data preprocessing includes data anomalies, data missing, data inconsistency, data incompleteness, and data non-standardization;

[0009] S2: Obtain K sample sets through the K-fold cross-validation method, and divide the training data set and the test data set respectively;

[0010] Divide the initial data obtained in step S1 into K sample subsets of equal size, traverse these K subsets in turn, and each time use the current subset as the test set and all the remaining samples as the training set for model training and evaluation; the above operations form K sample sets A1, A2... A k ;

[0011] S3: Use the multi-model fusion method to obtain the important feature set;

[0012] On the K sample sets obtained in step S2, use the multi-model fusion method of five machine learning algorithms, namely decision tree, random forest, XGBoost, AdaBoost, and GBDT, to obtain the important feature set;

[0013] S4: Select the important feature set data composed of N important features in step S3, and use the decision tree algorithm, random forest algorithm, XGBoost algorithm, AdaBoost algorithm, and GBDT algorithm respectively to train the five algorithms using the training set;

[0014] Obtain the trained decision tree prediction model, random forest prediction model, XGBoost prediction model, AdaBoost prediction model, and GBDT prediction model;

[0015] S5: Use the test data set as the input of the five trained models to obtain the prediction output results of the five models and perform weighted processing; integrate the weighted prediction output results in the K sample sets and divide them into training data set one and test data set one;

[0016] S6: Construct a BP neural network and train the BP neural network with the training data set one in step S5 to obtain the trained BP neural network;

[0017] S7: Use the test data set one as the input of the trained BP neural network to obtain the prediction output result one of the BP neural network;

[0018] S8: Compare the predicted output result 1 with the true values in the test data set 1, and determine whether the fusion model composed of the decision tree model, random forest model, XGBoost model, AdaBoost model, GBDT model, and BP neural network meets the prediction accuracy requirements according to the comparison result:

[0019] If the prediction accuracy requirements are not met, repeat steps S4 - S8 or repeat steps S6 - S8;

[0020] If the prediction accuracy requirements are met, use the fusion model to predict hypertensive populations.

[0021] Preferably, in step S3, a multi - model fusion method of five machine learning algorithms is used to obtain an important feature set:

[0022] S31: For each sample set A1, A2... A k , apply five machine learning algorithms of decision tree, random forest, XGBoost, AdaBoost, and GBDT to respectively obtain the relative importance score and average absolute SHAP value of each feature;

[0023] The importance scores and average absolute SHAP values of m features calculated using the above 5 machine learning algorithms form matrix A;

[0024] S32: According to the AUC values of the five prediction models, perform weighted processing on matrix A to obtain weighted relative importance scores and weighted average absolute SHAP values, forming matrix A1;

[0025] S33: Normalize the calculated weighted relative importance scores and weighted average absolute SHAP values to obtain normalized index values B;

[0026] S34: Based on the normalized index values B, calculate the mean to obtain the mean index C of each feature;

[0027] S35: Based on each sample set A1, A2... A in K data sets k , obtain the total mean index of each feature; that is, the results of K similar C matrices are summarized in the form of matrix D; and calculate the mean of each row of matrix D to obtain the total mean index E of each feature;

[0028] S36: In the total mean index E of each feature above, obtain an important feature set according to the preset number N of important features.

[0029] Preferably, in step S1, the data pre - processing includes the following specific operations:

[0030] S11: Identify and process outliers in the data, and use statistical methods (such as Z-score or IQR) to remove or correct outliers;

[0031] S12: Handle missing data, and use methods such as interpolation, mean filling, or KNN filling to fill in the missing values;

[0032] S13: Standardize data inconsistencies and irregularities, including encoding of categorical variables and standardization of numerical variables;

[0033] S14: Perform unified format conversion on data from different data sources to ensure data consistency.

[0034] Preferably, in step S8, the prediction accuracy includes multiple indicators, accuracy, recall rate, precision, F1-score, and the area under the ROC curve.

[0035] Preferably, in step S6, the structural optimization of the BP neural network includes using techniques such as Dropout and BatchNormalization to improve the generalization ability of the model.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The prediction method of the present invention obtains an important feature set based on five machine learning model fusion methods, re-uses the K-fold cross-validation method, and uses a multi-model fusion learning method based on five machine learning algorithms to establish a screening model for hypertensive nephropathy. This feature screening method based on multi-model fusion finds important features to establish a model, which is more conducive to improving the rationality and interpretability of the model.

[0037] 2. The K data sets generated by using the K-fold cross-validation method traverse more data combinations, making the selected important features universal and representative, and improving the generalization ability of the screening model.

[0038] 3. Different models have different adaptabilities to data. By fusing multiple models, it is possible to better cope with data diversity, improve the robustness of the model in different situations, integrate the advantages of different models, and improve performance and accuracy.

[0039] 4. The method proposed in this application can quickly obtain important features, reducing the time and labor costs required for modeling, facilitating improving the prediction efficiency and prediction accuracy of the model, and being able to reduce the cost of collecting data and improve the disease screening efficiency in screening applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of a prediction method for hypertensive nephropathy based on multi-model fusion feature screening;

[0041] Figure 2Flow chart for obtaining an important feature set by a multi-model fusion method. Detailed implementation manners

[0042] To further understand the purpose, structure, features and functions of the present invention, the following is a detailed description in conjunction with embodiments.

[0043] Please refer to Figure 1 and Figure 2 , the present invention provides a hypertension nephropathy prediction method for screening features based on multi-model fusion. The specific steps of the prediction method are as follows:

[0044] S1: Collect relevant information of hypertensive patients, including patients with hypertensive nephropathy and non-hypertensive nephropathy patients, and preprocess the relevant information to obtain initial data;

[0045] The relevant information covers demographic characteristics, physical measurement characteristics, examination and inspection characteristics, symptom characteristics, family history and past history;

[0046] Data preprocessing includes data anomalies, data missing, data inconsistency, data incompleteness and data non-standardization;

[0047] S11: Identify and process outliers in the data, and use statistical methods (such as Z-score or IQR) to remove or correct outliers;

[0048] S12: Process missing data, and use methods such as interpolation, mean filling or KNN filling to fill in the missing values;

[0049] S13: Standardize data inconsistency and non-standardization, including encoding of categorical variables and standardization of numerical variables;

[0050] S14: Perform unified format conversion on data from different data sources to ensure data consistency.

[0051] S2: Obtain K sample sets through the K-fold cross-validation method, and divide the training data set and the test data set respectively;

[0052] Divide the initial data obtained in step S1 into K sample subsets of equal size, traverse these K subsets in turn, and each time use the current subset as the test set and all other samples as the training set for model training and evaluation; the above operations form K sample sets A1, A2...A k ;

[0053] S3: Use a multi-model fusion method to obtain an important feature set;

[0054] On the K sample sets obtained in step S2, five machine learning multi-model fusion methods, including decision tree, random forest, XGBoost, AdaBoost, and GBDT, are used to obtain important feature sets;

[0055] S31: For each sample set A1, A2...A k , five machine learning algorithms, including decision tree, random forest, XGBoost, AdaBoost, and GBDT, were used to obtain the relative importance score and mean absolute SHAP value of each feature;

[0056] The relative importance score is calculated using the scikit-learn library in Python to train machine learning models (decision trees, random forests, XGBoost, AdaBoost, GBDT), where the feature_importances_ attribute provides a direct quantitative indicator (feature importance score) to measure the influence of each feature on the model's prediction results.

[0057] The relative average absolute SHAP value is calculated using TreeExplainer in the SHAP library to calculate the SHAP value of the test set samples, showing the contribution of each feature to the prediction results. The importance of each feature is obtained by taking the average of the absolute values of the SHAP value samples;

[0058] Table 1 Eigenvalues in each algorithm

[0059]

[0060] Among them, ai1 represents the relative importance score of the ith feature calculated using the decision tree algorithm, and there are m medical feature variables in total. The same applies to other variables.

[0061] The importance scores and mean absolute SHAP values of the m features calculated using the above five machine learning algorithms form the matrix A;

[0062]

[0063] aij represents the importance score or average absolute SHAP value of the i-th feature calculated using the above five machine learning algorithms, where i is a natural number from 1 to m and j is a natural number from 1 to 10.

[0064] S32: According to the AUC values of the five prediction models, the above matrix A is weighted to obtain a weighted relative importance score and a weighted average absolute SHAP value to form a matrix A1;

[0065]

[0066] Among them, w1 represents the AUC value of the decision tree model on the test set, w2 represents the AUC value of the random forest model on the test set, w3 represents the AUC value of the XGBoost model on the test set, w4 represents the AUC value of the AdaBoost model on the test set, and w5 represents the AUC value of the GBDT model on the test set.

[0067] S33: Normalize the calculated weighted relative importance scores and weighted average absolute SHAP values to obtain the normalized index value B;

[0068] Using Min - Max normalization, also known as linear normalization or maximum - minimum normalization, it scales the data into the interval [0, 1]. This is one of the simplest normalization methods; formula:

[0069] ;

[0070] where X is the original data, Xmin and Xmax are the minimum and maximum values in the dataset respectively, and Xnorm is the normalized data.

[0071] The A1 matrix is transformed into the normalized index value B through the above formula;

[0072] ;

[0073] where , i.e., the minimum value in the j - th column of the A1 matrix, i.e., the maximum value in the j - th column of the A1 matrix.

[0074] S34: Based on the normalized index value B, calculate the mean to obtain the mean index C of each feature;

[0075]

[0076] S35: Based on each sample set A1, A2...A k in each of the K datasets, obtain the total mean index of each feature; that is, the results of K similar C matrices are summarized in the form of a D matrix;

[0077]

[0078] where ciq represents the mean index of the i - th feature in the Aq sample set. Here, i takes natural numbers from 1 to m, and q takes natural numbers from 1 to K.

[0079] And calculate the mean of each row of the D matrix, that is, obtain the total mean index E of each feature;

[0080]

[0081] Among them, ei represents the total mean index of feature i.

[0082] S36: Among the total mean indexes E of the above-mentioned features, according to the preset number N of important features, an important feature set is obtained.

[0083] In the embodiment of the present invention, the feature selection is the selection of the number of important features. The total mean index values of each feature are sorted in descending order; the candidate features corresponding to the first N total mean index values before sorting are intercepted to obtain an important feature set, where N is the number corresponding to the quantity selection strategy.

[0084] The selection of N is adjusted according to the accuracy of the final fusion model.

[0085] S4: For the important feature set data composed of N important features selected in step S3, the decision tree algorithm, random forest algorithm, XGBoost algorithm, AdaBoost algorithm, and GBDT algorithm are respectively used to train the five algorithms using the training set.

[0086] A trained decision tree prediction model, random forest prediction model, XGBoost prediction model, AdaBoost prediction model, and GBDT prediction model are obtained.

[0087] Using the sklearn package of the python development language, five algorithms of decision tree, random forest, XGBoost, AdaBoost, and GBDT are selected to establish a screening model for hypertensive nephropathy respectively; an appropriate screening parameter set capable of discriminating hypertensive nephropathy is established, and the data is trained and iteratively trained in the five models, and the screening parameter set for hypertensive nephropathy is optimized. Finally, an appropriate screening parameter set for hypertensive nephropathy capable of discriminating hypertensive nephropathy is obtained. The optimal hyperparameter combination of each model during the training process is obtained by the grid search method.

[0088] S5: The test data set is used as the input of the five trained models, and the prediction output results of the five models are obtained and weighted; the weighted prediction output results in the K sample sets are integrated and divided into a training data set one and a test data set one;

[0089] In this embodiment, the sample subsets A1, A2... A k In the sample subset, the prediction output results m1, m2, m3, m4, and m5 of the five models on the test data set (n rows of data) are integrated into an n-row and 5-column data Mn*5, and then weighted calculation is performed according to the number of samples correctly predicted by each model. Similarly, for the K sample sets, that is, the K results are integrated into a K*n-row and 5-column data as the new feature set P, and it is divided into a training data set one and a test data set one.

[0090] The output result m1: the prediction result of the decision tree prediction model on the test data set, where mij is the data in the i-th row, and the probability value of the true label 1 (hypertensive nephropathy) of the output sample is in the range of 0 to 1, and the values of m2, m3, m4, and m5 are similar;

[0091]

[0092] The Mn*5 matrix is:

[0093]

[0094] The Nn*5 matrix is:

[0095]

[0096] Where w1 is the number of samples correctly predicted by the decision tree model / the sum of the number of samples correctly predicted by the above five models*100%. w2, w3, w4, w5 and so on. For K sample sets, K N are generated, namely: N1, N2, N3...NK.

[0097] The P matrix is:

[0098] .

[0099] S6: constructing a BP neural network, and training the BP neural network with the training data set 1 in step S5 to obtain a trained BP neural network;

[0100] The BP algorithm is an error back propagation algorithm. It is mainly divided into two parts: forward propagation and back propagation. In forward propagation, the output of each neuron is calculated layer by layer from the input layer to the output layer through the weighted connection between neurons to obtain the predicted value. In back propagation, the error of each layer is calculated based on the difference between the predicted value and the true value, and the parameters of the network are adjusted by the gradient descent method to minimize the error. This process is repeated many times until the error of the network is less than a certain threshold or other termination conditions are met.

[0101] The present invention adopts BP neural network to fuse the prediction outputs of decision tree model, random forest model, XGBoost model, AdaBoost model and GBDT model, and can use its excellent fitting ability to fuse model results and output more accurate prediction values. At the same time, its simple structure will not increase the complexity of the model, so that the fusion model has higher stability.

[0102] S7: using the test data set 1 as the input of the trained BP neural network to obtain a prediction output result 1 of the BP neural network;

[0103] S8: Compare the first predicted output result with the true values in the first test data set, and determine whether the fusion model composed of the decision tree model, random forest model, XGBoost model, AdaBoost model, GBDT model, and BP neural network meets the prediction accuracy requirement according to the comparison result:

[0104] If the prediction accuracy requirement is not met, repeat steps S4 - S8 or repeat steps S6 - S8;

[0105] If the prediction accuracy requirement is met, use the fusion model to predict hypertensive patients.

[0106] In a specific embodiment, when comparing the first predicted output result with the true values in the first test data set, the AUC value is used as the evaluation index.

[0107] In this embodiment, an AUC value prediction error threshold is set according to the prediction accuracy requirement. When the AUC value of the trained fusion model is greater than or equal to the AUC value prediction error threshold, the fusion model is the final fusion model.

[0108] Regarding the above-mentioned AUC value, AUC is the area under the ROC curve. The ROC curve uses the true positive rate (TPR) and false positive rate (FPR) of the hypertensive nephropathy screening model in the test data as the horizontal and vertical coordinates to describe the performance of the screening model at different thresholds; the AUC value is used to measure the performance of the screening model. The closer the AUC value is to 1, the better the performance of the screening model; conversely, the closer the AUC value is to 0, the worse the performance of the screening model. Through the ROC curve, we can intuitively understand the performance of the screening model at different thresholds; and through the AUC value, we can quantitatively evaluate the overall performance of the screening model.

[0109] Preferably, in step S8, the prediction accuracy includes multiple indicators, accuracy, recall rate, precision, F1-score, and the area under the ROC curve.

[0110] Accuracy is the ratio of the number of correctly predicted samples to the total number of samples. Its calculation formula is: Accuracy = Number of correctly predicted samples / Total number of samples;

[0111] Recall rate, also known as true positive rate, represents the proportion of positive samples that the model can correctly identify among all positive samples. Its calculation formula is: Recall rate = Number of true samples / (Number of true samples + Number of false negative samples);

[0112] Precision refers to the proportion of samples predicted as positive samples by the model that are actually positive samples. Its calculation formula is: Precision = Number of true positive samples / (Number of true positive samples + Number of false positive samples);

[0113] F1-score is the harmonic mean of precision and recall, and is often used to balance the weights of the two. Its calculation formula is: F1 = 2×(Precision × Recall) / (Precision + Recall);

[0114] The ROC curve (Receiver Operating Characteristic Curve) plots the relationship between the false positive rate (FPR) and the true positive rate (TPR). By changing the classification threshold, different FPRs and TPRs can be obtained, thus forming a curve.

[0115] AUC (Area Under the Curve) is the area under this curve, and its value range is between 0 and 1. AUC = 1: Perfect model. AUC = 0.5: Random guessing model. AUC < 0.5: The model performs worse than random guessing.

[0116] Preferably, in the step S6, the structural optimization of the BP neural network includes adopting techniques such as Dropout and BatchNormalization to improve the generalization ability of the model.

[0117] The present invention has been described by the above related embodiments. However, the above embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, modifications and refinements made without departing from the spirit and scope of the present invention fall within the scope of patent protection of the present invention.

Claims

1. A method for predicting hypertensive nephropathy by screening features based on multi-model fusion, characterized in that: The specific steps of the above-mentioned prediction method are as follows: S1: Collect relevant information of hypertensive patients, including patients with hypertensive nephropathy and non-hypertensive nephropathy patients, and preprocess the relevant information to obtain initial data; The relevant information covers demographic characteristics, physical measurement characteristics, test and examination characteristics, symptom characteristics, family history, and past history; Data preprocessing includes data anomalies, data missing, data inconsistency, data incompleteness, and data non-standardization; S2: Obtain K sample sets through the K-fold cross-validation method, and divide the training data set and the test data set respectively; The initial data obtained through step S1 is divided into K sample subsets of equal size. These K subsets are traversed in sequence. Each time, the current subset is used as the test set, and all the remaining samples are used as the training set for model training and evaluation. The above operations form K sample sets A1, A2... A k ; S3: Use the multi-model fusion method to obtain the important feature set; On the K sample sets obtained in step S2, use the multi-model fusion method of five machine learning algorithms, namely decision tree, random forest, XGBoost, AdaBoost, and GBDT, to obtain the important feature set; S4: Select the data of the important feature set composed of N important features in step S3, and use the decision tree algorithm, random forest algorithm, XGBoost algorithm, AdaBoost algorithm, and GBDT algorithm respectively to train the five algorithms using the training set; Obtain the trained decision tree prediction model, random forest prediction model, XGBoost prediction model, AdaBoost prediction model, and GBDT prediction model; S5: Use the test data set as the input of the five trained models to obtain the prediction output results of the five models and perform weighted processing; Integrate the weighted prediction output results in the K sample sets, and divide them into training data set one and test data set one; S6: Construct a BP neural network, and train the BP neural network with the training data set one in step S5 to obtain a trained BP neural network; S7: Use the test data set one as the input of the trained BP neural network to obtain the prediction output result one of the BP neural network; S8: Compare the prediction output result one with the true value in the test data set one, and judge whether the fusion model composed of the decision tree model, random forest model, XGBoost model, AdaBoost model, GBDT model, and BP neural network meets the prediction accuracy requirements according to the comparison result: If the prediction accuracy requirements are not met, repeat steps S4-S8 or repeat steps S6-S8; If the prediction accuracy requirements are met, use the fusion model to predict hypertensive patients.

2. The method for predicting hypertensive nephropathy by screening features based on multi-model fusion according to claim 1, wherein: In step S3, the multi-model fusion method of five machine learning algorithms is used to obtain the important feature set as follows: S31: For each sample set A1, A2... A k , use five machine learning algorithms: decision tree, random forest, XGBoost, AdaBoost, and GBDT to obtain the relative importance score and average absolute SHAP value of each feature respectively; Use the importance scores and average absolute SHAP values of m features calculated by the above five machine learning algorithms to form matrix A; S32: According to the AUC values of the five prediction models, perform weighted processing on matrix A to obtain weighted relative importance scores and weighted average absolute SHAP values, and form matrix A1; S33: Normalize the calculated weighted relative importance scores and weighted average absolute SHAP values to obtain the normalized index value B; S34: Based on the normalized index value B, calculate the mean value to obtain the mean index C of each feature; S35: Based on each sample set A1, A2... A in K data sets k , the total mean index of each feature is obtained; that is, the results of K similar C matrices are summarized in the form of a D matrix; and the mean of each row of the D matrix is calculated to obtain the total mean index E of each feature; S36: Among the overall mean indexes E of the above features, obtain the set of important features according to the preset number N of important features.

3. The method for predicting hypertensive nephropathy based on multi-model fusion feature screening according to claim 1, wherein: In step S1, the data preprocessing includes the following specific operations: S11: Identify and process the outliers in the data, and use statistical methods to eliminate or correct the outliers; S12: Process the missing data, and use methods such as interpolation, mean filling or KNN filling to fill the missing values; S13: Standardize the inconsistent and non-standard data, including the encoding of categorical variables and the standardization of numerical variables; S14: Perform unified format conversion on the data from different data sources to ensure data consistency.

4. The method for predicting hypertensive nephropathy based on multi-model fusion for feature screening according to claim 1, characterized in that: In step S8, the prediction accuracy includes multiple indexes, such as accuracy, recall rate, precision rate, F1-score and the area under the ROC curve.

5. The method for predicting hypertensive nephropathy based on multi-model fusion feature screening according to claim 1, wherein: In step S6, the structure optimization of the BP neural network includes adopting techniques such as Dropout and Batch Normalization to improve the generalization ability of the model.

Images