Ischemic stroke prediction method based on ensemble learning
By using a two-layer ensemble learning model, combining genetic algorithms and stacked ensemble models, the problem of insufficient model accuracy and generalization ability in ischemic stroke risk prediction is solved, achieving higher prediction accuracy and lower generalization error.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DALIAN UNIV OF TECH
- Filing Date
- 2022-11-07
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies lack model accuracy and generalization ability in ischemic stroke risk prediction, making it difficult to effectively uncover the interrelationships between features. The accuracy and generalization ability of traditional ensemble learning methods need to be improved.
A two-layer ensemble learning model is adopted. First, a genetic algorithm is used to select features and generate multiple base classifiers. Then, in the second layer, a stacked ensemble model is used to fuse the prediction results of multiple base classifiers. The model parameters are optimized by using a loss function and a penalty term to improve the prediction accuracy.
It effectively removes noise features, uncovers factors highly correlated with ischemic stroke, simplifies computational overhead, improves model prediction accuracy, and reduces generalization error.
Smart Images
Figure CN116092676B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for predicting ischemic stroke based on ensemble learning, belonging to the field of data mining. Background Technology
[0002] Ischemic stroke, commonly known as "cerebral infarction," is a cerebrovascular disease that easily leads to the interruption of blood flow in cerebral arteries, causing local brain tissue hypoxia and ischemic necrosis, resulting in corresponding neurological deficits. This disease is an acute cerebrovascular disease and ranks first among the causes of death and disability in adults in my country, characterized by high incidence, high morbidity, high mortality, high recurrence rate, and numerous complications. Because ischemic stroke is irreversible, difficult to cure, and has high nursing costs, the medical burden is showing an increasing trend. Therefore, conducting research on the prevention, diagnosis, treatment, and prognosis of this disease is of great significance for providing precision medicine and clinical and community care. With the vigorous development of AI technology in medicine, many researchers at home and abroad have begun to use machine learning methods to conduct research on the screening and prevention of ischemic stroke to assist in clinical treatment.
[0003] In recent years, intelligent medical research based on big data and AI has received increasing attention. Research in the area of ischemic stroke risk prediction focuses more on the tracking and processing of patients' electronic health records, striving to build high-precision ischemic stroke risk prediction models through data mining and machine learning. Regarding improving model accuracy, the Devaki team, in their paper "An Ensemble Framework for Improving Brain Stroke Prediction Performance," points out that feature selection and ensemble learning play crucial roles from both data and model perspectives. Feature selection can eliminate irrelevant or redundant features, thereby improving model accuracy and reducing runtime. Ensemble learning combines multiple weakly supervised models to obtain a better and more comprehensive strongly supervised model. Zhou, in his paper "Ensemble Learning," mentions that the underlying idea of ensemble learning is that even if one weak classifier makes an incorrect prediction, other weak classifiers can correct the error.
[0004] Specifically, ensemble methods can be categorized into three types: Bagging, Boosting, and Stacking. Bagging trains multiple weak learners simultaneously, while Boosting iteratively trains on a single weak learner, retaining the results from each generation for ensemble. Both generally use a majority vote of all weak learners' results as the final prediction. Ashley et al., in their paper "Stacked generalization: an introduction to super learning," mentioned that Stacking is similar to a multi-layer neural network, where one model is trained to combine other models. In Stacking, multiple different models are first trained, and then the outputs of the previously trained models are used as input to train a single model to obtain a final output. This method is not limited to the two-layer structures of Bagging and Boosting, and each layer can learn, offering greater flexibility. It also typically considers weakly supervised models with low correlation, improving accuracy while reducing generalization error.
[0005] In the research on high-precision models for ischemic stroke prediction based on ensemble learning, domestic work is still very limited. Therefore, proposing a solution to this problem has great practical significance. Summary of the Invention
[0006] To effectively improve the accuracy of prediction models, this invention proposes an ensemble learning-based method for predicting ischemic stroke. The method first imputes missing values and encodes categorical data into integers for Electrical Health Record (EHR) data. Then, the data is copied k times, and k base classifiers are selected as the first layer of the ensemble model. Feature selection based on a genetic algorithm is performed on each data set, and the results are used to train the corresponding base classifier. After this process, the prediction results of the k base classifiers are used as k evaluation metrics as input data for the second layer of the ensemble model—the ensemble classifier—and this data is used to train the ensemble classifier. Finally, a two-layer ensemble model is obtained to predict the risk of ischemic stroke.
[0007] The technical solution of the present invention:
[0008] An ensemble learning-based method for predicting ischemic stroke, comprising the following steps:
[0009] (1) Data cleaning stage:
[0010] (1.1) Impute missing values in the EHR dataset using the following formula:
[0011]
[0012] Among them, v i Let be the value of the i-th sample without missing data, and N be the total number of samples without missing data.
[0013] (1.2) Replace the category data in the EHR dataset with 0, 1, 2, 3... in the order in which each category appears;
[0014] (1.3) Copy the processed EHR data into k copies and select k base learners to correspond to them;
[0015] (2) Feature selection stage:
[0016] (2.1) For any dataset EHR i We select f features to form a feature subset F. This selection method is represented by a vector, denoted as s. i ;
[0017]
[0018] (2.2) Repeat step (2.1) n times to obtain n feature subsets and selection methods. The selection method set is denoted as S.
[0019] (2.3) Use F to train the dataset EHR i The base learner corresponding to step (1.3) is used to obtain the base classifier M, and the fitness corresponding to this selection method is calculated. The fitness formula is as follows:
[0020]
[0021] Where fit(j) is the fitness of j selection methods, p i y is the predicted category of the i-th data point. i It is the actual category of the i-th data;
[0022] (2.4) Based on the retention probability of each selection method, create a selection roulette wheel and select n times to form the set of n selection methods for the next generation; the retention probability formula is as follows:
[0023]
[0024] Where P(i) is the i-th selection method s i The retention probability, where n is the total number of selection methods;
[0025] (2.5) Treat adjacent elements in S as a pair, and each element is paired only once. The two vectors in each pair have a certain probability of being interchanged, as follows:
[0026] a′=[a1,a2,…,b i ,bi+1 ,…,b j ,a j+1 ,…,a l ]
[0027] b′=[b1,b2,…,a i ,a i+1 ,…,a j ,b j+1 ,…,b l ]
[0028] Where a′ and b′ are the vectors obtained by exchanging vectors a and b, and a i b i It is the i-th dimension of vectors a and b before the swap;
[0029] (2.6) Suppose that each element in S has a certain probability of undergoing a mutation, such that a certain dimension of the vector changes from 0 to 1 or from 1 to 0.
[0030] (2.7) Select the feature subset F with the highest fitness. best and the corresponding base classifier M best ;
[0031] (2.8) Repeat steps (1.3)-(1.7) L times;
[0032] (2.9) From the final set of selection methods S, select the feature selection method S with the highest fitness. best The corresponding feature subset F best With base classifier M best ;
[0033] (2.10) Repeat steps (2.1)-(2.9) until all k classifiers have been trained;
[0034] (3) Integrated model generation
[0035] (3.1) For any feature subset F obtained in step (2.9) best In the corresponding base classifier M best Based on this training, the base classifier M of the ensemble model is obtained. * ;
[0036] (3.2) Repeat step (3.1) k times until all base classifiers have been trained;
[0037] (3.3) Using k base classifiers M * Make predictions for the original dataset separately and save the prediction results;
[0038] (3.4) Merge the results into a new dataset, as follows:
[0039] xi =[p 1i ,p 2i ,p 3i ,…,p ki ]
[0040] Where, x i p represents the i-th new data. ki This represents the prediction result of the k-th classifier for the i-th original data.
[0041] (3.5) Using this new data as input, calculate the disease risk value. The risk value formula is as follows:
[0042] f(x i )=ω T x i +b
[0043] Where, f(x) i ) is x i Risk value, ω T =[ω1,ω2,ω3,…,ω k ] and b are undetermined coefficients;
[0044] (3.6) Calculate the loss value of the model. The loss function is as follows:
[0045] L(f(x i ),y i )=max{0,1-y i f(x i )}
[0046] Among them, y i It is the true result of the i-th data;
[0047] (3.7) Calculate the penalty value for model complexity. The penalty function is as follows:
[0048]
[0049] Where ρ is a proportionality constant, set to 0.15; ω i Let ω be the i-th dimension of the vector.
[0050] (3.8) Based on the input data, ω is updated as follows:
[0051]
[0052] Where η is the learning rate, y i It is the true result of the i-th data;
[0053] (3.9) Based on the input data, b is updated as follows:
[0054]
[0055] (3.10) Repeat steps (3.5)-(3.9) t times to obtain the ensemble classifier;
[0056] (4) Stroke risk prediction
[0057] (4.1) After case data is input into the system, the data is transformed into a format that conforms to the system input.
[0058] (4.2) The data is copied into 4 copies and input into 4 base classifiers respectively to obtain 4 results. These results are then concatenated into a 4-dimensional vector w. The calculation method is as follows:
[0059] w = [p1, p2, p3, ..., p k ]
[0060] Where w represents a 4-dimensional vector, p k This represents the prediction result of the k-th classifier on the original data;
[0061] (4.3) Use the above x as input, calculate the disease risk value f(x) mentioned in step (3.5), and output it.
[0062] The beneficial effects of this invention are as follows: The diversity and complexity of the pathogenic factors of ischemic stroke make it impossible for traditional classifier models to fully explore the interrelationships between features. In view of this, the prediction model based on ensemble learning proposed in this method can effectively remove noisy features, explore factors that are highly correlated with ischemic stroke, and at the same time simplify computational overhead and improve model running efficiency.
[0063] Furthermore, traditional ensemble approaches like Bagging and Boosting integrate a single base learner and build the model through a voting mechanism, resulting in lower accuracy and generalization ability. Stacked ensemble models, however, combine multiple different base learners in the first layer, fully leveraging the strengths of each classifier. The second layer introduces learners instead of voting, allowing for a more organic combination of different classifiers. This makes the model more comprehensive, improving prediction accuracy while reducing generalization error. Attached Figure Description
[0064] Figure 1 This is a structural diagram of the prediction method described in this invention.
[0065] Figure 2 This is a flowchart of the data cleaning process described in this invention.
[0066] Figure 3 This is a flowchart illustrating the feature selection process described in this invention.
[0067] Figure 4This is a flowchart illustrating the generation of the integrated model described in this invention.
[0068] Figure 5 This is a flowchart illustrating the ischemic stroke risk prediction method described in this invention. Detailed Implementation
[0069] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below through embodiments and accompanying drawings.
[0070] An ensemble learning-based method for predicting ischemic stroke is proposed, comprising feature selection, ensemble model generation, and stroke risk prediction.
[0071] Reference Figure 2 The specific process of data cleaning is as follows:
[0072] Step 1. Impute missing values in the EHR dataset using the following formula:
[0073]
[0074] Where v i Let be the value of the i-th sample without missing data, and N be the total number of samples without missing data.
[0075] Step 2. Replace the category data in the EHR dataset with numbers 0, 1, 2, 3, etc., in the order in which each category appears.
[0076] Step 3. Copy the processed EHR data into k copies and select k base learners, each corresponding to a different dataset. In the experiment, k was 4.
[0077] Reference Figure 3 The specific process of data feature selection is as follows:
[0078] Step 4. Using one of the data sets, EHR i For example, select f features to form a feature subset F. This selection method is represented by a vector, denoted as s.
[0079]
[0080] Step 5. Repeat step 4 n times to obtain n feature subsets and selection methods. The set of selection methods is denoted as S. In the experiment, n is taken as 100.
[0081] Step 6. Use F to train the EHR dataset i The base learner from step 3 is used to obtain the base classifier M, and the fitness of the selection method is calculated. The fitness formula is as follows:
[0082]
[0083] Where fit(j) is the fitness of the j-th selection method, p i y is the predicted category of the i-th data point. i It is the actual category of the i-th data.
[0084] Step 7. Based on the retention probability of each selection method, create a selection roulette wheel, and select n times to form the next generation of n selection methods. The retention probability formula is as follows:
[0085]
[0086] Where P(i) is the i-th selection method S i The retention probability is given by n, where n is the total number of selection methods, and fit is as described in step 5.
[0087] Step 8. Treat adjacent elements in S as a pair, with each element paired only once. The two vectors in each pair have a certain probability of being interchanged; in the experiment, this probability is set to 0.5. The interchange method is as follows:
[0088] a′=[a1,a2,…,b i ,b i+1 ,…,b j ,a j+1 ,…,a l ]
[0089] b′=[b1,b2,…,a i ,a i+1 ,…,a j ,b j+1 ,…,b l ]
[0090] Where a′ and b′ are the vectors obtained by exchanging vectors a and b, and a i b i It is the i-th dimension of vectors a and b before the swap;
[0091] Step 9. Suppose that each element in S has a certain probability of undergoing a mutation, such that a certain dimension of the vector changes from 0 to 1 (or from 1 to 0). In the experiment, this probability is set to 0.05.
[0092] Step 10. Repeat steps 6–9 multiple times, set to 100 times in the experiment.
[0093] Step 11. On the final S, select the feature selection method S with the highest fitness. best The corresponding feature subset F best With base classifier M best .
[0094] Step 12. Repeat steps 4–11 until all k classifiers have been trained.
[0095] Reference Figure 4 The specific process of generating the ensemble model is as follows:
[0096] Step 13. Using one of the k feature subsets F obtained in Step 11 best For example, in the corresponding base classifier M best Based on this training, the base classifier M of the ensemble model is obtained. * .
[0097] Step 14. Repeat step 13 k times until all base classifiers have been trained.
[0098] Step 15. Use k base classifiers M * Make predictions for the original dataset separately and save the prediction results.
[0099] Step 16. Merge the results of the k base classifiers into a new dataset, as follows:
[0100] x i =[p 1i ,p 2i ,p 3i ,…,p ki ]
[0101] Where x i p represents the i-th new data. ki This represents the prediction result of the k-th classifier for the i-th original data.
[0102] Step 17. Using this new data as input, calculate the disease risk value. The risk value formula is as follows:
[0103] f(x i )=ω T x i +b
[0104] Where f(x) i ) is x i Risk value, ω T =[ω1,ω2,ω3,…,ω k ] and b are undetermined coefficients, x i Same as described in (3.4).
[0105] Step 18. Calculate the loss value of the model. The loss function is as follows:
[0106] L(f(x i ),y i )=max{0,1-y i f(x i )}
[0107] Where y i It is the actual result of the i-th data.
[0108] Step 19. Calculate the penalty value for model complexity. The penalty function is as follows:
[0109]
[0110] Where ρ is a proportionality constant, set to 0.15 in the experiment, and ω i Let ω be the i-th dimension of the vector.
[0111] Step 20. Based on the input data, ω is updated as follows:
[0112]
[0113] Where η is the learning rate, and y i These are the actual results for the i-th data point, R(ω) and L(f(x). i ),y i The definition is the same as above.
[0114] Step 21. Based on the input data, b is updated as follows:
[0115]
[0116] Step 22. On the dataset obtained in step 15, repeat the process of steps 16–20 t times to obtain the ensemble classifier.
[0117] Reference Figure 4 The specific operational process of stroke risk prediction is as follows:
[0118] Step 23. After inputting case data into the system, the data is transformed into a format consistent with the system input.
[0119] Step 24. Copy the data four times and input each copy into one of the four base classifiers to obtain four results. Concatenate these results into a 4-dimensional vector w. The calculation method is as follows:
[0120] w = [p1, p2, p3, ..., p k ]
[0121] Where w represents a 4-dimensional vector, p k This represents the prediction result of the k-th classifier for the original data.
[0122] Step 25. Use the above x as input, calculate the disease risk value f(x) mentioned in step (3.5), and output it.
Claims
1. A method for predicting ischemic stroke based on ensemble learning, characterized in that, The steps are as follows: (1) Data cleaning stage: (1.1) Impute missing values in the EHR dataset using the following formula: Among them, v i Let be the value of the i-th sample without missing data, and N be the total number of samples without missing data. (1.2) Replace the category data in the EHR dataset with 0, 1, 2, 3... in the order in which each category appears; (1.3) Copy the processed EHR data into k copies and select k base learners to correspond to them; (2) Feature selection stage: (2.1) For any dataset EHR i We select f features to form a feature subset F. This selection method is represented by a vector, denoted as s. i ; (2.2) Repeat step (2.1) n times to obtain n feature subsets and selection methods. The selection method set is denoted as S. (2.3) Use F to train the dataset EHR i The base learner corresponding to step (1.3) is used to obtain the base classifier M, and the fitness corresponding to this selection method is calculated. The fitness formula is as follows: Where fit(j) is the fitness of j selection methods, p i y is the predicted category of the i-th data point. i It is the actual category of the i-th data; (2.4) Based on the retention probability of each selection method, create a selection roulette wheel and select n times to form the set of n selection methods for the next generation; the retention probability formula is as follows: Where P(i) is the i-th selection method s i The probability of retention, where n is the total number of selection methods; (2.5) Treat adjacent elements in S as a pair, and each element is paired only once; the two vectors in each pair have a certain probability of being crossed, and the crossing method is as follows: a′=[a1,a2,...,b i ,b i+1 ,...,b j ,α j+1 ,...,a l ] b′=[b1,b2,...,a i ,a i+1 ,...,a j ,b j+1 ,...,b l ] Where a′ and b′ are the vectors obtained by exchanging vectors a and b, and a i b i It is the i-th dimension of vectors a and b before the swap; (2.6) Suppose that each element in S has a certain probability of undergoing a mutation, such that a certain dimension of the vector changes from 0 to 1 or from 1 to 0. (2.7) Select the feature subset F with the highest fitness. best and the corresponding base classifier M best ; (2.8) Repeat steps (1.3)-(1.7) L times; (2.9) From the final set of selection methods S, select the feature selection method S with the highest fitness. best The corresponding feature subset F best With base classifier M best ; (2.10) Repeat steps (2.1)-(2.9) until all k classifiers have been trained; (3) Integrated model generation (3.1) For any feature subset F obtained in step (2.9) best In the corresponding base classifier M best Based on this training, the base classifier M of the ensemble model is obtained. * ; (3.2) Repeat step (3.1) k times until all base classifiers have been trained; (3.3) Using k base classifiers M * Make predictions for the original dataset separately and save the prediction results; (3.4) Merge the results into a new dataset, as follows: x i =[p 1i ,p 2i ,p 3i ,...,p ki ] Where, x i p represents the i-th new data. ki This represents the prediction result of the k-th classifier for the i-th original data. (3.5) Using this new data as input, calculate the disease risk value. The risk value formula is as follows: f(x i )=ω T x i +b Where, f(x) i ) is x i Risk value, ω T =[ω1,ω2,ω3,...,ω k ] and b are undetermined coefficients; (3.6) Calculate the loss value of the model. The loss function is as follows: L(f(x i ),y i )=max{0,1-y i f(x i )} Among them, y i It is the true result of the i-th data; (3.7) Calculate the penalty value for model complexity. The penalty function is as follows: Where ρ is a proportionality constant, set to 0.15; ω i Let ω be the i-th dimension of the vector. (3.8) Based on the input data, ω is updated as follows: Where η is the learning rate, y i It is the true result of the i-th data; (3.9) Based on the input data, b is updated as follows: (3.10) Repeat steps (3.5)-(3.9) t times to obtain the ensemble classifier; (4) Stroke risk prediction (4.1) After case data is input into the system, the data is transformed into a format that conforms to the system input. (4.2) The data is copied into 4 copies and input into 4 base classifiers respectively to obtain 4 results. These results are then concatenated into a 4-dimensional vector w. The calculation method is as follows: w=[p1,p2,p3,...,p k ] Where w represents a 4-dimensional vector, p k This represents the prediction result of the k-th classifier on the original data; (4.3) Use the above x as input, calculate the disease risk value f(x) mentioned in step (3.5), and output it.