Construction method of integrated learning model based on genetic algorithm and comprehensive evaluation method

Abstract

The application provides an integrated learning model construction method based on a genetic algorithm and a comprehensive evaluation method, relates to the technical field of digital technology, and constructs an integrated learning model construction framework based on the genetic algorithm and the comprehensive evaluation method to improve the classification performance and prediction accuracy of the integrated model. By constructing a basic learning device, the R- CIEM score is used to hierarchically divide the basic learning device formed by the used machine learning method, the hierarchical results of the basic learning device are obtained, the basic learning device with the level of "excellent" is selected to combine and construct an integrated learning model, the integrated learning model is used, and the basic information and clinical test data of the same type of clinical patients in the test set are used to predict the clinical evaluation information and clinical prognosis information of the patients.

Description

Technical Field

[0001] This invention relates to the field of digital technology, and in particular to a method for constructing an ensemble learning model based on genetic algorithms and a comprehensive evaluation method. Background Technology

[0002] With the continuous development of clinical medicine, the application of artificial intelligence (AI) technology is gradually demonstrating its enormous potential in clinical diagnosis. AI technologies, represented by machine learning, are increasingly being used in clinical diagnosis, playing a crucial role in improving diagnostic accuracy, optimizing treatment plans, and improving patient prognosis. As medical data accumulates and medical knowledge evolves, machine learning technology can analyze and mine large-scale clinical data, helping doctors make quick and accurate diagnoses and provide personalized treatment recommendations. Therefore, introducing machine learning technology into the field of clinical diagnosis is of great significance and is expected to promote further development of clinical medicine and improve patient treatment outcomes.

[0003] Ensemble learning is a crucial research area in machine learning. Its main idea is to improve classification performance and prediction accuracy by combining multiple different classifier models. Soft voting, as an excellent ensemble learning method, has two requirements for the selection of base learners: First, the performance differences between base learners should not be too large; if the performance difference between a base learner and others is too significant, that base learner is likely to become noise. Second, the base learners should have relatively low homogeneity. In existing research in this field, ensemble learning models based on genetic algorithms and multiple machine learning models have been widely applied. However, current research still has shortcomings in model optimization, model selection, and model fusion. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for constructing ensemble learning models based on genetic algorithms and a comprehensive evaluation method, thereby improving the classification performance and prediction accuracy of ensemble models. Specifically, hyperparameter optimization using genetic algorithms can provide high-quality base learners for the ensemble learning model. The comprehensive evaluation method, R-CIEM, can reduce performance differences between base learners, mitigating noise interference to the ensemble model caused by excessive performance disparities among base learners.

[0005] An ensemble learning model construction method based on genetic algorithms and comprehensive evaluation methods includes the following steps:

[0006] Step 1: Collect basic information, clinical test data, clinical assessment information, and clinical prognosis information of a certain type of clinical patient from any clinical public database. Divide the collected information into a dataset according to the patient ratio of 7:3, with the training set accounting for 70% and the validation set accounting for 30%. Then collect basic information, clinical test data, clinical assessment information, and clinical prognosis information of the same type of clinical patient from other clinical public databases as the test set.

[0007] The basic information in step 1 includes gender and age, clinical test data is the concentration value of laboratory tests, clinical assessment information includes the severity of the disease, and clinical prognosis information includes survival status and death status. Survival status is represented by a value of 0, and death status is represented by a value of 1.

[0008] Step 2: Using the training set, optimize the hyperparameters of several pre-defined machine learning methods using a Python-based genetic algorithm;

[0009] The hyperparameters of the pre-defined machine learning methods are adjusted using a genetic algorithm to find the optimal parameter settings. Then, for each machine learning method, the machine learning model is trained and built using the corresponding fit function in Python based on the training set and the found optimal parameter settings. The built machine learning model is then used as the base learner.

[0010] Step 3: On the validation set, each base learner is evaluated using five-fold cross-validation to obtain the corresponding evaluation metrics: Accuracy, F1, Precision, Recall, C-index, and AUC. Then, the R-CIEM comprehensive evaluation method is used to comprehensively analyze the obtained evaluation metrics Accuracy, F1, Precision, Recall, C-index, and AUC to obtain the R-CIEM score. The R-CIEM score is then used to hierarchically divide the base learners composed of the machine learning methods used to obtain the hierarchical results of the base learners.

[0011] The R-CIEM comprehensive evaluation method includes the following steps:

[0012] Step S1: Integrate and analyze the evaluation metrics of each basic learner using the comprehensive index evaluation method to obtain the comprehensive evaluation index Z. i ;

[0013] Step S2: Place Z i Compile a WRSR frequency distribution table in ascending order, listing the frequency f of each individual evaluation index. i Calculate the cumulative frequency F of each indicator. i Calculate the cumulative frequency p i Convert to the probability unit Probit, where n is the total number of samples, i.e., the number of basic learners;

[0014] Step S3: Use Probit to stratify and rank the evaluation objects to obtain the stratification results of the basic learner; if Probit≤4: poor, if 4<Probit<6: good, if Probit≥6: excellent.

[0015] The evaluation object is the basic learner, and the excellent, good and poor are hierarchical classifications of the performance of the evaluation object, i.e., the basic learner, based on the Probit value.

[0016] Step 4: Based on the hierarchical results of the basic learners obtained in Step 3, select the basic learners with the "excellent" level; then, based on the soft voting method of ensemble learning, combine the basic learners with the "excellent" level to construct an ensemble learning model.

[0017] The ensemble learning model is constructed by weighting the prediction results of each base learner according to certain weights to obtain the final ensemble prediction result.

[0018] Step 5: Using the ensemble learning model obtained in Step 4, and using the basic information and clinical test data of the same type of clinical patients in the test set, predict the clinical assessment information and clinical prognosis information of the patients. Based on the existing clinical assessment information and clinical prognosis information of the patients, evaluate the ensemble model by calculating evaluation indicators such as Accuracy, F1, Precision, Recall, C-index and AUC.

[0019] The beneficial effects of adopting the above technical solution are as follows:

[0020] This invention provides a method for constructing an ensemble learning model based on genetic algorithms and a comprehensive evaluation method, which can effectively improve classification performance and prediction accuracy. An improved comprehensive evaluation method, R-CIEM, is proposed, which combines the advantages of the comprehensive index evaluation method and the RSR evaluation method, overcoming the shortcomings of the comprehensive index evaluation method in classifying samples and the insufficient information utilization of the RSR method. Furthermore, this method is more stable and reliable in selecting the base learner; repeated cross-validation is used to make the ensemble learning model more reliable and stable. The method for constructing an ensemble learning model based on the genetic algorithm and comprehensive evaluation method provided by this invention has practicality and broad application prospects in predicting clinical assessment information and clinical prognostic information of clinical patients. It can be used for prediction and prognosis problems of different categories of patients, exhibiting high accuracy and stability. Attached Figure Description

[0021] Figure 1This is a flowchart of the method in an embodiment of the present invention; Detailed Implementation

[0022] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0023] Ensemble learning model construction methods based on genetic algorithms and comprehensive evaluation methods, such as Figure 1 As shown, it includes the following steps:

[0024] Step 1: Collect basic information, clinical test data, clinical assessment information, and clinical prognosis information of a certain type of clinical patient from any clinical public database. Divide the collected information into a dataset according to the patient ratio of 7:3, with the training set accounting for 70% and the validation set accounting for 30%. Then collect basic information, clinical test data, clinical assessment information, and clinical prognosis information of the same type of clinical patient from other clinical public databases as the test set.

[0025] The basic information in step 1 includes gender and age, clinical test data is the concentration value of laboratory tests, clinical assessment information includes the severity of the disease, and clinical prognosis information includes survival status and death status. Survival status is represented by a value of 0, and death status is represented by a value of 1.

[0026] Step 2: Using the training set, optimize the hyperparameters of several pre-defined machine learning methods using a Python-based genetic algorithm;

[0027] The hyperparameters of the pre-defined machine learning methods are adjusted using a genetic algorithm to find the optimal parameter settings, thereby improving the performance of the machine learning model to be built. Then, for each machine learning method, the machine learning model is trained and built using the corresponding fit function in Python based on the training set and the found optimal parameter settings. The built machine learning model is then used as the base learner.

[0028] Step 3: On the validation set, each base learner is evaluated using 10 repetitions of 5-fold cross-validation to obtain the corresponding evaluation metrics: Accuracy, F1, Precision, Recall, C-index, and AUC. Then, the R-CIEM comprehensive evaluation method is used to comprehensively analyze the obtained evaluation metrics Accuracy, F1, Precision, Recall, C-index, and AUC to obtain the R-CIEM score. The R-CIEM score is then used to hierarchically divide the base learners constructed using the machine learning methods used, thus obtaining the hierarchical results of the base learners.

[0029] The R-CIEM comprehensive evaluation method includes the following steps:

[0030] Step S1: Integrate and analyze the evaluation metrics of each basic learner using the comprehensive index evaluation method to obtain the comprehensive evaluation index Z. i ;

[0031] The comprehensive index evaluation method is a commonly used evaluation method widely applied in decision-making and assessment across various fields. It is a method that comprehensively considers multiple indicators or factors. By weighting or standardizing each indicator, it synthesizes them into a comprehensive index, which is used to evaluate the overall performance or ranking of an object.

[0032] Step S2: Place Z i Compile a WRSR frequency distribution table in ascending order, listing the frequency f of each individual evaluation index. i Calculate the cumulative frequency F of each indicator. i Calculate the cumulative frequency p i Convert to the probability unit Probit, where n is the total number of samples, i.e., the number of basic learners;

[0033] Step S3: Use Probit to stratify and rank the evaluation objects to obtain the stratification results of the basic learner; if Probit≤4: poor, if 4<Probit<6: good, if Probit≥6: excellent.

[0034] The evaluation object is the basic learner, and the excellent, good and poor are hierarchical classifications of the performance of the evaluation object, i.e., the basic learner, based on the Probit value.

[0035] Step 4: Based on the hierarchical results of the basic learners obtained in Step 3, select those basic learners that are rated as "excellent". Then, based on the soft voting method of ensemble learning, combine these "excellent" basic learners to construct an ensemble learning model. When constructing the ensemble learning model, the prediction results of each basic learner are weighted according to certain weights to obtain the final ensemble prediction result. In this way, the ensemble learning model can comprehensively utilize the predictive capabilities of multiple high-performing basic learners, thereby improving the accuracy and reliability of clinical assessment and prognostic information for clinical patients.

[0036] Step 5: Using the ensemble learning model obtained in Step 4, and using the basic information and clinical test data of the same type of clinical patients in the test set, predict the clinical assessment information and clinical prognosis information of the patients. Based on the existing clinical assessment information and clinical prognosis information of the patients, evaluate the ensemble model by calculating evaluation indicators such as Accuracy, F1, Precision, Recall, C-index and AUC to ensure that the constructed ensemble learning model is a model with high accuracy and strong generalization ability.

[0037] Based on the above steps, a predictive model was finally constructed that can predict the clinical assessment information and clinical prognosis information of a certain type of clinical patients. This model has the characteristics of high accuracy and strong generalization ability, which can help doctors make a quick and accurate diagnosis.

[0038] This embodiment constructs a diagnostic and prognostic model for COVID-19 patients using an ensemble learning framework based on genetic algorithms and a comprehensive evaluation method. To provide a diagnostic and prognostic model more suitable for COVID-19 patients with varying degrees of severity, this application uses publicly available COVID-19 patient data from a university, including data from Hospital A (1126 cases) and Hospital B (395 cases). First, routine data preprocessing is performed to remove outliers (including handling missing and correlated values). The data is then divided into training and testing sets. Subsequently, the construction of the ensemble model begins. First, basic learners were constructed based on the training set using 10 repeated five-fold cross-validation, a genetic algorithm, and 22 machine learning methods (Adaboost (Ada), Bagging, BernoulliNB (BNB), Decision Tree (DT), ExtraTree, ExtraTrees, Gradient Boosting (GB), GaussianNB (GNB), Gaussian Process (GP), K-Nearest Neighbor (KNN), Linear Discriminant Analysis (LDA), LinearSVC, Logistic RegressionCV (LRCV), NuSVC, Passive Aggressive (PA), Perceptron, Quadratic Discriminant Analysis (QDA), Random Forest (RF), RidgeCV, Stochastic Gradient Descent (SGD), SVC, and XGBoost). Then, various evaluation metrics for these basic learners were calculated, including Accuracy, F1, Precision, Recall, C-index, and AUC. Simultaneously, these indicators were comprehensively analyzed using the R-CIEM comprehensive evaluation method to derive comprehensive evaluation indicators for these basic learners, and these basic learners were then classified. The classification results are shown in Table 1. Finally, based on a soft voting strategy, the basic learners classified as "excellent" were selected to construct an ensemble learning model. This model exhibits high stability and accuracy. The performance of this model can be verified on a test set. Furthermore, ablation experiments can be used to verify the necessity of the genetic algorithm and the R-CIEM comprehensive evaluation method. The R-CIEM classification data in this embodiment are shown in Table 1:

[0039] Table 1. R-CIEM Classification

[0040]

[0041]

[0042] The probability unit Probit here is the standard normal deviation +5 corresponding to the cumulative frequency.

[0043] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for constructing an ensemble learning model based on genetic algorithms and comprehensive evaluation methods, characterized in that, Includes the following steps: Step 1: Collect basic information, clinical test data, clinical assessment information, and clinical prognosis information of a certain type of clinical patients from any clinical public database, and divide the collected information into datasets according to the set proportion of patients; Step 2: Using the training set, optimize the hyperparameters of several pre-defined machine learning methods using a Python-based genetic algorithm; Step 3: Construct the basic learner and obtain the hierarchical results of the basic learner; Step 3 specifically involves: on the validation set, obtaining the evaluation index corresponding to each basic learner based on five-fold cross-validation; conducting a comprehensive analysis of the evaluation index based on the R-CIEM comprehensive evaluation method to obtain the R-CIEM score; and using the R-CIEM score to hierarchically divide the basic learners composed of the machine learning methods used to obtain the hierarchical results of the basic learners. The evaluation metrics include Accuracy, F1, Precision, Recall, C-index, and AUC; The R-CIEM comprehensive evaluation method includes the following steps: Step S1: Integrate and analyze the evaluation metrics of each basic learner using the comprehensive index evaluation method to obtain the comprehensive evaluation index. ; Step S2: Compile a WRSR frequency distribution table in ascending order, listing the frequency of each individual evaluation indicator. Calculate the cumulative frequency of each indicator. Calculate the cumulative frequency ,Will Convert to the probability unit Probit, where n is the total number of samples, i.e., the number of basic learners; Step S3: Use Probit to stratify and rank the evaluation objects to obtain the stratification results of the basic learner; if Probit≤4: poor, if 4<Probit<6: good, if Probit≥6: excellent. The evaluation object is the basic learner, and the excellent, good and poor are hierarchical classifications of the performance of the evaluation object, i.e., the basic learner, based on the Probit value. Step 4: Based on the hierarchical results of the basic learners, select the basic learners with the "excellent" level; combine the basic learners with the "excellent" level using the soft voting method based on ensemble learning to construct an ensemble learning model; Step 5: Using an ensemble learning model, the basic information and clinical test data of the same type of clinical patients in the test set are used to predict the clinical assessment information and clinical prognosis information of the patients. The ensemble model is then evaluated based on the existing clinical assessment information and clinical prognosis information of the patients using evaluation indicators.

2. The method for constructing an ensemble learning model based on genetic algorithm and comprehensive evaluation method according to claim 1, characterized in that, The basic information in step 1 includes gender and age, clinical test data is the concentration value of laboratory tests, clinical assessment information includes the severity of the disease, and clinical prognosis information includes survival status and mortality status. Survival status is represented by a value of 0, and mortality status is represented by a value of 1.

3. The method for constructing an ensemble learning model based on genetic algorithm and comprehensive evaluation method according to claim 1, characterized in that, The dataset division in step 1 is as follows: the training set accounts for 70% and the validation set accounts for 30%; then, basic information, clinical test data, clinical assessment information and clinical prognosis information of the same type of clinical patients are collected from other clinical public databases as the test set.

4. The method for constructing an ensemble learning model based on genetic algorithm and comprehensive evaluation method according to claim 1, characterized in that, Step 2 specifically involves adjusting the hyperparameters of a pre-defined machine learning method using a genetic algorithm to find the optimal parameter settings. Then, for each machine learning method, the machine learning model is trained and constructed using the corresponding fit function in Python based on the training set and the found optimal parameter settings. The constructed machine learning model is then used as the base learner.

5. The method for constructing an ensemble learning model based on genetic algorithm and comprehensive evaluation method according to claim 1, characterized in that, In step 4, the ensemble learning model is constructed by weighting the prediction results of each base learner according to certain weights to obtain the final ensemble prediction result.

Images