Type II diabetes causal feature selection method

Abstract

The invention relates to a causal feature selection method for type II diabetes mellitus. The execution process of the method is described as follows: (1) preprocessing and standardizing original data; (2) introducing a type II diabetes risk factor reinforcement learning causal discovery method, and constructing a type II diabetes characteristic causal structure; (3) executing a forward selection stage, taking causal information gain as a selection strategy, and taking a feature set with high score of a strategy function as an output data set; and (4) executing a backward deletion stage, and deleting repeated redundant features by taking a correlation coefficient as a deletion strategy to obtain a final feature set. Existing feature selection methods are mostly based on correlation among variables and have the defect that causal relationships among the variables are difficult to reflect. The invention designs a new type II diabetes feature selection method based on the combination of causal information gain and correlation coefficients, and the method can effectively reduce the model training time, improve the model robustness, and provide a new thought for diagnosis and prevention research of type II diabetes.

Description

Technical Field

[0001] The invention relates to the field of biological diagnosis engineering, and in particular to a method for selecting causal features of type II diabetes. Background Art

[0002] Type II diabetes is a chronic disease with a complex etiology and a high incidence, and its prevalence is increasing year by year. If not treated and controlled in time, patients may develop a series of complications, so early diagnosis and prediction of type II diabetes are of great significance for its treatment and avoidance of complications.

[0003] In recent years, researchers have found that machine learning algorithms can help with early detection and analysis of diseases. As machine learning research continues to deepen, more and more machine learning prediction models are being used for type 2 diabetes. However, when the amount of sample data is large, model training becomes very difficult and data preprocessing is usually required.

[0004] Feature selection is an important part of data preprocessing methods, which can eliminate useless data, reduce model training time and improve the accuracy of machine learning models. The primary consideration in the construction of a prediction model for type II diabetes is the design rationality of the feature selection method. Existing feature selection algorithms often select features based on the correlation between prediction features and the correlation between prediction features and class variables, and do not attempt to capture the causal relationship between them. It has been proven that understanding the causal relationship between variables has potential benefits for establishing an interpretable and robust prediction model, because the causal relationship directly reflects the inherent mechanism of the system. Therefore, research on the feature selection method of type II diabetes risk factors based on causality has attracted more and more attention. Based on this, the present invention provides a causal feature selection method for type II diabetes, which provides assistance for diabetes diagnosis and prevention research.

[0005] In addition, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making the present invention, but due to space limitations, not all details and contents are listed in detail. However, this does not mean that the present invention does not have the characteristics of these prior arts. On the contrary, the present invention already has all the characteristics of the prior art, and the applicant reserves the right to add relevant prior art to the background technology. Summary of the invention

[0006] The present invention relates to a method for selecting a type II diabetes feature. Considering that most existing feature selection methods are based on the correlation between variables and are unable to reflect the causal relationship between variables, the present invention proposes for the first time a method for selecting a type II diabetes causal feature based on causal information gain and combined with a correlation coefficient.

[0007] In order to achieve the purpose of the above invention, the technical solution adopted by the present invention is as follows:

[0008] A causal feature selection method for type II diabetes, such as Figure 1 As shown, the following steps are included:

[0009] (1) Preprocess and standardize the raw data;

[0010] (2) Introducing the reinforcement learning causal discovery method for risk factors of type 2 diabetes and constructing the characteristic causal structure of type 2 diabetes;

[0011] (3) Execute the forward selection phase, use causal information gain as the selection strategy, and use the feature set with a high score of the strategy function as its output data set;

[0012] (4) Execute the backward deletion phase, use the correlation coefficient as the deletion strategy, delete duplicate and redundant features, and obtain the final feature set.

[0013] According to a preferred embodiment, the raw data is preprocessed and the Z-score standardization is calculated as follows:

[0014]

[0015] Where x is the individual observation, μ is the mean of the population, and σ is the standard deviation of the population.

[0016] According to a preferred embodiment, a method for constructing a characteristic causal structure of type II diabetes comprises: using a codec model to generate a directed acyclic graph, scoring the obtained directed acyclic graph using a BIC scoring function, tending to obtain a causal adjacency matrix with a high score; in continuous loop iterations, using the causal adjacency matrix with the highest score as output.

[0017] To describe the causal strength between two characteristic variables, the inverse information entropy causal strength is defined as follows:

[0018]

[0019] Among them, S(p x，y ), S(p x，y ) are the standardized data information entropy of variables, respectively.

[0020] The causal structure of the initial features is then obtained, expressed as an adjacency matrix (the input of the forward addition stage) as follows:

[0021]

[0022] Among them, T l，i Represents variable x l , x i The causal strength between them, i represents the number of features in the observed data set.

[0023] According to a preferred embodiment, the forward selection phase is performed, and causal information gain is used as the selection strategy, and the feature set with a high strategy function score is used as its output data set. The features in the original data set are added to the output data set in the order of information gain. The condition for adding is to determine whether the strategy score increases. If the strategy score increases, the current feature is added to the output data set; otherwise, the current feature is skipped and other features are judged. First, the information gain of each feature relative to the classification variable c is calculated, and the features are sorted according to the information gain size; secondly, the causal information gain of the current feature is calculated in combination with the causal structure generated by causal discovery; finally, the current feature is judged according to the strategy score function to determine whether it should be added to the output data set.

[0024] The information gain calculation formula is as follows:

[0025] Gain(DX)=Ent(D)-Ent(D|X) (4)

[0026] Among them, Ent(D) represents the information entropy of set D, and Ent(D|X) represents the conditional information entropy of set D under the given feature x.

[0027] According to a preferred embodiment, causal information gain is used to indicate the contribution of the current iterative feature to the output data set. The existence of a causal relationship between features indicates that the two features have an intrinsic connection, which is reflected in the similarity and redundancy of specific data, so it is necessary to select features with high information gain and few causal connections. When the causal information gain is positive, it means that the current feature is helpful in the construction of the output data set, and this feature needs to be added to the output data set. The specific expression is as follows:

[0028] Gain(DX) N =Gain(CX)-∑T N (X) (5)

[0029] Among them, Gain(C, X) represents the information gain of the iterative feature X for the classification variable C, ∑T N (X) represents the sum of the causal strengths of feature X in this iteration and other features in the output dataset.

[0030] The strategy score function is used to characterize the quality of the current output data set. The higher the score, the better the quality of the data set, and the more suitable it is for training the prediction model. The score function of the current output data set is defined as follows:

[0031] S(F)=S(F -1 )+Gain(D,X) N (6)

[0032] Among them, S(F -1 ) represents the score of the output dataset of the last iteration.

[0033] According to a preferred embodiment, deleting redundant features generated in the previous stage is, for example, determining the significant correlation between variables in the output data set in reverse order, deleting significant correlation features with low information gain, and avoiding repeated redundant features in the data set. The step of deleting redundant features generated in the previous stage includes: calculating the significant correlation of features in the output data set; deleting features in reverse order according to the significant correlation; and obtaining a final feature set.

[0034] When executing the backward deletion phase, the correlation coefficient is used as the deletion strategy to delete the redundant features generated in the forward selection phase and obtain the final feature set. The significant correlation between variables is determined in reverse order from the output data set, and the significantly correlated features with low information gain are deleted to avoid duplicate redundant features in the data set.

[0035] Preferably, the step of deleting redundant features generated in the previous stage comprises:

[0036] (1) Calculate the significant correlation of the output data set features. The Spearman correlation coefficient calculation method is used, and the correlation coefficient ρ is expressed as follows:

[0037]

[0038] Among them, d i It represents the rank difference between the i-th values ​​of two features, and n is the number of samples of the feature.

[0039] (2) Delete features in reverse order according to significant correlation. Determine whether to delete the current feature based on the correlation coefficient between features and the significant P value. When the correlation coefficient between this feature and other features is greater than 0.7 and the P value is less than 0.05, it is considered that this feature has a significant strong correlation with other features, so the current feature is deleted to reduce the repeated and redundant features in the feature set.

[0040] (3) Obtain the final feature set. According to the different execution processes, it can be divided into standard feature selection and cyclic feature selection.

[0041] Preferably, if Figure 2 As shown in the figure, the standard feature selection algorithm process or standard feature selection includes the following steps: in the forward execution phase, the data set with the highest score is selected through a scoring function based on causal information gain; in the backward execution phase, the redundant similar features in the data set selected in the previous phase are deleted using significant correlation; and the final data set is output.

[0042] Preferably, if Figure 3As shown in the figure, the cyclic feature selection algorithm process or cyclic feature selection includes the following steps: filter out a feature through the forward stage; use the backward stage to determine whether this feature should be deleted, and loop until all features are traversed; output the final data set.

[0043] The risk factors for type II diabetes are numerous and related. Most existing feature selection methods are based on the correlation between variables, which is difficult to reflect the causal relationship between risk factor variables. The feature selection method involved in the present invention uses a method combining causal information gain with a correlation coefficient to screen features with high contribution to the prediction model. The feature selection method can effectively reduce the model training time, improve the model robustness, and provide new ideas for the prevention and research of type II diabetes. The beneficial effects of the present invention relative to the prior art are:

[0044] The causal feature selection method for type 2 diabetes proposed by combining causal information gain and correlation coefficient can effectively reduce model training time, improve model robustness, and provide new ideas for the diagnosis and prevention research of type 2 diabetes. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Select a flow chart for standard features;

[0046] Figure 2 Select a flow chart for the loop feature;

[0047] Figure 3 The process of predictive model for type II diabetes based on causal feature selection method;

[0048] Figure 4 This is a comparison chart of the accuracy of the standard feature selection method random forest model;

[0049] Figure 5 This is a comparison chart of the accuracy of the logistic regression model using the standard feature selection method;

[0050] Figure 6 This is a comparison chart of the accuracy of the K nearest neighbor model of the standard feature selection method;

[0051] Figure 7 This is a comparison chart of the accuracy of the decision tree model using the standard feature selection method;

[0052] Figure 8 This is a comparison chart of the training time of each model of the standard feature selection method;

[0053] Fig. 9 This is a comparison chart of the accuracy of the random forest model using the cyclic feature selection method;

[0054] Fig.10 This is a comparison chart of the accuracy of the logistic regression model using the cyclic feature selection method;

[0055] Fig.11 This is a comparison chart of the accuracy of the K nearest neighbor model of the cyclic feature selection method;

[0056] Fig.12 This is a comparison chart of the accuracy of the decision tree model for the cyclic feature selection method;

[0057] Fig.13 This is a comparison chart of the training time of each model of the cyclic feature selection method;

[0058] Fig.14 This is a comparison chart of examples of standard feature selection methods;

[0059] Fig.15 A comparison chart of examples of cyclic feature selection methods. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. For those skilled in the art, the specific meanings of the terms in the present invention can be understood in specific circumstances.

[0061] This example provides a causal feature selection method for type II diabetes. To verify the feasibility and effectiveness of the present invention, three groups of experiments were conducted, namely, prediction model comparison experiment, different feature selection method comparison experiment and example experiment. To reduce random errors, the model training was repeated 1000 times. The evaluation indicators AUC, training set accuracy, test set accuracy, accuracy, recall rate, F1 value and model training time involved in the experiment were all average values ​​of 1000 times.

[0062] There are three experimental data: ① The Pima people (PIMA) diabetes dataset with a sample size of 768; ② The Iraqi national diabetes dataset published by the Laboratory of Medical City Hospital (LMCH), with a sample size of 947; ③ The diabetes dataset synthesized by searching the National Health and Nutrition Examination Survey (NHANES) website, with a sample size of 5087.

[0063] Among them, the PIMA data set contains 8 variables: number of pregnancies, 2-hour glucose concentration, diastolic blood pressure, triceps skin fold thickness, 2-hour insulin content, body mass index, diabetes pedigree function, and age; the LMCH data set contains 11 variables: gender, age, urea, creatinine ratio, glycosylated hemoglobin, cholesterol, triglycerides, high-density lipoprotein, low-density lipoprotein, very low-density lipoprotein, and body index; the NHANES data set contains 48 variables: gender, age, race, education level, marital status, energy, protein, carbohydrates, fat, saturated fatty acids, monounsaturated fatty acids, polyunsaturated fatty acids, cholesterol intake, dietary fiber, calcium, phosphorus, magnesium, iron, zinc, copper, sodium, potassium, selenium, caffeine, theobromine, alcohol, water, pulse, whether the pulse is regular, systolic blood pressure, diastolic blood pressure, weight, height, BMI, arm circumference, waist circumference, albumin, creatinine, triglycerides, low-density lipoprotein, cholesterol, high-density lipoprotein, glycosylated hemoglobin, fasting blood sugar, insulin, drinking, smoking, and whether there is anyone smoking at home.

[0064] (1) Prediction model comparison experiment

[0065] The above three data sets were used to train and compare four typical diabetes classification prediction models, including random forest, logistic regression, K nearest neighbor and decision tree.

[0066] ① Comparative analysis of standard feature selection methods

[0067] Comparison of prediction model accuracy with standard feature selection Figures 4 to 7 As shown in the figure, the accuracy of the four models on the three data sets remains at a level comparable to that of the original features, and the AUC value also does not decrease. It can be seen that standard feature selection does not reduce the performance of the prediction model, and since the AUC value does not decrease, it shows that the model has strong applicability.

[0068] At the same time, by Figures 4 to 7 It can be seen that the prediction model after feature selection has a certain degree of improvement in training accuracy and test accuracy compared to the original data prediction model, the difference between training accuracy and test accuracy has become smaller, and the AUC value has also increased. This is because after feature selection, features with large information gain and small mutual connection are selected, which reduces the mutual interference between features, reduces the degree of model overfitting, and improves the model generalization ability.

[0069] Figure 8 This is a comparison chart of the training time of each model. Figure 8It can be seen that after feature selection, the training time of each model is reduced accordingly, especially with the increase of sample size and feature dimension, the reduction of training time is more obvious. However, the training time changes will be different for different prediction models. For example, the random forest method can be calculated in parallel, and the unique sample sampling and feature selection method determines that it can effectively process high-dimensional data, and its training time reduction is not obvious; the training time reduction of the other three classification algorithms is very obvious.

[0070] ②Comparative analysis of cyclic feature selection methods

[0071] Comparison of prediction model accuracy of cyclic feature selection Figures 9 to 12 As shown. Figures 9 to 12 It can be seen that after feature selection, the prediction model has not decreased in terms of accuracy and AUC value. Similarly, in most cases, the training accuracy, test accuracy and AUC value have increased to a certain extent. Therefore, the cyclic feature selection method also has the characteristics of reducing the degree of model overfitting and increasing the generalization ability of the model.

[0072] The training time of each model is compared. Fig.13 As shown. Fig.13 It can be seen that the training time of the three data sets has been reduced, and the effect is similar to that of the standard feature selection method. In the experiments of this paper, the PIMA and LMCH data sets obtained the same features in the two feature selection method processes, so no comparative analysis was performed; for the NHANES data set, the cyclic feature selection method screened more features than the standard feature selection method, so the degree of reduction in training time is not as good as the standard feature selection method. For example, under the decision tree model, the former has 39% less training time, and the latter has 49% less training time.

[0073] (2) Comparative experiments on different feature selection methods

[0074] The random forest prediction model is used to compare the performance of the proposed method with four common feature selection methods, including recursive feature elimination, mRMR, RRliefF, and principal component analysis; the NHANES data set is selected. The number of features selected by the standard feature selection method and the cyclic feature selection method are 30 and 33, respectively, and the experimental comparison is carried out respectively. The experimental results are shown in Tables 1 and 2.

[0075] As can be seen from Tables 1 and 2, compared with the other four feature selection methods, the two feature selection processes proposed in this paper have advantages in accuracy. In Table 1, the standard feature selection method is slightly lower than the mRMR and RReliefF methods in accuracy, but has a clear advantage in recall rate. Because accuracy and recall rate affect each other, an increase in recall rate may lead to a decrease in accuracy. Therefore, in general, the standard feature selection method has the best performance. In Table 2, the accuracy of the cyclic feature selection method is slightly lower than that of the recursive feature elimination, mRMR and RReliefF methods, and it also has a clear advantage in recall rate. Therefore, the cyclic feature selection method has the best performance. It can be seen that the feature selection method proposed in this paper has obvious advantages in algorithm performance indicators and training time.

[0076] (3) Example experiment

[0077] AdaBoost is a high-performance diabetes prediction model. Through the continuous iteration of Boosting technology, it has satisfactory prediction accuracy. However, its disadvantage is that the training time of the model is relatively long. Therefore, this model is selected as the tuning model, and experimental analysis is carried out in combination with the feature selection method proposed in this paper.

[0078] Depend on Figures 14-15 It can be seen that compared with the original AdaBoost model, its overfitting degree and training time are significantly reduced. In the PIMA and NHANES datasets, the AUC values ​​are improved, especially in the NHANES dataset, the training time is reduced more significantly; in the LMCH dataset, due to the simple data structure of the dataset, the model accuracy is not improved, and the overfitting phenomenon of the model is reduced to a certain extent.

[0079] Table 1

[0080]

[0081] Table 2

[0082]

[0083] It should be noted that the above-mentioned specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the protection scope of the present invention. Those skilled in the art should understand that the present invention specification and its drawings are illustrative and do not constitute a limitation of the claims. The scope of protection of the present invention is defined by the claims and their equivalents. The present invention specification contains multiple inventive concepts, such as "preferably", "according to a preferred embodiment" or "optionally", all of which indicate that the corresponding paragraph discloses an independent concept, and the applicant reserves the right to file a divisional application based on each inventive concept. Throughout the text, the features guided by "preferably" are only an optional method and should not be understood as being required. Therefore, the applicant reserves the right to abandon or delete the relevant preferred features at any time.

Claims

1. A method for selecting causal features of type II diabetes, characterized in that: The steps include: (1) Preprocess and standardize the raw data; (2) Introducing the reinforcement learning causal discovery method for risk factors of type 2 diabetes and constructing the characteristic causal structure of type 2 diabetes; (3) Execute the forward selection phase, use causal information gain as the selection strategy, and use the feature set with a high score of the strategy function as its output data set; (4) Execute the backward deletion phase, use the correlation coefficient as the deletion strategy, delete duplicate and redundant features, and obtain the final feature set.

2. The method for selecting causal features of type II diabetes according to claim 1, characterized in that: The raw data is preprocessed and the Z-score standardization is calculated as follows: Where x is the individual observation, μ is the mean of the population, and σ is the standard deviation of the population.

3. The method for selecting causal features of type II diabetes according to claim 2, characterized in that: The method for constructing the characteristic causal structure of type II diabetes includes the following steps: The encoder-decoder model is used to generate a directed acyclic graph, and the BIC score function is used to score the resulting directed acyclic graph, which tends to obtain a causal adjacency matrix with a high score; In the continuous loop iteration, the causal adjacency matrix with the highest score is taken as the output, where the inverse information entropy causal strength is defined as follows to describe the causal strength between two feature variables: in, are the standardized data information entropy of variables x1 and x2 respectively, Then the causal structure of the initial features is obtained, represented by the adjacency matrix O G，T (The input to the forward selection phase) is as follows: Among them, T 1，i It represents the causal strength between variables x1 and x2, and i represents the number of features in the observed data set.

4. The method for selecting causal features of type II diabetes according to claim 3, characterized in that: In the forward selection phase, causal information gain is used as the selection strategy, and the feature set with a high strategy function score is used as its output data set. The features in the original data set are added to the output data set in the order of information gain. The condition for adding is to determine whether the strategy score increases. If the strategy score increases, the current feature is added to the output data set; otherwise, the current feature is skipped and other features are judged, that is, the information gain of each feature relative to the classification variable c is calculated, and the features are sorted according to the size of the information gain; the causal information gain of the current feature is calculated in combination with the causal structure generated by causal discovery; the current feature is judged based on the strategy score function to determine whether it should be added to the output data set.

5. The method for selecting causal features of type II diabetes according to claim 4, characterized in that: The information gain calculation formula is as follows: Gain(D,X)=Ent(D)-Ent(D|X) (4) Among them, Ent(D) represents the information entropy of set D, and Ent(D|X) represents the conditional information entropy of set D under the given feature x. When the causal information gain is positive, it means that the current feature is helpful for constructing the output data set, and this feature needs to be added to the output data set. The specific expression is as follows: Gain(D.X) N =Gain(C.X)-∑T N (X) (5) Among them, Gain(C, X) represents the information gain of the iterative feature X for the classification variable C, ∑T N (X) represents the sum of the causal strengths of feature X in this iteration and other features in the output dataset.

6. The method for selecting causal features of type II diabetes according to claim 5, characterized in that: The strategy score function is used to characterize the quality of the current output data set. The higher the score, the better the quality of the data set, and the more suitable it is for training the prediction model. The score function of the current output data set is defined as follows: S(F)=S(F -1 )+Gain(D,X) N (6) Among them, S(F -1 ) represents the score of the output dataset of the last iteration.

7. The method for selecting causal features of type II diabetes according to claim 6, characterized in that: The backward deletion phase is executed, and the correlation coefficient is used as the deletion strategy to delete repeated redundant features and obtain the final feature set, which is characterized by being used to delete the redundant features generated in the claim 4 phase.

8. The method for selecting causal features of type II diabetes according to claim 7, characterized in that: Determine the significant correlation between variables in reverse order from the output data set, delete the significant correlation features with low information gain, and avoid repeated redundant features in the data set. The specific steps include: (1) Calculate the significant correlation of the output data set features: Use the Spearman correlation coefficient calculation method, and the correlation coefficient ρ is expressed as follows: Among them, d i It represents the rank difference between the i-th values ​​of two features, and n is the number of samples of the feature. (2) Deleting features in reverse order based on significant correlation: Determine whether to delete the current feature based on the correlation coefficient between features and the significant P value. When the correlation coefficient between this feature and other features is greater than 0.7 and the P value is less than 0.05, it is considered that this feature has a significant strong correlation with other features, so the current feature is deleted to reduce duplicate and redundant features in the feature set; (3) Obtain the final feature set.

9. The method for selecting causal features of type II diabetes according to claim 8, characterized in that: The type II diabetes causal feature selection method comprises standard feature selection, which comprises the following steps: in the forward execution phase, a data set with the highest score is selected by a score function based on causal information gain; in the backward execution phase, redundant similar features in the data set selected in the previous phase are deleted by using significant correlation; Finally, the final dataset is output.

10. The method for selecting causal features of type II diabetes according to claim 9, characterized in that: The type II diabetes causal feature selection method includes cyclic feature selection, which includes the following steps: screening out a feature through a forward stage; using a backward stage to determine whether the feature is deleted, looping until all features are traversed; and outputting a final data set.

Images