A chronic disease feature selection method based on standard deviation and mutual information
By introducing standard deviation and interactive information in the selection of chronic disease characteristics, the problem of ignoring the dynamic changes of interactive characteristics and information in traditional methods is solved, and more accurate feature selection is achieved, improving the accuracy and economic benefits of disease prediction.
Patent Information
- Application Number
- CN202111485486.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-07
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-12-07
AI Technical Summary
The traditional filtered feature selection method ignores dynamic changes in interactive features and information in the feature selection of chronic diseases, resulting in insufficient feature selection and inability to distinguish the same scores during feature evaluation, affecting classification performance.
A feature selection method based on standard deviation and interactive information is adopted to calculate the mutual information and conditional information of candidate characteristics and categories, and combine standard deviations to select features that are highly relevant to categories to ensure the accuracy and comprehensiveness of feature selection.
Based on the consideration of synergies between features, a subset of features that have a decisive effect on chronic diseases can be selected more accurately, which improves the accuracy and economic benefits of disease prediction, and overcomes the large number of features and accuracy in traditional methods. Not high problem.
Smart Images

Figure CN114358989B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer science and technology, and specifically relates to a chronic disease feature selection method based on standard deviation and mutual information. Background Art
[0002] With the early arrival of the aging era, the health problems of the elderly have attracted more and more attention from the society. Among them, chronic diseases such as hypertension, diabetes and coronary heart disease have become the primary cause of threats to the health of the elderly, which has attracted great attention from experts and scholars in many fields. In recent years, in order to reduce the incidence of chronic diseases in the elderly population and timely predict and warn the occurrence and development of chronic diseases, the influencing factors of chronic diseases have been widely studied.
[0003] The fields of clinical medicine and public health have a variety of data related to chronic diseases. In the past, most of the research on chronic disease-related factors borrowed traditional statistical methods. However, with the advent of the big data era, traditional statistical methods have many limitations in high-dimensional data modeling. At the same time, machine learning algorithms began to emerge, and algorithms such as support vector machines are widely used in various fields, but machine learning algorithms also have some difficulties in modeling high-dimensional data, so many feature selection methods have been born.
[0004] Feature selection is to remove irrelevant or redundant features from a high-dimensional data set with many features, and select some features that are highly relevant to the category label. They constitute a set of optimal features, that is, the optimal feature subset. The information expressed by this set can well replace all the original features. Based on the classification performance, this optimal feature subset can also reduce the dimension of the data, reduce the computational and storage overhead, and construct an efficient classifier.
[0005] The essence of feature selection is to obtain an optimal subdomain of the original feature domain, and the classification performance of the sample in this optimal subspace will not decrease compared with the classification performance in the original space, and the number of features in this optimal subspace is minimized as much as possible. According to the relationship between the feature selection method and the classifier, it can be divided into: filtering, encapsulation and embedded feature selection methods. Among them, the filtering feature selection method defines the importance of the features according to the contribution of each feature to the classification and sorts them. Due to its high efficiency and scalability in dimensionality reduction, it is widely used in various fields.
[0006] However, traditional filtering feature selection methods have some shortcomings. First, they only consider the amount of information between candidate features and categories. However, when a new feature is added to the selected feature set, the amount of information between each target feature and the category in the set changes, so it is necessary to consider the dynamic changes of information between the target feature and the category. Secondly, feature selection research often focuses on removing irrelevant and redundant features, but ignores the existence of interactive features, which are those that seem to have little correlation with the category, but when combined with other features, the interactive information generated makes the feature highly correlated with the category. Therefore, the discovery of interactive features is an indispensable part of feature selection. In addition, according to the principle of filtering feature selection, the evaluation function is the final indicator to measure the correlation between features and categories. However, traditional feature selection methods rarely consider the case of the same score. When two or more features have the same score, it is a necessary step to select the feature that is more important to the category.
[0007] In summary, a feature selection method is needed that takes into account both the dynamic changes of target features and information between categories and the synergy between features. This method can select features with higher correlation with categories under the condition of the target feature set, thereby achieving a more comprehensive and accurate selection of feature subsets that are decisive for categories, obtaining the influencing factors of chronic diseases, and preventing or predicting chronic diseases through statistical analysis of the influencing factors. Summary of the invention
[0008] In view of the shortcomings of the prior art, the present invention proposes a chronic disease feature selection method based on standard deviation and mutual information, the method comprising: acquiring the user's physical examination data in real time, preprocessing the data, inputting the preprocessed data into a chronic disease feature selection model based on standard deviation and mutual information, obtaining the influencing factors of the chronic disease and marking the influencing factors;
[0009] The training process of the chronic disease feature selection model based on standard deviation and mutual information includes:
[0010] S1: Get raw data;
[0011] S2: Divide the original data into 10 sets, randomly select 9 sets from the 10 sets as training sets, and the remaining set as the test set;
[0012] S3: Use the training set as the candidate feature set; initialize the target feature set to be empty, and set the initial number of selected features K;
[0013] S4: Calculate the mutual information value between each feature and the category in the candidate feature set;
[0014] S5: Filter the feature with the largest mutual information value with the category, add the feature to the target feature set S, and delete the feature from the candidate feature set to obtain the filtered candidate feature set;
[0015] S6: Use the evaluation function to calculate the score of each feature in the filtered candidate feature set, filter the features according to the scores of each feature, add the filtered features to the target feature set S, and delete the corresponding features in the candidate feature set;
[0016] S7: Repeat steps S5 to S6 until the number of features in the target feature set S is K;
[0017] S8: Calculate the average classification accuracy based on the target feature set S;
[0018] S9: Iterate steps S2 to S8, stop iteration when the average classification accuracy is the highest, and obtain a classification model with optimal classification accuracy; classify the features in the test set according to the current classification model, complete model training, and obtain the influencing factors of chronic diseases.
[0019] Preferably, the process of preprocessing the data includes: cleaning the data to obtain cleaned data; discretizing the cleaned data to obtain discretized data; and normalizing the discretized data to obtain preprocessed data.
[0020] Furthermore, data cleaning includes: deleting variables, filling variables, and processing outliers; the deleted variables include: variables with missing values greater than 30%, highly unbalanced categorical variables, all physical examination information variables except chronic diseases, and version time and home address variables.
[0021] Furthermore, the normalization formula is:
[0022]
[0023] Among them, F * represents the normalized data, F represents the unnormalized data, and F max is the maximum value in the original feature set, F min is the minimum value in the original feature set.
[0024] Furthermore, dividing the original data set into a training set and a test set includes: using a ten-fold cross-validation method, which randomly splits the original data set into 10 data sets with approximately the same sample size, each time selecting 9 data sets as training data sets and leaving one as a test data set.
[0025] Preferably, the evaluation function is:
[0026]
[0027] Among them, F m represents the mth candidate feature, F j represents the jth target feature, C represents the category, S represents the target feature set, J(F m ) represents the characteristic score; I(F m ; C|F j ) is the conditional mutual information, indicating that given the target feature F j The candidate feature F under the condition m The amount of information provided for category C; I(F j ; C|F m ) is the conditional mutual information, indicating that given the candidate feature F m Under the condition of target feature F j The amount of information provided for category C; I(F m ; F j ; C) represents the candidate feature F m and the target feature F j The interactive information provided by category C, I(F j ; F m ) represents the candidate feature F m and the target feature F j of redundant information.
[0028] Furthermore, the calculation formula of the interaction information is:
[0029] I(F m ; F j ; C)=I(F m ,F j ; C)-[I(F m ; C)+I(F j ; C)]
[0030] Among them, I(F m ,F j ; C) represents the candidate feature F m and the target feature F j The joint mutual information of m ; C) represents the candidate feature F m The amount of information provided for category C, I(F j ; C) represents the target feature F j The amount of information provided for category C.
[0031] Preferably, the process of screening features according to the scores of each feature is:
[0032] If there is only one maximum score, then the feature with the maximum score is selected and added to the target feature set S;
[0033] If there are at least two maximum scores, the standard deviation of the features corresponding to the scores is calculated, and the feature corresponding to the minimum standard deviation is selected and added to the target feature set S.
[0034] Furthermore, the calculation formula of standard deviation is:
[0035]
[0036]
[0037] Where μ represents F m The average value of the additional information that can be provided, δ represents the standard deviation, I(F m ; C|F j ) is the conditional mutual information, indicating that given the target feature F j Under the condition that m The amount of information provided for category C; I(F j ; C|F m ) is the conditional mutual information, indicating that given the candidate feature F m Under the condition of j The amount of information provided for category C; I(F m ; F j ; C) represents the candidate feature F m and the target feature F j The interactive information provided by category C, I(F j ; F m ) represents the candidate feature F m and the target feature F j The amount of redundant information.
[0038] The beneficial effects of the present invention are as follows: In the present invention, not only the dynamic changes of information between target features and categories are considered, but also the synergy between target features and candidate features is considered. In addition, the standard deviation is introduced to identify candidate features that are more important to categories under the same score, so that the calculation of feature correlation is more accurate and comprehensive; the present invention can effectively screen and mark the influencing factors of chronic diseases such as hypertension and diabetes, and users can prevent or predict chronic diseases by analyzing and counting the influencing factors. The present invention provides assistance for early warning of chronic diseases, overcomes the defects of existing chronic disease predictions that require more features and low accuracy, and has good economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 This is a training flow chart of a chronic disease feature selection model based on standard deviation and mutual information in the present invention;
[0040] Figure 2It is a feature selection flow chart of the chronic disease feature selection method based on standard deviation and mutual information in the present invention;
[0041] Figure 3 It is a relationship diagram between features and categories in the present invention. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0043] The present invention proposes a chronic disease feature selection method based on standard deviation and mutual information, such as Figure 1 As shown, the method includes: acquiring the user's physical examination data in real time, preprocessing the data, inputting the preprocessed data into a chronic disease feature selection model based on standard deviation and mutual information, obtaining chronic disease influencing factors and marking the influencing factors;
[0044] The training process of the chronic disease feature selection model based on standard deviation and mutual information includes:
[0045] S1: Get raw data;
[0046] S2: Divide the original data into 10 sets, randomly select 9 sets from the 10 sets as training sets, and the remaining set as the test set;
[0047] S3: Use the training set as the candidate feature set; initialize the target feature set to be empty, and set the initial number of selected features K;
[0048] S4: Calculate the mutual information value between each feature and the category in the candidate feature set;
[0049] S5: Filter the feature with the largest mutual information value with the category, add the feature to the target feature set S, and delete the feature from the candidate feature set to obtain the filtered candidate feature set;
[0050] S6: Use the evaluation function to calculate the score of each feature in the filtered candidate feature set, filter the features according to the scores of each feature, add the filtered features to the target feature set S, and delete the corresponding features in the candidate feature set;
[0051] S7: Repeat steps S5 to S6 until the number of features in the target feature set S is K;
[0052] S8: Calculate the average classification accuracy based on the target feature set S;
[0053] S9: Iterate steps S2 to S8, stop iteration when the average classification accuracy is the highest, and obtain a classification model with optimal classification accuracy; classify the features in the test set according to the current classification model, complete model training, and obtain the influencing factors of chronic diseases.
[0054] like Figure 2 As shown, the process of preprocessing data includes: cleaning the data to obtain cleaned data; discretizing the cleaned data to obtain discretized data; and normalizing the discretized data to obtain preprocessed data.
[0055] Data cleaning includes: deleting variables, filling variables, and processing outliers; the deleted variables include: variables with missing values greater than 30%, highly unbalanced categorical variables, all physical examination information variables except chronic diseases, and version time and home address variables.
[0056] The normalization formula is:
[0057]
[0058] Among them, F * represents the normalized data, F represents the unnormalized data, and F max is the maximum value in the original feature set, F min is the minimum value in the original feature set.
[0059] The original data is divided into 10 sets, 9 sets are randomly selected as training sets, and the remaining set is used as a test set. The method includes: using a ten-fold cross-validation method, which randomly splits the original data set into 10 data sets with approximately the same sample size, selects 9 data sets as training data sets, and the remaining one is used as a test data set.
[0060] A specific implementation method of selecting a target feature set is:
[0061] Existing feature evaluation methods based on mutual information do not consider the dynamic changes in the information value between the target feature and the category. They only focus on the correlation between the candidate feature and the category, but ignore the correlation between the target feature and the category, which leads to inaccurate calculation of the correlation between the feature and the category. For example,
[0062] Table 1 Artificial dataset
[0063]
[0064] Table 2 Information between features and categories
[0065]
[0066] In Table 1, S = {F1, F2, F3} is a set of three features, and C is a category. In this set, it is assumed that F1 is the target feature, F2 and F3 are two candidate features, and the values of mutual information (MI), conditional mutual information (CMI) and joint mutual information (JMI) are calculated respectively, as shown in Table 2; from the data in Table 2, it can be observed that I(F1; C|F2) < I(F1; C) and I(F1; C|F3) < I(F1; C). This result shows that when a candidate feature is given, the amount of information between the category and the target feature is different, and this phenomenon is caused by the candidate feature. Therefore, when introducing candidate features, it is necessary to analyze the dynamic changes in the amount of information between the category and the target feature.
[0067] To solve the above problems, considering the dynamic changes of information between categories and target features, the present invention introduces conditional mutual information I(F j ; C|F m ), conditional mutual information is used to replace the mutual information used in traditional feature correlation measurement; where F j represents the jth target feature, F m represents the mth candidate feature, and C represents the category.
[0068] I(F j ; C|F m ) measures the dynamic changes of information between target features and categories. There are three cases, as shown below:
[0069] 1)I(F j ; C|F m )<I(F j ; C) indicates that the addition of candidate features cannot provide new information for the category and generates redundant information, and the amount of information provided by the target feature for the category is reduced;
[0070] 2)I(F j ; C|F m )=I(F j ; C) indicates that the target feature and the candidate feature do not share common information, and the amount of information provided by the target feature does not change;
[0071] 3)I(F j ; C|F m )>I(F j ; C) indicates that the addition of candidate features provides new information for the category, and the amount of information provided by the target feature for the category increases.
[0072] From the above analysis, we can see that the candidate feature F m The more important the category is, the more important I(F j ; C|F m) value is larger.
[0073] In addition, traditional feature selection methods rarely consider interactive information, and the correlation between features and categories is not calculated accurately, thus affecting the accuracy of classification; Figure 3 As shown, the candidate feature F m The information content contains three parts:
[0074] 1)I(F m ; C|F j ): Conditional mutual information, indicating that given the target feature F j , candidate feature F m The amount of information provided for Category C;
[0075] 2)I(F j ; C|F m ): Conditional mutual information, indicating that given the candidate feature F m , target feature F j The amount of information provided for Category C;
[0076] 3)I(F m ; F j ; C): Interactive information, representing the candidate feature F m and the target feature F j The mutual information provided for category C, i.e., candidate feature F m and the target feature F j The resulting interactive correlation plays a synergistic role.
[0077] The calculation formula of the interaction information is:
[0078] I(F m ; F j ; C)=I(F m ,F j ; C)-[I(F m ; C)+I(F j ; C)]
[0079] Among them, I(F m ,F j ; C) represents the candidate feature F m and the target feature F j The joint mutual information of m ; C) represents the candidate feature F m The amount of information provided for category C, I(F j ; C) represents the target feature F j Amount of information provided for category C.
[0080] There are three types of interactive information:
[0081] 1)I(Fm ,F j ;C)>I(F m ; C)+I(F j ; C) indicates that the candidate features and the target features jointly generate new information. There is a positive synergy between the candidate features and the target features;
[0082] 2)I(F m ,F j ; C)=I(F m ; C)+I(F j ; C) indicates that the candidate feature and the target feature are independent of each other, the target feature and the candidate feature do not share common information, and the amount of information provided by the target feature does not change;
[0083] 3)I(F m ,F j ;C)<I(F m ; C)+I(F j ; C) indicates that the candidate feature and the target feature provide the same information, resulting in redundancy. The candidate feature and the target feature produce a negative synergy.
[0084] From the above analysis, we can see that the more important the candidate feature is to the category, the higher the I(F m ; F j ; The larger the value of C).
[0085] In this invention, a feature selection method is proposed that not only considers the information change between the target feature and the category, but also considers the synergy between the target feature and the candidate features. The evaluation function is defined as follows:
[0086]
[0087] Among them, S represents the target feature set, J(F m ) represents the feature score, that is, the importance of the feature; I(F m ; C|F j ) indicates that in the target feature F j Under the premise of existence, the candidate feature F m The amount of information related to the category, measuring the candidate feature F m Contribution to Category C; I(F j ; C|F m ) indicates that as the candidate feature F m The target feature F j and the change in the amount of information of category C. The larger the value, the more conducive it is to classification; I(F m ; F j ; C) represents the candidate feature F m and the target feature F j Synergistic effect; I(Fj ; F m ) represents the candidate feature F m and the target feature F j The amount of redundant information, that is, the target feature F j and candidate features F m The redundancy between them.
[0088] Although many methods apply mutual information and conditional mutual information, they are not always effective as global functions. For example, in Table 3, S = {F1, F2, F3} is a set of three features, and C is a category. In this set, assuming that F1 is the target feature, F2 and F3 are two candidate features, the values of mutual information (MI), conditional mutual information (CMI) and mutual information (II) are calculated respectively, as shown in Table 4.
[0089] Table 3 Artificial dataset
[0090]
[0091] Table 4 Information between features and categories
[0092]
[0093] As shown in Table 4, I(F2; C|F1) = I(F3; C|F1), I(F1; C|F2) = I(F1; C|F3) and I(F1; F2) = I(F1; F3), it can be seen that the mutual information and conditional mutual information are the same. Under this condition, the traditional feature selection method cannot distinguish the features to be selected next.
[0094] The traditional feature evaluation function only considers the total amount of additional information provided between the candidate feature and all target features. On this basis, the present invention introduces the difference in the amount of additional information provided between the candidate feature and each target feature, taking into account both the overall and individual effects, thereby identifying the candidate feature that is more important to the category when the scores are the same, that is, calculating the standard deviation of the feature when the scores are the same; the calculation process is as follows:
[0095]
[0096]
[0097] Where μ represents F m The average value of the additional information that can be provided, δ represents the standard deviation, which reflects the difference between individuals in the group. In this method, it reflects the candidate feature F m And each feature F in the target feature set S j The difference in the additional classification information provided. If the standard deviation is smaller, it means that the given target feature Fj Under the condition of m The closer the additional information provided to category C is to μ, the better the candidate feature F m For each target feature F j The higher the stability of the feature in providing additional information for classification, the more important the candidate feature is for classification.
[0098] A specific embodiment of screening features according to the scores of each feature is:
[0099] If there is only one maximum score, then the feature with the maximum score is selected and added to the target feature set S;
[0100] If there are at least 2 maximum scores, the standard deviation of the features corresponding to the scores is calculated, and the feature corresponding to the minimum standard deviation is selected and added to the target feature set S. After each iteration of the training set obtains a complete target feature set, the average classification accuracy of the target feature set is calculated; the average classification accuracy of each iteration is calculated iteratively, and the iteration is stopped when the average classification accuracy is the highest, and the classification model with the optimal classification accuracy is obtained; the features in the test set are classified according to the current classification model, the model training is completed, and the type of chronic disease is obtained to achieve the purpose of predicting chronic diseases.
[0101] The present invention uses conditional mutual information to replace the mutual information used in traditional feature correlation measurement, and measures the dynamic changes of information between target features and categories through conditional mutual information; at the same time, the present invention not only takes into account feature correlation and redundancy, but also measures the synergy between target features and candidate features by introducing mutual information; in addition, the standard deviation is introduced as a method of discrimination, the purpose of which is to select features with higher correlation with categories under the condition of the target feature set. Compared with traditional feature selection methods, this method can more comprehensively and accurately select feature subsets that have a decisive effect on categories, and obtain the influencing factors of chronic diseases. Users can prevent or predict chronic diseases by analyzing and statistically analyzing the influencing factors; the present invention provides assistance for early warning of chronic diseases, overcomes the defects of existing chronic disease predictions that require more features and low accuracy, and has good economic benefits.
[0102] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A chronic disease feature selection method based on standard deviation and mutual information, characterized in that: Obtain the user's physical examination data in real time, pre-process the data, input the pre-processed data into the chronic disease feature selection model based on standard deviation and mutual information, obtain the influencing factors of chronic diseases and mark the influencing factors; The training process of the chronic disease feature selection model based on standard deviation and mutual information includes: S1: Get raw data; S2: Divide the original data into 10 sets, randomly select 9 sets from the 10 sets as training sets, and the remaining set as the test set; S3: Use the training set as the candidate feature set; initialize the target feature set to be empty, and set the initial number of selected features K; S4: Calculate the mutual information value between each feature and the category in the candidate feature set; S5: Filter the feature with the largest mutual information value with the category, add the feature to the target feature set S, and delete the feature from the candidate feature set to obtain the filtered candidate feature set; S6: Use the evaluation function to calculate the score of each feature in the filtered candidate feature set, filter the features according to the scores of each feature, add the filtered features to the target feature set S, and delete the corresponding features in the candidate feature set; The evaluation function is: Among them, F m represents the mth candidate feature, F j represents the jth target feature, C represents the category, S represents the selected candidate set, J(F m ) represents the characteristic score; I(F m ; C|F j ) is the conditional mutual information, indicating that given the target feature F j The candidate feature F under the condition m The amount of information provided for category C; I(F j ; C|F m ) is the conditional mutual information, indicating that given the candidate feature F m Under the condition of target feature F j The amount of information provided for category C; I(F m ; F j ; C) represents the candidate feature F m and the target feature F j The interactive information provided by category C, I(F j ; F m ) represents the candidate feature F m and the target feature F j The mutual information of The calculation formula of the interaction information is: I(F m ;F j ;C)=I(F m ,F j ;C)-[I(F m ;C)+I(F j ;C)] Among them, I(F m ,F j ; C) represents the candidate feature F m and the target feature F j The joint mutual information of m ; C) represents the candidate feature F m The amount of information provided for category C, I(F j ; C) represents the target feature F j The amount of information provided for Category C; The process of filtering features according to the scores of each feature is: If there is only one maximum score, then the feature with the maximum score is selected and added to the target feature set S; If there are at least 2 maximum scores, calculate the standard deviation of the features corresponding to the scores, select the feature corresponding to the minimum standard deviation and add it to the target feature set S; The formula for calculating the standard deviation is: Where μ represents F m The average value of the additional information that can be provided, δ represents the standard deviation, I(F j ; F m ) represents the candidate feature F m and the target feature F j The amount of redundant information; S7: Repeat steps S5 to S6 until the number of features in the target feature set S is K; S8: Calculate the average classification accuracy based on the target feature set S; S9: Iterate steps S2 to S8, stop iteration when the average classification accuracy is the highest, and obtain a classification model with optimal classification accuracy; classify the features in the test set according to the current classification model, complete model training, and obtain the influencing factors of chronic diseases.
2. According to claim 1, a chronic disease feature selection method based on standard deviation and mutual information is characterized in that: The process of preprocessing data includes: cleaning the data to obtain cleaned data; discretizing the cleaned data to obtain discretized data; normalizing the discretized data to obtain preprocessed data.
3. The method for selecting chronic disease features based on standard deviation and mutual information according to claim 2, characterized in that: Data cleaning includes: deleting variables, filling variables, and processing outliers; the deleted variables include: variables with missing values greater than 30%, highly unbalanced categorical variables, all physical examination information variables except chronic diseases, and version time and home address variables.
4. The method for selecting chronic disease features based on standard deviation and mutual information according to claim 2, characterized in that: The normalization formula is: Among them, F * represents the normalized data, F represents the unnormalized data, and F max is the maximum value in the original feature set, F min is the minimum value in the original feature set.
