Feature selection method and device based on information entropy, equipment and medium

Abstract

The invention provides a feature selection method based on information entropy, which can be applied to the field of cloud computing or big data. The method comprises the steps of obtaining a to-be-processed data set, wherein the to-be-processed data set comprises a plurality of high-dimensional features; dividing the to-be-processed data set into an unbalanced data set and a balanced data set based on the category distribution of the to-be-processed data set; for the unbalanced data set, calculating a joint entropy between high-dimensional features in the unbalanced data set, and iteratively selecting an optimal feature in the unbalanced data set according to the joint entropy to obtain a first optimal feature subset; and for the balanced data set, calculating the information entropy, the joint entropy, the conditional entropy and the information gain of the high-dimensional features in the balanced data set, and iteratively selecting the optimal features in the balanced data set according to the information entropy, the joint entropy, the conditional entropy and the information gain to obtain a second optimal feature subset. The invention further provides an information entropy-based feature selection device, equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of cloud computing or big data, and more specifically to a feature selection method, device, equipment, storage medium and program product based on information entropy. Background Art

[0002] At present, feature extraction faces a series of challenges when processing data sets containing multiple high-dimensional features. These data sets usually contain a large number of continuous attributes and discrete attributes, and the relationship between the attributes is intricate, forming a highly complex data structure. In the process of feature extraction, traditional feature extraction methods, such as principal component analysis, are mainly suitable for processing data sets with continuous attributes. In addition, due to the confidentiality and privacy restrictions of data, some data sets have been specialized. At the same time, different data sets may have different characteristics and requirements. How to be closer to practical applications and use algorithms to select the optimal subset of the data set to meet the variability in future practical applications has become an urgent problem to be solved in multi-dimensional feature engineering. Summary of the invention

[0003] In view of the above problems, the embodiments of the present disclosure provide a feature selection method, apparatus, device, medium and program product based on information entropy.

[0004] According to a first aspect of the present disclosure, a feature selection method based on information entropy is provided, the method comprising: obtaining a data set to be processed, the data set to be processed comprising a plurality of high-dimensional features; dividing the data set to be processed into an unbalanced data set and a balanced data set based on the category distribution of the data set to be processed; for the unbalanced data set, calculating the joint entropy between the high-dimensional features in the unbalanced data set, and iteratively selecting the optimal features in the unbalanced data set according to the joint entropy to obtain a first optimal feature subset; and for the balanced data set, calculating the information entropy, joint entropy, conditional entropy and information gain of the high-dimensional features in the balanced data set, and iteratively selecting the optimal features in the balanced data set according to the information entropy, the joint entropy, the conditional entropy and the information gain to obtain a second optimal feature subset.

[0005] According to an embodiment of the present disclosure, calculating the joint entropy of high-dimensional features in the unbalanced data set includes: calculating the information entropy of each high-dimensional feature in the unbalanced data set for each high-dimensional feature of the unbalanced data set; and calculating the joint entropy of each pair of high-dimensional features based on the information entropy of each high-dimensional feature of the unbalanced data set.

[0006] According to an embodiment of the present disclosure, iteratively selecting the optimal features in the unbalanced data set according to the joint entropy to obtain a first optimal feature subset includes: selecting a high-dimensional feature with the largest information entropy from the unbalanced data set, and storing the high-dimensional feature with the largest information entropy to a first selected feature subset; iteratively selecting a high-dimensional feature whose joint entropy with the high-dimensional features in the first selected feature subset is maximized from the remaining high-dimensional features of the unbalanced data set, and storing it to the first selected feature subset; and when the number of high-dimensional features in the first selected feature subset is equal to a first preset threshold, stopping the iteration to obtain the first optimal feature subset.

[0007] According to an embodiment of the present disclosure, for the balanced data set, calculating the information entropy, joint entropy, conditional entropy and information gain of high-dimensional features in the balanced data set includes: for each high-dimensional feature of the balanced data set, calculating the information entropy of each high-dimensional feature, and calculating the information gain based on the relationship between the high-dimensional feature and the category label, wherein the category label is used to characterize the attribute of the category distribution of the high-dimensional feature; and traversing the high-dimensional feature pairs formed by each high-dimensional feature in the balanced data set, calculating the joint entropy of each high-dimensional feature pair based on the information entropy of the high-dimensional feature, as well as the conditional entropy under the condition of a given preset counterpart value.

[0008] According to an embodiment of the present disclosure, the iterative selection of the optimal features in the balanced data set according to the information entropy, the joint entropy, the conditional entropy and the information gain to obtain the second optimal feature subset includes: selecting a high-dimensional feature with the largest weighted sum of information entropy and information gain from the balanced data set, and storing the high-dimensional feature with the largest information entropy and information gain in the second selected feature subset; iterative selection based on the relationship between the remaining high-dimensional features of the balanced data set and the joint entropy, conditional entropy and information gain of the high-dimensional features in the second selected feature subset, and storing the selected high-dimensional features in the second selected feature subset; and when the number of high-dimensional features in the second selected feature subset is equal to a second preset threshold, stopping the iteration to obtain the second optimal feature subset.

[0009] According to an embodiment of the present disclosure, the iterative selection based on the relationship between the joint entropy, conditional entropy and information gain of the remaining high-dimensional features of the balanced data set and the high-dimensional features in the second selected feature subset, and the selected high-dimensional features are stored in the second selected feature subset, including: calculating the weighted sum of the joint entropy and information gain between the remaining high-dimensional features of the balanced data set and the high-dimensional features of the second selected feature subset; and selecting the high-dimensional features with the largest weighted sum among the remaining high-dimensional features of the balanced data set while minimizing the conditional entropy, and storing them in the second selected feature subset.

[0010] According to an embodiment of the present disclosure, the method also includes: performing data cleaning on the data set to be processed, detecting and removing missing values, outliers and duplicate data; and performing feature encoding on the high-dimensional features of the data set to be processed after data cleaning, and performing standardization or normalization processing.

[0011] A second aspect of the present disclosure provides a feature selection device based on information entropy, the device comprising: an acquisition module, used to acquire a data set to be processed, the data set to be processed comprising multiple high-dimensional features; a division module, used to divide the data set to be processed into an unbalanced data set and a balanced data set based on the category distribution of the data set to be processed; a first selection module, used to calculate the joint entropy between the high-dimensional features in the unbalanced data set for the unbalanced data set, and iteratively select the optimal features in the unbalanced data set according to the joint entropy to obtain a first optimal feature subset; and a second selection module, used to calculate the information entropy, joint entropy, conditional entropy and information gain of the high-dimensional features in the balanced data set for the balanced data set, and iteratively select the optimal features in the balanced data set according to the information entropy, the joint entropy, the conditional entropy and the information gain to obtain a second optimal feature subset.

[0012] The third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above-mentioned feature selection method based on information entropy.

[0013] The fourth aspect of the present disclosure also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned feature selection method based on information entropy when executed by a processor.

[0014] The fifth aspect of the present disclosure also provides a computer program product, including a computer program, which implements the steps of the above-mentioned feature selection method based on information entropy when executed by a processor.

[0015] In an embodiment of the present disclosure, for a data set containing a large number of attribute variables, it is divided into a balanced data set and an unbalanced data set, and different feature selections are performed on the data sets in these two states according to the information entropy theory, so that the optimal feature subsets can be selected separately within a limited time, thereby improving the efficiency of the selection. Ultimately, it can not only effectively reduce the dimension of the data set, reduce the complexity and computational cost of subsequent model training, but also significantly improve the accuracy and robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0017] Figure 1 A diagram schematically shows an application scenario of a feature selection method, apparatus, device, medium, and program product based on information entropy according to an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of a feature selection method based on information entropy according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 3 A schematic diagram of a feature selection method based on information entropy in an unbalanced data set according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 4 Schematic diagram of a feature selection method based on information entropy in a balanced data set according to an embodiment of the present disclosure;

[0021] Figure 5 schematically shows a structural block diagram of a feature selection device based on information entropy according to an embodiment of the present disclosure; and

[0022] Figure 6 The figure schematically shows a block diagram of an electronic device according to the feature selection method based on information entropy according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0024] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0025] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0026] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0027] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0028] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0029] Figure 1 The application scenario diagram of the feature selection method, apparatus, device, medium and program product based on information entropy according to the embodiments of the present disclosure is schematically shown.

[0030] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0031] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0032] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0033] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0034] It should be noted that the feature selection method based on information entropy provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the feature selection device based on information entropy provided in the embodiment of the present disclosure can generally be set in the server 105. The feature selection method based on information entropy provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the feature selection device based on information entropy provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0035] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0036] Figure 2 The flowchart of the feature selection method based on information entropy according to an embodiment of the present disclosure is schematically shown.

[0037] like Figure 2 As shown, the feature selection method based on information entropy of this embodiment includes operations S210 to S240.

[0038] In operation S210, a data set to be processed is obtained, where the data set to be processed includes multiple high-dimensional features.

[0039] In an embodiment of the present disclosure, before obtaining the data set to be processed, the user's consent or authorization is obtained. For example, before operation S210, a request to obtain the data set to be processed is issued to the user. If the user agrees or authorizes that the data set to be processed can be obtained, operation S210 is performed.

[0040] The high-dimensional nature of a dataset represents the large number of features (or variables) contained in the dataset, each of which represents a different aspect or attribute of the data.

[0041] For example, when performing credit card fraud detection or other similar data analysis tasks, you can first obtain a dataset containing rich information to be processed. This dataset usually comes from multiple channels such as financial institutions' transaction records, user behavior logs, credit score reports, etc., and covers multiple high-dimensional features to fully reflect users' transaction habits and potential risks.

[0042] After the data set to be processed is obtained, operations S211 to S212 are performed.

[0043] In operation S211 , data cleaning is performed on the data set to be processed to detect and remove missing values, outliers, and duplicate data.

[0044] In operation S212, feature encoding is performed on the high-dimensional features of the data set to be processed after data cleaning, and standardization or normalization is performed.

[0045] In the process of processing the data set to be processed, you can first clean the data, detect and remove missing values, for example, by filling, interpolating or deleting the records where the missing values ​​are located. At the same time, correct the errors and outliers in the data, and remove duplicate records. Then extract relevant features from the original data set to be processed, such as the transaction amount, transaction time, transaction location and user account status in the credit card fraud detection scenario. According to business needs, derivative features can be further generated, such as transaction frequency and transaction category distribution. Feature encoding is performed on the high-dimensional features of the data set to be processed after data cleaning. For example, for categorical variables, one-hot encoding or label encoding can be used for conversion. Then, in order to eliminate the dimensional differences between different features, the features can be standardized or normalized.

[0046] It is understandable that data cleaning of the processed data set can ensure the integrity, stability and consistency of the data set, simplify the data set, improve computational efficiency, and avoid model overfitting. Feature encoding helps to unify the data format and improve the compatibility of the model. Standardization or normalization can eliminate the dimensional differences between different features and make them have the same scale range, thereby accelerating the convergence of the model and improving the stability and accuracy of the model.

[0047] In operation S220 , the data set to be processed is divided into an unbalanced data set and a balanced data set based on the category distribution of the data set to be processed.

[0048] According to the category distribution of the data set, the data set to be processed can be divided into an unbalanced data set and a balanced data set. An unbalanced data set refers to a data set with extremely unbalanced category distribution, while a balanced data set refers to a data set with relatively balanced category distribution. Category labels are used to characterize the properties of the category distribution of high-dimensional features, that is, category labels can reveal the category or group to which a data point belongs in the high-dimensional feature space.

[0049] Exemplarily, the credit card dataset to be processed is divided into two states: a balanced dataset and an unbalanced dataset. A balanced dataset means that the number of fraudulent transactions is similar to that of non-fraudulent transactions, while an unbalanced dataset means that the number of fraudulent transactions is far less than that of non-fraudulent transactions.

[0050] In operation S230, for the unbalanced data set, the joint entropy between high-dimensional features in the unbalanced data set is calculated, and according to the joint entropy, the optimal features in the unbalanced data set are iteratively selected to obtain a first optimal feature subset.

[0051] In operation S240, for the balanced data set, the information entropy, joint entropy, conditional entropy and information gain of the high-dimensional features in the balanced data set are calculated, and the optimal features in the balanced data set are iteratively selected based on the information entropy, joint entropy, conditional entropy and information gain to obtain a second optimal feature subset.

[0052] In the embodiment of the present disclosure, a corresponding operation portal is provided for the user to choose to agree or reject the automated decision result. That is, before the unbalanced data set selection and balanced data set processing are performed, the user is input through the corresponding operation portal to agree or reject the processing / decision. If the user agrees to perform the processing / decision, the unbalanced data set selection and balanced data set processing / decision are performed, that is, step S240 is executed. If the user refuses to perform the processing / decision, the expert decision process is entered.

[0053] In some exemplary embodiments, starting from the credit card fraud detection application scenario, a credit card data set with more data attribute variables is obtained. In the credit card fraud detection data set, fraudulent transactions generally account for a small proportion of the data set, resulting in an unbalanced data set. Whether the data set is balanced or not will directly affect the results of the feature engineering algorithm output, and further affect the performance difference of the model on the classification problem. Therefore, in response to whether the data set is unbalanced, the embodiments of the present disclosure construct different feature selection methods based on information entropy theory to extract the optimal feature subsets under different conditions.

[0054] From a mathematical point of view, the set Indicates that it contains Features and The data set of data nodes belongs to a certain feature category, where Indicates The first data node The purpose of feature selection is to select a subset of features from the original feature set that can enable the algorithm to achieve optimal performance, that is, Select from the features Representative nodes. The feature subset is represented as , , and the feature subset will not deteriorate the performance of the classification algorithm.

[0055] In some embodiments, the data set to be processed is divided into two states, balanced and unbalanced, that is, divided into unbalanced data sets and balanced data sets. For data sets in different states, the approximate calculation formula of the feature selection algorithm is derived through the mathematical theory of information entropy. Among them, corresponding to the unbalanced data set, since the number of samples in some categories is much less than that in other categories, the joint entropy is mainly calculated based on the information entropy, and the correlation between the features is considered to identify those feature combinations that contribute more to the classification task, that is, the optimal features, thereby forming the first optimal feature subset. For the balanced data set, since the number of samples in each category is similar, the information entropy, joint entropy, conditional entropy and information gain of the high-dimensional features are mainly calculated, wherein the information entropy measures the uncertainty of the data distribution; the joint entropy considers the common uncertainty of multiple features; the conditional entropy measures the degree of reduction of the uncertainty of other features after a certain feature value is given; and the information gain reflects the degree of improvement of the classification task performance of a certain feature. Based on the calculation results of the above entropy value and information gain, the optimal feature can be selected by the same iterative method. After multiple iterations and optimizations, the final second optimal feature subset is obtained. The first optimal feature subset and the second optimal feature subset together form the optimal feature subset of the data set to be processed.

[0056] It is understandable that a data set may exist in two states: balanced and unbalanced. The embodiments of the present disclosure analyze the impact of these two states on feature engineering respectively, perform different feature selections based on information entropy theory, and address the problem of both discrete attributes and continuous attributes in a data set. Based on modern information theory, it not only achieves characteristics such as minimum intra-class redundancy and maximum inter-class correlation, but also maintains a measure of the importance of the attribute itself, thereby meeting the need to efficiently obtain the optimal feature subset in practical applications.

[0057] Based on the above embodiments, in this embodiment, for an unbalanced data set, calculating the joint entropy of high-dimensional features in the unbalanced data set includes: for each high-dimensional feature of the unbalanced data set, calculating the information entropy of each high-dimensional feature in the unbalanced data set; and based on the information entropy of each high-dimensional feature of the unbalanced data set, calculating the joint entropy of each pair of high-dimensional features.

[0058] For example, the credit card fraud detection dataset has a class imbalance problem, resulting in an overwhelming amount of normal transaction data, which is close to the number of the entire dataset, so that the effect of the class label on the feature (attribute) can be ignored, that is, entropy , thus obtaining the information gain and etc., among which , Represent the unselected features and the selected features respectively. is the category label.

[0059] In some embodiments, although the distribution of the category labels of the imbalanced data set may be extremely imbalanced, the information entropy of each feature itself is still meaningful. Therefore, for each high-dimensional feature in the imbalanced data set, its information entropy can be calculated first. The larger the information entropy, the more uncertain the data distribution of the feature is, that is, the greater the amount of information contained. Then, based on the information entropy of each high-dimensional feature, the joint entropy of each pair of high-dimensional features is further calculated to measure the uncertainty of the common distribution of multiple features, taking into account the correlation between features.

[0060] It is understandable that for unbalanced data sets, ignoring the influence of category labels can avoid the interference of category imbalance on the feature selection process, focus on the information content of the data itself and the correlation between features, so that a more representative feature subset can be selected. At the same time, by calculating the information entropy of each high-dimensional feature and the joint entropy between features, feature subsets with large information content and low correlation can be identified, which helps to reduce the dimension of the data set, reduce the amount of calculation, and improve the training efficiency of the model.

[0061] On the basis of the above embodiments, in this embodiment, according to the joint entropy, iteratively selecting the optimal features in the unbalanced data set to obtain the first optimal feature subset includes: selecting high-dimensional features with the largest information entropy from the unbalanced data set, and storing the high-dimensional features with the largest information entropy in the first selected feature subset; iteratively selecting high-dimensional features whose joint entropy with the high-dimensional features in the first selected feature subset is maximized from the remaining high-dimensional features of the unbalanced data set, and storing them in the first selected feature subset; and when the number of high-dimensional features in the first selected feature subset is equal to a first preset threshold, stopping the iteration to obtain the first optimal feature subset.

[0062] Figure 3 A schematic diagram of a feature selection method based on information entropy in an unbalanced data set according to an embodiment of the present disclosure is schematically shown.

[0063] like Figure 3 As shown, according to the actual application needs, in the information processing stage of feature engineering, the embodiment of the present disclosure ignores the influence of inter-class correlation, starts from the minimum intra-class redundancy characteristics, and maximizes the joint entropy between two attributes according to the principle of information entropy, that is,

[0064] <m> argmax < / m> [ ∑ x j ∈ S H ( x t , x j ) ]

[0065] Complete the iterative selection of the best features, where and Represent the selected and unselected feature sets respectively, that is, Figure 3 Calculate the maximum sum of the shaded parts of ①, ② and ③.

[0066] In some exemplary embodiments, for the full attribute set X to be processed and the number of features (attributes) k to be selected, first initialize the data set X containing all high-dimensional features, and then clear a data set of features to be selected. Next, select the first feature, that is, select the feature with the largest information entropy calculated above, and store the feature in the cleared data set. At this time, the data set is the first selected feature subset. Then greedily select features. In this process, the number of selected features is determined based on the number of features k, and the feature with the largest joint entropy is selected and stored in the first selected feature subset according to the joint entropy between the new feature and the selected feature. When the number of features in the first selected feature subset is greater than 1, when iteratively selecting new features, the sum of the joint entropy of the new feature and the selected feature in the first selected feature subset can be calculated to select the feature with the largest sum of the joint entropy and store it in the first selected feature subset. Until the number of high-dimensional features in the first selected feature subset is equal to the first preset threshold k, or the increase in the joint entropy is no longer significant, the first optimal feature subset is finally obtained.

[0067] It should be noted that when selecting features based on the joint entropy, a method of selecting while calculating can also be adopted. Specifically, when selecting each new feature, the joint entropy between the feature and the selected features can be calculated first, and then the selection is performed based on the joint entropy.

[0068] It can be understood that in the process of iteratively selecting an unbalanced data set, selecting high-dimensional features that maximize the joint entropy with the high-dimensional features in the first selected feature subset ensures that the newly added features have a strong correlation or complementarity with the existing features, thereby enhancing the overall expressiveness of the feature subset.

[0069] On the basis of the above embodiments, in this embodiment, for a balanced data set, calculating the information entropy, joint entropy, conditional entropy and information gain of high-dimensional features in the balanced data set includes: for each high-dimensional feature of the balanced data set, calculating the information entropy of each high-dimensional feature, and calculating the information gain based on the relationship between the high-dimensional feature and the category label, wherein the category label is used to characterize the attribute of the category distribution of the high-dimensional feature; and traversing the high-dimensional feature pairs formed by each high-dimensional feature in the balanced data set, calculating the joint entropy of each high-dimensional feature pair based on the information entropy of the high-dimensional feature, and the conditional entropy under a given preset counterpart value.

[0070] Information gain is a measure of the contribution of a feature to the classification task. In a balanced data set, information gain can be calculated based on the relationship between high-dimensional features and category labels. Specifically, first, the information entropy of the entire data set on the category label is calculated to reflect the overall uncertainty of the data set. Then, for each high-dimensional feature, the conditional information entropy of the data set on the category label is calculated given the feature to reflect the degree of reduction in the uncertainty of the data set after the feature value is known. Finally, using the formula for information gain, the overall information entropy of the data set and the conditional information entropy of the feature are substituted into the formula to calculate the information gain of each high-dimensional feature. The larger the value of information gain, the greater the contribution of the feature to the classification task. Conditional entropy is a measure of the uncertainty of another feature (or feature set) given a certain feature (or feature set). In a balanced data set, the conditional entropy of other high-dimensional features can be calculated given a certain high-dimensional feature (or feature subset). In this process, the conditional probability of other high-dimensional features taking each possible value is first calculated given a certain value of a certain high-dimensional feature (or feature subset). Then, using the formula for conditional entropy, the calculated conditional probability is substituted into the formula to calculate the conditional entropy of other high-dimensional features given a certain high-dimensional feature (or feature subset).

[0071] For example, for a balanced data set containing 1000 samples, each sample has 5 high-dimensional features (F1, F2, F3, F4, F5) and a category label C (C1, C2), the information entropy, joint entropy, conditional entropy and information gain of each high-dimensional feature can be calculated according to the following steps. First, for each high-dimensional feature Fi (i=1,2,3,4,5), calculate its information entropy Hi. For example, for feature F1, count the frequency of each value of F1, and then calculate the information entropy H1 of F1 based on the frequency. Calculate joint entropy: First traverse all high-dimensional feature pairs (Fi, Fj) and calculate their joint entropy Hij. For example, for the feature pair (F1, F2), count the frequency of each value combination of F1 and F2, and then calculate the joint entropy H12 of (F1, F2) based on the frequency. Calculate conditional entropy: For each high-dimensional feature Fi (i=1,2,3,4,5), calculate the conditional entropy Hi|others of Fi given other features (or feature subsets). For example, for feature F1, given feature F2, we calculate the conditional entropy H1|F2 of F1. Calculate information gain: For each high-dimensional feature Fi (i=1,2,3,4,5), calculate the information gain IG(Fi, C) between it and the category label C. For example, for feature F1 and category label C, we first calculate the information entropy HC of the dataset on C, then calculate the conditional entropy HC|F1 of C given F1, and finally calculate the information gain I (F1,C) of F1 to C based on HC and HC|F1.

[0072] It is understandable that for balanced data sets, by calculating information entropy, joint entropy and conditional entropy, we can have a deeper understanding of the intrinsic structure of the data and the correlation between features. In addition, information gain can be used as an important basis for feature selection. In classification tasks, selecting features with larger information gain as input features can improve the performance of the classification model.

[0073] On the basis of the above embodiments, in this embodiment, according to information entropy, joint entropy, conditional entropy and information gain, iteratively selecting the optimal features in the balanced data set to obtain the second optimal feature subset includes: selecting a high-dimensional feature with the largest weighted sum of information entropy and information gain from the balanced data set, and storing the high-dimensional feature with the largest information entropy and information gain in the second selected feature subset; iteratively selecting based on the relationship between the remaining high-dimensional features of the balanced data set and the joint entropy, conditional entropy and information gain of the high-dimensional features in the second selected feature subset, and storing the selected high-dimensional features in the second selected feature subset; and when the number of high-dimensional features in the second selected feature subset is equal to the second preset threshold, stopping the iteration to obtain the second optimal feature subset.

[0074] The embodiment of the present disclosure is different from previous studies that only consider the information gain problem. In the context of the equilibrium state data set, the characteristic of the feature itself is introduced. , while considering the minimum intra-class redundancy of the balanced dataset and the maximum inter-class correlation ,in is a feature set, expressed as , is the number of features, , , For the influence coefficient, seek to implement the following formula:

[0075] argmax x t ∈ X − S [ α ⋅ H ( x t ) + β ⋅ I ( x t , C ) − δ ⋅ I ( x t , X ) ] (1)

[0076] Specific reference Figure 4 , that is, calculate the shaded part that maximizes [①+②-③-④].

[0077] In order to calculate formula (1), we need to first implement , and in order to improve the calculation speed, it is necessary to obtain Approximate calculation formula, where and , the specific mathematical theory derivation process is as follows.

[0078] Theorem 1: Assume any , hour, , then

[0079] The proof process is as follows.

[0080] when When , can be obtained

[0081] (2)

[0082] when When , can be obtained

[0083] (3)

[0084] ∵ (4)

[0085] and (5)

[0086] ∴(3)= (4) - (5)

[0087]

[0088]

[0089]

[0090] (6)

[0091] when When , can be obtained

[0092]

[0093] = [ ∑ i = 1 , 3 I ( x i , x t ) − I ( x 1 , x 3 ) + I ( x 1 , x 3 | x t ) ] + I ( x 2 , x t ) − [ ∑ i = 1 , 3 I ( x i , x 2 ) − I ( x 1 , x 3 ) + I ( x 1 , x 3 | x 2 ) ] + [ ∑ i = 1 , 3 I ( x i , x 2 | x t ) − I ( x 1 , x 3 | x t ) + I ( x 1 , x 3 | x 2 , x t ) ]

[0094] (7)

[0095] according to ,Right now

[0096] ∵

[0097] ∴(7)

[0098]

[0099]

[0100] (8)

[0101] when When , can be obtained

[0102] (9)

[0103] set up , according to Theorem 1, we can get:

[0104] argmax x t ∈ X − S [ α ⋅ H ( x t ) + β ⋅ I ( x t , C ) − δ ⋅ I ( x t , X ) ]

[0105] ≈ <m> argma x < / m> { α ⋅ H ( x t ) + β ⋅ I ( x t , C ) − 1 m ∑ i = 1 , t ≠ i m [ I ( x t , x i ) − ∑ j = i + 1 m [ I ( x i , x j ) − I ( x i , x j | x t ) ] ] } (10)

[0106] ∵

[0107] and and

[0108] ∴

[0109] Available

[0110]

[0111] − H ( x j | x t ) + H ( x i , x j )]]}

[0112]

[0113] − H ( x j | x t )]]} (11)

[0114] From (11), we can see that we only need to calculate (gains in features and class labels), (Information Entropy), (conditional entropy) and (Joint Entropy).

[0115] In some exemplary embodiments, for the full set of attributes to be processed And the number of features (attributes) k that need to be selected, parameters . First, initialize the data set X containing all high-dimensional features, and then clear a data set S of features to be selected. Next, initialize the information entropy, conditional entropy, joint entropy between features, and the information gain between features and category labels. In this process, the entropy of the feature and the gain of the feature and category label, as well as the conditional entropy and joint entropy of the feature, can be calculated, and the information gain between features can also be calculated. Then, select the first feature, that is, select the high-dimensional feature with the largest information entropy and information gain calculated above, and store the feature in the cleared data set. At this time, the data set is the second selected feature subset. Features are selected greedily again until the number of high-dimensional features in the second selected feature subset is equal to the second preset threshold k, or the increase in joint entropy is no longer significant, and finally the second optimal feature subset is obtained.

[0116] It should be noted that the information gain between high-dimensional features is calculated to deeply understand the interaction between features, so as to select the feature combination that is most important for the prediction target, and finally achieve feature dimensionality reduction and simplify the model.

[0117] It is understandable that for a balanced data set, the optimal feature subset is selected based on information entropy, joint entropy, conditional entropy and information gain, so that the model can focus more on learning the truly important features, thereby improving the prediction accuracy and generalization ability of the model. At the same time, removing redundant and unnecessary features can significantly reduce the computational complexity of the model and speed up training.

[0118] On the basis of the above embodiments, in this embodiment, based on the relationship between the joint entropy, conditional entropy and information gain of the remaining high-dimensional features of the balanced data set and the high-dimensional features in the second selected feature subset, iterative selection is performed, and the selected high-dimensional features are stored in the second selected feature subset, including: calculating the weighted sum of the joint entropy and information gain between the remaining high-dimensional features of the balanced data set and the high-dimensional features of the second selected feature subset; and selecting the high-dimensional features with the largest weighted sum among the remaining high-dimensional features of the balanced data set while minimizing the conditional entropy, and storing them in the second selected feature subset.

[0119] In some exemplary embodiments, in the process of greedily selecting features for a balanced data set, the number of selected features is first determined based on the number of features k, and the weighted sum of the joint entropy and information gain between the new features and the selected features is calculated, such as , select the feature with the largest joint entropy and store it in the first selected feature subset. Then, calculate , maximize cs-cs1, that is, minimize the conditional entropy, and select the features that meet the maximum condition and store them in the second selected feature subset.

[0120] It can be understood that by comprehensively considering joint entropy, information gain and conditional entropy, the importance of each feature can be evaluated more accurately, thereby selecting a more representative feature subset. At the same time, minimizing conditional entropy helps reduce redundant features in the feature subset and improve the effectiveness and generalization ability of the feature subset.

[0121] Figure 5 The structural block diagram of the feature selection device based on information entropy according to an embodiment of the present disclosure is schematically shown.

[0122] like Figure 5 As shown, the feature selection device 500 based on information entropy according to this embodiment includes an acquisition module 510, a division module 520, a first selection module 530 and a second selection module 540.

[0123] The acquisition module 510 is used to acquire a data set to be processed, wherein the data set to be processed includes multiple high-dimensional features. In one embodiment, the acquisition module 510 can be used to perform the operation S210 described above, which will not be described in detail here.

[0124] The partitioning module 520 is used to partition the to-be-processed data set into an unbalanced data set and a balanced data set based on the category distribution of the to-be-processed data set. In one embodiment, the partitioning module 520 can be used to perform the operation S220 described above, which will not be described in detail here.

[0125] The first selection module 530 is used to calculate the joint entropy between the high-dimensional features in the imbalanced data set for the imbalanced data set, and iteratively select the optimal features in the imbalanced data set according to the joint entropy to obtain a first optimal feature subset. In one embodiment, the first selection module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0126] The second selection module 540 is used to calculate the information entropy, joint entropy, conditional entropy and information gain of the high-dimensional features in the balanced data set, and iteratively select the optimal features in the balanced data set according to the information entropy, the joint entropy, the conditional entropy and the information gain to obtain a second optimal feature subset. In one embodiment, the second selection module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0127] In an embodiment of the present disclosure, the first selection module 530 is specifically used to: calculate the information entropy of each high-dimensional feature of the unbalanced data set for each high-dimensional feature of the unbalanced data set; and calculate the joint entropy of each pair of high-dimensional features based on the information entropy of each high-dimensional feature of the unbalanced data set.

[0128] In an embodiment of the present disclosure, the first selection module 530 is specifically used to: select the high-dimensional feature with the largest information entropy from the unbalanced data set, and store the high-dimensional feature with the largest information entropy to the first selected feature subset; iteratively select the high-dimensional feature with the maximum joint entropy with the high-dimensional features in the first selected feature subset from the remaining high-dimensional features of the unbalanced data set, and store it to the first selected feature subset; and when the number of high-dimensional features in the first selected feature subset is equal to a first preset threshold, stop the iteration to obtain the first optimal feature subset.

[0129] In an embodiment of the present disclosure, the second selection module 540 is specifically used to: calculate the information entropy of each high-dimensional feature of the balanced data set, and calculate the information gain based on the relationship between the high-dimensional feature and the category label, wherein the category label is used to characterize the attribute of the category distribution of the high-dimensional feature; and traverse the high-dimensional feature pairs formed by each high-dimensional feature in the balanced data set, and calculate the joint entropy of each high-dimensional feature pair based on the information entropy of the high-dimensional feature, as well as the conditional entropy under a given preset counterpart value.

[0130] In an embodiment of the present disclosure, the second selection module 540 is specifically used to: select a high-dimensional feature with the largest weighted sum of information entropy and information gain from the balanced data set, and store the high-dimensional feature with the largest information entropy and information gain in a second selected feature subset; perform iterative selection based on the relationship between the remaining high-dimensional features of the balanced data set and the joint entropy, conditional entropy and information gain of the high-dimensional features in the second selected feature subset, and store the selected high-dimensional features in the second selected feature subset; and when the number of high-dimensional features in the second selected feature subset is equal to a second preset threshold, stop the iteration to obtain the second optimal feature subset.

[0131] In an embodiment of the present disclosure, the second selection module 540 is specifically used to: calculate the weighted sum of the joint entropy and information gain between the remaining high-dimensional features of the balanced data set and the high-dimensional features of the second selected feature subset; and select the high-dimensional features with the largest weighted sum among the remaining high-dimensional features of the balanced data set while minimizing the conditional entropy, and store them in the second selected feature subset.

[0132] In an embodiment of the present disclosure, the acquisition module 510 can also be used to: perform data cleaning on the data set to be processed, detect and remove missing values, outliers and duplicate data; and perform feature encoding on the high-dimensional features of the data set to be processed after data cleaning, and perform standardization or normalization processing.

[0133] According to an embodiment of the present disclosure, any multiple modules among the acquisition module 510, the division module 520, the first selection module 530 and the second selection module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 510, the division module 520, the first selection module 530 and the second selection module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the acquisition module 510, the division module 520, the first selection module 530 and the second selection module 540 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0134] Figure 6A block diagram of an electronic device according to an information entropy-based feature selection method according to an embodiment of the present disclosure is schematically shown.

[0135] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0136] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0137] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0138] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0139] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0140] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the feature selection method based on information entropy provided by the embodiment of the present disclosure.

[0141] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 601. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0142] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0143] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0144] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0145] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0146] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0147] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A feature selection method based on information entropy, characterized in that: The method comprises: Obtaining a data set to be processed, wherein the data set to be processed includes multiple high-dimensional features; Based on the category distribution of the data set to be processed, dividing the data set to be processed into an unbalanced data set and a balanced data set; For the unbalanced data set, calculating the joint entropy between high-dimensional features in the unbalanced data set, and iteratively selecting the optimal features in the unbalanced data set according to the joint entropy to obtain a first optimal feature subset; and For the balanced data set, the information entropy, joint entropy, conditional entropy and information gain of the high-dimensional features in the balanced data set are calculated, and according to the information entropy, the joint entropy, the conditional entropy and the information gain, the optimal features in the balanced data set are iteratively selected to obtain a second optimal feature subset.

2. The method according to claim 1, characterized in that The step of calculating the joint entropy of high-dimensional features in the imbalanced data set includes: For each high-dimensional feature of the unbalanced data set, calculating the information entropy of each high-dimensional feature in the unbalanced data set; and Based on the information entropy of each high-dimensional feature of the imbalanced data set, the joint entropy of each pair of high-dimensional features is calculated.

3. The method according to claim 2, characterized in that The iterative selection of the optimal features in the imbalanced data set according to the joint entropy to obtain a first optimal feature subset includes: Selecting a high-dimensional feature with the largest information entropy from the unbalanced data set, and storing the high-dimensional feature with the largest information entropy into a first selected feature subset; Iteratively select high-dimensional features that maximize the joint entropy with the high-dimensional features in the first selected feature subset from the remaining high-dimensional features of the imbalanced data set, and store them in the first selected feature subset; and When the number of high-dimensional features in the first selected feature subset is equal to a first preset threshold, the iteration is stopped to obtain a first optimal feature subset.

4. The method according to claim 1, characterized in that: The step of calculating the information entropy, joint entropy, conditional entropy and information gain of high-dimensional features in the balanced data set includes: For each high-dimensional feature of the balanced data set, calculating the information entropy of each high-dimensional feature, and calculating the information gain according to the relationship between the high-dimensional feature and the category label, wherein the category label is used to characterize the attribute of the category distribution of the high-dimensional feature; and The high-dimensional feature pairs formed by the high-dimensional features in the balanced data set are traversed, and the joint entropy of each high-dimensional feature pair is calculated according to the information entropy of the high-dimensional features, as well as the conditional entropy under a given preset counterpart value.

5. The method according to claim 1 or 4, characterized in that: The iterative selection of the optimal features in the balanced data set according to the information entropy, the joint entropy, the conditional entropy and the information gain to obtain the second optimal feature subset includes: Selecting a high-dimensional feature with the largest weighted sum of information entropy and information gain from the balanced data set, and storing the high-dimensional feature with the largest information entropy and information gain in a second selected feature subset; Iteratively select the remaining high-dimensional features of the balanced data set based on the relationship between the joint entropy, conditional entropy and information gain of the high-dimensional features in the second selected feature subset, and store the selected high-dimensional features in the second selected feature subset; and When the number of high-dimensional features in the second selected feature subset is equal to a second preset threshold, the iteration is stopped to obtain a second optimal feature subset.

6. The method according to claim 5, characterized in that The iterative selection based on the relationship between the remaining high-dimensional features of the balanced data set and the joint entropy, conditional entropy and information gain of the high-dimensional features in the second selected feature subset, and storing the selected high-dimensional features in the second selected feature subset includes: Calculating a weighted sum of joint entropy and information gain between the remaining high-dimensional features of the balanced data set and the high-dimensional features of the second selected feature subset; and Under the condition of minimizing the conditional entropy, the high-dimensional feature with the largest weighted sum among the remaining high-dimensional features of the balanced data set is selected and stored in the second selected feature subset.

7. The method according to any one of claims 1, 2 and 4, characterized in that The method further comprises: Performing data cleaning on the data set to be processed to detect and remove missing values, outliers and duplicate data; and The high-dimensional features of the data set to be processed after data cleaning are feature encoded and standardized or normalized.

8. A feature selection device based on information entropy, characterized in that: The device comprises: An acquisition module is used to acquire a data set to be processed, wherein the data set to be processed includes multiple high-dimensional features; A partitioning module, configured to partition the data set to be processed into an unbalanced data set and a balanced data set based on the category distribution of the data set to be processed; a first selection module, configured to calculate, for the imbalanced data set, a joint entropy between high-dimensional features in the imbalanced data set, and iteratively select optimal features in the imbalanced data set according to the joint entropy to obtain a first optimal feature subset; and The second selection module is used to calculate the information entropy, joint entropy, conditional entropy and information gain of the high-dimensional features in the balanced data set, and iteratively select the optimal features in the balanced data set according to the information entropy, the joint entropy, the conditional entropy and the information gain to obtain a second optimal feature subset.

9. An electronic device, comprising: one or more processors; a storage device for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images