Classification model feature selection method, device and equipment
By using the method of attribute correlation and difference analysis in small samples, the input features of the classification model are selected, which solves the problem of inaccurate feature selection in the prior art and improves the stability and accuracy of the model.
Patent Information
- Application Number
- CN202210344015.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-02
- Publication Date
- 2025-03-14
- Estimated Expiration
- 2042-04-02
AI Technical Summary
The feature selection of classification models in the prior art is inaccurate in the case of small samples, resulting in the impact of model stability and accuracy.
By obtaining the attribute data and classification annotation data of multiple training samples used to train the classification model, the attribute vectors of each attribute are formed, the correlation between any two attributes is determined for the combination of attributes, and the target attribute is selected as the input feature of the classification model according to the correlation.
Effectively remove invalid and duplicate features, reduce interference to the stability and accuracy of the classification model, and improve the stability and accuracy of the model.
Smart Images
Figure CN114764594B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and specifically to a classification model feature selection method and device, and an electronic device. Background Art
[0002] A classification model is a model used to predict the probability that a data object belongs to a certain category. The input is information related to the data object, and the output is the category to which the data object belongs. Taking the training of a small sample classification model as an example, there are 20,000 samples in total, and more than 100 attribute values of the training samples are selected as input features.
[0003] Since it is impossible to identify the validity of input features from an application perspective, when training the model with input features, there are many invalid and similar input features, which leads to large differences in the results of multiple model runs. These input features need to be filtered. Currently, the following feature selection methods are mainly used:
[0004] 1) Variance analysis method, that is, calculating the variance between input features and retaining features with large variance. This method has large errors, and features with small variance may also be effective features.
[0005] 2) Pearson correlation coefficient univariate analysis method, that is, calculate the correlation between the input features and the classification target, and retain the features with higher correlation with the classification target. This method also has errors. In model training, the features and classification targets are generally not linearly correlated; and this method is affected by the classification target and is unstable in the case of small samples.
[0006] 3) Recursive feature elimination method, that is, after the model training is completed, the weight of each feature on the model is evaluated, and the features with low weight are continuously deleted. This method has a large amount of computation and requires multiple recursive training of the model; at the same time, it will be unstable in the case of small samples.
[0007] In summary, the existing technology has the problem of inaccurate feature selection, which affects the stability and accuracy of the model in the case of small samples. Summary of the invention
[0008] The present application provides a classification model feature selection method to solve the problem of inaccurate feature selection in the prior art. The present application also provides a classification model feature selection device and an electronic device.
[0009] The present application provides a classification model feature selection method, comprising:
[0010] Obtaining attribute data and classification labeling data of multiple training samples for training a classification model;
[0011] According to the attribute data of multiple training samples on each attribute, an attribute vector of each attribute is formed;
[0012] For each combination of attributes, determine the correlation between any two attributes according to the attribute vector;
[0013] Determine, according to the correlation, a first target attribute and a combination of attributes to be deduplicated;
[0014] Select a second target attribute from the attribute combination to be deduplicated;
[0015] A third target attribute is selected from the first target attribute and the second target attribute as an input feature of the classification model.
[0016] Optionally, determining the first target attribute and the attribute combination to be deduplicated according to the correlation includes:
[0017] Taking the attribute in the attribute combination whose relevance is less than the relevance threshold as the first target attribute;
[0018] The attribute combination whose correlation is greater than or equal to the correlation threshold is used as the attribute combination to be deduplicated.
[0019] Optionally, determining the first target attribute and the attribute combination to be deduplicated according to the correlation includes:
[0020] Determining relevance ranking data of the attribute combination according to the relevance;
[0021] A first target attribute is selected according to the ranking data and the attribute combination selection ratio.
[0022] Optionally, selecting a second target attribute from the attribute combination to be deduplicated includes:
[0023] Determine the attribute difference between different classification samples of the attribute to be deduplicated and / or the non-correlation between the attribute to be deduplicated and the first target attribute;
[0024] According to the attribute difference and / or the non-correlation of the attribute to be deduplicated, an attribute is selected from the attribute combination to be deduplicated as the second target attribute.
[0025] Optionally, the attribute difference is obtained in the following manner:
[0026] According to the sample classification status corresponding to the attribute data, the sum of the attribute data of each type of sample is obtained;
[0027] For each combination of sample classifications, the difference in the sum of attribute data between the two categories of samples is determined according to the sum of attribute data;
[0028] The statistical data of the gaps corresponding to the combinations of two or more of the categories are used as the attribute difference.
[0029] Optionally, the non-correlation degree is obtained in the following manner:
[0030] Determine the non-correlation between the attribute to be deduplicated and each first target attribute;
[0031] The statistical data of the non-correlation degree corresponding to each first target attribute is used as the non-correlation degree between the attribute to be deduplicated and the first target attribute.
[0032] Optionally, the selecting a second target attribute from the attribute combination to be deduplicated further includes:
[0033] Determine the correlation between the attribute to be deduplicated and the classification state of the training sample and / or the variance of the attribute to be deduplicated;
[0034] The second target attribute is determined according to the attribute difference and / or the non-correlation of the attribute to be deduplicated, and the correlation and / or the variance with the classification state of the training sample.
[0035] Optionally, selecting a third target attribute from the first target attribute and the second target attribute includes:
[0036] For the first target attribute and the second target attribute, obtaining the attribute difference of the attribute and / or the correlation between the attribute and the classification state of the training sample according to the attribute vector;
[0037] A third target attribute is selected from the first target attribute and the second target attribute according to the attribute difference and / or the correlation between the attribute and the classification state of the training sample.
[0038] The present application also provides a classification model feature selection method, including:
[0039] Obtaining attribute data and classification labeling data of multiple training samples for training a classification model;
[0040] According to the characteristic data of multiple training samples on each attribute, an attribute vector of each attribute is formed;
[0041] For each combination of attributes, determining the attribute difference between different classification samples according to the feature vector;
[0042] According to the attribute difference, a target attribute is selected from the attribute combination as an input feature of the classification model.
[0043] Optionally, also include:
[0044] Determine the relevance of the features to the classification status of the training samples;
[0045] A target attribute is selected from the attribute combination according to the attribute difference and the correlation with the classification state of the training sample.
[0046] The present application also provides a classification model feature selection device, comprising:
[0047] A training sample acquisition unit, used to acquire attribute data and classification label data of a plurality of training samples used to train a classification model;
[0048] An attribute vector generating unit, used for forming an attribute vector of each attribute according to the attribute data of multiple training samples on each attribute;
[0049] An attribute correlation determination unit, for determining the correlation between any two attributes according to the attribute vector for each combination of attributes;
[0050] A first selection unit, configured to determine a first target attribute and a combination of attributes to be deduplicated according to the correlation;
[0051] A second selection unit, used for selecting a second target attribute from the attribute combination to be deduplicated;
[0052] The third selection unit is used to select a third target attribute from the first target attribute and the second target attribute as an input feature of the classification model.
[0053] The present application also provides a classification model feature selection device, comprising:
[0054] A training sample acquisition unit, used to acquire attribute data and classification label data of a plurality of training samples used to train a classification model;
[0055] An attribute vector generating unit, used for forming an attribute vector of each attribute according to the feature data of multiple training samples on each attribute;
[0056] An attribute difference determination unit, used for determining the attribute difference between different classification samples according to the feature vector for each combination of attributes;
[0057] The feature selection unit is used to select a target attribute from the attribute combination according to the attribute difference as an input feature of the classification model.
[0058] The present application also provides an electronic device, including:
[0059] Processor and memory;
[0060] The memory is used to store a program for implementing the feature selection method according to the above classification model. The device is powered on and runs the program of the method through the processor.
[0061] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned various methods.
[0062] The present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the above-mentioned various methods.
[0063] Compared with the prior art, this application has the following advantages:
[0064] The classification model feature selection method provided by the embodiment of the present application is to obtain the attribute data and classification annotation data of multiple training samples for training the classification model; form the attribute vector of each attribute according to the attribute data of multiple training samples on each attribute; determine the correlation between any two attributes according to the attribute vector for the pairwise combination of attributes; determine the first target attribute and the attribute combination to be deduplicated according to the correlation; select the second target attribute from the attribute combination to be deduplicated; select the third target attribute from the first target attribute and the second target attribute as the input feature of the classification model. By adopting this processing method, the attributes of the pairwise combination with lower correlation (the first target feature) are first selected, and then the attributes of the pairwise combination with higher correlation (the attribute combination to be deduplicated) are automatically filtered based on the attribute difference between different classification samples of each attribute, the non-correlation between the attribute and the first target attribute, and the correlation between the attribute and the classification state of the sample, so as to effectively remove invalid and repeated features, thereby reducing the interference to the stability and accuracy of the classification model, and thus improving the stability and accuracy of the classification model.
[0065] The classification model feature selection method provided in the embodiment of the present application obtains the attribute data and classification annotation data of multiple training samples used to train the classification model; forms the attribute vector of each attribute according to the attribute data of multiple training samples on each attribute; determines the attribute difference between different classification samples according to the attribute vector for each combination of attributes; and selects the target attribute from the attribute combination according to the attribute difference as the input feature of the classification model. This processing method can automatically filter the input features of the classification model based on the attribute difference analysis between different classification samples of the attributes of the training samples, and can effectively remove invalid and repeated features, thereby reducing interference with the stability and accuracy of the classification model, and thus improving the stability and accuracy of the classification model. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 Schematic diagram of the process of selecting features of the classification model provided in this application;
[0067] Figure 2A schematic diagram of a specific process of the classification model feature selection method provided in this application;
[0068] Figure 3 A schematic diagram of a specific process of the classification model feature selection method provided in this application;
[0069] Figure 4 A flowchart of the classification model feature selection method provided in this application. DETAILED DESCRIPTION
[0070] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.
[0071] In the present application, a classification model feature selection method and device, and an electronic device are provided. In the following embodiments, various solutions are described in detail one by one.
[0072] First embodiment
[0073] Please refer to Figure 1 , which is a flow chart of an embodiment of the classification model feature selection method of the present application. In this embodiment, the method may include the following steps:
[0074] Step S101: Acquire attribute data and classification label data of multiple training samples for training a classification model.
[0075] The classification model is a machine learning model that predicts the category of a data object based on the multi-faceted feature data of the data object. The data object may be a user, a commodity, a voice, a text, etc. The feature data of the data object is derived from the attribute data (also known as the attribute value) of the data object. Among all the attribute data of the data object, some of the attribute data are quite different and have a relatively important impact on the classification of the data object, and another part of the attribute data has a similar impact on the classification of the data object as the previous part of the attribute data. By using the method provided in the embodiment of the present application, the previous part of the attribute data can be automatically screened out from all the attribute data of the data object, and this type of attribute data is called the feature data of the data object. The automatically screened feature data is used as the input data of the classification model, and the classification model outputs the category prediction information of the data object.
[0076] The classification model can be learned by machine learning based on the feature data and classification annotation data of multiple training samples. The training samples are also data objects. The classification model of this embodiment is a supervised machine learning model. The data of the training samples include multiple feature data and classification annotation data of the data objects. The classification annotation data can be obtained by manual annotation. According to the classification annotation data, the training samples can be divided into different categories. If there are only two categories, the training samples can be divided into positive samples and negative samples; if there are multiple categories, the training samples can be divided into multiple categories. The classification model can be a classification model based on a neural network or a classification model using other structures.
[0077] The classification model includes but is not limited to a small sample classification model, that is, a small number of training samples. For example, a classification model is a model used to predict whether a user will sign a contract, and the classification model is trained based on about 20,000 samples. In this case, the data object is a user, and the attribute information of the data object includes user information, such as user gender, age, education level, etc. The classification annotation data is whether the user has signed a contract. The training sample data of this embodiment is shown in Table 1 below:
[0078]
[0079] Table 1. Training sample data
[0080] Step S103: forming an attribute vector of each attribute according to the attribute data of multiple training samples on each attribute.
[0081] Each training sample includes a plurality of attribute data (attribute values), and the data of the same attribute of the plurality of training samples are used to form an attribute vector of the attribute, thereby forming an attribute vector of each attribute.
[0082] This step can be formally expressed as: select M samples to train the classification model, each sample contains m attribute data, and the data of the same attribute of the M samples are combined into an attribute vector including M attribute data, forming a total of m attribute vectors.
[0083] For example, if a classification model for determining whether a user will sign a contract is to be trained based on 20,000 user samples, after executing this step, the gender attribute can be obtained: {0, 1, 1, 1, 1, 0, ... 1}. This vector includes the gender values of 20,000 users, with males represented by 0 and females by 1. In addition, attribute vectors corresponding to other attributes such as education level can also be obtained, which will not be described here.
[0084] Step S105: for any combination of attributes, determine the correlation between any two attributes according to the attribute vector.
[0085] In this step, all attributes of the training samples are combined in pairs. For any set of attribute combinations, the correlation between the two attributes is calculated based on the attribute vectors of the two attributes to measure the correlation between the two attributes.
[0086] The correlation may be a Pearson correlation coefficient, or other methods may be used to calculate the correlation between two attributes.
[0087] In this embodiment, the Pearson correlation coefficient between the attribute vectors of the two attributes is calculated. The Pearson correlation coefficient is used to measure the correlation between two variables X and Y, and its value is between -1 and 1. The Pearson correlation coefficient between the two attribute vectors is the quotient of the covariance and the standard deviation between the two attribute vectors.
[0088] The Pearson correlation coefficient can be expressed as follows:
[0089]
[0090] in σX is the sum of the i The standard score, sample mean and sample standard deviation of the sample. In this embodiment, X represents an attribute vector of an attribute, and the dimension of the attribute vector is the number of training samples.
[0091] Step S107: Determine a combination of the first target attribute and the attribute to be deduplicated according to the correlation.
[0092] In this step, based on the correlation between any two attributes, the attribute combinations are divided into highly correlated attribute combinations and low correlated attribute combinations, and the two attributes in the low correlated attribute combinations are used as the first target attributes. For example, age and education level are low correlated attributes, and these two attributes are used as the first target attributes; age and date of birth are highly correlated attributes, and the combination of these two attributes is used as the attribute combination to be deduplicated.
[0093] In one example, a highly correlated attribute combination may be an attribute combination in which the correlation between the two attributes is greater than or equal to a correlation threshold, and a lowly correlated attribute combination may be an attribute combination in which the correlation between the two attributes is less than the correlation threshold. The correlation threshold may be set according to application requirements. For example, if the correlation threshold is set to 50%, then if the correlation between the two attributes reaches 50%, it means that the two attributes are highly correlated and need to be de-duplicated; if the correlation between the two attributes is below 50%, it means that the two attributes are unrelated, and the two unrelated attributes are used as the first target attributes.
[0094] In another example, step S107 can be implemented as follows: according to the correlation, determine the correlation ranking data of the feature combination; according to the ranking data and the feature combination selection ratio, select the first target feature. For example, set the proportion of high correlation number p=5%, the correlation ranking threshold N=floor(m*(m-1) / 2*p), and floor is a rounding function. Assuming there are one hundred attributes, the proportion of high correlation number p=5%, then N=247, the attributes ranked before 247 in pairs are highly correlated attribute combinations, and the attributes ranked after 247 are low-correlated attribute combinations. With this processing method, the high and low attribute correlations can be divided according to the proportion. No matter how large or small the sample size is, a certain proportion of attribute combinations can be selected for subsequent deduplication processing; therefore, the computational complexity and accuracy of feature selection can be effectively balanced.
[0095] In one example, the method may further include the following step: obtaining a correlation threshold or a high correlation ratio specified by a user. In this processing method, the user may specify relevant parameters, which can effectively improve the flexibility of feature selection.
[0096] Step S109: Select a second target attribute from the attribute combination to be deduplicated.
[0097] In this step, two attributes in attribute combinations with high correlation (attribute combinations to be deduplicated) are compared, and one attribute is selected from each pair of attribute combinations as the second target attribute.
[0098] like Figure 2 As shown, in one example, step S109 may include the following sub-steps:
[0099] Step S109 - 1 : Determine the attribute difference between different classification samples of the attribute to be deduplicated and / or the non-correlation between the attribute to be deduplicated and the first target attribute.
[0100] The attributes in the attribute combination to be deduplicated are called attributes to be deduplicated. This step can determine the attribute difference between the attributes to be deduplicated in different classification samples, and can also determine the non-correlation between the attributes to be deduplicated and the first target attribute. It can also simultaneously determine the attribute difference between the attributes to be deduplicated in different classification samples and the non-correlation between the attributes to be deduplicated and the first target attribute.
[0101] 1) Determine the attribute difference between different classification samples of the attributes to be deduplicated.
[0102] The attribute combination to be deduplicated includes two attributes (attribute X and attribute Y). The attribute difference between samples of different categories for any attribute can be defined as: obtaining the data sum of the attribute corresponding to the samples of each category; calculating the difference between the attribute data sums of the two categories for each combination of categories; calculating the statistical value of the difference between the attribute data sums corresponding to each combination of categories, such as the average value, maximum value or minimum value. The greater the attribute difference of an attribute, the greater the influence of the attribute on the sample classification, and the more it should be retained.
[0103] In one example, the attribute difference can be obtained in the following manner: according to the sample classification status corresponding to the attribute data, the sum of the attribute data of the attribute in each category of samples is obtained; for each pairwise combination of sample classifications, the difference in the sum of the attribute data between two categories of samples is determined according to the sum of the attribute data; and the statistical data of the difference corresponding to the pairwise combination of multiple classifications is used as the attribute difference.
[0104] In one example, the attribute difference can be obtained in the following manner: normalize the attribute vector; obtain the sum of the normalized attribute data of the attribute in each category of samples based on the normalized attribute data and the sample classification status corresponding to the attribute data; for each pairwise combination of sample classifications, determine the absolute value of the difference between the sum of the attribute data of two categories of samples based on the sum of the normalized attribute data; and take the average of the absolute values of the differences corresponding to multiple pairwise combinations as the attribute difference.
[0105] In practical applications, the data of two attributes in an attribute combination may have different data magnitudes, and there is no direct comparability between the two attribute data. In order to make different data comparable, in specific implementation, the attributes that need to be deduplicated are differentiated and calculated separately. The attribute vector of the attribute can be normalized first. Each attribute after normalization includes M normalized attribute data. The normalized attribute data are summed according to the classification status of the training sample. Then, for multiple classifications, the difference of the sum of the corresponding attribute data is calculated. Finally, the absolute value of the difference corresponding to the multiple classifications is averaged.
[0106] For example, attribute X and attribute Y are two attributes that need to be deduplicated, and their corresponding attribute vectors are [x1,x2...xn], [y1,y2...yn] respectively; after normalization, these two attribute vectors are [x1 / sum(x),x2 / sum(x)...xn / sum(x)], [y1 / sum(y),y2 / sum(y)...yn / sum(y)] respectively; according to the classification status of each training sample (such as the first category, the second category, the third category, etc.), these two attribute vectors are split. Assuming that x1~xm, y1~ym are the first category (such as contract signed), xm+1~xn, ym+1~yn are the second category (such as non-contract signed), then the sum of the first category corresponding to attribute X is sum(x1 / s The sum of the first category corresponding to attribute Y is sum(y1 / sum(y)~ym / sum(y)), and the sum of the second category is sum(xm+1 / sum(x)~xn / sum(x)); the sum of the first category corresponding to attribute Y is sum(y1 / sum(y)~ym / sum(y)), and the sum of the second category is sum(ym+1 / sum(y)~yn / sum(y)); the absolute value of the difference corresponding to the first category is abs(sum(x1 / sum(x)~xm / sum(x))-sum(xm+1 / sum(x)~xn / sum(x))), and the absolute value of the difference corresponding to the second category is abs(sum(y1 / sum(y)~ym / sum(y))-sum(ym+1 / sum(y)~yn / sum(y))). In specific implementation, there can also be a third category and its corresponding absolute value of the difference.
[0107] 2) Determine the non-correlation between the attribute to be deduplicated and the first target attribute.
[0108] The non-correlation refers to the degree of non-correlation between the attribute to be deduplicated and the first target attribute. The non-correlation can be obtained in the following manner: determining the non-correlation between the attribute to be deduplicated and each first target attribute; and using the statistical data of the non-correlation corresponding to each first target attribute as the non-correlation between the attribute to be deduplicated and the first target attribute.
[0109] In this embodiment, for the attributes to be deduplicated, their correlation with the first target attribute (also called the retained feature) is calculated respectively, and after the absolute value of the correlation is subtracted from 1, the absolute values (the non-correlation between the two attributes) are summed and averaged (a statistical data, which may also be a maximum value or a minimum value, etc.), as the average non-correlation between the attribute and the retained attribute. When calculating the correlation between the two attributes, the Pearson correlation coefficient may be used.
[0110] For example, attribute X and attribute Y are two attributes that need to be deduplicated, and the correlations between these two attributes and the first target attributes A, B, and C are [Pxa, Pxb, Pxc], [Pya, Pyb, Pyc], respectively, where Pxa represents the correlation between attribute X and attribute A, and Pxb represents the correlation between attribute X and attribute B. The rest are not repeated here. After obtaining the correlation between the attributes to be deduplicated and the first target attribute, the absolute value of the correlation is subtracted from 1: [1-abs(Pxa), 1-abs(Pxb), 1-abs(Pxc)], [1-abs(Pya), 1-abs(Pyb), 1-abs(Pyc)]; then the sum and average are: for attribute X, the corresponding difference average is (1-abs(Pxa)+1-abs(Pxb)+1-abs(Pxc)) / 3, which represents the average non-correlation between attribute X and the retained attributes A, B, and C; for attribute Y, the corresponding difference average is (1-abs(Pya)+1-abs(Pyb)+1-abs(Pyc)) / 3, which represents the average non-correlation between attribute Y and the retained attributes A, B, and C.
[0111] Step S109 - 3 : selecting an attribute from the attribute combination to be deduplicated according to the attribute difference degree and / or the non-correlation degree of the attribute to be deduplicated as a second target attribute.
[0112] In this step, the second target attribute may be selected according to the attribute difference, or according to the non-correlation, or according to both the attribute difference and the non-correlation.
[0113] In specific implementation, step S1093 can be implemented in the following manner: determine the score of the attribute to be deduplicated based on the attribute difference and / or the non-correlation of the attribute to be deduplicated; select an attribute from the attribute combination to be deduplicated based on the score as the second target attribute. When calculating the score, different weights can be set for the attribute difference and the non-correlation, so that the score of the attribute to be deduplicated is a weighted average or a weighted sum. In this embodiment, an attribute with a high score is selected as the second target attribute, that is, an attribute with a greater attribute difference between different classification samples and a higher non-correlation with the first target attribute is selected.
[0114] In one example, step S109 may further include the following sub-steps:
[0115] Step S109 - 2 : Determine the correlation between the attribute to be deduplicated and the classification state of the training sample and / or the variance of the attribute to be deduplicated.
[0116] 1) Determine the correlation between the attributes to be deduplicated and the classification status of the training samples.
[0117] In this embodiment, the correlation between the attribute to be deduplicated and the classification state of the training sample can be calculated in the following manner: determine the correlation between the attribute vector of the attribute to be deduplicated and the sample classification vector. The sample classification vector includes classification annotation data of each training sample.
[0118] In specific implementation, the correlation between the classification status of the M training samples and the attributes to be deduplicated can be calculated, and the absolute value of the calculation result can be taken, and the Pearson correlation coefficient can be used. For example, the attribute vector of attribute X is {X1, X2...Xn}, and the vector of the classification status is {Y1, Y2...Yn}, then the correlation between the attribute vector and the classification vector can be calculated using the Pearson correlation coefficient:
[0119]
[0120] Where X represents the attribute vector of an attribute, and Y represents the sample classification vector.
[0121] 2) Determine the variance of the attribute to be deduplicated.
[0122] In specific implementation, the variance of the attribute to be deduplicated may be calculated using an existing method, which will not be described in detail here.
[0123] Accordingly, step S109-3 may be implemented in the following manner: determining the second target attribute according to the attribute difference and / or the non-correlation of the attribute to be deduplicated, and the correlation and / or the variance with the classification state of the training sample.
[0124] In this embodiment, the score of the attribute to be deduplicated is determined based on the attribute difference, the non-correlation, the correlation with the classification state of the training sample and the variance. The weighted average calculation is performed on the differential calculated value of the attribute to be deduplicated (the attribute difference), the average non-correlation with the retained attribute (the first target attribute) (the non-correlation), the correlation with the classification state and the normalized variance to obtain the weighted average of each attribute to be deduplicated, in order of importance from large to small: the attribute difference, the non-correlation, the correlation with the classification state, and the weight distribution of 45%, 35%, and 20% can be used.
[0125] For example, attribute X and attribute Y are two attributes that need to be deduplicated; the attribute difference of attribute X is 50, the average non-correlation corresponding to attribute X is 80, the correlation between attribute X and the classification state is 15, and the normalized variance of attribute X is 10; the attribute difference of attribute Y is 80, the average non-correlation corresponding to attribute X is 75, the correlation between attribute X and the classification state is 50, and the normalized variance of attribute X is 20; then the weighted average of attribute X is less than the weighted average of attribute Y, attribute Y is selected as the second target attribute, and attribute X is filtered out.
[0126] Step S111: Select a third target attribute from the first target attribute and the second target attribute as an input feature of the classification model.
[0127] This step combines the first target attribute and the second target attribute, and then selects the third target feature as the input feature of the classification model.
[0128] In one example, step S111 may include the following sub-steps:
[0129] Step S111 - 1 : for the first target attribute and the second target attribute, according to the attribute vector, obtain the attribute difference of the attribute and / or the correlation between the attribute and the classification state of the training sample.
[0130] The attribute difference of the second target attribute and the correlation between the classification status of the training sample have been obtained in the above steps S109-1 and S109-2. The attribute difference of the first target attribute and the correlation between the classification status of the training sample can be obtained by the same processing method, which will not be repeated here.
[0131] This step may obtain the attribute difference of each attribute, or obtain the correlation between each attribute and the classification state of the training sample, or may simultaneously obtain the attribute difference of each attribute and the correlation between the classification state of the training sample.
[0132] Step S111 - 3 : selecting a third target attribute from the first target attribute and the second target attribute according to the attribute difference and / or the correlation between the attribute and the classification state of the training sample.
[0133] In this step, the third target attribute can be selected based on the attribute difference of each attribute, or based on the correlation between each attribute and the classification state of the training sample. The third target attribute can also be selected based on both the attribute difference of each attribute and the correlation with the classification state of the training sample.
[0134] In one example, step S111-3 may include the following sub-steps:
[0135] Step S111-3-1: Obtain attribute difference ranking data, and determine the attribute difference threshold value according to the attribute difference ranking data and the attribute selection ratio.
[0136] In this step, the attribute difference threshold is screened according to the attribute difference of all attributes in the first target attribute and the second target attribute; the attribute difference threshold can be the value of the first 90% (attribute selection ratio) of all attributes. For example, the attribute difference of all attributes is arranged from large to small, and the first 90% of the attributes are selected.
[0137] Step S111-3-3: Obtain the ranking data of the correlation with the classification state of the training sample, and determine the threshold of the correlation with the classification state of the training sample according to the ranking data of the correlation with the classification state of the training sample and the attribute selection ratio.
[0138] Step S111 - 3 - 5 : Selecting a third target attribute from the first target attribute and the second target attribute according to the attribute difference threshold and the threshold of the correlation with the classification state of the training sample.
[0139] For example, the attributes whose attribute difference and the correlation with the classification state of the training sample are both lower than the corresponding threshold are removed, and the remaining attributes are three-target attributes.
[0140] In this embodiment, the attribute data of the third target attribute and the classification status of the corresponding sample are brought into the small sample classification model for training, and the trained small sample classification model is used to predict the classification status of the data to be classified.
[0141] From the above embodiments, it can be seen that the classification model feature selection method provided by the embodiment of the present application obtains the attribute data and classification annotation data of multiple training samples for training the classification model; forms the attribute vector of each attribute according to the attribute data of multiple training samples on each attribute; for the pairwise combination of attributes, determines the correlation between any two attributes according to the attribute vector; determines the first target attribute and the attribute combination to be deduplicated according to the correlation; selects the second target attribute from the attribute combination to be deduplicated; selects the third target attribute from the first target attribute and the second target attribute as the input feature of the classification model. By adopting this processing method, the attributes of the pairwise combination with lower correlation (the first target feature) are selected first, and then the attributes of the pairwise combination with higher correlation (the attribute combination to be deduplicated) are selected. Based on the attribute difference between different classification samples of each attribute, the non-correlation between the attribute and the first target attribute, and the correlation between the attribute and the classification state of the sample, the input features of the classification model are automatically filtered, which can effectively remove invalid and repeated features, thereby reducing the interference to the stability and accuracy of the classification model, and thus improving the stability and accuracy of the classification model.
[0142] Second embodiment
[0143] In the above embodiment, a feature selection method is provided, and correspondingly, the present application also provides a feature selection device. The device corresponds to the embodiment of the above method. The parts of this embodiment that are the same as the first embodiment are not repeated here, please refer to the corresponding parts in the first embodiment.
[0144] A feature selection device provided by the present application includes:
[0145] A training sample acquisition unit, used to acquire attribute data and classification label data of a plurality of training samples used to train a classification model;
[0146] An attribute vector generating unit, used for forming an attribute vector of each attribute according to the attribute data of multiple training samples on each attribute;
[0147] An attribute correlation determination unit, for determining the correlation between any two attributes according to the attribute vector for each combination of attributes;
[0148] A first selection unit, configured to determine a first target attribute and a combination of attributes to be deduplicated according to the correlation;
[0149] A second selection unit, used for selecting a second target attribute from the attribute combination to be deduplicated;
[0150] The third selection unit is used to select a third target attribute from the first target attribute and the second target attribute as an input feature of the classification model.
[0151] Optionally, the first selection unit is specifically configured to select an attribute in the attribute combination whose correlation is less than a correlation threshold as a first target attribute; and select an attribute combination whose correlation is greater than or equal to the correlation threshold as an attribute combination to be deduplicated.
[0152] Optionally, the first selection unit is specifically used to determine relevance ranking data of the attribute combination according to the relevance; and select the first target attribute according to the ranking data and the attribute combination selection ratio.
[0153] Optionally, the second selection unit is specifically used to determine the attribute difference between the attribute to be deduplicated and different classification samples and / or the non-correlation between the attribute to be deduplicated and the first target attribute; and select an attribute from the attribute combination to be deduplicated as the second target attribute based on the attribute difference and / or the non-correlation of the attribute to be deduplicated.
[0154] Optionally, the attribute difference is obtained in the following manner:
[0155] According to the sample classification status corresponding to the attribute data, the sum of the attribute data of each type of sample is obtained;
[0156] For each combination of sample classifications, the difference in the sum of attribute data between the two categories of samples is determined according to the sum of attribute data;
[0157] The statistical data of the gaps corresponding to the combinations of two or more of the categories are used as the attribute difference.
[0158] Optionally, the non-correlation degree is obtained in the following manner:
[0159] Determine the non-correlation between the attribute to be deduplicated and each first target attribute;
[0160] The statistical data of the non-correlation degree corresponding to each first target attribute is used as the non-correlation degree between the attribute to be deduplicated and the first target attribute.
[0161] Optionally, the second selection unit is also used to determine the correlation between the attribute to be deduplicated and the classification status of the training samples and / or the variance of the attribute to be deduplicated; determine the second target attribute based on the attribute difference and / or the non-correlation of the attribute to be deduplicated, as well as the correlation with the classification status of the training samples and / or the variance.
[0162] Optionally, the third selection unit is specifically used to obtain, for the first target attribute and the second target attribute, the attribute difference and / or the correlation between the attribute and the classification status of the training sample according to the attribute vector; and select the third target attribute from the first target attribute and the second target attribute according to the attribute difference and / or the correlation between the attribute and the classification status of the training sample.
[0163] Third embodiment
[0164] In the above embodiment, a feature selection method is provided. Correspondingly, the present application also provides a feature selection method. The method corresponds to the embodiment of the above method. The parts of this embodiment that are the same as the first embodiment are not repeated here. Please refer to the corresponding parts in the first embodiment.
[0165] A feature selection method provided in this application includes:
[0166] Step S401: Acquire attribute data and classification label data of multiple training samples for training a classification model;
[0167] Step S403: forming an attribute vector of each attribute according to the feature data of multiple training samples on each attribute;
[0168] Step S405: for each combination of attributes, determine the attribute difference between different classification samples according to the feature vector;
[0169] Step S407: According to the attribute difference, a target attribute is selected from the attribute combination as an input feature of the classification model.
[0170] In one example, the method may further include the following steps:
[0171] Step S406: Determine the correlation between the feature and the classification status of the training sample;
[0172] Accordingly, step S405 may be implemented in the following manner: selecting a target attribute from the attribute combination according to the attribute difference and the correlation with the classification state of the training sample.
[0173] It can be seen from the above embodiments that the classification model feature selection method provided in the embodiments of the present application obtains the attribute data and classification annotation data of multiple training samples used to train the classification model; forms the attribute vector of each attribute according to the attribute data of multiple training samples on each attribute; determines the attribute difference between different classification samples according to the attribute vector for each combination of attributes; and selects the target attribute from the attribute combination according to the attribute difference as the input feature of the classification model. This processing method can automatically filter the input features of the classification model based on the attribute difference analysis between different classification samples of the attributes of the training samples, and can effectively remove invalid and repeated features, thereby reducing interference with the stability and accuracy of the classification model, and thus improving the stability and accuracy of the classification model.
[0174] Fourth embodiment
[0175] In the above embodiment, a feature selection method is provided, and correspondingly, the present application also provides a feature selection device. The device corresponds to the embodiment of the above method. The parts of this embodiment that are the same as the first embodiment are not repeated here, please refer to the corresponding parts in the first embodiment.
[0176] A feature selection device provided by the present application includes:
[0177] A training sample acquisition unit, used to acquire attribute data and classification label data of a plurality of training samples used to train a classification model;
[0178] An attribute vector generating unit, used for forming an attribute vector of each attribute according to the feature data of multiple training samples on each attribute;
[0179] An attribute difference determination unit, used for determining the attribute difference between different classification samples according to the feature vector for each combination of attributes;
[0180] The feature selection unit is used to select a target attribute from the attribute combination according to the attribute difference as an input feature of the classification model.
[0181] Optionally, the device further comprises:
[0182] A correlation determination unit, used to determine the correlation between the feature and the classification state of the training sample;
[0183] The feature selection unit is specifically used to select a target attribute from the attribute combination according to the attribute difference and the correlation with the classification state of the training sample.
[0184] Fifth embodiment
[0185] The present application also provides an electronic device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described below is only illustrative.
[0186] An electronic device of this embodiment includes: a processor and a memory; the memory is used to store a program for implementing a feature selection method, and the device is powered on and runs the program of the method through the processor.
[0187] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.
[0188] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0189] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0190] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.
[0191] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
Claims
1. A classification model feature selection method, characterized in that: include: Acquire attribute data and classification label data of a plurality of training samples for training a classification model, wherein the training sample is one of the following data objects: speech, text; According to the attribute data of multiple training samples on each attribute, an attribute vector of each attribute is formed; For each combination of attributes, determine the correlation between any two attributes according to the attribute vector; Determine, according to the correlation, a first target attribute and a combination of attributes to be deduplicated; Select a second target attribute from the attribute combination to be deduplicated; For the first target attribute and the second target attribute, according to the attribute vector, the attribute difference of the attribute and / or the correlation between the attribute and the classification state of the training sample are obtained; according to the attribute difference and / or the correlation between the attribute and the classification state of the training sample, a third target attribute is selected from the first target attribute and the second target attribute as the input feature of the classification model.
2. The method according to claim 1, characterized in that The determining, according to the correlation, a first target attribute and a combination of attributes to be deduplicated includes: Taking the attribute in the attribute combination whose relevance is less than the relevance threshold as the first target attribute; The attribute combination whose correlation is greater than or equal to the correlation threshold is used as the attribute combination to be deduplicated.
3. The method according to claim 1, characterized in that The determining, according to the correlation, a first target attribute and a combination of attributes to be deduplicated includes: Determining relevance ranking data of the attribute combination according to the relevance; A first target attribute is selected according to the ranking data and the attribute combination selection ratio.
4. The method according to claim 1, characterized in that: The selecting a second target attribute from the attribute combination to be deduplicated includes: Determine the attribute difference between different classification samples of the attribute to be deduplicated and / or the non-correlation between the attribute to be deduplicated and the first target attribute; According to the attribute difference and / or the non-correlation of the attribute to be deduplicated, an attribute is selected from the attribute combination to be deduplicated as the second target attribute.
5. The method according to claim 4, characterized in that The attribute difference is obtained in the following way: According to the sample classification status corresponding to the attribute data, the sum of the attribute data of each type of sample is obtained; For each combination of sample classifications, the difference in the sum of attribute data between the two categories of samples is determined according to the sum of attribute data; The statistical data of the gaps corresponding to the combinations of two or more of the categories are used as the attribute difference.
6. The method according to claim 4, characterized in that The non-correlation degree is obtained in the following manner: Determine the non-correlation between the attribute to be deduplicated and each first target attribute; The statistical data of the non-correlation degree corresponding to each first target attribute is used as the non-correlation degree between the attribute to be deduplicated and the first target attribute.
7. The method according to claim 4, characterized in that The selecting a second target attribute from the attribute combination to be deduplicated further includes: Determine the correlation between the attribute to be deduplicated and the classification state of the training sample and / or the variance of the attribute to be deduplicated; The second target attribute is determined according to the attribute difference and / or the non-correlation of the attribute to be deduplicated, and the correlation and / or the variance with the classification state of the training sample.
8. A classification model feature selection method, characterized in that: include: Acquire attribute data and classification label data of a plurality of training samples for training a classification model, wherein the training sample is one of the following data objects: speech, text; According to the characteristic data of multiple training samples on each attribute, an attribute vector of each attribute is formed; For each combination of attributes, determining the attribute difference between different classification samples according to the attribute vector; According to the attribute difference, a target attribute is selected from the attribute combination as an input feature of the classification model.
9. The method according to claim 8, characterized in that Also includes: Determine the relevance of the features to the classification status of the training samples; A target attribute is selected from the attribute combination according to the attribute difference and the correlation with the classification state of the training sample.
10. A classification model feature selection device, characterized in that: include: A training sample acquisition unit, used to acquire attribute data and classification label data of a plurality of training samples for training a classification model, wherein the training sample is one of the following data objects: speech, text; An attribute vector generating unit, used for forming an attribute vector of each attribute according to the attribute data of multiple training samples on each attribute; An attribute correlation determination unit, for determining the correlation between any two attributes according to the attribute vector for each combination of attributes; A first selection unit, configured to determine a first target attribute and a combination of attributes to be deduplicated according to the correlation; A second selection unit, used for selecting a second target attribute from the attribute combination to be deduplicated; The third selection unit is used to obtain, for the first target attribute and the second target attribute, the attribute difference and / or the correlation between the attribute and the classification state of the training sample according to the attribute vector; and select the third target attribute from the first target attribute and the second target attribute as the input feature of the classification model according to the attribute difference and / or the correlation between the attribute and the classification state of the training sample.
11. A classification model feature selection device, characterized in that: include: A training sample acquisition unit, used to acquire attribute data and classification label data of a plurality of training samples for training a classification model, wherein the training sample is one of the following data objects: speech, text; An attribute vector generating unit, used for forming an attribute vector of each attribute according to the feature data of multiple training samples on each attribute; An attribute difference determination unit, for determining attribute differences between different classification samples according to the attribute vectors for each combination of attributes; The feature selection unit is used to select a target attribute from the attribute combination according to the attribute difference as an input feature of the classification model.
12. An electronic device, characterized in that: include: Processor and memory; A memory for storing a program for implementing the classification model feature selection method according to any one of claims 1 to 9, the device is powered on and runs the program of the method through the processor.
Citation Information
Patent Citations
Text segment recognition model training method, text segment recognition method, equipment and medium
CN115033691A