Feature screening method and device, computer equipment and storage medium

Abstract

The invention relates to a feature screening method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: firstly, carrying out first feature screening on initial to-be-screened feature information, storing features meeting a first type of set conditions into a first screening set, then carrying out second feature screening on the features in the first screening set, storing features meeting a second type of set conditions into a second screening set, and carrying out second feature screening on the features in the second screening set; inputting the features in the second screening set into a plurality of machine learning models, scoring each feature to obtain a plurality of scoring results, performing weighted calculation to obtain a total score of each feature, ranking the total scores, and storing the screened features into a final feature pool. Wherein the calculated quantities corresponding to the first feature screening, the second feature screening and the screening based on the plurality of machine learning models are progressive in sequence. By adopting the method, the feature screening efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a feature screening method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of data processing technology, feature screening methods have emerged. This technology has important application value in the fields of machine learning and financial quantification. The current feature screening methods usually input all features into the model for screening.

[0003] However, the current feature selection method consumes a lot of time and computing resources when the number of features is large, reducing the efficiency of the entire feature selection process. Therefore, a more efficient method is needed that can quickly screen out the most important features and improve the efficiency of feature selection while ensuring accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a feature screening method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides a feature screening method, comprising:

[0006] Acquire initial feature information to be screened, perform a first feature screening on the feature information to be screened, select features whose first correlation index with the target variable in the feature information to be screened meets a first set of set conditions, and store the features in the first screening set in the memory;

[0007] Acquire the first screening set from the memory, perform second feature screening on the features in the first screening set, select features in the first screening set whose second correlation index with other features satisfies the second set condition, and store the features in the second screening set in the memory;

[0008] Obtaining the second filtered set from the memory, inputting the features in the second filtered set into multiple machine learning models, obtaining the scoring results of the multiple machine learning models for each feature, performing weighted calculation to obtain a total score for each feature, and filtering the features in the second filtered set according to the total score, and storing the filtered features in a final feature pool in the memory;

[0009] Among them, the calculation amounts corresponding to the first feature screening, the second feature screening, and the screening based on the multiple machine learning models are progressive.

[0010] In one embodiment, the first correlation index includes two types of indicators: a linear correlation index and a non-linear correlation index; the first feature screening is performed on the feature information to be screened, and the features whose first correlation indicators with the target variable in the feature information to be screened meet the first type of setting conditions are selected and stored in the first screening set in the memory, including: for each feature in the feature information to be screened, determining whether the linear correlation index of each feature meets the first type of setting conditions and whether the non-linear correlation index meets the first type of setting conditions; selecting any one of the linear correlation index and the non-linear correlation index in the feature information to be screened that meets the first type of setting conditions, and storing them in the first screening set in the memory.

[0011] In one embodiment, there are multiple linear correlation indicators; the separately determining whether the linear correlation indicator of each feature satisfies the first type of setting conditions and whether the nonlinear correlation indicator satisfies the first type of setting conditions includes: for each feature in the feature information to be screened, calculating multiple linear correlations of the feature under the multiple linear correlation indicators, and the nonlinear correlation of the feature under the nonlinear correlation indicator; when the multiple linear correlations satisfy corresponding conditions, determining that the linear correlation indicator corresponding to the feature satisfies the first type of setting conditions; and, when the nonlinear correlation satisfies corresponding conditions, determining that the nonlinear correlation indicator corresponding to the feature satisfies the first type of setting conditions.

[0012] In one embodiment, the multiple linear correlation indicators include a first Pearson coefficient correlation and a result credibility; when the linear correlations all meet the corresponding conditions, it is determined that the linear correlation indicator corresponding to the feature meets the first type of setting conditions, including: when the first Pearson coefficient correlation of the feature in the feature information to be screened is greater than a first setting value and the result credibility is less than a second setting value, it is determined that the feature meets the first type of setting conditions.

[0013] In one embodiment, the second correlation index includes two types of indicators: a collinearity index and a low variance index; the second feature screening of the features in the first screening set, selecting the features in the first screening set whose second correlation index with other features meets the second type of setting conditions and storing them in the second screening set in the memory, includes: for each feature in the first screening set, determining whether the collinearity index between each feature and other features meets the second type of setting conditions and whether the low variance index meets the second type of setting conditions; selecting the features in the first screening set whose collinearity index and the low variance index both meet the second type of setting conditions, and storing them in the second screening set in the memory.

[0014] In one embodiment, the collinearity indicator is the second Pearson coefficient between the features in the first screening set and any other features; and determining whether the collinearity indicator between each feature and any other features satisfies the second type of setting conditions and whether the low variance indicator satisfies the second type of setting conditions respectively includes: if the second Pearson coefficient between the features in the first screening set and any other features is less than a third setting value and the low variance of the features in the first screening set is greater than a fourth setting value, then determining that the corresponding features in the first screening set meet the second type of setting conditions.

[0015] In one embodiment, the multiple machine learning models include two types of models: linear models and nonlinear models; the features in the second screening set are input into multiple machine learning models, and for each feature, the scoring results of the multiple machine learning models are obtained for weighted calculation to obtain the total score of each feature, including: for each feature in the second screening set, obtaining the scoring result of the linear model for the feature, and obtaining the scoring result of the nonlinear model for the feature; based on the same weight of the scoring results of the two types of models, the linear model and the nonlinear model, the scoring results are weighted to obtain the total score of each feature.

[0016] In one embodiment, the linear model and the nonlinear model respectively include multiple different scoring models; for each feature in the second screening set, obtaining the scoring result of the linear model for the feature, and obtaining the scoring result of the nonlinear model for the feature, includes: obtaining the scores of multiple different linear models for each feature in the second screening set, and obtaining the scores of multiple different nonlinear models for each feature in the second screening set; calculating the sum of the scores of the multiple different linear models for the feature as the score of the linear model for the feature; and calculating the sum of the scores of the multiple different nonlinear models for the feature as the score of the nonlinear model for the feature.

[0017] In a second aspect, the present application also provides a feature screening device, comprising:

[0018] A first screening module, used for obtaining initial feature information to be screened, performing a first feature screening on the feature information to be screened, selecting features whose first correlation index with the target variable in the feature information to be screened meets a first set of set conditions, and storing the features in a first screening set in a memory;

[0019] A second screening module, configured to obtain the first screening set from the memory, perform a second feature screening on the features in the first screening set, select features in the first screening set whose second correlation index with other features satisfies a second set condition, and store the features in the second screening set in the memory;

[0020] a final feature confirmation module, configured to obtain the second screening set from the memory, input the features in the second screening set into a plurality of machine learning models, obtain the scoring results of the plurality of machine learning models for each feature, perform weighted calculation, obtain a total score for each feature, screen the features in the second screening set according to the total score, and store the screened features in a final feature pool in the memory;

[0021] Among them, the calculation amounts corresponding to the first feature screening, the second feature screening, and the screening based on the multiple machine learning models are progressive.

[0022] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the feature screening of the method described in any one of the embodiments of the present application when executing the computer program.

[0023] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the feature screening of the method described in any one of the embodiments of the present application is implemented.

[0024] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the feature screening of the method described in any one of the embodiments of the present application.

[0025] The above-mentioned feature screening method, device, computer equipment, storage medium and computer program product have the following beneficial effects during feature screening: by obtaining the initial feature information to be screened, performing a first feature screening on the feature information to be screened, selecting the features whose first correlation index between the feature information to be screened and the target variable meets the first type of set conditions and storing them in the first screening set in the memory; then obtaining the first screening set from the memory, performing a second feature screening on the features in the first screening set, selecting the features whose second correlation index between the first screening set and other features meets the second type of set conditions and storing them in the second screening set in the memory; then obtaining the second screening set from the memory, inputting the features in the second screening set into multiple machine learning models, obtaining the scoring results of multiple machine learning models for each feature, performing weighted calculation, obtaining the total score of each feature, and screening the features in the second screening set according to the total score, and storing the screened features in the final feature pool in the memory; wherein, the calculation amount corresponding to the first feature screening, the second feature screening and the screening based on multiple machine learning models in this scheme are progressive in sequence. Therefore, this solution can save the computing overhead of the processor while ensuring accuracy, and then quickly screen out the most important features, improve the efficiency of feature selection, and meet business needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 An application environment diagram of a feature screening method in an embodiment;

[0028] Figure 2 A schematic diagram of a feature screening method in one embodiment;

[0029] Figure 3 is a detailed schematic diagram of a feature screening step in an embodiment;

[0030] Figure 4 A schematic diagram of the screening methods used in each stage of the feature screening method in one embodiment;

[0031] Figure 5 is a structural block diagram of a feature screening device in one embodiment;

[0032] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] The feature screening method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. In the application scenario of the present application, the server 104 obtains initial feature information to be screened from the terminal 102, performs a first feature screening on the information to be screened, and stores the features whose first correlation index between the information to be screened and the target variable satisfies the first type of set conditions into a first screening set in the memory. Then, the server 104 obtains the first screening set from the memory, and performs a second feature screening on the features in the first screening set, and stores the features whose second correlation index between the first screening set and other features satisfies the second type of set conditions into a second screening set in the memory. Finally, after the server 104 obtains the second screening set from the memory, it inputs the features in the second screening set into multiple machine learning models, obtains the scoring results of multiple machine learning models for each feature, performs weighted calculation, and obtains the total score of each feature. The features in the second screening set are screened according to the total score, and the screened features are stored in the final feature pool in the memory. The calculation amounts corresponding to the above-mentioned first feature screening, second feature screening, and screening based on multiple machine learning models are progressive in sequence. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0035] In an exemplary embodiment, Figure 2 As shown, a feature screening method is provided, which is applied to Figure 1 Taking the processor in the server 104 as an example, the method includes the following steps 202 to 206. Among them:

[0036] Step 202, obtaining initial feature information to be screened, performing a first feature screening on the feature information to be screened, selecting features whose first correlation index with the target variable in the feature information to be screened meets the first set condition, and storing them in a first screening set in the memory.

[0037] The initial feature information to be screened refers to all feature data that can be obtained before feature screening; the first feature screening refers to the target filtering method, which is the process of preliminary screening of the feature information to be screened.

[0038] Optionally, the processor obtains feature information to be screened from a data source or a memory, and the feature information may be data of various variables or attributes. The first round of screening of the feature information to be screened may be achieved by statistical analysis methods or machine learning algorithms. The correlation index between the feature information to be screened and the target variable is calculated, and features with a certain correlation with the target variable are screened according to the first set conditions, and the features obtained by the first feature screening are stored in the first screening set in the memory to facilitate subsequent feature screening and analysis.

[0039] Step 204, obtaining the first screening set from the memory, performing a second feature screening on the features in the first screening set, selecting the features in the first screening set whose second correlation index with other features satisfies the second set condition, and storing them in the second screening set in the memory.

[0040] Among them, the second feature screening refers to inter-feature filtering, which is a process of further screening the features in the first screening set.

[0041] Exemplarily, the processor reads the first filtering set from the memory and performs a second round of screening on the features in the first filtering set. The purpose of this step is to select features in the first filtering set whose correlation with other features meets specific conditions. Collinearity and low variance can be used to calculate the correlation index between features, and feature screening is performed according to the second type of set conditions. The features obtained through the second feature screening are stored in the second filtering set in the memory.

[0042] Step 206, obtain a second filtered set from the memory, input the features in the second filtered set into multiple machine learning models, obtain the scoring results of multiple machine learning models for each feature, perform weighted calculation, obtain the total score of each feature, and filter the features in the second filtered set according to the total score, and store the filtered features in the final feature pool in the memory.

[0043] Among them, the machine learning model refers to the machine learning model that scores features, and the scoring result refers to the evaluation result of the machine learning model on the feature, which can be in the form of probability, score, etc. The total score of each feature refers to the comprehensive score obtained after weighted calculation of the model scores of different categories of the feature in the machine learning model; the final feature pool is used to store features filtered by the total score, among which the features with the highest total score are the features that pass the screening and are stored in the final feature pool.

[0044] Optionally, the processor uses multiple machine learning models to evaluate and weight the features in the second screening set to obtain a total score for each feature, and screens the features according to the total score. Finally, the screened features are stored in a final feature pool.

[0045] The calculation amount corresponding to the first feature screening, the second feature screening, and the screening based on multiple machine learning models is progressive. This application determines the final feature screening result after three rounds of screening from simple calculation to complex calculation.

[0046] In the feature screening method of the above embodiment, the processor obtains the initial feature information to be screened, performs a first feature screening on the feature information to be screened, selects the features whose first correlation index between the feature information to be screened and the target variable meets the first set of set conditions and stores them in the first screening set in the memory; then obtains the first screening set from the memory, performs a second feature screening on the features in the first screening set, selects the features whose second correlation index between the first screening set and other features meets the second set of set conditions and stores them in the second screening set in the memory; then obtains the second screening set from the memory, inputs the features in the second screening set into multiple machine learning models, obtains the scoring results of multiple machine learning models for each feature, performs weighted calculation, obtains the total score of each feature, and screens the features in the second screening set according to the total score, and stores the screened features in the final feature pool in the memory; wherein, the calculation amount corresponding to the first feature screening, the second feature screening and the screening based on multiple machine learning models in this scheme is progressive in sequence. Therefore, under the premise of ensuring accuracy, this scheme can save the computing overhead of the processor, and then quickly screen out the most important features, improve the efficiency of feature selection, and meet business needs.

[0047] In one embodiment, in step 202, a first feature screening is performed on the feature information to be screened, and features whose first correlation indicators with the target variable in the feature information to be screened meet the first type of setting conditions are selected and stored in a first screening set in the memory, which may include: for each feature in the feature information to be screened, determining whether the linear correlation indicator of each feature meets the first type of setting conditions and whether the nonlinear correlation indicator meets the first type of setting conditions; selecting features whose linear correlation indicators and nonlinear correlation indicators in the feature information to be screened meet the first type of setting conditions, and storing them in the first screening set in the memory.

[0048] In this embodiment, the linear correlation index of each feature refers to the correlation degree of the Pearson correlation coefficient between each feature and the target variable and the credibility of the result. The nonlinear correlation index of each feature refers to the mutual information coefficient between each feature and the target variable. By calculating the linear correlation index and the nonlinear correlation index of each feature in the feature information to be screened, the features that meet any type of indicators are obtained as the first screening set, which can improve the efficiency of feature screening.

[0049] In one embodiment, determining whether the linear correlation index of each feature satisfies the first type of setting conditions and whether the nonlinear correlation index of each feature satisfies the first type of setting conditions respectively includes:

[0050] Step 1: for each feature in the feature information to be screened, multiple linear correlations of the feature under multiple linear correlation indicators and multiple nonlinear correlation indicators are calculated;

[0051] Among them, the calculation method of multiple linear correlations of features under multiple linear correlation indicators and nonlinear correlation of features under nonlinear correlation indicators can be achieved by statistical analysis methods: such as correlation coefficient, chi-square test, or machine learning algorithms: such as feature importance evaluation, feature selection algorithm.

[0052] Step 2: When multiple linear correlations all meet corresponding conditions, determine that the linear correlation index corresponding to the feature meets the first type of setting conditions; and multiple nonlinear correlation indicators all meet the first type of setting conditions.

[0053] The linear correlation index and the nonlinear correlation index each include multiple specific indicators. When multiple specific indicators under the linear correlation index or the nonlinear correlation index are met, one type of indicator is considered to be met.

[0054] Through this embodiment, it is explained that the linear correlation index includes multiple specific indicators, and multiple specific indicators must be met before the linear correlation index is considered to be satisfied; the specific indicators under the nonlinear correlation index are met before the nonlinear correlation index is considered to be satisfied, thereby improving the accuracy of feature screening.

[0055] Further, in one embodiment, multiple linear correlation indicators include a first Pearson coefficient correlation degree and a result credibility degree; correspondingly, determining that the linear correlation indicator corresponding to the feature in the feature information to be screened satisfies the first type of setting conditions includes: when the first Pearson coefficient correlation degree of the feature in the feature information to be screened is greater than a first setting value and the result credibility degree is less than a second setting value, determining that the feature satisfies the first type of setting conditions.

[0056] Specifically, the Pearson coefficient is used to measure whether two data sets are on the same line. It is used to measure the linear relationship between interval variables. The p-value in the multiple linear regression model refers to the confidence level, which indicates the credibility level of the analysis results. In this application, when the first Pearson coefficient correlation is greater than the first set value, and the credibility of the result corresponding to each feature is less than the second set value, it is considered to pass the linear correlation index screening, where the first set value can be 0.1 and the second set value can be 0.05.

[0057] In the above embodiment, when screening a large amount of feature information to be screened, a round of screening is first performed using the target screening method. This method has a fast screening speed, a more general and broad algorithm logic, and does not filter out valuable features, which can save feature screening time and effectively improve the feature screening efficiency of the processor; at the same time, the features that pass the first round of screening are stored in the first screening set in the memory. Other features can be cleared from the memory, which is also conducive to saving storage resources.

[0058] In one embodiment, the second correlation index includes two types of indicators: collinearity index and low variance index. Correspondingly, in the above steps, the second feature screening is performed on the features in the first screening set, and the features whose second correlation indexes with other features in the first screening set meet the second type of setting conditions and are stored in the memory. Specifically, the second screening set may include: for each feature in the first screening set, determining whether the collinearity index between each feature and other features meets the second type of setting conditions and whether the low variance index meets the second type of setting conditions; selecting the features whose collinearity index and low variance index meet the second type of setting conditions in the first screening set, and storing them in the second screening set in the memory.

[0059] Specifically, only when both the collinearity index and the low variance index of the feature are satisfied, the feature can be stored in the second screening set.

[0060] Through the above embodiments, in the above embodiments, when screening the features in the first screening set, the screening is performed by filtering between features. This method further refines the classification of feature screening, performs feature screening from multiple indicators, increases the accuracy of feature screening, and reduces the loss of time and resources. Effectively improve the efficiency of feature screening of the processor; at the same time, the features that pass the second round of screening are stored in the second screening set in the memory. Other features can be cleared from the memory, which is conducive to saving storage resources.

[0061] Furthermore, in one embodiment, the collinearity index is the second Pearson coefficient between the feature in the first screening set and any other feature; and determining whether the collinearity index between each feature and the other features satisfies the second type of setting conditions and whether the low variance index satisfies the second type of setting conditions respectively includes:

[0062] If the second Pearson coefficient between the feature in the first screening set and any other feature is less than the third set value and the low variance of the feature is greater than the fourth set value, it is determined that the feature meets the second type of setting conditions.

[0063] Among them, low variance means that the differences between samples or features in the data set are small, that is, the values ​​of the data are relatively concentrated or similar.

[0064] Specifically, the threshold of the second Pearson coefficient in the present application can be 0.8. When the Pearson correlation coefficient of the two features is greater than the threshold value 0.8, the feature with the lower correlation coefficient with the predicted target is deleted. If the Pearson correlation coefficient of the two features is less than the threshold value 0.8, both features are screened. In feature selection, low variance features refer to features with smaller variances. Such features vary less in the data set, have less information, and contribute less to the training and prediction capabilities of the model, so they need to be eliminated. In one embodiment, if the variance of the feature is less than 0.1, it can be determined as a low variance feature and eliminated. Among them, low variance features may be caused by the presence of a large number of repeated or similar values ​​in the data, or by the small differences between samples.

[0065] Through the above embodiments, eliminating low-variance features can reduce the risk of model overfitting, reduce computational complexity, and improve the generalization ability of the model. According to the second Pearson coefficient corresponding to the collinearity between features, highly correlated feature pairs can be removed, and features with high correlation with the predicted target can be retained, thereby reducing the collinearity between features. The amount of calculation is increased compared to the above-mentioned first feature screening, but the calculation of the processor is still relatively simple, and a part of the features can be further screened out, effectively improving the efficiency of the feature screening of the processor. In addition, other features that have not passed the screening can be cleared from the memory, reducing the amount of calculation for the subsequent feature screening step.

[0066] In one embodiment, the multiple machine learning models include two types of models: linear models and nonlinear models. In the above embodiment, the features in the second screening set are input into multiple machine learning models, and for each feature, the scoring results of the multiple machine learning models are obtained for weighted calculation to obtain the total score of each feature, which may specifically include: for each feature in the second screening set, obtaining the scoring result of the linear model for the feature, and obtaining the scoring result of the nonlinear model for the feature; based on the same weight of the scoring results of the two types of models, the linear model and the nonlinear model, the scoring results are weighted to obtain the total score of each feature.

[0067] Specifically, after the two steps of feature screening, namely target filtering and feature screening, the passed features are input into the third step: model scoring. The scoring in this step can include two types of models: linear model and nonlinear model. If the feature passes the screening of one of the two types of models, the score of the feature in the model is 1 point, and if it fails, it is 0 point.

[0068] Through the above embodiment, by giving the same weight to the scoring results based on the linear model and the nonlinear model, the contributions of the two models can be balanced. The linear model can capture the linear relationship between features, while the nonlinear model can better capture the nonlinear relationship. Therefore, by weighting, we can comprehensively consider the advantages of the two models to obtain more accurate feature screening results.

[0069] Furthermore, in one embodiment, the linear model and the nonlinear model respectively include a plurality of different scoring models; for each feature in the second screening set, the steps of obtaining the scoring result of the linear model for the feature and obtaining the scoring result of the nonlinear model for the feature specifically include:

[0070] Step A: obtaining scores of multiple different linear models for each feature in the second screening set, and obtaining scores of multiple different nonlinear models for each feature in the second screening set.

[0071] The linear models can include the Granger causality test model and three linear machine learning models, namely, the stepwise regression model, the minimum absolute value convergence and selection operator algorithm model, and the elastic network regression model. The nonlinear models can include a random forest composed of many decision trees and a distributed gradient boosting framework based on the decision tree algorithm.

[0072] Step B: Calculate the sum of the scores of multiple different linear models for the feature as the score of the linear model for the feature; and calculate the sum of the scores of multiple different nonlinear models for the feature as the score of the nonlinear model for the feature.

[0073] Specifically, the sum of the scores of the Granger causality test model and the three linear machine learning models for this feature is calculated as the score of the linear model for this feature; the sum of the scores of the random forest and the distributed gradient boosting framework based on the decision tree algorithm for this feature is calculated as the score of the nonlinear model for this feature; finally, the scores of the two types of models for this feature are combined to obtain the total score of this feature.

[0074] Through the above embodiment, by calculating the scoring results of the linear model and the nonlinear model, the evaluation results of the two models can be comprehensively considered to obtain a more comprehensive and accurate feature evaluation. This feature screening step can improve the accuracy and reliability of feature selection.

[0075] In order to better understand the above data processing timeliness process, combined with Figure 3 As shown, the following is a detailed description of the specific process of timely data processing for this application, including the following steps:

[0076] S302, use the target filtering method to perform the first feature screening on the initial feature information to be screened and store the features that pass the screening into the first screening set. The target filtering method includes two types of parallel indicators, linear and nonlinear. If the test of any category is passed, the next step can be entered, and the indicators in the same category must be met at the same time. The nonlinear correlation indicator is the mutual information coefficient. When the mutual information coefficient is greater than 0.5, the feature passes the screening of the nonlinear correlation indicator.

[0077] Combination Figure 4 As shown, that is, for each feature, if the person coefficient and P value between the feature and the preset target quantity meet the verification conditions, it is determined that the feature passes the verification of the linear correlation index; if the normalized mutual information coefficient meets the verification conditions, it is determined that the feature passes the verification of the nonlinear correlation index; when either the linear correlation index or the nonlinear correlation index passes the verification, it is determined that the feature passes the screening of the target filtering method and is stored in the first screening set, and other features are cleared.

[0078] S304, use inter-feature filtering to perform a second feature screening on the features in the first screening set and store the features that pass the screening into the second screening set. Only features that meet both collinearity and low variance requirements can pass the screening in this step.

[0079] Combination Figure 4 As shown, that is, for each feature, if the collinearity and low variance indicators between the feature and other features meet the verification conditions, it is determined that the feature passes the screening between features and is stored in the second screening set, and other features are cleared.

[0080] S306, after the first two steps of screening, the features that have passed are input into the third step: model scoring. The scoring in this step includes two types of models: linear model and nonlinear model. When a feature value passes the screening of a model, the score of the corresponding model is set to 1 point, and if it fails, it is set to 0 point.

[0081] Combination Figure 4 As shown in the figure, for each feature, multiple linear models are used to score the feature, where the linear model indicators include stepwise regression model, Granger, Lasso and ElasticNet; for each feature, multiple nonlinear models are used to score the feature, where the nonlinear model indicators include RandomForest and LightGBM; when a feature value is screened by a certain model, the feature value is scored corresponding to the model score, and other features are deleted.

[0082] Specifically, stepwise regression in linear models is a combination of forward selection and backward selection, that is, selecting and eliminating at the same time, and finally obtaining an optimal set of variables. This model reflects the linear gain of features. If a feature is in the optimal variable set, it is considered to pass, otherwise it fails; Granger causality model is a statistical model used to analyze causal relationships in time series data. It compares the historical data of different time series variables to determine whether a variable can be used to predict another variable. This model reflects the causal relationship between features and target variables. Features with a significant p value of less than 0.05 with the target variable are considered to pass the model, otherwise they fail; Lasso is a statistical model used to effectively screen out the most influential variables among more variables that may affect the target variable, while acting on other insignificant variables, thereby achieving feature selection. The model uses L1 regularization to screen features, where L1 regularization is also called Lasso regularization. It is a regularization method used for feature selection and model sparsification. Features whose corresponding Lasso model coefficients are not zero are considered to have passed the model, otherwise they will not pass. The elastic network is a model that combines Lasso and ridge regression. It is suitable for data sets with multiple related features. It can more accurately screen out important features and reduce redundancy. The model uses L1 and L2 regularization to screen features. Features whose corresponding elastic network model coefficients are not zero are considered to have passed the model, otherwise they will not pass.

[0083] When the relationship between the feature and the response variable is nonlinear, tree-based methods can be used in nonlinear models: decision trees, random forests, or extended linear models, etc. Among them, the tree-based method is better for modeling nonlinear relationships, and does not require too much debugging, which is a common method. In this application, the module selects two representative algorithms of different integration methods, and performs feature screening according to the importance of the trained features. The random forest is composed of many decision trees, and there is no association between each decision tree. After the forest is randomly established, when a new sample enters, each decision tree will be judged separately, and then the classification result is given based on the voting method, in which the average impurity of the random forest is greater than 0.85. The feature is considered to pass the model, otherwise it will not pass; the decision tree algorithm is also a decision tree-based integration algorithm like the random forest, but the regular term is added to its objective function to control the complexity of the model, so that the learned model is simpler and prevent overfitting. When the feature importance is greater than zero, it is considered to pass the model, otherwise it will not pass.

[0084] S308, according to the model scoring results, weighted according to a certain weight to obtain the final feature evaluation result. The model is divided into two categories according to linear model and nonlinear model, and each category is given the same weight. The final weighted score calculation formula for each feature is as follows:

[0085] Score=(Granger+Lasso+StepWise+ElasticNet)+(2*RandomForest+2*LightGBM)

[0086] Taking feature one as an example, Granger is the score of the Granger causality model for the output of feature one, Lasso is the score of the minimum absolute value convergence and selection operator algorithm model for the output of feature one, StepWise is the score of the stepwise regression model for the output of feature one, ElasticNet is the score of the elastic network model for the output of feature one, RandomForest is the score of the random forest model for the output of feature one, and LightGBM is the score of the distributed gradient boosting framework model based on the decision tree algorithm for the output of feature one.

[0087] Combination Figure 4 As shown, for the scores of multiple linear models and multiple nonlinear models for each feature, the scores of different models are weighted to calculate the total score of each feature.

[0088] S310, selecting a number of top-ranked features according to the weighted scores to form a final feature pool.

[0089] Through the above embodiments, linear and nonlinear indicators and models are taken into consideration at the same time, wherein linear indicators / models can capture linear relationships, while nonlinear indicators / models can process more complex data structures and associations, and combine them to screen features more comprehensively and accurately, thereby improving the effect of feature selection; and, the three steps of the present application respectively evaluate the features from simple indicators to complex indicators, which can quickly filter out features that have no obvious correlation with the target variable or have little change in the early stage of feature screening, thereby reducing the time and resource consumption of subsequent further calculations, greatly improving the speed of feature selection, and improving the efficiency of feature selection.

[0090] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0091] In one embodiment, Figure 5 As shown, based on the same inventive concept, the embodiment of the present application also provides a feature screening device for implementing the feature screening method involved above, including: a first screening module 501, a second screening module 502 and a final feature confirmation module 503, wherein:

[0092] The first screening module 501 is used to obtain initial feature information to be screened, perform a first feature screening on the feature information to be screened, select features whose first correlation index between the feature information to be screened and the target variable meets the first set condition, and store them in a first screening set in the memory.

[0093] The second screening module 502 is used to obtain the first screening set from the memory, perform a second feature screening on the features in the first screening set, select the features in the first screening set whose second correlation index with other features meets the second type of set conditions and store them into the second screening set in the memory.

[0094] The final feature confirmation module 503 is used to obtain the second filtering set from the memory, input the features in the second filtering set into multiple machine learning models, obtain the scoring results of the multiple machine learning models for each feature, perform weighted calculation, obtain the total score of each feature, and filter the features in the second filtering set according to the total score, and store the filtered features in the final feature pool in the memory.

[0095] Among them, the calculation amounts corresponding to the first feature screening, the second feature screening, and the screening based on the multiple machine learning models are progressive.

[0096] In one embodiment, the first screening module 501 is specifically used to determine, for each feature in the feature information to be screened, whether the linear correlation index of each feature satisfies the first type of set conditions and whether the nonlinear correlation index satisfies the first type of set conditions; select the feature whose linear correlation index and nonlinear correlation index in the feature information to be screened satisfy the first type of set conditions, and store them in the first screening set in the memory.

[0097] In one embodiment, the first screening module 501 is specifically used to calculate, for each feature in the feature information to be screened, multiple linear correlations of the feature under the multiple linear correlation indicators, and the non-linear correlation of the feature under the non-linear correlation indicator; when the multiple linear correlations meet the corresponding conditions, determine that the linear correlation indicator corresponding to the feature meets the first type of set conditions; and when the non-linear correlation meets the corresponding conditions, determine that the non-linear correlation indicator corresponding to the feature meets the first type of set conditions.

[0098] In one embodiment, the first screening module 501 is specifically configured to determine that the feature in the feature information to be screened satisfies the first type of setting condition when the first Pearson coefficient correlation degree of the feature is greater than a first set value and the result credibility is less than a second set value.

[0099] In one embodiment, the second screening module 502 is specifically used to determine, for each feature in the first screening set, whether the collinearity index between each feature and other features meets the second type of setting conditions, and whether the low variance index meets the second type of setting conditions; select the features in the first screening set whose collinearity index and low variance index both meet the second type of setting conditions, and store them in the second screening set in the memory.

[0100] In one embodiment, the second screening module 502 is specifically used to determine that the corresponding feature in the first screening set meets the second type of setting conditions if the second Pearson coefficient between the feature in the first screening set and any other feature is less than the third set value and the low variance of the feature in the first screening set is greater than the fourth set value.

[0101] In one embodiment, the final feature confirmation module 503 is specifically used to obtain, for each feature in the second screening set, a scoring result of the linear model for the feature, and a scoring result of the nonlinear model for the feature; the same weight is given to the scoring results based on the linear model and the nonlinear model, and the scoring results are weighted to obtain the total score of each feature.

[0102] In one embodiment, the final feature confirmation module 503 is specifically used to obtain scores of multiple different linear models for each feature in the second screening set, and to obtain scores of multiple different nonlinear models for each feature in the second screening set; to calculate the sum of the scores of multiple different linear models for the feature as the score of the linear model for the feature; and, to calculate the sum of the scores of multiple different nonlinear models for the feature as the score of the nonlinear model for the feature.

[0103] Each module in the above-mentioned feature screening device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0104] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store feature screening data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a feature screening method is implemented.

[0105] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0106] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0107] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0109] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0110] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A feature screening method, characterized in that: Applied to a processor, the method comprises: Acquire initial feature information to be screened, perform a first feature screening on the feature information to be screened, select features whose first correlation index with the target variable in the feature information to be screened meets a first set of set conditions, and store the features in the first screening set in the memory; Acquire the first screening set from the memory, perform second feature screening on the features in the first screening set, select features in the first screening set whose second correlation index with other features satisfies the second set condition, and store the features in the second screening set in the memory; Obtaining the second filtered set from the memory, inputting the features in the second filtered set into multiple machine learning models, obtaining the scoring results of the multiple machine learning models for each feature, performing weighted calculation to obtain a total score for each feature, and filtering the features in the second filtered set according to the total score, and storing the filtered features in a final feature pool in the memory; Among them, the calculation amounts corresponding to the first feature screening, the second feature screening, and the screening based on the multiple neural network models are progressive.

2. The method according to claim 1, characterized in that: The first correlation index includes two types of indicators: a linear correlation index and a nonlinear correlation index; The first feature screening is performed on the feature information to be screened, and features whose first correlation index between the feature information to be screened and the target variable meets the first set condition are selected and stored in the first screening set in the memory, including: For each feature in the feature information to be screened, determining whether the linear correlation index of each feature satisfies the first type of setting conditions and whether the nonlinear correlation index satisfies the first type of setting conditions; Any feature of the linear correlation index and the nonlinear correlation index in the feature information to be screened that meets the first type of setting conditions is selected and stored in a first screening set in the memory.

3. The method according to claim 2, characterized in that There are multiple linear correlation indicators; The respectively determining whether the linear correlation index of each feature satisfies the first type of setting conditions and whether the nonlinear correlation index satisfies the first type of setting conditions comprises: For each feature in the feature information to be screened, calculating multiple linear correlations of the feature under the multiple linear correlation indicators, and a nonlinear correlation of the feature under the nonlinear correlation indicator; When the multiple linear correlations meet the corresponding conditions, it is determined that the linear correlation index corresponding to the feature meets the first type of setting conditions; and when the nonlinear correlations meet the corresponding conditions, it is determined that the nonlinear correlation index corresponding to the feature meets the first type of setting conditions.

4. The method according to claim 3, characterized in that The multiple linear correlation indicators include a first Pearson coefficient correlation degree and a result credibility degree; When the linear correlations all meet the corresponding conditions, determining that the linear correlation index corresponding to the feature meets the first type of setting conditions includes: When the first Pearson coefficient correlation degree of the feature in the feature information to be screened is greater than the first set value and the result credibility is less than the second set value, it is determined that the feature meets the first type of set conditions.

5. The method according to claim 1, characterized in that The second correlation index includes two types of indicators: collinearity index and low variance index; The performing second feature screening on the features in the first screening set, selecting features in the first screening set whose second correlation index with other features satisfies the second type of setting conditions and storing them in the second screening set in the memory, comprises: For each feature in the first screening set, determining whether the collinearity index between each feature and other features meets the second type of setting conditions and whether the low variance index meets the second type of setting conditions; The features of the collinearity index and the low variance index in the first screening set that both meet the second set conditions are selected and stored in the second screening set in the memory.

6. The method according to claim 5, characterized in that The collinearity index is the second Pearson coefficient between the first screening set feature and any other feature; The respectively determining whether the collinearity index between each feature and other features satisfies the second type of setting conditions and whether the low variance index satisfies the second type of setting conditions includes: If the second Pearson coefficient between the feature in the first screening set and any other feature is less than the third set value and the low variance of the feature in the first screening set is greater than the fourth set value, it is determined that the corresponding feature in the first screening set meets the second type of setting conditions.

7. The method according to claim 1, characterized in that The multiple machine learning models include two types of models: linear models and nonlinear models; The step of inputting the features in the second screening set into multiple machine learning models, obtaining the scoring results of the multiple machine learning models for each feature, performing weighted calculation, and obtaining a total score of each feature includes: For each feature in the second screening set, obtaining a scoring result of the feature by the linear model, and obtaining a scoring result of the feature by the nonlinear model; Based on the same weight of the scoring results of the linear model and the nonlinear model, the scoring results are weighted to obtain the total score of each feature.

8. The method according to claim 7, characterized in that The linear model and the nonlinear model respectively include a plurality of different scoring models; For each feature in the second screening set, obtaining a scoring result of the feature by the linear model, and obtaining a scoring result of the feature by the nonlinear model, includes: Obtaining scores of multiple different linear models for each feature in the second screening set, and obtaining scores of multiple different nonlinear models for each feature in the second screening set; The sum of the scores of the multiple different linear models on the feature is calculated as the score of the linear model on the feature; and the sum of the scores of the multiple different nonlinear models on the feature is calculated as the score of the nonlinear model on the feature.

9. A feature screening device, characterized in that: The device comprises: A first screening module, used for obtaining initial feature information to be screened, performing a first feature screening on the feature information to be screened, selecting features whose first correlation index with the target variable in the feature information to be screened meets a first set of set conditions, and storing the features in the first screening set in the memory; A second screening module, configured to obtain the first screening set from the memory, perform a second feature screening on the features in the first screening set, select features in the first screening set whose second correlation index with other features satisfies a second set condition, and store the features in the second screening set in the memory; a final feature confirmation module, configured to obtain the second screening set from the memory, input the features in the second screening set into a plurality of machine learning models, obtain the scoring results of the plurality of machine learning models for each feature, perform weighted calculation, obtain a total score for each feature, screen the features in the second screening set according to the total score, and store the screened features in a final feature pool in the memory; Among them, the calculation amounts corresponding to the first feature screening, the second feature screening, and the screening based on the multiple machine learning models are progressive.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images