Feature screening method, device, equipment and medium

Abstract

The invention provides a feature screening method and device, equipment and a medium, and relates to big data analysis, artificial intelligence and feature engineering technologies. The method comprises the following steps: calculating a correlation index between a feature data sequence of each feature and feature data sequences of other features to obtain a correlation index table; selecting features i, j corresponding to the association degree index with the highest absolute value from the association degree index table; if the correlation degree index with the highest absolute value exceeds a preset threshold value, determining a feature to be deleted from the features i and j according to the correlation between the features i and j and other features; deleting the to-be-deleted features to obtain a new feature data table; and for the new feature data table, returning to execute the step of respectively calculating the association degree indexes of the feature data sequence of each feature and the feature data sequences of other features until the association degree index with the highest absolute value in the association degree index table of the new feature data table does not exceed the preset threshold value. And the remaining features at the moment are used as target features obtained by screening.

Description

Technical Field

[0001] The present application relates to the fields of big data analysis, artificial intelligence and feature engineering technology, and in particular to a feature screening method, device, equipment and medium. Background Art

[0002] Feature engineering technology is the foundation of modern artificial intelligence models. Especially for regression and classification problems, the quality of the feature variable data used by the model directly determines the prediction effect of the model.

[0003] Traditional feature engineering technology mainly relies on the modeler's own modeling experience and lacks technology that can intelligently screen features, thus affecting the efficiency of modeling. Summary of the invention

[0004] The present application provides a feature screening method, device, equipment and medium to solve the problem that intelligent feature screening cannot be achieved in the prior art.

[0005] In a first aspect, the present application provides a feature screening method, comprising:

[0006] Obtain a feature data table of n samples under m dimensional features, and use n feature data under any dimensional feature as a feature data sequence;

[0007] Calculate the correlation index of the feature data sequence of each feature and the feature data sequences of other features respectively to obtain a correlation index table;

[0008] Selecting the feature i and feature j corresponding to the correlation index with the highest absolute value from the correlation index table;

[0009] If the correlation index with the highest absolute value exceeds a preset threshold, the feature to be deleted is determined from the feature i and the feature j according to the correlation between the feature i and the feature j and other features;

[0010] Deleting the feature to be deleted and its feature data sequence to obtain a new feature data table;

[0011] For the new feature data table, return to the step of respectively calculating the correlation index of the feature data sequence of each feature with the feature data sequences of other features, until the correlation index with the highest absolute value in the correlation index table of the new feature data table does not exceed the preset threshold, and the remaining features at this time are used as the target features obtained by screening.

[0012] In a second aspect, the present application provides a feature screening device, comprising:

[0013] A feature data acquisition module is used to obtain a feature data table of n samples under m dimensional features, and use the n feature data under any dimensional feature as a feature data sequence;

[0014] A correlation index calculation module is used to calculate the correlation index of the feature data sequence of each feature and the feature data sequence of other features respectively to obtain a correlation index table;

[0015] A selection module, used for selecting the feature i and feature j corresponding to the correlation index with the highest absolute value from the correlation index table;

[0016] A feature to be deleted determining module, configured to determine a feature to be deleted from the feature i and the feature j according to the correlation between the feature i and the feature j and other features if the correlation index with the highest absolute value exceeds a preset threshold;

[0017] A feature deletion module is used to delete the feature to be deleted and its feature data sequence to obtain a new feature data table;

[0018] A loop processing module is used to return to the step of respectively calculating the correlation index of the feature data sequence of each feature with the feature data sequences of other features for the new feature data table, until the correlation index with the highest absolute value in the correlation index table of the new feature data table does not exceed the preset threshold value, and the remaining features at this time are used as the target features obtained by screening.

[0019] In a third aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the feature screening method as described in any one of the embodiments of the present application is implemented.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a feature screening method as described in any one of the embodiments of the present application.

[0021] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements a feature screening method as described in any one of the embodiments of the present application.

[0022] The feature screening method, device, equipment and medium provided in the present application measure the correlation between different features through a correlation index, first calculate the correlation index between any two features, then select the correlation index with the highest absolute value, and then determine the corresponding two features, feature i and feature j. Next, the overall correlation between feature i and feature j and other features is examined respectively, and the features to be deleted are determined from feature i and feature j according to the overall correlation for deletion. Thus, the above operation is performed cyclically based on the deleted features until the absolute value of the correlation index between any two features exceeds the preset threshold, and the final feature set can be obtained. By using the above method to intelligently screen the features, the obtained feature set is more streamlined, and the feature set contains more information, which greatly improves the effectiveness of feature screening. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flow chart of a feature screening method provided in an embodiment of the present application;

[0025] Figure 2 A flow chart for determining features to be deleted in the feature screening method provided in an embodiment of the present application;

[0026] Figure 3 A flow chart of a method for correcting features in a feature screening method provided in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of the structure of a feature screening device provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0029] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0030] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0031] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. The purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or must use the relevant content of the solution.

[0032] Figure 1 Flow chart of the feature screening method provided in the embodiment of the present application. This embodiment is applicable to the case of screening feature data used for training models. The method can be performed by a feature screening device, which can be implemented in software and / or hardware, preferably configured in an electronic device, such as a computer device or a server. Figure 1 As shown, the method specifically includes:

[0033] S101. Obtain a feature data table of n samples under m dimensional features, and use n feature data under any dimensional feature as a feature data sequence.

[0034] S102 , respectively calculating the correlation index between the feature data sequence of each feature and the feature data sequences of other features to obtain a correlation index table.

[0035] S103 . Select the feature i and feature j corresponding to the correlation index with the highest absolute value from the correlation index table.

[0036] S104: Determine whether the correlation index with the highest absolute value exceeds a preset threshold. If yes, execute S105; if no, execute S107.

[0037] S105 . Determine the features to be deleted from the features i and j according to the correlations between the features i and j and other features.

[0038] S106, deleting the feature to be deleted and its feature data sequence to obtain a new feature data table, and returning to execute S102 for the new feature data table.

[0039] S107, taking the remaining features at this time as the target features obtained by screening.

[0040] The embodiment of the present application screens features based on the correlation between features. Specifically, first obtain a feature data table of n samples under m dimensional features, and use the n feature data under any dimensional feature as a feature data sequence. Exemplarily, the feature data table is represented as follows:

[0041] serial number Feature 1 Feature 2 Feature 3 Feature 4 Feature 5 …… Feature m-1 Feature m 1 <![CDATA[x 11 ]]> <![CDATA[x 12 ]]> <![CDATA[x 13 ]]> <![CDATA[x 14 ]]> <![CDATA[x 15 ]]> …… <![CDATA[x 1(m-1) ]]> <![CDATA[x 1m ]]> 2 <![CDATA[x 21 ]]> <![CDATA[x 22 ]]> <![CDATA[x 23 ]]> <![CDATA[x 24 ]]> <![CDATA[x 25 ]]> …… <![CDATA[x 2(m-1) ]]> <![CDATA[x 2m ]]> 3 <![CDATA[x 31 ]]> <![CDATA[x 32 ]]> <![CDATA[x 33 ]]> <![CDATA[x 34 ]]> <![CDATA[x 35 ]]> …… <![CDATA[x 3(m-1) ]]> <![CDATA[x 3m ]]> 4 <![CDATA[x 41 ]]> <![CDATA[x 42 ]]> <![CDATA[x 43 ]]> <![CDATA[x 44 ]]> <![CDATA[x 45 ]]> …… <![CDATA[x 4(m-1) ]]> <![CDATA[x 4m ]]> …… …… …… …… …… …… …… …… …… n-1 <![CDATA[x (n-1)1 ]]> <![CDATA[x (n-1)2 ]]> <![CDATA[x (n-1)3 ]]> <![CDATA[x (n-1)4 ]]> <![CDATA[x (n-1)5 ]]> …… <![CDATA[x (n-1)(m-1) ]]> <![CDATA[x (n-1)m ]]> n <![CDATA[x n1 ]]> <![CDATA[x n2 ]]> <![CDATA[x n3 ]]> <![CDATA[x n4 ]]> <![CDATA[x n5 ]]> …… <![CDATA[x n(m-1) ]]> <![CDATA[x nm ]]>

[0042] Among them, n represents the number of observation samples, m represents the number of feature dimensions, each row in the above table represents the feature data of an observation sample under each feature dimension, and each column represents the feature data sequence of a feature variable. For example, suppose a financial institution wants to predict the willingness of customers to purchase new financial products and needs to perform machine learning model modeling and analysis. There are currently 10,000 customer observation samples, corresponding to n = 10,000 in the above table, and each customer's data contains 10 features, corresponding to m = 10 in the above table.

[0043] Different features refer to describing the observed samples from different dimensions or angles. However, the same changes may occur between feature variables of different dimensions. The more such feature variables there are, the more complex the model will be, but the more data information the model can obtain. Therefore, it is necessary to intelligently screen the features of m dimensions. The target features obtained after screening can be used to train the current target model to improve the efficiency of model training.

[0044] First, the correlation index of the feature data sequence of each feature and the feature data sequence of other features is calculated respectively to obtain a correlation index table. For example, correlation coefficients such as the Pearson correlation coefficient, the Spearman correlation coefficient, or the Kendall rank correlation coefficient can be used as correlation indicators to measure the degree of correlation between features. For example, a correlation index table that can represent the correlation between two features can be constructed as follows:

[0045] 1 2 3 …… D 1 <![CDATA[ρ 11 ]]> <![CDATA[ρ 12 ]]> <![CDATA[ρ 13 ]]> …… <![CDATA[ρ 1D ]]> 2 <![CDATA[ρ 21 ]]> <![CDATA[ρ 22 ]]> <![CDATA[ρ 23 ]]> …… <![CDATA[ρ 2D ]]> 3 <![CDATA[ρ 31 ]]> <![CDATA[ρ 32 ]]> <![CDATA[ρ 33 ]]> …… <![CDATA[ρ 3D ]]> …… …… …… …… …… …… D <![CDATA[ρ D1 ]]> <![CDATA[ρ D2 ]]> <![CDATA[ρ D3 ]]> …… <![CDATA[ρ DD ]]>

[0046] Among them, ρ ij represents the correlation coefficient between the feature data sequences of the i-th feature and the j-th feature, and ρ ij =ρ ji , for ρ ii ,i=1,…,D (D represents the number of features), which represents the correlation coefficient between feature i and itself and is fixed to 1.

[0047] Then, select the feature i and feature j corresponding to the correlation index with the highest absolute value from the correlation index table. In the correlation index table, if the absolute value of the correlation index of the feature data sequence of feature i and feature j is the highest, it means that the correlation between feature i and feature j is the highest compared to any other pair of features. Therefore, if the correlation index of feature i and feature j does not exceed the preset threshold, then the correlation index of any other pair of features must not exceed the threshold either, so all features do not need to be deleted. On the contrary, if the correlation index of feature i and feature j exceeds the preset threshold, then you need to select one to delete to achieve the purpose of streamlining features.

[0048] In the embodiment of the present application, the feature to be deleted is determined from feature i and feature j based on the correlation between feature i and feature j and other features. That is, if one of feature i and feature j has a higher overall correlation with other features, it means that the information of the feature is more easily expressed by other features, so the feature can be deleted. The overall correlation between feature i and feature j and other features can also be measured by the correlation coefficient method, and the present application does not make any limitation on this.

[0049] After deleting the feature to be deleted and its feature data sequence, a new feature data table can be obtained, and then the new feature data table is continued to be executed back to S102. That is, if there are two features in the new feature data table whose absolute value of correlation index is the highest and exceeds the preset threshold, then one needs to be selected for deletion, and the cycle is iterated until the correlation index with the highest absolute value in a new feature data table does not exceed the preset threshold, and the remaining features at this time are used as the target features obtained by screening.

[0050] The feature screening method, device, equipment and medium provided in the embodiments of the present application measure the correlation between different features through a correlation index, first calculate the correlation index between any two features, then select the correlation index with the highest absolute value, and then determine the corresponding two features, feature i and feature j. Next, the overall correlation between feature i and feature j and other features is examined respectively, and the features to be deleted are determined from feature i and feature j according to the overall correlation for deletion. Thus, the above operation is performed cyclically based on the deleted features until the absolute value of the correlation index between any two features exceeds the preset threshold, and the final feature set can be obtained. The features are intelligently screened by the above method, and the obtained feature set is more streamlined, and the feature set contains more information, which greatly improves the effectiveness of feature screening.

[0051] In one embodiment, the correlation index includes at least one of the following: Pearson correlation coefficient, Spearman correlation coefficient, Kendall rank correlation coefficient and synchronous change rate, wherein the synchronous change rate is used to observe the same frequency phenomenon of data changes within a certain time span.

[0052] Specifically, different correlation coefficients reflect different aspects of correlation. For example, the Pearson correlation coefficient is a linear correlation coefficient, which reflects linear correlation, while the Spearman correlation coefficient and Kendall rank correlation coefficient are both rank correlation coefficients, which reflect whether the order of size of two features is consistent, and then determine whether the two increase or decrease synchronously.

[0053] In the embodiment of the present application, the above three kinds of correlation coefficients and the synchronous change rate can be considered at the same time, so that the correlation between different features can be measured from multiple angles and aspects, so that the result of the correlation analysis is more accurate, and the features with high correlation that are not easy to find can be excavated, so as to better perform feature screening. Specifically, any one of the above three kinds of correlation coefficients and the synchronous change rate can be selected as the correlation index of the feature for calculation, and a round of screening is performed according to the feature screening method in the above embodiment based on the correlation index table. Then, for the feature set obtained by the first round of screening, another kind of correlation index is selected for calculation, and the same screening method is adopted for the second round of screening. By analogy, a total of four rounds of screening are performed to obtain the final target feature set. It should be noted that the embodiment of the present application does not make any restriction on the order of the correlation index selected for each round of screening.

[0054] Furthermore, the synchronous change rate in the embodiment of the present application is a set of evaluation metrics based on the consistency of synchronous changes of different features, which is different from the correlation coefficient. Since the correlation coefficient between features is only for the comparison between data in the same period, but the correlation between many data often has a lag, in this case there is no way to detect the correlation coefficient. For example, two sets of data (x1, y1), (x2, y2), ..., (x1, y1) of feature X and feature Y collected in the past n days n ,y n ), where the subscripts of x and y are dates. If the correlation coefficient is used, the analysis process is to combine the feature X data x1 and the feature Y data y1 of the first day into a data pair (x1, y1), and then include it in the correlation analysis for consideration. However, the actual situation may be that the data of feature X and feature Y on the same day are not correlated, but the data one day apart are highly correlated, that is, x1 and y1 are unrelated, but x1 and y2 are highly correlated. In this case, no matter which correlation coefficient is used, it will be wrongly judged that feature X and feature Y are unrelated.

[0055] The synchronous change rate in the embodiment of the present application is not analyzed by the change correlation between the data at a fixed single time point, but by observing whether the data has a high degree of change frequency phenomenon within a certain time span area, as follows:

[0056] Assume that any two sets of feature X and feature Y have data (x1, y1), (x2, y2), …, (x n ,y n ), the synchronous change rate can be constructed:

[0057]

[0058] dx i =(x i -x i-1 ) / x i-1 , i = 2, ..., n

[0059] dy i =(y i -y i-1 ) / y i-1 , i = 2, ..., n

[0060] Among them, tc XY Indicates the synchronous change rate of feature X and feature Y, dx i anddy i Respectively represent the data x in the feature data sequence of feature X and feature Y i , data y i The increase ratio;

[0061] is a synchronous characteristic function, when dx i With dy i ,dy i-1 ,dy i+1 If any of the two with the same sign and the difference in the increase ratio between them does not exceed the target multiple (such as 1.5 times), then I is 1, otherwise I is 0. For example, suppose dx i =0.2,dy i =0.3,dy i-1 =0.25,dy i+1 =-0.3, obviously, dx i With dy i anddy i-1 The signs are the same, that is, there is dy i anddy i-1 Two values ​​and dx i The signs are the same, so we only need to compare dy i anddy i-1 Neutralize dx iIs the largest difference more than 1.5 times? i and dx i The difference is 0.1, dy i-1 and dx i The difference is 0.05, dy i and dx i The difference is greater than dy i-1 and dx i The gap is large, and The condition that the difference in the increase ratio does not exceed the target multiple is met, so I is 1.

[0062] It should be noted that due to tc XY The particularity of the formula construction may exist XY ≠tc YX Therefore, we can also use the method of calculating the mean as the final synchronous change rate of feature X and feature Y, that is:

[0063] Before screening the features, invalid features may be removed. That is, before respectively calculating the correlation index between the feature data sequence of each feature and the feature data sequence of other features, the method of the embodiment of the present application further includes:

[0064] If the data missing rate of the feature data sequence of any feature among the m dimensional features meets the first set condition, the feature and its feature data are deleted;

[0065] Otherwise, the data homogeneity degree of the feature data sequence of the feature is determined;

[0066] If the data homogeneity degree meets the second set condition, the missing data of the feature data sequence of the feature is filled.

[0067] For example, if the data missing rate of a feature exceeds 30%, it means that the information loss of the feature is high and it is difficult to fill and correct it through data supplementation methods. Therefore, the feature and its data can be directly deleted. If the missing rate does not exceed 30%, further analysis is required, that is, in the feature data sequence of the feature, after removing the missing feature data, observe whether the data homogeneity of the remaining samples of the feature is high. If the data homogeneity is high, the feature can also be deleted. Otherwise, the missing data of the feature data sequence of the feature can be filled.

[0068] In one embodiment, the degree of data homogeneity can be measured by: the volatility of data distribution, whether the distribution is even, and whether the data is concentrated in a set range. For example, assuming a column feature k, whose feature data sequence is represented by x 1k , x 2k, ..., x nk , there are P samples missing, so after removing the missing samples, the remaining samples are x 1k , x 2k , ..., x (n-p)k , then we can judge by measuring the homogeneity of the remaining samples, which is specifically expressed as follows:

[0069]

[0070] Where σ is x 1k , x 2k , ..., x (n-p)k The standard deviation of is its mean, q 0.95 and q 0.05 is its 95% and 5% quantiles, m △ That is, q 0.95 and q 0.05 The bin spacing for equally spaced bins.

[0071] Based on s, m △ That is, we can analyze whether the remaining sample data distribution volatility is large enough, whether the distribution is even, and whether the data values ​​are concentrated in a small range. If the volatility of the remaining samples is small, or most of the data is concentrated in a small range, then the quality of such data is very low and should not be included in the model. For example, the following judgment criteria can be constructed:

[0072] s>0.25

[0073]

[0074] Among them, P(q 0.95 +i*m △ ) means that the sample falls in the interval [q 0.95 +(i-1)*m △ ,q 0.95 +i*m △ ], the above three formulas are satisfied, then it can be considered that the homogeneity of the remaining sample data is not high, and this feature can be retained, so it can be introduced into the model after filling and correcting the sample data.

[0075] Next, the meaning and rationality of the above three formulas are explained. The essence of s is the coefficient of variation, which is used to measure the proportion of standard deviation to sample mean. Generally speaking, the distribution characteristics of a relatively dispersed and balanced data set follow the 4 sigma criterion, that is, the interval Approximately matches the 95% confidence interval of the sample. If s≤0.25, then the 95% confidence interval of the sample The coverage range will be too small, and the fluctuation range of its upper and lower limits will be compressed. This is very unfavorable for the discretization of data distribution and will seriously interfere with the subsequent model fitting work. It reflects that q 0.95 and q 0.05 After equally spaced bins, each box contains the maximum probability of sample data. It is the minimum value of the probability of containing sample data after binning. Too large or If the value is too small, it indicates that there is a serious accumulation of probabilities, that is, most of the data are concentrated in a small range, and some intervals have almost no samples, that is, the degree of homogeneity is too high, which is extremely bad for the balance of data distribution and has no benefit for the fitting of subsequent models.

[0076] Therefore, the embodiments of the present application can create a quantitative system for evaluating the homogeneity of feature data by analyzing the missing rate of data missing features, conduct preliminary screening of features based on this, select data with sufficient information for subsequent modeling, and improve the quality of feature data.

[0077] Figure 2 This is a flow chart of determining features to be deleted in the feature screening method provided in the embodiment of the present application. This embodiment further optimizes how to determine features to be deleted based on any of the above embodiments. Figure 2 As shown, the method includes:

[0078] S201 , respectively obtain the correlation indexes of feature i and feature j with feature data sequences of other features to obtain a correlation index sequence i and a correlation index sequence j.

[0079] S202 , determining a feature to be deleted from feature i and feature j according to the size of each correlation index in correlation index sequence i and correlation index sequence j and a preset threshold.

[0080] Among them, the size of each correlation index in the correlation index sequence is used to measure which feature i and feature j has a greater overall correlation with all other features, and the feature with a greater overall correlation is used as the feature to be deleted.

[0081] In one embodiment, the number of association indicators whose absolute values ​​exceed a preset threshold in the association indicator sequence i and the association indicator sequence j can be determined respectively, and the features corresponding to the larger number are used as features to be deleted. That is, in the association indicator sequences i and j, the one with the larger number of association indicators whose absolute value exceeds the preset threshold has the corresponding feature used as the feature to be deleted.

[0082] In another embodiment, if the number of correlation indices whose absolute values ​​exceed the preset threshold in correlation index sequence i and correlation index sequence j is the same, the cumulative sum of the absolute values ​​of each correlation index in correlation index sequence i and correlation index sequence j is calculated respectively, and the feature corresponding to the larger cumulative sum is used as the feature to be deleted. If the cumulative sums are still the same, a feature is randomly selected as the feature to be deleted.

[0083] For example, first, for the Pearson correlation coefficient P, we use the correlation coefficient table notation above and record the Pearson correlation coefficient table of each feature as follows:

[0084] 1 2 3 …… D 1 <![CDATA[pρ 11 ]]> <![CDATA[pρ 12 ]]> <![CDATA[pρ 13 ]]> …… <![CDATA[pρ 1D ]]> 2 <![CDATA[pρ 21 ]]> <![CDATA[pρ 22 ]]> <![CDATA[pρ 23 ]]> …… <![CDATA[pρ 2D ]]> 3 <![CDATA[pρ 31 ]]> <![CDATA[pρ 32 ]]> <![CDATA[pρ 33 ]]> …… <![CDATA[pρ 3D ]]> …… …… …… …… …… …… D <![CDATA[pρ D1 ]]> <![CDATA[pρ D2 ]]> <![CDATA[pρ D3 ]]> …… <![CDATA[pρ DD ]]>

[0085] The i-th row in the table represents the Pearson correlation coefficient between the i-th feature and other features. The solution of the embodiment of the present application is to first select the two features with the highest absolute value of the Pearson correlation coefficient, and then select one of them as the feature to be deleted based on other related characteristics reflected by these two features. In order to avoid interference, it is necessary to add the correlation coefficient {pρ ii ,i=1,…,D} is eliminated because its value is 1, which has reached the upper limit of the correlation coefficient and will interfere with the screening of highly correlated features, so it is eliminated.

[0086] In one embodiment, assuming that the two features with the highest absolute value of the correlation coefficient in the Pearson correlation coefficient table are feature i and feature j, if the absolute value of the Pearson correlation coefficient of the two features does not exceed 0.5, it indicates that the correlation between all features is not strong and should be retained. On the contrary, if it exceeds 0.5, continue to observe the correlation coefficients of feature i and feature j with other features to obtain the Pearson correlation coefficient sequence h i and the Pearson correlation coefficient sequence h j , respectively expressed as follows:

[0087] h i =(pρ i1 , pρ i2 , …, pρ i(i-1) , pρ i(i+1) , …, pρ iD}

[0088] h j ={p j1 , pρ j2 , …, pρ j(j-1) , pρ j(j+1) , …, pρ jD}

[0089] h i With h jWhichever feature has a correlation coefficient with an absolute value exceeding 0.5 and has more data, the corresponding feature is selected as the feature to be deleted. i There are 5 absolute values ​​of Pearson correlation coefficients exceeding 0.5, h j If 7 of the correlation coefficients have an absolute value greater than 0.5, it means that feature j has a higher overall correlation with other features, and the information of feature j is more easily expressed by other features, so feature j should be deleted. i With h j If the absolute value of the correlation coefficient in the same number exceeds 0.5, then h i With h j The absolute values ​​of the Pearson correlation coefficients in are accumulated respectively, and the features with higher accumulated sums are selected as the features to be deleted. If the accumulated sums are still consistent, a feature is randomly selected as the feature to be deleted.

[0090] The Pearson correlation coefficient table of the sample is reconstructed for the remaining features after deletion, and the same method as described above in the embodiment of the present application is used to continue to determine whether there are new features to be deleted, and the process is iterated in this way until no two features with absolute values ​​of correlation coefficients exceeding 0.5 are found in the remaining features, which is equivalent to the first round of feature screening using the Pearson correlation coefficient.

[0091] For example, the features after the first round of Pearson correlation coefficient screening can be sorted and summarized, and a Spearman correlation coefficient table can be constructed for them. The same logic and method as the Pearson correlation coefficient screening can be used to select the two features with the highest absolute values ​​of the Spearman correlation coefficient and select one to delete, until no two features with an absolute value of the Spearman correlation coefficient exceeding 0.5 are found among all the remaining features, which is equivalent to a second round of feature screening using the Spearman correlation coefficient.

[0092] Then, the features filtered by the Spearman correlation coefficient are sorted and summarized again, and the Kendall rank correlation coefficient table is constructed for them. Similarly, the two features with the highest absolute values ​​of the Kendall rank correlation coefficient are selected and one of them is eliminated, until no two features with absolute values ​​of the Kendall rank correlation coefficient exceeding 0.5 are found among all the remaining features. This is equivalent to the third round of feature screening using the Kendall rank correlation coefficient.

[0093] Finally, the features filtered by the Kendall rank correlation coefficient are sorted and summarized again, and a synchronous change rate table is constructed. Based on the synchronous change rate table, feature selection is performed again according to the above screening logic, which is equivalent to the fourth round of feature screening using the synchronous change rate.

[0094] The final features obtained from the above four rounds of screening are sorted out for subsequent model fitting. Thus, by constructing a correlation analysis method and creating a synchronous change rate analysis mechanism, a four-fold feature screening and filtering system is formed to achieve intelligent feature selection, greatly improving the effectiveness of feature screening.

[0095] Figure 3 This is a flow chart of a method for modifying features in a feature screening method provided in an embodiment of the present application. This embodiment further optimizes how to modify features based on any of the above embodiments. The number of dimensions of the target feature is represented as k. Figure 3 As shown, the method includes:

[0096] S301, inputting feature data of n samples under k dimensional features into the target model for fitting.

[0097] S302. Calculate the evaluation index of the influence degree of each dimensional feature among the k dimensional features on the target variable of the target model.

[0098] S303. Randomly shuffle the numerical order of the target variable a target number of times, and calculate the mean of the evaluation index of the degree of influence of each dimensional feature of the k dimensional features on the target variable after the target number of times of shuffling.

[0099] S304, comparing the influence evaluation index of each dimensional feature before scrambling, the relationship between the mean of the influence evaluation index after scrambling and the set reasonable range, and modifying the k dimensional features according to the comparison results.

[0100] The k-dimensional features obtained after screening are input into the model for fitting. During the fitting process, the model will also feedback a lot of information about the use effect of the features. This information can be used to further correct the features and delete inappropriate features to achieve the purpose of further streamlining the features. Among them, this application does not limit the type of target model. For example, it can be a prediction model for predicting the purchase intention of financial products in financial institutions.

[0101] Through model fitting, we can obtain the evaluation results of the impact of each feature on the target variable of the target model, that is, the impact evaluation index. The impact evaluation index includes at least one of the following: IV value used to reflect the predictive ability of the feature variable on the target variable, importance value used to measure the impact of the feature variable on the change of the target variable, and correlation coefficient between the feature and the target variable.

[0102] Among them, the full name of IV value is Information Value, which can be obtained by binning the value range of a feature, calculating the WOE (Weight of Evidence) value of each bin and weighted accumulation. The Importance value is a unique measurement indicator of the tree model such as the integrated algorithm LightGBM (Light Gradient Boosting Machine) to measure the impact of feature variables on the change of target variables. The full name is feature_importance. It should be noted that for tree models, when selecting features based on Importance, the selection threshold is not defined based on the specific Importance value between each feature and the target variable, but is determined based on the ranking of the Importance value of feature i among all features. If the model requires 20 feature variables, the principle of selecting features is whether their ranking is in the top 20. If it is in the top 20, the importance of feature i is marked. i =1, otherwise 0. The correlation coefficient between the feature and the target variable includes but is not limited to the Pearson correlation coefficient p, the Spearman correlation coefficient sp and the Kendall rank correlation coefficient tau. In one embodiment, in order to reflect the comprehensive effect, the average of the three correlation coefficients can be taken as the final correlation coefficient ρ * .

[0103] In one implementation, all indicators reflecting the relationship between features and target variables are evaluated in a mixed manner. Taking LightGBM as an example, IV value, importance value and ρ are used to evaluate * Comprehensive consideration is used for feedback correction. Using the above notation, assume that the relationship between k features and the target variable y is as follows:

[0104]

[0105]

[0106] Extract each column of feature data from the above table separately and perform IV, importance and ρ on the target variable y. * For example, the value of any column feature i {x 1i ,x 2i ,…,x ni} and {y1,y2,…,y n The result of the calculation is marked as IV i 、importance i and Then the target variables {y1,y2,…,y n} values ​​are shuffled, and after each shuffle, the new IV, importance, and ρ about feature i and the shuffled target variable y can be calculated. * value, repeat this shuffle operation for the target number of times, such as 1000 times, and each time you will get different IV, importance, and ρ about feature i and the shuffled target variable y * The average of 1000 results is taken as the mean of the evaluation index of the influence degree between feature i and target variable y, which is recorded as and

[0107] Then, for the impact evaluation index of each dimension feature before scrambling, if any feature’s impact evaluation index is not within the set reasonable range, the feature and its feature data will be deleted. For any feature, if its impact evaluation index before scrambling is within the set reasonable range, but its average impact evaluation index after scrambling is higher than its impact evaluation index before scrambling, the feature and its feature data will be deleted.

[0108] Therefore, illustratively, the following evaluation rules can be constructed:

[0109] (1) Start cyclic screening from the first feature (i=1):

[0110] (2) Compare “IV i 、importance i , "and The following judgment rules are used to determine whether feature i should be deleted:

[0111] When “IV i >0.5” or “IV i When <0.1”, feature i is discarded;

[0112] When "0.1≤IV i ≤0.3” and When , feature i is discarded;

[0113] When “importance i =0", feature i is discarded;

[0114] When “importance i =1” and When , feature i is discarded;

[0115] when When , feature i is discarded;

[0116] when and When , feature i is discarded;

[0117] (3) After feature i is judged, the next feature loop is started, that is, i=i+1, and the loop judgment steps in (2) are repeated;

[0118] (4) When i=k and the loop judgment step in (2) is completed, the entire loop screening process ends;

[0119] The reason for doing this is that if a feature is really important to the model, then first the IV, importance, and ρ corresponding to the feature * They should all be within a reasonable range. For example, the reasonable range of IV is 0.1-0.5, and the corresponding importance when the feature is very important is 0.1-0.5. i The reasonable value should be 1, ρ * The reasonable range should be at least greater than 0.1. However, it is not enough to be within the reasonable range. If a feature is really important, then after the order of the target variable y is disrupted, it is normal that the original data is seriously disturbed. At this time, the feature should change from an important feature to an unimportant feature. Therefore, further analysis and comparison, if the IV value is still 0.1≤IV before the order is disrupted i ≤0.3 is a reasonable range, but after the disruption That is, the IV value is further increased, which means that disrupting the order of the target variable y not only does not reduce the importance of the feature, but makes the feature more important, which is obviously inconsistent with the facts. Therefore, there is the above statement "when 0.1≤IV i ≤0.3 and Similarly, when importance i =1 and When discarding feature i" and "when and The judgment rules for "discarding feature i" are based on the same logic.

[0120] The technical solution of the embodiment of the present application creates a model fitting feedback correction mechanism, performs model fitting on the screened features, and performs feedback correction on these features based on the model fitting effect. The rationality of feature screening is further improved by analyzing the changing characteristics of feature importance after disrupting the order of target variables, laying the foundation for real-time updating of model features and greatly reducing the threshold for beginners to use model technology.

[0121] Figure 4 This is a schematic diagram of the structure of the feature screening device provided in the embodiment of the present application. Figure 4 As shown, the feature screening device 40 comprises:

[0122] The feature data acquisition module 410 is used to acquire a feature data table of n samples under m dimensional features, and use the n feature data under any dimensional feature as a feature data sequence;

[0123] The correlation index calculation module 420 is used to calculate the correlation index of the feature data sequence of each feature and the feature data sequence of other features respectively, and obtain a correlation index table;

[0124] A selection module 430 is used to select, from the association index table, features i and j corresponding to the association index with the highest absolute value;

[0125] A feature to be deleted determining module 440 is used to determine a feature to be deleted from the feature i and the feature j according to the correlation between the feature i and the feature j and other features if the correlation index with the highest absolute value exceeds a preset threshold;

[0126] A feature deletion module 450 is used to delete the feature to be deleted and its feature data sequence to obtain a new feature data table;

[0127] The loop processing module 460 is used to return to the step of calculating the correlation index of the feature data sequence of each feature with the feature data sequences of other features for the new feature data table, until the correlation index with the highest absolute value in the correlation index table of the new feature data table does not exceed the preset threshold value, and the remaining features at this time are used as the target features obtained by screening.

[0128] In some embodiments, the to-be-deleted feature determination module 440 includes:

[0129] A correlation index sequence acquisition unit is used to respectively acquire the correlation indexes of feature i and feature j with feature data sequences of other features, and obtain a correlation index sequence i and a correlation index sequence j;

[0130] A feature-to-be-deleted determining unit, configured to determine a feature to be deleted from the feature i and the feature j according to the magnitude of each correlation index in the correlation index sequence i and the correlation index sequence j and the preset threshold;

[0131] In some embodiments, the feature to be deleted determining unit is specifically used to:

[0132] The number of association indexes whose absolute values ​​exceed the preset threshold in the association index sequence i and the association index sequence j is determined respectively, and the feature corresponding to the larger number is used as the feature to be deleted.

[0133] In some embodiments, the to-be-deleted feature determination unit is further configured to:

[0134] If the number of association indicators whose absolute values ​​exceed the preset threshold in the association indicator sequence i and the association indicator sequence j is the same, the cumulative sum of the absolute values ​​of each association indicator in the association indicator sequence i and the association indicator sequence j is calculated respectively, and the feature corresponding to the larger cumulative sum is used as the feature to be deleted.

[0135] In some embodiments, the correlation index includes at least one of the following: Pearson correlation coefficient, Spearman correlation coefficient, Kendall rank correlation coefficient and synchronous change rate, wherein the synchronous change rate is used to observe the same frequency phenomenon of data changes within a certain time span.

[0136] In some embodiments, the synchronization change rate is expressed by the following formula:

[0137]

[0138] dx i =(x i -x i-1 ) / x i-1 , i = 2, ..., n

[0139] dy i =(y i -y i-1 ) / y i-1 , i = 2, ..., n

[0140] Among them, X and Y represent any two features, tc XY Indicates the synchronous change rate of feature X and feature Y, dx i anddy i Respectively represent the data x in the feature data sequence of feature X and feature Y i , data y i The increase ratio;

[0141] is a synchronous characteristic function, when dx i With dy i ,dy i-1 ,dy i+1 If any two numbers with the same positive and negative signs differ by no more than the target multiple, then I is 1, otherwise I is 0.

[0142] In some embodiments, the dimension of the target feature is represented as k, and the device further includes a correction module, which includes:

[0143] A fitting unit, used for inputting the feature data of the n samples under the k dimensional features into the target model for fitting;

[0144] An influence degree evaluation index calculation unit, used to respectively calculate an influence degree evaluation index of each dimensional feature of the k dimensional features on the target variable of the target model;

[0145] A sequence shuffling unit, used to randomly shuffle the numerical sequence of the target variable a target number of times, and respectively calculate the mean of the evaluation index of the degree of influence of each dimensional feature of the k dimensional features on the target variable after the target number of times of shuffling;

[0146] The correction unit is used to compare the influence evaluation index of each dimensional feature before scrambling, the relationship between the mean value of the influence evaluation index after scrambling and the set reasonable range, and correct the k dimensional features according to the comparison result.

[0147] In some embodiments, the correction unit is specifically used to:

[0148] For the impact evaluation indexes of the features of each dimension before the disruption, if any of the impact evaluation indexes of the features is not within the set reasonable range, the feature and its feature data will be deleted; and

[0149] For any feature, if its impact evaluation index before disruption is within the set reasonable range, but its average impact evaluation index after disruption is higher than its impact evaluation index before disruption, the feature and its feature data will be deleted.

[0150] In some embodiments, the impact evaluation index includes at least one of the following:

[0151] The IV value used to reflect the predictive ability of the feature variable on the target variable, the importance value used to measure the impact of the feature variable on the change of the target variable, and the correlation coefficient between the feature and the target variable.

[0152] In some embodiments, the device further includes a data preprocessing module, and the data preprocessing module is specifically used to:

[0153] Before the correlation index calculation module 420 respectively calculates the correlation index between the feature data sequence of each feature and the feature data sequences of other features, if the data missing rate of the feature data sequence of any feature in the m dimensional features meets the first set condition, the feature and its feature data are deleted;

[0154] Otherwise, the data homogeneity degree of the feature data sequence of the feature is determined;

[0155] If the data homogeneity degree meets the second set condition, the missing data of the feature data sequence of the feature is filled.

[0156] In some embodiments, the degree of data homogeneity is determined by at least one of the following methods:

[0157] The volatility of the data distribution, whether the distribution is even, and whether the data is concentrated within a set range.

[0158] In some embodiments, the features obtained through the correction are used to train the target model.

[0159] The feature screening device provided in the embodiment of the present application can be used to execute the technical solution of the feature screening method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0160] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the correlation index calculation module 420 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above correlation index calculation module 420. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0161] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0162] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the scheme in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0163] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0164] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0165] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0166] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0167] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the feature screening method of the above embodiment.

[0168] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the feature screening method in the above embodiment can be implemented.

[0169] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present application, and the programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and also includes a conventional procedural programming language, such as "C" language or similar programming language. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0170] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A feature screening method, characterized in that: include: Obtain a feature data table of n samples under m dimensional features, and use n feature data under any dimensional feature as a feature data sequence; Calculate the correlation index of the feature data sequence of each feature and the feature data sequences of other features respectively to obtain a correlation index table; Selecting the feature i and feature j corresponding to the correlation index with the highest absolute value from the correlation index table; If the correlation index with the highest absolute value exceeds a preset threshold, the feature to be deleted is determined from the feature i and the feature j according to the correlation between the feature i and the feature j and other features; Deleting the feature to be deleted and its feature data sequence to obtain a new feature data table; For the new feature data table, return to the step of respectively calculating the correlation index of the feature data sequence of each feature with the feature data sequences of other features, until the correlation index with the highest absolute value in the correlation index table of the new feature data table does not exceed the preset threshold, and the remaining features at this time are used as the target features obtained by screening.

2. The method according to claim 1, characterized in that Determining the features to be deleted from the features i and j according to the correlations between the features i and j and all other features, includes: Respectively obtain the correlation indexes of feature i and feature j with the feature data sequences of other features, and obtain the correlation index sequence i and the correlation index sequence j; According to the magnitude of each correlation index in the correlation index sequence i and the correlation index sequence j and the preset threshold, a feature to be deleted is determined from the feature i and the feature j.

3. The method according to claim 2, characterized in that The step of determining the feature to be deleted from the feature i and the feature j according to the size of each correlation index in the correlation index sequence i and the correlation index sequence j and the preset threshold value comprises: The number of association indexes whose absolute values ​​exceed the preset threshold in the association index sequence i and the association index sequence j is determined respectively, and the feature corresponding to the larger number is used as the feature to be deleted.

4. The method according to claim 3, characterized in that The step of determining the feature to be deleted from the feature i and the feature j according to the size of each correlation index in the correlation index sequence i and the correlation index sequence j and the preset threshold value further includes: If the number of association indicators whose absolute values ​​exceed the preset threshold in the association indicator sequence i and the association indicator sequence j is the same, the cumulative sum of the absolute values ​​of each association indicator in the association indicator sequence i and the association indicator sequence j is calculated respectively, and the feature corresponding to the larger cumulative sum is used as the feature to be deleted.

5. The method according to any one of claims 1 to 4, characterized in that The correlation index includes at least one of the following: Pearson correlation coefficient, Spearman correlation coefficient, Kendall rank correlation coefficient and synchronous change rate, wherein the synchronous change rate is used to observe the same frequency phenomenon of data changes within a certain time span.

6. The method according to claim 5, characterized in that The synchronous change rate is expressed by the following formula: dx i =(x i -x i-1 ) / x i-1 ,i=2,…,n of i (y i -the i-1 ) / the i-1 ,i=2,…,n Among them, X and Y represent any two features, tc XY Indicates the synchronous change rate of feature X and feature Y, dx i anddy i Respectively represent the data x in the feature data sequence of feature X and feature Y i , data y i The increase ratio; I {dxi,dyi,dyi-1,dyi+1} is a synchronous characteristic function, when dx i With dy i ,dy i-1 ,dy i+1 If any two numbers with the same positive and negative signs differ by no more than the target multiple, then I is 1, otherwise I is 0.

7. The method according to claim 1, characterized in that The dimension of the target feature is represented as k, and the method further includes: Inputting the feature data of the n samples under k dimensional features into the target model for fitting; Calculate the evaluation index of the influence degree of each dimensional feature of the k dimensional features on the target variable of the target model respectively; Randomly shuffle the numerical order of the target variable a target number of times, and calculate the mean of the evaluation index of the degree of influence of each dimensional feature in the k dimensional features on the target variable after the target number of times of shuffling; Compare the impact evaluation index of each dimensional feature before scrambling, the relationship between the mean value of the impact evaluation index after scrambling and the set reasonable range, and modify the k dimensional features according to the comparison results.

8. The method according to claim 7, characterized in that The comparison of the impact evaluation index of each dimensional feature before the scrambling and the relationship between the average of the impact evaluation index after the scrambling and the set reasonable range, and the correction of the k dimensional features according to the comparison result, includes: For the impact evaluation indexes of the features of each dimension before the disruption, if any of the impact evaluation indexes of the features is not within the set reasonable range, the feature and its feature data will be deleted; and For any feature, if its impact evaluation index before disruption is within the set reasonable range, but its average impact evaluation index after disruption is higher than its impact evaluation index before disruption, the feature and its feature data will be deleted.

9. The method according to claim 7 or 8, characterized in that: The impact evaluation index includes at least one of the following: The IV value used to reflect the predictive ability of the feature variable on the target variable, the importance value used to measure the impact of the feature variable on the change of the target variable, and the correlation coefficient between the feature and the target variable.

10. The method according to claim 1, characterized in that Before respectively calculating the correlation index between the feature data sequence of each feature and the feature data sequences of other features, the method further includes: If the data missing rate of the feature data sequence of any feature among the m dimensional features meets the first set condition, the feature and its feature data are deleted; Otherwise, the data homogeneity degree of the feature data sequence of the feature is determined; If the data homogeneity degree meets the second set condition, the missing data of the feature data sequence of the feature is filled.

11. The method according to claim 10, characterized in that The degree of data homogeneity is determined by at least one of the following methods: The volatility of the data distribution, whether the distribution is even, and whether the data is concentrated within a set range.

12. The method according to claim 7, characterized in that The features obtained through the correction are used to train the target model.

13. A feature screening device, characterized in that: include: A feature data acquisition module is used to obtain a feature data table of n samples under m dimensional features, and use the n feature data under any dimensional feature as a feature data sequence; A correlation index calculation module is used to calculate the correlation index of the feature data sequence of each feature and the feature data sequence of other features respectively to obtain a correlation index table; A selection module, used for selecting the feature i and feature j corresponding to the correlation index with the highest absolute value from the correlation index table; A feature to be deleted determining module, configured to determine a feature to be deleted from the feature i and the feature j according to the correlation between the feature i and the feature j and other features if the correlation index with the highest absolute value exceeds a preset threshold; A feature deletion module is used to delete the feature to be deleted and its feature data sequence to obtain a new feature data table; A loop processing module is used to return to the step of respectively calculating the correlation index of the feature data sequence of each feature with the feature data sequences of other features for the new feature data table, until the correlation index with the highest absolute value in the correlation index table of the new feature data table does not exceed the preset threshold value, and the remaining features at this time are used as the target features obtained by screening.

14. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the feature screening method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the feature screening method according to any one of claims 1 to 12.

16. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the feature screening method described in any one of claims 1 to 12.

Images