Variable screening method, device, nonvolatile storage medium and processor

Abstract

The application discloses a variable screening method and device, a nonvolatile storage medium and a processor. The method comprises the following steps: obtaining independent variables for evaluating a target variable, wherein the target variable corresponds to one or more independent variables; using a preset Pearson correlation coefficient model to evaluate the linear relationship between the independent variables and the target variable, and determining the linear correlation degree; in the case that the target variable corresponds to multiple independent variables with a linear correlation degree higher than a preset correlation degree threshold, selecting the independent variable with the highest linear correlation degree as a sample variable of the target variable, wherein the sample variable and the target variable are used as training data for training a target prediction model, and the target prediction model is used for analyzing the independent variables to determine a prediction variable. The application solves the technical problem of low efficiency in determining a consumer portrait due to the inability to screen the independent variables for determining the consumer portrait.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically, to a variable selection method, apparatus, non-volatile storage medium, and processor. Background Technology

[0002] In recent years, with the rise of big data, research on consumer behavior analysis has flourished. Scholars from many fields, including databases and data mining, information systems and information management, image processing and computer vision, social network analysis, and e-commerce, have joined the ranks of consumer behavior researchers. Simultaneously, this research field has also attracted significant attention from enterprises operating in the digital economy, such as e-commerce and social networks. Consumer behavior analysis is considered an effective means for enterprises to understand their consumers and conduct marketing activities in the digital economy. In these emerging fields, consumer behavior research is referred to as consumer profiling, and it also holds an important position in research areas such as social computing.

[0003] However, existing technologies cannot filter the independent variables used to determine consumer profiles, thus affecting the efficiency of consumer profile determination due to the angles of the independent variables used.

[0004] There is currently no effective solution to the problem of low efficiency in determining consumer profiles due to the inability to filter independent variables. Summary of the Invention

[0005] This invention provides a variable filtering method, apparatus, non-volatile storage medium, and processor to at least solve the technical problem of low efficiency in determining consumer profiles due to the inability to filter independent variables for determining consumer profiles.

[0006] According to one aspect of the present invention, a variable selection method is provided, comprising: obtaining independent variables for evaluating a target variable, wherein the target variable corresponds to one or more independent variables; evaluating the linear relationship between the independent variables and the target variable using a preset Pearson correlation coefficient model to determine the linear correlation degree; and selecting the independent variable with the highest linear correlation degree as a sample variable of the target variable when the target variable corresponds to multiple independent variables with linear correlation degrees higher than a preset correlation degree threshold, wherein the sample variable and the target variable serve as training data for training a target prediction model, and the target prediction model is used to analyze the independent variables to determine predictor variables.

[0007] Optionally, after selecting the independent variable with the highest linear correlation as the sample variable of the target variable, the method further includes: identifying the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable, and binary variable; if the independent variable belongs to the categorical variable, using a pre-set chi-square test model to test the correlation between the independent variable and the predictor variable of the target prediction model; if the independent variable does not belong to the categorical variable, using a pre-set regression model to test the correlation between the independent variable and the predictor variable of the target prediction model.

[0008] Optionally, obtaining independent variables for evaluating the target variable includes: obtaining attribute values ​​of the independent variables; analyzing the attribute values ​​of the independent variables using a preset evaluation algorithm to determine the predictive value of the independent variables, wherein the predictive value is used to represent the degree of conformity between the predictive variable determined based on the independent variables and the target variable corresponding to the independent variables; and selecting independent variables whose predictive values ​​are higher than a preset value threshold as independent variables for evaluating the target variable.

[0009] Optionally, analyzing the attribute values ​​of the independent variable using a preset evaluation algorithm to determine the predictive value of the independent variable includes: identifying the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable, and binary variable; if the independent variable belongs to the interval variable, binning the independent variable into multiple interval variables; and analyzing the attribute values ​​of each interval variable using the preset evaluation algorithm to determine the predictive value of the interval variable.

[0010] Optionally, analyzing the attribute values ​​of each interval variable using the preset evaluation algorithm to determine the predictive value of the interval variable includes: analyzing each interval variable using a preset evidence weight algorithm to determine the evidence weight of each interval variable, wherein the evidence weight is used to represent the logarithm of the ratio of the proportion of good variables to the proportion of bad variables in the interval variable, the proportion of good variables is the proportion of good variables in each interval variable relative to the proportion of good variables in all interval variables, and the proportion of bad variables is the proportion of bad variables in each interval variable relative to the proportion of bad variables in all interval variables; and analyzing the attribute values ​​of each interval variable using the preset evaluation algorithm to determine the predictive value of the interval variable.

[0011] Optionally, analyzing the attribute values ​​of each interval variable using the preset evaluation algorithm to determine the predicted value of the interval variable includes: analyzing the evidence weights of each interval variable using a preset information value evaluation model to determine the information value of each interval variable, and using the information value as the predicted value, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; or analyzing the evidence weights of each interval variable using a preset Gini index model to determine the Gini index of each interval variable, and using the Gini index as the predicted value, wherein the Gini index is used to assess the impurity of the interval variable.

[0012] Optionally, analyzing the attribute values ​​of each interval variable using the preset evaluation algorithm to determine the predictive value of the interval variable includes: analyzing the evidence weights of each interval variable using a preset information value evaluation model to determine the information value of each interval variable, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; analyzing the evidence weights of each interval variable using a preset Gini index model to determine the Gini index of each interval variable, wherein the Gini index is used to assess the impurity of the interval variable; determining the product of the information value and a first preset weight to determine a first value; determining the product of the information value and a second preset weight to determine a second value; and determining the predictive value based on the sum of the first value and the second value.

[0013] According to another aspect of the present invention, a variable screening device is also provided, comprising: an acquisition module, configured to acquire independent variables for evaluating a target variable, wherein the target variable corresponds to one or more independent variables; an evaluation module, configured to evaluate the linear relationship between the independent variables and the target variable using a preset Pearson correlation coefficient model, and determine the linear correlation degree; and a selection module, configured to select the independent variable with the highest linear correlation degree as a sample variable of the target variable when the target variable corresponds to multiple independent variables whose linear correlation degree is higher than a preset correlation degree threshold, wherein the sample variable and the target variable serve as training data for training a target prediction model, and the target prediction model is used to analyze the independent variables to determine predictor variables.

[0014] According to another aspect of the present invention, a non-volatile storage medium is also provided, wherein a program is stored in the non-volatile storage medium, wherein the program controls the device where the non-volatile storage medium is located to execute the variable filtering method described above when the program is running.

[0015] According to another aspect of the present invention, an electronic device is also provided, comprising: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the variable filtering method described above.

[0016] In this embodiment of the invention, independent variables for evaluating the target variable are obtained, wherein the target variable corresponds to one or more independent variables; a preset Pearson correlation coefficient model is used to evaluate the linear relationship between the independent variables and the target variable to determine the linear correlation degree; when the target variable corresponds to multiple independent variables with linear correlation degrees higher than a preset correlation degree threshold, the independent variable with the highest linear correlation degree is selected as the sample variable of the target variable, wherein the sample variable and the target variable are used as training data for training the target prediction model, and the target prediction model is used to analyze the independent variables to determine the predictor variable, thereby achieving the purpose of screening the independent variables, reducing the amount of training data required to train the target prediction model, and thus achieving the technical effect of improving the training efficiency of the target prediction model, thereby solving the technical problem of low efficiency in determining consumer profiles due to the inability to screen the independent variables for determining consumer profiles. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of a variable selection method according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of a variable screening device according to an embodiment of the present invention;

[0020] Figure 3 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] According to an embodiment of the present invention, a variable screening method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0024] Figure 1 This is a flowchart of a variable selection method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0025] Step S102: Obtain the independent variables used to evaluate the target variable, wherein the target variable corresponds to one or more independent variables;

[0026] Step S104: Use a pre-set Pearson correlation coefficient model to evaluate the linear relationship between the independent variable and the target variable, and determine the linear correlation degree;

[0027] Step S106: When the target variable corresponds to multiple independent variables with linear correlation higher than the preset correlation threshold, the independent variable with the highest linear correlation is selected as the sample variable of the target variable. The sample variable and the target variable are used as training data for training the target prediction model. The target prediction model is used to analyze the independent variables to determine the predictor variable.

[0028] In this embodiment of the invention, independent variables for evaluating the target variable are obtained, wherein the target variable corresponds to one or more independent variables; a preset Pearson correlation coefficient model is used to evaluate the linear relationship between the independent variables and the target variable to determine the linear correlation degree; when the target variable corresponds to multiple independent variables with linear correlation degrees higher than a preset correlation degree threshold, the independent variable with the highest linear correlation degree is selected as the sample variable of the target variable, wherein the sample variable and the target variable are used as training data for training the target prediction model, and the target prediction model is used to analyze the independent variables to determine the predictor variable, thereby achieving the purpose of screening the independent variables, reducing the amount of training data required to train the target prediction model, and thus achieving the technical effect of improving the training efficiency of the target prediction model, thereby solving the technical problem of low efficiency in determining consumer profiles due to the inability to screen the independent variables for determining consumer profiles.

[0029] The variable selection method described above in this invention can be used in e-commerce environments, where consumer profiles are used to represent users' consumption behavior during the e-commerce shopping process.

[0030] In step S102 above, the target variable can represent "whether the user purchases a certain product", and the independent variable can represent "the user's browsing history" or "the user's purchase history".

[0031] For example, if a user browses a product multiple times, it indicates that the user intends to purchase the product. Therefore, by using the number of times a user browses a product as the independent variable and whether the user purchases the product as the target variable, we can predict whether a user will purchase the product based on the number of times the user browses it.

[0032] For example, if a user browses a product for a long time, it indicates that the user intends to buy the product. Therefore, by using the browsing time of a user to browse a product as the independent variable and whether the user buys the product as the target variable, we can predict whether the user will buy the product based on the browsing time of a user.

[0033] For example, if a user purchases product A, and product B is a related product of product A, then the user's purchase of product A indicates a high probability that the user will purchase product B. Therefore, by taking the user's purchase of product A as the independent variable and whether the user purchases product B as the target variable, we can predict whether the user will purchase product B based on the user's purchase of product A.

[0034] In step S102 above, the target variable can represent "whether the user has engaged in credit fraud" and the independent variable can represent "the user's annual income".

[0035] For example, a user's credit fraud will affect their credit score. If a user has a high and stable annual income, their credit fraud will have a significant negative impact on their credit score. The benefits of fraud are not proportional to the costs of fraud. Therefore, users with high and stable annual income are less likely to engage in credit fraud. Thus, by using the user's annual income as the independent variable and whether the user engages in credit fraud as the target variable, we can predict whether the user will engage in credit fraud based on their annual income.

[0036] In step S104 above, the preset Pearson correlation coefficient model is mainly used to describe the linear relationship between proportional variables, between interval variables, and between binary variables and interval variables.

[0037] In step S104 above, the Pearson correlation coefficient model is preset as follows:

[0038] ,

[0039] Where x represents the independent variable and y represents the target variable, the correlation coefficient r of linear correlation ranges from [-1, +1]. Based on experience, different values ​​of r indicate different degrees of linear correlation. |r| represents the degree of linear correlation. |r| < 0.3 indicates low linear correlation; 0.3 < |r| < 0.5 indicates low to medium linear correlation; 0.5 < |r| < 0.8 indicates medium linear correlation; and 0.8 < |r| < 1 indicates high linear correlation.

[0040] Alternatively, if |r|>0.6 and there are multiple independent variables used to evaluate the target variable, only one independent variable needs to be retained.

[0041] Optionally, independent variables with a linear correlation |r| greater than a preset correlation threshold can be selected as independent variables for evaluating the target variable.

[0042] In step S106 above, the target prediction model is used to predict the predictor variable related to the known independent variable.

[0043] For example, if a user browses a product multiple times, it indicates that the user intends to purchase the product. Therefore, if the number of times a user browses a product is used as the independent variable, the target prediction model can be used to predict the independent variable and determine the predictor variable. The predictor variable can then be represented as the user's intention to purchase the product.

[0044] Optionally, during the training phase of the target prediction model, the independent variables and the target variable are used as known training data for model training; when using the target prediction model to analyze the independent variables, the unknown predictor variables can be determined based on the known independent variables.

[0045] Optionally, both the target variable and the predictor variable can be used as related variables of the independent variable. If the related variable of the independent variable already exists, then the related variable is the target variable; if the related variable of the independent variable has not yet occurred, then the related variable is the predictor variable.

[0046] As an optional embodiment, after selecting the independent variable with the highest linear correlation as the sample variable of the target variable, the method further includes: identifying the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable, and binary variable; if the independent variable is a categorical variable, using a pre-set chi-square test model to test the correlation between the independent variable and the predictor variable of the target prediction model; if the independent variable is not a categorical variable, using a pre-set regression model to test the correlation between the independent variable and the predictor variable of the target prediction model.

[0047] Optionally, the preset chi-square test model is:

[0048]

[0049] in, The observed values ​​representing the frequencies of each cross-classification (e.g., independent variables). Let $\mathbf{ ... When the sample size is large, The statistic approximately follows a sequence with (R⁻¹)(C⁻¹) degrees of freedom. (Chi-square) distribution; The larger the value, the stronger the correlation between the independent variable and the target variable.

[0050] In the above embodiments of the present invention, the chi-square test is used to measure the correlation between qualitative variables such as categorical variables, including ordinal variables, and to compare the sample rates of two or more variables.

[0051] Optionally, the pre-defined regression model can be constructed based on the R-squared method, which uses the R-squared method to draw on the analytical algorithm of multiple linear regression to identify and select independent variables that have important predictive significance and value for the target variable.

[0052] Optionally, a regression model is preset:

[0053] in, This indicates the goodness of fit of the regression equation. (0,1), A larger R value indicates a better fit between the regression equation and the sample observations (such as independent variables). R is also known as the sample multiple correlation coefficient between the target variable Y and the independent variables X1, X2, ..., Xp, which represents the linear relationship between the overall X1, X2, ..., Xp and Y.

[0054] Wherein, SSR represents the regression square, SSE represents the residual sum of squares, and SST represents the total sum of squares.

[0055] In the above embodiments of the present invention, when the independent variable is a categorical variable, a pre-set chi-square test model can be used to test the correlation between the independent variable and the predictor variable of the target prediction model; when the independent variable is not a categorical variable, a pre-set regression model can be used to test the correlation between the independent variable and the predictor variable of the target prediction model. Thus, by using the pre-set chi-square test model and the pre-set regression model, the correlation between different types of independent variables and predictor variables can be determined, thereby realizing the testing of the correlation between the independent variable and the predictor variable in training the target prediction model and ensuring that the independent variable used in training the target prediction model has a high correlation with the target variable.

[0056] As an optional embodiment, obtaining independent variables for evaluating the target variable includes: obtaining attribute values ​​of the independent variables; analyzing the attribute values ​​of the independent variables using a preset evaluation algorithm to determine the predictive value of the independent variables, wherein the predictive value is used to represent the degree of conformity between the predictive variable determined based on the independent variables and the target variable corresponding to the independent variables; and selecting independent variables with predictive values ​​higher than a preset value threshold as independent variables for evaluating the target variable.

[0057] In the above embodiments of the present invention, a preset evaluation algorithm is used to analyze the attribute values ​​of independent variables to determine the predictive value of the independent variables. Based on the predictive value, independent variables with higher predictive value can be selected from multiple independent variables used to evaluate the target variable. Then, the target prediction model can be trained using independent variables with higher predictive value to ensure the accuracy of the trained target prediction model.

[0058] Optionally, the independent variables that need to be trained in the target prediction model can be selected with high predictive value through IV, WOE and Gini and then put into the target prediction model for training, so as to provide more accurate analysis and prediction of potential value in data mining commercial scenarios.

[0059] As an optional embodiment, analyzing the attribute values ​​of independent variables using a preset evaluation algorithm to determine the predictive value of independent variables includes: identifying the variable type of independent variables, wherein the variable type includes at least: proportional variables, interval variables, categorical variables, and binary variables; if the independent variable is an interval variable, binning the independent variable into multiple interval variables; and analyzing the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable.

[0060] In the above embodiments of the present invention, when the independent variable is an interval variable, the interval variable can be divided into multiple interval variables by binning, and then the predictive value of the interval variables can be determined by a preset evaluation algorithm.

[0061] Optionally, in a project to "predict whether a user is suspected of credit fraud in credit card usage," the target variable is "whether there is credit fraud," a binary variable (0,1), where 0 represents no fraud and 1 represents fraud. Simultaneously, one of the independent variables is "user's annual income." In the original records of the data warehouse, this field is an interval variable. If we use the Weight of Evidence (WOE) and Information Value (IV) metrics to determine its measurability—that is, whether it is suitable as an independent variable for prediction—we need to transform this interval variable "user's annual income" into a categorical weighted variable (ordinal weighted variable). For example, "binning" becomes a categorical variable with four intervals: less than 20,000 yuan, [20,000, 60,000), [60,000, 100,000), and more than 100,000 yuan, for a total of four categories. The above four types of intervals are also known as the four attribute values ​​of the independent variable "user's annual income". For each attribute value, the weight of evidence (WOE) in the sample data can be calculated.

[0062] As an optional embodiment, analyzing the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable includes: analyzing each interval variable using a preset evidence weight algorithm to determine the evidence weight of each interval variable, wherein the evidence weight is used to represent the logarithm of the ratio of the proportion of good variables to the proportion of bad variables in the interval variable, the proportion of good variables is the proportion of good variables in each interval variable relative to the proportion of good variables in all interval variables, and the proportion of bad variables is the proportion of bad variables in each interval variable relative to the proportion of bad variables in all interval variables; and analyzing the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable.

[0063] Optionally, the preset evidence weighting algorithm is:

[0064]

[0065]

[0066] in, This represents the number of good variables within the interval variable. This represents the number of good variables across all interval variables. The number of bad variables in an interval variable. This represents the number of bad variables in the entire range. For good variable proportions, The proportion of bad variables.

[0067] Optionally, good variables are predictable events, and bad variables are unpredictable events.

[0068] As an optional embodiment, analyzing the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable includes: analyzing the evidence weights of each interval variable using a preset information value evaluation model to determine the information value of each interval variable, and using the information value as the predictive value, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; or analyzing the evidence weights of each interval variable using a preset Gini index model to determine the Gini index of each interval variable, and using the Gini index as the predictive value, wherein the Gini index is used to assess the impurity of the interval variable.

[0069] In the above embodiments of the present invention, a preset information value assessment model can be used to determine the information value of each interval variable, and a preset Gini index model can be used to determine the Gini index of each interval variable. One of the preset information value assessment model and the preset Gini index model is selected to determine the predictive value of the interval variable, thereby realizing the value assessment of the interval variable.

[0070] Optionally, a pre-defined information value assessment model is used to measure the predictive power of each independent variable on the target variable y, and is used to select independent variables.

[0071] Optionally, the preset information value assessment model is:

[0072] .

[0073] Optionally, the preset Gini index model is:

[0074] .

[0075] Optionally, the Gini index represents the probability that a randomly selected sample in the sample set will be misclassified. The smaller the Gini index, the lower the probability that the selected sample in the set will be misclassified, which means that the purity of the set is higher. Conversely, the larger the Gini index, the less pure the set is. When all samples in the set belong to the same class, the Gini index is 0.

[0076] In the above embodiments of the present invention, when using Information Value (IV), Weight of Evidence (WOE), and Gini index as indicators, the following objectives can be achieved in data mining practice:

[0077] 1. Adjust the optimal binning threshold by changing the weight of evidence (WOE). The usual approach is to first divide an interval variable into 10-20 temporary intervals, calculate the WOE value for each interval, and then merge them according to the trend of WOE changes in each interval to achieve a more reasonable interval division.

[0078] 2. Select independent variables with high predictive value through information value (IV) or Gini index and input them into the model training.

[0079] As an optional embodiment, analyzing the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable includes: analyzing the evidence weights of each interval variable using a preset information value evaluation model to determine the information value of each interval variable, where the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; analyzing the evidence weights of each interval variable using a preset Gini index model to determine the Gini index of each interval variable, where the Gini index is used to assess the impurity of the interval variable; determining the product of the information value and a first preset weight to determine a first value; determining the product of the Gini index and a second preset weight to determine a second value; and determining the predictive value based on the sum of the first value and the second value.

[0080] In the above embodiments of the present invention, the information value of each interval variable can be determined by using a preset information value assessment model, and the Gini index of each interval variable can be determined by using a preset Gini index model. The predicted value is determined by combining the preset information value assessment model and the preset Gini index model, thereby realizing the value assessment of the interval variable.

[0081] This invention utilizes AI combined with linear correlation indicators as a data screening scheme. The Pearson correlation coefficient is primarily used to describe the linear relationship between proportional variables, interval variables, and binary variables. The R-squared method, borrowing from multiple linear regression analysis algorithms, is used to identify and select independent variables with significant predictive value for the target variable. Then, the chi-square test is employed to assess the correlation between categorical variables, including ordinal variables and other qualitative variables, and to compare sample rates across two or more variables. Variables to be included in the model are screened using IV, WOE, and Gini coefficients to select independent variables with high predictive value for model training. This provides more accurate analysis and prediction of the potential value in commercial data mining scenarios.

[0082] According to an embodiment of the present invention, a variable screening device embodiment is also provided. It should be noted that the variable screening device can be used to execute the variable screening method in the embodiment of the present invention, and the variable screening method in the embodiment of the present invention can be executed in the variable screening device.

[0083] Figure 2 This is a schematic diagram of a variable screening device according to an embodiment of the present invention, such as... Figure 2 As shown, the device may include: an acquisition module 22, used to acquire independent variables for evaluating the target variable, wherein the target variable corresponds to one or more independent variables; an evaluation module 24, used to evaluate the linear relationship between the independent variables and the target variable using a preset Pearson correlation coefficient model, and determine the linear correlation degree; and a selection module 26, used to select the independent variable with the highest linear correlation degree as the sample variable of the target variable when the target variable corresponds to multiple independent variables with linear correlation degrees higher than a preset correlation degree threshold, wherein the sample variable and the target variable serve as training data for training the target prediction model, and the target prediction model is used to analyze the independent variables to determine the predictor variable.

[0084] It should be noted that the acquisition module 22 in this embodiment can be used to execute step S102 in this application embodiment, the evaluation module 24 in this embodiment can be used to execute step S104 in this application embodiment, and the selection module 26 in this embodiment can be used to execute step S106 in this application embodiment. The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments.

[0085] In this embodiment of the invention, independent variables for evaluating the target variable are obtained, wherein the target variable corresponds to one or more independent variables; a preset Pearson correlation coefficient model is used to evaluate the linear relationship between the independent variables and the target variable to determine the linear correlation degree; when the target variable corresponds to multiple independent variables with linear correlation degrees higher than a preset correlation degree threshold, the independent variable with the highest linear correlation degree is selected as the sample variable of the target variable, wherein the sample variable and the target variable are used as training data for training the target prediction model, and the target prediction model is used to analyze the independent variables to determine the predictor variable, thereby achieving the purpose of screening the independent variables, reducing the amount of training data required to train the target prediction model, and thus achieving the technical effect of improving the training efficiency of the target prediction model, thereby solving the technical problem of low efficiency in determining consumer profiles due to the inability to screen the independent variables for determining consumer profiles.

[0086] As an optional embodiment, the device further includes: an identification module, used to identify the variable type of the independent variable after selecting the sample variable with the highest linear correlation as the target variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable, and binary variable; a first testing module, used to test the correlation between the independent variable and the predictor variable of the target prediction model using a preset chi-square test model when the independent variable belongs to the categorical variable; and a second testing module, used to test the correlation between the independent variable and the predictor variable of the target prediction model using a preset regression model when the independent variable does not belong to the categorical variable.

[0087] As an optional embodiment, the acquisition module includes: an acquisition unit for acquiring the attribute values ​​of independent variables; a determination unit for analyzing the attribute values ​​of independent variables using a preset evaluation algorithm to determine the predictive value of the independent variables, wherein the predictive value is used to represent the degree of conformity between the predictive variable determined based on the independent variables and the target variable corresponding to the independent variables; and a selection unit for selecting independent variables with predictive values ​​higher than a preset value threshold as independent variables used to evaluate the target variable.

[0088] As an optional embodiment, the determining unit includes: an identification unit for identifying the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable, and binary variable; a binning unit for binning the independent variable into multiple interval variables if the independent variable belongs to the interval variable; and an analysis unit for analyzing the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable.

[0089] As an optional embodiment, the analysis unit includes: a first analysis subunit, used to analyze each interval variable using a preset evidence weighting algorithm to determine the evidence weight of each interval variable, wherein the evidence weight is used to represent the logarithm of the ratio of the proportion of good variables to the proportion of bad variables in the interval variable, the proportion of good variables is the proportion of good variables in each interval variable relative to the proportion of good variables in all interval variables, and the proportion of bad variables is the proportion of bad variables in each interval variable relative to the proportion of bad variables in all interval variables; and a second analysis subunit, used to analyze the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable.

[0090] As an optional embodiment, the second analysis subunit includes: a third analysis subunit, used to analyze the evidence weights of each interval variable using a preset information value assessment model, determine the information value of each interval variable, and use the information value as the predicted value, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; or a fourth analysis subunit, used to analyze the evidence weights of each interval variable using a preset Gini index model, determine the Gini index of each interval variable, and use the Gini index as the predicted value, wherein the Gini index is used to assess the impurity of the interval variable.

[0091] As an optional embodiment, the second analysis subunit includes: a fifth analysis subunit, used to analyze the evidence weights of each interval variable using a preset information value assessment model to determine the information value of each interval variable, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; a sixth analysis subunit, used to analyze the evidence weights of each interval variable using a preset Gini index model to determine the Gini index of each interval variable, wherein the Gini index is used to assess the impurity of the interval variable; a first determination subunit, used to determine the product of the information value and the first preset weight to determine the first value; a second determination subunit, used to determine the product of the Gini index and the second preset weight to determine the second value; and a third determination subunit, used to determine the predictive value based on the sum of the first value and the second value.

[0092] Embodiments of the present invention can provide a computer terminal, which can be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the computer terminal can also be replaced by a mobile terminal or other terminal device.

[0093] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0094] In this embodiment, the computer terminal described above can execute the program code for the following steps in the variable screening method: obtaining independent variables used to evaluate the target variable, wherein the target variable corresponds to one or more independent variables;

[0095] The linear relationship between the independent and target variables is evaluated using a pre-defined Pearson correlation coefficient model to determine the degree of linear correlation.

[0096] When the target variable corresponds to multiple independent variables with linear correlations higher than a preset correlation threshold, the independent variable with the highest linear correlation is selected as the sample variable of the target variable. The sample variable and the target variable are used as training data for training the target prediction model, and the target prediction model is used to analyze the independent variables to determine the predictor variables.

[0097] Optionally, Figure 3 This is a structural block diagram of a computer terminal according to an embodiment of the present invention. Figure 3 As shown, the computer terminal 30 may include one or more (only one is shown in the figure) processors 32 and memory 34.

[0098] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the variable filtering method and apparatus in this embodiment of the invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the variable filtering method described above. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal 30 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0099] The processor can access information and applications stored in memory via a transmission device to perform the following steps: obtaining independent variables for evaluating the target variable, wherein the target variable corresponds to one or more independent variables; evaluating the linear relationship between the independent variables and the target variable using a preset Pearson correlation coefficient model to determine the linear correlation degree; and selecting the independent variable with the highest linear correlation degree as a sample variable of the target variable when the target variable corresponds to multiple independent variables with linear correlation degrees higher than a preset correlation degree threshold, wherein the sample variable and the target variable serve as training data for training the target prediction model, and the target prediction model is used to analyze the independent variables to determine the predictor variables.

[0100] Optionally, the processor may also execute program code that performs the following steps: after selecting the independent variable with the highest linear correlation as the sample variable of the target variable, identify the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable and binary variable; if the independent variable is a categorical variable, use a pre-set chi-square test model to test the correlation between the independent variable and the predictor variable of the target prediction model; if the independent variable is not a categorical variable, use a pre-set regression model to test the correlation between the independent variable and the predictor variable of the target prediction model.

[0101] Optionally, the processor may also execute program code that performs the following steps: obtains the attribute values ​​of the independent variables; analyzes the attribute values ​​of the independent variables using a preset evaluation algorithm to determine the predictive value of the independent variables, wherein the predictive value is used to represent the degree of agreement between the predictive variable determined based on the independent variables and the target variable corresponding to the independent variables; and selects independent variables with predictive values ​​higher than a preset value threshold as independent variables used to evaluate the target variable.

[0102] Optionally, the processor may also execute program code that performs the following steps: identifying the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable and binary variable; if the independent variable is an interval variable, binning the independent variable into multiple interval variables; and using a preset evaluation algorithm to analyze the attribute values ​​of each interval variable to determine the predictive value of the interval variable.

[0103] Optionally, the processor may also execute program code for the following steps: analyzing each interval variable using a preset evidence weighting algorithm to determine the evidence weight of each interval variable, wherein the evidence weight is used to represent the logarithm of the ratio of the proportion of good variables to the proportion of bad variables in the interval variable, the proportion of good variables is the proportion of good variables in each interval variable relative to the proportion of good variables in all interval variables, and the proportion of bad variables is the proportion of bad variables in each interval variable relative to the proportion of bad variables in all interval variables; and analyzing the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable.

[0104] Optionally, the processor may also execute program code that performs the following steps: analyzes the evidence weights of each interval variable using a preset information value assessment model, determines the information value of each interval variable, and uses the information value as the predicted value, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; or analyzes the evidence weights of each interval variable using a preset Gini index model, determines the Gini index of each interval variable, and uses the Gini index as the predicted value, wherein the Gini index is used to assess the impurity of the interval variable.

[0105] Optionally, the processor may also execute program code for the following steps: analyzing the evidence weights of each interval variable using a preset information value assessment model to determine the information value of each interval variable, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; analyzing the evidence weights of each interval variable using a preset Gini index model to determine the Gini index of each interval variable, wherein the Gini index is used to assess the impurity of the interval variable; determining the product of the information value and the first preset weight to determine the first value; determining the product of the Gini index and the second preset weight to determine the second value; and determining the predicted value based on the sum of the first value and the second value.

[0106] This invention provides a variable selection scheme. It involves obtaining independent variables for evaluating a target variable, where the target variable corresponds to one or more independent variables; using a pre-set Pearson correlation coefficient model to evaluate the linear relationship between the independent variables and the target variable, determining the linear correlation degree; when the target variable corresponds to multiple independent variables with linear correlation degrees higher than a pre-set correlation threshold, selecting the independent variable with the highest linear correlation degree as a sample variable of the target variable. The sample variable and the target variable serve as training data for training a target prediction model. The target prediction model is used to analyze the independent variables and determine the predictor variable, thus achieving the purpose of selecting independent variables, reducing the amount of training data required to train the target prediction model, and thereby improving the training efficiency of the target prediction model. This solves the technical problem of low efficiency in determining consumer profiles due to the inability to select independent variables for determining consumer profiles.

[0107] Those skilled in the art will understand that Figure 3 The structure shown is for illustrative purposes only. The computer terminal can also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a mobile internet device (MID), a PAD, and other terminal devices. Figure 3 This does not limit the structure of the aforementioned electronic device. For example, computer terminal 30 may also include components that are more advanced than those described above. Figure 3 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 3 The different configurations shown.

[0108] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0109] Embodiments of the present invention also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the variable filtering method provided in the above embodiments.

[0110] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0111] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining independent variables for evaluating the target variable, wherein the target variable corresponds to one or more independent variables; evaluating the linear relationship between the independent variables and the target variable using a preset Pearson correlation coefficient model, and determining the linear correlation degree; when the target variable corresponds to multiple independent variables with linear correlation degrees higher than a preset correlation degree threshold, selecting the independent variable with the highest linear correlation degree as the sample variable of the target variable, wherein the sample variable and the target variable serve as training data for training the target prediction model, and the target prediction model is used to analyze the independent variables to determine the predictor variable.

[0112] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: after selecting the sample variable with the highest linear correlation as the target variable, identifying the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable, and binary variable; if the independent variable is a categorical variable, using a preset chi-square test model to test the correlation between the independent variable and the predictor variable of the target prediction model; if the independent variable is not a categorical variable, using a preset regression model to test the correlation between the independent variable and the predictor variable of the target prediction model.

[0113] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: obtaining the attribute values ​​of the independent variables; analyzing the attribute values ​​of the independent variables using a preset evaluation algorithm to determine the predictive value of the independent variables, wherein the predictive value is used to represent the degree of conformity between the predictive variable determined based on the independent variables and the target variable corresponding to the independent variables; selecting independent variables with predictive values ​​higher than a preset value threshold as independent variables used to evaluate the target variable.

[0114] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: identifying the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable, and binary variable; if the independent variable is an interval variable, binning the independent variable into multiple interval variables; and using a preset evaluation algorithm to analyze the attribute values ​​of each interval variable to determine the predictive value of the interval variable.

[0115] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: analyzing each interval variable using a preset evidence weighting algorithm to determine the evidence weight of each interval variable, wherein the evidence weight is used to represent the logarithm of the ratio of the proportion of good variables to the proportion of bad variables in the interval variable, the proportion of good variables is the proportion of good variables in each interval variable relative to the proportion of good variables in all interval variables, and the proportion of bad variables is the proportion of bad variables in each interval variable relative to the proportion of bad variables in all interval variables; analyzing the attribute values ​​of each interval variable using a preset evaluation algorithm to determine the predictive value of the interval variable.

[0116] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: analyzing the evidence weights of each interval variable using a preset information value assessment model, determining the information value of each interval variable, and using the information value as the predicted value, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; or analyzing the evidence weights of each interval variable using a preset Gini index model, determining the Gini index of each interval variable, and using the Gini index as the predicted value, wherein the Gini index is used to assess the impurity of the interval variable.

[0117] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: analyzing the evidence weights of each interval variable using a preset information value assessment model to determine the information value of each interval variable, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; analyzing the evidence weights of each interval variable using a preset Gini index model to determine the Gini index of each interval variable, wherein the Gini index is used to assess the impurity of the interval variable; determining the product of the information value and a first preset weight to determine a first value; determining the product of the Gini index and a second preset weight to determine a second value; and determining a predicted value based on the sum of the first value and the second value.

[0118] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0119] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0121] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0122] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0123] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0124] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A variable selection method, applied in an e-commerce environment, characterized in that, include: Obtaining independent variables for evaluating a target variable includes: obtaining attribute values ​​of the independent variables; analyzing the attribute values ​​of the independent variables using a preset evaluation algorithm to determine the predictive value of the independent variables, wherein identifying the variable type of the independent variables, wherein the variable type includes at least: proportional variables, interval variables, categorical variables, and binary variables; if the independent variable belongs to the interval variable type, binning the independent variable into multiple interval variables; analyzing the attribute values ​​of each interval variable using the preset evaluation algorithm to determine the predictive value of the interval variable; the predictive value is used to represent the degree of conformity between the predictive variable determined based on the independent variable and the target variable corresponding to the independent variable; selecting independent variables whose predictive value is higher than a preset value threshold as independent variables for evaluating the target variable, wherein the target variable corresponds to one or more independent variables, and the independent variables include at least one of the following: the number of times a user browses a certain product, the browsing duration of a user browsing a certain product, and whether a user purchases related products of a product; The linear relationship between the independent variable and the target variable is evaluated using a pre-defined Pearson correlation coefficient model to determine the degree of linear correlation. When the target variable corresponds to multiple independent variables with linear correlations higher than a preset correlation threshold, the independent variable with the highest linear correlation is selected as a sample variable of the target variable. The sample variable and the target variable are used as training data for training the target prediction model. The target prediction model is used to analyze the independent variables to determine the predictor variable. The predictor variable is used to represent the user's consumption behavior during the e-shopping process.

2. The method according to claim 1, characterized in that, After selecting the independent variable with the highest linear correlation as the sample variable of the target variable, the method further includes: Identify the variable type of the independent variable, wherein the variable type includes at least: proportional variable, interval variable, categorical variable, and binary variable; When the independent variable belongs to the categorical variable, the correlation between the independent variable and the predictor variable in the target prediction model is tested using a pre-defined chi-square test model. If the independent variable does not belong to the categorical variable, a pre-defined regression model is used to test the correlation between the independent variable and the predictor variable in the target prediction model.

3. The method according to claim 1, characterized in that, The preset evaluation algorithm is used to analyze the attribute values ​​of each interval variable to determine the predictive value of the interval variable, including: Each interval variable is analyzed using a preset evidence weighting algorithm to determine the evidence weight of each interval variable. The evidence weight is used to represent the logarithm of the ratio of the proportion of good variables to the proportion of bad variables in the interval variable. The proportion of good variables is the proportion of good variables in each interval variable relative to the proportion of good variables in all interval variables. The proportion of bad variables is the proportion of bad variables in each interval variable relative to the proportion of bad variables in all interval variables. The preset evaluation algorithm is used to analyze the attribute values ​​of each interval variable to determine the predictive value of the interval variable.

4. The method according to claim 3, characterized in that, The preset evaluation algorithm is used to analyze the attribute values ​​of each interval variable to determine the predictive value of the interval variable, including: The evidence weights of each interval variable are analyzed using a pre-defined information value assessment model to determine the information value of each interval variable, and this information value is used as the predicted value. The information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; or The evidence weights of each interval variable are analyzed using a pre-defined Gini index model to determine the Gini index of each interval variable, and the Gini index is used as the predicted value, wherein the Gini index is used to assess the impurity of the interval variable.

5. The method according to claim 3, characterized in that, The preset evaluation algorithm is used to analyze the attribute values ​​of each interval variable to determine the predictive value of the interval variable, including: The evidence weights of each interval variable are analyzed using a pre-defined information value assessment model to determine the information value of each interval variable, wherein the information value represents the ability of the independent variable to distinguish between events and non-events in the target variable; The evidence weights of each interval variable are analyzed using a pre-defined Gini index model to determine the Gini index of each interval variable, wherein the Gini index is used to assess the impurity of the interval variable; Determine the first value by multiplying the information value by a first preset weight; The second value is determined by multiplying the Gini index and the second preset weight. The predicted value is determined based on the sum of the first value and the second value.

6. A variable filtering device, applied in an e-commerce environment, characterized in that, include: An acquisition module is used to acquire independent variables for evaluating a target variable, including: acquiring the attribute values ​​of the independent variables; analyzing the attribute values ​​of the independent variables using a preset evaluation algorithm to determine the predictive value of the independent variables, wherein the variable type of the independent variables is identified, and the variable type includes at least: proportional variables, interval variables, categorical variables, and binary variables; if the independent variable belongs to the interval variable type, binning the independent variable into multiple interval variables; analyzing the attribute values ​​of each interval variable using the preset evaluation algorithm to determine the predictive value of the interval variable; the predictive value is used to represent the degree of conformity between the predictive variable determined based on the independent variable and the target variable corresponding to the independent variable; selecting independent variables whose predictive value is higher than a preset value threshold as independent variables for evaluating the target variable, wherein the target variable corresponds to one or more independent variables, and the independent variables include at least one of the following: the number of times a user browses a certain product, the browsing duration of a user browsing a certain product, and whether a user purchases related products of a product; The evaluation module is used to evaluate the linear relationship between the independent variable and the target variable using a preset Pearson correlation coefficient model, and to determine the degree of linear correlation. The selection module is used to select the independent variable with the highest linear correlation as the sample variable of the target variable when the target variable corresponds to multiple independent variables with linear correlation higher than a preset correlation threshold. The sample variable and the target variable are used as training data for training the target prediction model. The target prediction model is used to analyze the independent variables to determine the predictor variable. The predictor variable is used to represent the user's consumption behavior during the e-shopping process.

7. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores a program, wherein when the program is executed, it controls the device where the non-volatile storage medium is located to execute the variable screening method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, performs the variable filtering method according to any one of claims 1 to 5.

Images