Prediction method for Internet click-through rate prediction based on shadow feature screening

Abstract

A prediction method for Internet click-through rate prediction based on shadow feature screening includes a data preprocessing step, a training set generation step, and the model training and model establishment step; wherein, the training set generation step includes feature engineering processing, shadow feature construction, and feature screening based on feature shadows. Therefore, the present invention retains or deletes each original feature based on the feature screening strategy, whether the mean of the importance of each original feature after M repetitions is greater than the shadow feature, and whether there is a significant difference in the distribution of the original feature and the shadow feature, and uses the retained original feature for the data features of the final training set and validation set of the model; that is, the present invention filters the original features by constructing shadow features, so that the filtered original features significantly reduce the overfitting risk of the tree model, improve the robustness of the algorithm, and have important significance for improving the click-through rate prediction results in actual business.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence in Internet marketing, and more specifically, to a prediction method for Internet click-through rate prediction based on shadow feature screening. Background Art

[0002] Click-Through Rate (CTR) prediction is a critical issue in internet computing advertising; the accuracy of user predictions directly impacts a company's marketing advertising revenue. Click-Through Rate (CTR) prediction is a critical issue in internet computing advertising; the accuracy of user predictions directly impacts a company's marketing advertising revenue.

[0003] The advertising process typically involves three steps: ad exposure, user clicks, and user registration (or conversion). A major goal of CTR estimation is to improve the efficiency from ad exposure to user clicks, that is, to increase click-through rate (CTR). This is because user clicks or non-clicks are generally defined as labels for user behavior.

[0004] Those skilled in the art are aware that traditional CTR estimation can be viewed as a supervised learning problem: that is, after knowing a series of features such as user click behavior, user clicks are recorded as 1 and user non-clicks are recorded as 0, thereby performing binary classification learning.

[0005] In click-through rate estimation, data features can generally be divided into:

[0006] ① User profile features, which mainly include the user's age, gender, income level, device type, etc.;

[0007] ②. Item / commodity characteristics, which mainly include user access DPI, user click frequency, etc.

[0008] ③. Data features can also be interacted and combined by combining specific business needs or through algorithms, including various statistical features.

[0009] In particular, the categorical features of certain products, after processing (such as one-hot vector expansion), often become very high-dimensional, and the composition of the data features is relatively complex. Furthermore, in many businesses, the contribution of data features to the predicted results is often a concern. Therefore, data feature screening is very beneficial in many aspects, such as extracting important features, reducing computational complexity, and mitigating the risk of overfitting.

[0010] Those skilled in the art know that for many tree-based algorithms, such as XGBoost and LightGBM, feature importance can be output to guide feature engineering. However, feature importance cannot be used for feature screening. The main reasons are as follows:

[0011] First of all, since feature importance can only indicate which features play a role in training, it cannot indicate that there is a dependency relationship between the features and the target variable.

[0012] Second, feature importance is generally calculated based on the reduction in node impurity before and after the decision tree split (such as the Gini coefficient). Therefore, it is particularly easy to overestimate the importance of numerical features and categorical features with high cardinality.

[0013] Finally, after outputting the feature importance, it is generally necessary to manually select a threshold to choose to retain or delete certain features. It is difficult to ensure that the filtered features can improve model performance. Summary of the Invention

[0014] The purpose of the present invention is to provide a prediction method for Internet click-through rate prediction based on shadow feature screening, which constructs shadow features, outputs feature importance and performs feature screening after training with original feature information through a tree model to improve user click-through rate prediction results.

[0015] To achieve the above object, the technical solution of the present invention is as follows:

[0016] A prediction method for Internet click-through rate prediction based on shadow feature screening, comprising a data preprocessing step S1, a training set generation step S2, and a model training and model establishment step S3;

[0017] The data preprocessing step S1 includes the following steps:

[0018] Step S11: Obtaining raw data information of N users, and extracting raw feature information from the raw data information of the users; wherein the raw feature information includes user ID, user province, user city, user access DPI, user access time, device type, and / or whether the user clicks; wherein the user province, user city, and user access DPI are categorical features, the user access time is a continuous feature, and N is greater than or equal to 2;

[0019] Step S12: performing abnormality detection and processing on the original feature information of the user;

[0020] Step S13: Process the categorical features through one-hot encoding, perform normalization and gradient descent processing according to business needs and solution model selection, or use the RankGauss method to adjust the sample distribution of continuous features;

[0021] The training set generation step S2 includes a feature engineering processing step S21, a shadow feature construction step S22, and a feature screening step S23 based on feature shadows;

[0022] Step S21 specifically includes the following steps:

[0023] Step S211: User access DPI processing step, creating a feature column of user ID and user access DPI;

[0024] Step S212: Processing the category features by one-hot encoding;

[0025] Step S213: Perform numerical feature transformation on continuous features;

[0026] Step S214: performing feature screening and feature construction to form new features;

[0027] Step S215: performing continuous feature discretization processing;

[0028] Step S22: Constructing the shadow feature specifically includes the following steps:

[0029] Construct corresponding shadow features for the continuous features and the discretized continuous features and the category features; splice the original features of the training data set and the obtained shadow features to obtain a mixed feature that is twice the original feature M of the training data;

[0030] Step S23 specifically includes the following steps:

[0031] Step S231: Calculate the distribution of M feature importances for each original feature and its corresponding shadow feature; retain or delete each original feature based on the feature screening strategy, whether the mean importance of each original feature after M repetitions is greater than that of the shadow feature, and whether there is a significant difference in the distribution of the original feature and the shadow feature;

[0032] Step S232: using the retained original features as features of the final training set and validation set data of the model; that is, using the retained original features as input to the neural network model based on the tree algorithm;

[0033] The model training and model building step S3 includes:

[0034] Step S31: Establishing a neural network initialization model based on a tree algorithm, using the original features of the retained original data, using the processed original data in the training set, and using whether or not it is clicked as a training label to obtain the output probability of whether each training sample is clicked;

[0035] Step S32: train and verify the neural network model based on the tree algorithm for each sample in the new training set to obtain the neural network model after parameter optimization, and use the verification set for verification to obtain the final prediction network model based on the tree algorithm.

[0036] Furthermore, the step S31 establishes an initialization model based on a tree algorithm as a gradient boosting decision tree model, and its implementation includes XGBoost or lightGBM.

[0037] Furthermore, the feature screening strategy includes:

[0038] ① The original features whose feature importance mean is less than that of their corresponding shadow features are directly deleted from the dataset, regardless of whether there is a significant difference between the two.

[0039] ② For original features whose feature importance mean is greater than that of their corresponding shadow features, the Mann-Whitney U test is used to determine whether the distributions of the two are significantly different; if the importance mean of the selected original feature is significantly greater than that of the corresponding shadow feature, it is retained; otherwise, the original feature and its corresponding shadow feature are directly deleted from the dataset.

[0040] Furthermore, the prediction method further includes a prediction step S4, which specifically includes:

[0041] Step S41: Obtaining a user group for which click prediction for a marketing campaign is to be made and user raw data information of the user group, and extracting raw feature information from the user raw data information; wherein the raw feature information includes user ID, user province, user city, user access DPI, user access time, device type, and / or whether the user clicks; wherein the user province, user city, and user access DPI are categorical features, the user access time is a continuous feature, and N is greater than or equal to 2;

[0042] Step S42: Perform anomaly detection and processing on the raw data information of each user; process the categorical features through one-hot encoding, perform normalization and gradient descent processing according to business needs and solution model selection, or use the RankGauss method to adjust the sample distribution of continuous features;

[0043] Step S43: Feature screening and feature construction are performed on the original data information of the user to form new features; continuous features are discretized according to multiple thresholds; for each feature, corresponding shadow features are constructed by random rearrangement, and each original feature is retained or deleted based on the feature screening strategy, whether the mean importance of each original feature after M repetitions is greater than that of the shadow feature, and whether there is a significant difference in the distribution of the original feature and the shadow feature;

[0044] Step S44: Provide a trained gradient boosting decision tree model, input the original data information of the users with the retained original features in sequence, and obtain the predicted values of all the users in the user group for click prediction of the marketing activity; wherein the model predicted value of the user is the click intention of the user.

[0045] Furthermore, the model prediction step S4 further includes:

[0046] Step S45: According to actual delivery needs, select all or part of the users whose model predicted values have a concentrated click intention of 1 to perform precision marketing tasks.

[0047] It can be seen from the above technical solutions that the prediction method for Internet click-through rate prediction based on shadow feature screening of the present invention develops a more reliable feature screening algorithm. It filters the original features by constructing shadow features, so that the filtered original features significantly reduce the overfitting risk of the tree model, improve the robustness of the algorithm, and are of great significance in improving the click-through rate prediction results in actual business. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 The figure shows a flow chart of a prediction method for Internet click-through rate prediction based on shadow feature screening in an embodiment of the present invention.

[0049] Figure 2 Shown is a schematic diagram of the construction method of the shadow feature in an embodiment of the present invention

[0050] Figure 3 Schematic diagram showing the distribution of the N feature importances of the original feature A and the M feature importances of the shadow feature A in an embodiment of the present invention

[0051] Figure 4 Schematic diagram showing the probability distribution before feature screening and after feature screening in an embodiment of the present invention DETAILED DESCRIPTION

[0052] The following is combined with Figure 1-4 , the specific implementation methods of the present invention are further described in detail.

[0053] In the following specific embodiments, when describing the embodiments of the present invention in detail, in order to clearly represent the structure of the present invention for the convenience of explanation, the structures in the accompanying drawings are not drawn according to general proportions, and are partially enlarged, deformed and simplified. Therefore, it should be avoided to understand this as a limitation of the present invention.

[0054] See also Figure 1 , Figure 1The figure shows a flow chart of a prediction method for Internet click-through rate prediction based on shadow feature screening according to the present invention. Figure 1 As shown, the prediction method includes a data preprocessing step S1, a training set generation step S2, a prediction model establishment step S3 and a marketing activity prediction step S4.

[0055] In the embodiment of the present invention, the data preprocessing step S1 is very important, and the data preprocessing step S1 includes the following steps:

[0056] Step S11: Obtain the original data information of N users, and extract original feature information from the original data information of the users; wherein the original feature information includes user ID (user_id), user's province (province), user's city (city), user's access DPI, user's access time, device type (phone_type) and / or whether the user clicks on the feature; wherein the user's province, user's city, and user's access DPI are categorical features, the user's access time is a continuous feature, and N is greater than or equal to 2.

[0057] Please refer to Table 1 below. Table 1 is a table description of the raw data information before preprocessing. The format of the raw data information before preprocessing is shown in Table 1 below:

[0058] Table 1

[0059]

[0060] Please note that at this time, for the label of the data, that is, whether the user clicks, only positive samples (1) and unlabeled samples (0) are distinguished.

[0061] In an embodiment of the present invention, the above-mentioned raw data information also needs to go through steps such as anomaly detection and processing, category feature processing, continuous feature processing and dimensionality reduction processing.

[0062] Step S12: performing abnormality detection and processing on the original data information of the user.

[0063] Anomaly detection and processing: In the process of integrating business requirements, missing values and oversized values in the raw data need to be deleted or filled. During the raw data collection process, due to the average user volume in the millions, missing values may occur. If the missing value is small, it can generally be directly eliminated. If it is difficult to determine whether the missing raw data will affect the final model training results, the missing values can generally be filled by taking the mean, mode, and median.

[0064] In addition, during the process of collecting raw data information, you may encounter problems with excessively large values. For example, a user visits the DPI tens of thousands of times in a day. This situation is generally not particularly helpful in improving the generalization ability of the model during the actual modeling process. Therefore, you can also use elimination or filling methods to perform corresponding processing.

[0065] Step S13: Process the categorical features through one-hot encoding, perform normalization and gradient descent processing according to business needs and solution model selection, or use the RankGauss method to adjust the sample distribution of continuous features.

[0066] As can be seen from the above table, compared with traditional user access data, each user ID corresponds to multiple DPI access records, and the access time is also given.

[0067] This embodiment of the present invention also includes processing user-accessed DPIs. Because each user accesses a unique DPI, the same user ID may appear multiple times in the original data, indicating that the user has multiple access records. Specifically, after deduplicating all DPIs, a feature column for user-accessed DPIs is created. If a user accesses several DPIs, the statistical count for those DPIs is incremented by one.

[0068] Step S2: The step of generating a training set, which includes a feature engineering processing step S21, a shadow feature construction step S22, and a feature screening step S23 based on feature shadows.

[0069] Step S21: Feature engineering processing step, which performs feature screening and feature construction on the user's original data information to form new features; it specifically includes the following steps:

[0070] Step S211: User access DPI processing step, creating a feature column of user ID and user access DPI. Because each user's accessed DPI is not unique, the same user ID may appear multiple times in the original data information, meaning that the user has multiple access records. Therefore, after deduplicating all DPIs, a feature column of user DPI is created. If a user accesses several DPIs, the statistical count of these DPIs is incremented by 1.

[0071] Step S212: One-hot encoding is used to process categorical features. Categorical features such as the user's province, city, and device type can be expanded using one-hot encoding. For example, the provinces are (HN, JS, LN, SH, JL, BJ). After one-hot encoding, the province of a particular user is 1, while the others are 0. For example, user ID 199990, whose province is HN, has a one-hot encoding vector of (1, 0, 0, 0, 0, 0). Correspondingly, the location vector for user 199991 is (0, 1, 0, 0, 0, 0).

[0072] Step S213: Perform numerical feature transformation processing on continuous features.

[0073] This step, called numerical feature transformation, can be tailored to business needs and model selection. For example, for models using gradient descent, such as logistic regression and neural networks, normalization generally results in smoother gradient descent and faster convergence. For another example, some machine learning models assume that features follow a Gaussian distribution. In this case, methods like GaussRank can be used to adjust the sample distribution.

[0074] Step S214: Perform feature screening and feature construction to form new features.

[0075] Feature screening: Based on the embedding method, a random forest-based learner is established to train the preprocessed data and output the importance of each feature. Several features with lower weights are eliminated, and then the next round of training is carried out based on the new feature set. The recursive elimination of features with lower importance is repeated M times. Preferably, M is 3.

[0076] Feature construction: Add common statistical features such as skewness, peakedness, etc.

[0077] Step S215: Discretize continuous features.

[0078] For some machine learning algorithms, such as logistic regression and feedforward neural networks, discretizing continuous features based on multiple thresholds can effectively increase the convergence speed of algorithm optimization, prevent the impact of outliers on model training, and provide a certain degree of regularization. For example, based on the different quantiles of the sample distribution on a continuous feature (assuming the quantiles are 0.25, 0.5, and 0.75), the samples can be divided into five intervals on that continuous feature. Each sample may be distributed in one of the five intervals, and the sample will take the value of 1 in the feature interval it falls in and the value of 0 in the remaining four intervals. The effect of discretizing a continuous feature is to expand it into a one-hot encoding.

[0079] After preprocessing and feature engineering, the data is available as shown in Table 2 below:

[0080]

[0081] Continuous features include statistical features such as DPI access frequency, kurtosis, and skewness. Categorical features include the one-hot encoding of DPI access, the one-hot encoding of the number's location, and the discretized DPI access frequency.

[0082] Step S22: shadow feature construction step;

[0083] In an embodiment of the present invention, a shadow feature is a new feature obtained by disrupting the original features in the training data set. It is worth noting that the characteristic of the shadow feature is that it theoretically has no statistical correlation with the training label. Therefore, when using the shadow feature to establish a statistical model, ideally the shadow feature has no contribution to the statistical model.

[0084] See also Figure 2 , Figure 2 FIG. 1 is a schematic diagram showing the construction method of the shadow feature in an embodiment of the present invention. Figure 2 As shown, in the embodiment of the present invention, corresponding shadow features can be constructed for continuous features and discretized continuous features and category features.

[0085] Assuming that the training data in the training dataset has M features, each feature is randomly rearranged to obtain the corresponding shadow feature, so the number of shadow features is M. The original training data features of the training dataset and the obtained shadow features are then concatenated to obtain 2M features, which are then trained together in a decision tree-based model (such as gradient boosted decision trees or random forests).

[0086] The decision tree-based model can output the corresponding feature importance (such as average split information gain or average number of splits) for 2M features. Since the construction of shadow features and algorithms such as random forests have a certain degree of randomness, the above process of obtaining feature importance is performed N times.

[0087] After repeated execution, each feature has N different feature importances, and the importance of all features forms a feature importance matrix as shown in Table 2 below:

[0088]

[0089] Step S23: Feature screening based on feature shadows.

[0090] Step S231: Form a distribution of M feature importances for each original feature and its corresponding shadow feature; retain or delete each original feature based on the feature screening strategy, whether the mean importance of each original feature after M repetitions is greater than that of the shadow feature, and whether there is a significant difference in the distribution of the original feature and the shadow feature.

[0091] In the embodiment of the present invention, the feature screening is based on the 2N feature importances obtained after N repeated modeling of the original feature A and its corresponding shadow feature A; the N feature importances of the original feature A and the N feature importances of the shadow feature A can form two distributions.

[0092] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram showing the distribution of the importance of the N features of the original feature A and the importance of the M features of the shadow feature A in an embodiment of the present invention. Figure 3 As shown in Figure 2, according to its distribution, it can be divided into four cases: Figure (A), Figure (B), Figure (C) and Figure (D).

[0093] Among them, Figure (A) and Figure (B) show that the mean importance of the original feature after M repetitions is greater than that of the shadow feature, while Figure (C) and Figure (D) show the opposite situation; the difference between Figure (A) and Figure (B) is that the distribution of the original feature and the shadow feature in Figure (A) is significantly different, while the distribution of the two in Figure (B) is not significantly different; the difference between Figure (C) and Figure (D) is also whether the difference is significant.

[0094] Therefore, for the above four situations in the embodiments of the present invention, the following feature screening strategies can be set:

[0095] ① The original features whose feature importance mean is less than that of their corresponding shadow features are directly deleted from the dataset, regardless of whether there is a significant difference between the two.

[0096] ② For original features whose mean importance is greater than that of their corresponding shadow features, use the Mann-Whitney U test to determine whether the two distributions are significantly different. This significance test is chosen because feature importance distributions do not necessarily follow a normal distribution, and the importance distributions of the original and shadow features do not necessarily follow the homoscedasticity assumption. This step selects features whose mean importance is significantly greater than that of the corresponding shadow feature.

[0097] Step S232: The retained original features are used as features of the final training set and validation set data of the model; that is, the retained original features are used as input of the neural network model based on the tree algorithm.

[0098] After the above data processing steps, the newly generated data is divided into a training set and a validation set, and the model training and model building steps S3 are performed:

[0099] The model building steps of the present invention are:

[0100] Step S31: Establish a neural network initialization model based on a tree algorithm, use the original features of the retained original data, use the processed original data in the training set, use whether it is clicked as a training label, and obtain the output probability of whether each training sample is clicked.

[0101] Specifically, all remaining category features of the user modeling data are used, with whether or not a click is used as a training label, to train and generate a neural network model based on a tree algorithm. Preferably, the neural network model based on the tree algorithm is a gradient boosting decision tree model (implemented in a manner such as XGBoost or lightGBM, without special requirements).

[0102] Step S32: train and verify the neural network model based on the tree algorithm for each sample in the new training set to obtain the neural network model after parameter optimization, and use the verification set for verification to obtain the final prediction network model based on the tree algorithm.

[0103] In an embodiment of the present invention, after having the above-mentioned gradient boosting decision tree model, the user data to be predicted is subjected to the same feature engineering processing and shadow feature construction and processing according to the above-mentioned method.

[0104] Specifically, after all the above processing steps are completed, the above features can be introduced into the trained user prediction model to screen out users with high willingness before advertising is delivered, and marketing ads can be accurately delivered to these users. The prediction step S4 specifically includes:

[0105] Step S41: Obtaining a user group for which click prediction for a marketing campaign is to be made and user raw data information of the user group, and extracting raw feature information from the user raw data information; wherein the raw feature information includes user ID, user province, user city, user access DPI, user access time, device type, and / or whether the user clicks; wherein the user province, user city, and user access DPI are categorical features, the user access time is a continuous feature, and N is greater than or equal to 2;

[0106] Step S42: Perform anomaly detection and processing on the raw data information of each user; process the categorical features through one-hot encoding, perform normalization and gradient descent processing according to business needs and solution model selection, or use the RankGauss method to adjust the sample distribution of continuous features;

[0107] Step S43: Feature screening and feature construction are performed on the original data information of the user to form new features; continuous features are discretized according to multiple thresholds; for each feature, corresponding shadow features are constructed by random rearrangement, and each original feature is retained or deleted based on the feature screening strategy, whether the mean importance of each original feature after M repetitions is greater than that of the shadow feature, and whether there is a significant difference in the distribution of the original feature and the shadow feature;

[0108] Step S44: Provide a trained gradient boosting decision tree model, input the original data information of the users with the retained original features in sequence, and obtain the predicted values of all the users in the user group for click prediction of the marketing activity; wherein the model predicted value of the user is the click intention of the user.

[0109] Step S45: According to actual delivery needs, select all or part of the users whose model predicted values have a concentrated click intention of 1 to perform precision marketing tasks.

[0110] See also Figure 4 , Figure 4 The diagram shows the probability distribution before feature selection and after feature selection in the embodiment of the present invention. Figure 4As shown in the figure below, the left figure shows the probability distribution before feature screening, and the right figure shows the probability distribution after feature screening. Comparing the two, it can be seen that there is a significant deviation between the distribution of the validation set before feature screening and the distribution of the training set. After the above feature screening, the predicted probability distributions of the validation set and the training set are more consistent. From the calculated evaluation metric AUC, the AUC of the training set and validation set before feature screening were: 0.7653 and 0.6657 respectively; after feature screening, the AUC of the two were: 0.6899 and 0.6568 respectively. These results show that after feature screening by this method, the risk of overfitting is greatly reduced, while the predictive performance on the validation set is almost unaffected.

[0111] The results show that through the above-mentioned feature screening scheme of the present invention, relevant experiments were carried out on certain actual click-through rate prediction data, and the original features were effectively screened by constructing shadow features; the screened features significantly reduced the overfitting risk of the tree model and improved the results of user click-through rate prediction.

[0112] The above descriptions are merely preferred embodiments of the present invention, and the embodiments are not intended to limit the scope of patent protection of the present invention. Therefore, any equivalent structural changes made using the description and drawings of the present invention should also be included in the scope of protection of the present invention.

Claims

1. A prediction method for Internet click-through rate prediction based on shadow feature screening, characterized in that: It includes a data preprocessing step S1, a training set generation step S2, and a model training and model establishment step S3; The data preprocessing step S1 includes the following steps: Step S11: Obtaining raw data information of N users, and extracting raw feature information from the raw data information of the users; wherein the raw feature information includes user ID, user province, user city, user access DPI, user access time, device type, and / or whether the user clicks; wherein the user province, user city, and user access DPI are categorical features, the user access time is a continuous feature, and N is greater than or equal to 2; Step S12: performing abnormality detection and processing on the original feature information of the user; Step S13: Process the categorical features through one-hot encoding, perform normalization and gradient descent processing according to business needs and solution model selection, or use the RankGauss method to adjust the sample distribution of continuous features; The training set generation step S2 includes a feature engineering processing step S21, a shadow feature construction step S22, and a feature screening step S23 based on feature shadows; Step S21 specifically includes the following steps: Step S211: User access DPI processing step, creating a feature column of user ID and user access DPI; Step S212: Processing the category features by one-hot encoding; Step S213: Perform numerical feature transformation on continuous features; Step S214: performing feature screening and feature construction to form new features; Step S215: performing continuous feature discretization processing; Step S22: Constructing the shadow feature specifically includes the following steps: Construct corresponding shadow features for the continuous features and the discretized continuous features and the category features; splice the original features of the training data set and the obtained shadow features to obtain a mixed feature that is twice the original feature M of the training data; Step S23 specifically includes the following steps: Step S231: Calculate the distribution of M feature importances for each original feature and its corresponding shadow feature; retain or delete each original feature based on the feature screening strategy, whether the mean importance of each original feature after M repetitions is greater than that of the shadow feature, and whether there is a significant difference in the distribution of the original feature and the shadow feature; Step S232: using the retained original features as features of the final training set and validation set data of the model; that is, using the retained original features as input to the neural network model based on the tree algorithm; The model training and model building step S3 includes: Step S31: Establishing a neural network initialization model based on a tree algorithm, using the original features of the retained original data, using the processed original data in the training set, and using whether or not it is clicked as a training label to obtain the output probability of whether each training sample is clicked; Step S32: train and verify the neural network model based on the tree algorithm for each sample in the new training set to obtain the neural network model after parameter optimization, and use the verification set for verification to obtain the final prediction network model based on the tree algorithm.

2. The prediction method according to claim 1, characterized in that The step S31 establishes an initialization model based on a tree algorithm as a gradient boosting decision tree model, and its implementation method includes XGBoost or lightGBM.

3. The prediction method according to claim 1, wherein: The feature screening strategy includes: ① The original features whose feature importance mean is less than that of their corresponding shadow features are directly deleted from the dataset, regardless of whether there is a significant difference between the two. ② For original features whose feature importance mean is greater than that of their corresponding shadow features, the Mann-Whitney U test is used to determine whether the distributions of the two are significantly different; if the importance mean of the selected original feature is significantly greater than that of the corresponding shadow feature, it is retained; otherwise, the original feature and its corresponding shadow feature are directly deleted from the dataset.

4. The prediction method according to claim 1, wherein: The method also includes a prediction step S4, which specifically includes: Step S41: Obtaining a user group for which click prediction for a marketing campaign is to be made and user raw data information of the user group, and extracting raw feature information from the user raw data information; wherein the raw feature information includes user ID, user province, user city, user access DPI, user access time, device type, and / or whether the user clicks; wherein the user province, user city, and user access DPI are categorical features, the user access time is a continuous feature, and N is greater than or equal to 2; Step S42: Perform anomaly detection and processing on the raw data information of each user; process the categorical features through one-hot encoding, perform normalization and gradient descent processing according to business needs and solution model selection, or use the RankGauss method to adjust the sample distribution of continuous features; Step S43: Feature screening and feature construction are performed on the original data information of the user to form new features; continuous features are discretized according to multiple thresholds; for each feature, corresponding shadow features are constructed by random rearrangement, and each original feature is retained or deleted based on the feature screening strategy, whether the mean importance of each original feature after M repetitions is greater than that of the shadow feature, and whether there is a significant difference in the distribution of the original feature and the shadow feature; Step S44: Provide a trained gradient boosting decision tree model, input the original data information of the users with the retained original features in sequence, and obtain the predicted values of all the users in the user group for click prediction of the marketing activity; wherein the model predicted value of the user is the click intention of the user.

5. The prediction method according to claim 4, characterized in that The model prediction step S4 further includes: Step S45: According to actual delivery needs, select all or part of the users whose model predicted values have a concentrated click intention of 1 to perform precision marketing tasks.

Images