User classification method and device, cascaded user classification model and equipment

Abstract

The present disclosure provides a user classification method, a cascaded user classification model and a device, and relates to the field of artificial intelligence technology. The method includes: training a first prediction model through first user data that has not been marketed historically, and training a second prediction model through second user data that has been marketed historically; determining combined data based on the first user data and the second user data; inputting the combined data into the trained first prediction model and the second prediction model to obtain the first order probability and the second order probability respectively; determining the interaction variable based on the first order probability and the second order probability, and determining the sample data of the third prediction model based on the interaction variable and the combined data; predicting the user data of the target user based on the trained first prediction model, the second prediction model and the third prediction model to obtain the prediction results, including the type that does not order without marketing and orders after marketing. This solution can effectively improve the marketing conversion rate.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a user classification method, a user classification device, a cascaded user classification model, and an electronic device for implementing the user classification method. Background Art

[0002] Related marketing activities may be carried out for all users that can be obtained. For example, the operator sends SMS to promote products or services to all users whose contact information can be obtained. In some other marketing activities, a single / single-level machine learning model is trained with user data to predict the probability of a customer making an order after contact. Then, marketing activities are carried out for users with a higher predicted probability of ordering.

[0003] However, the marketing conversion rate in the solutions provided by related technologies needs to be improved.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a user classification method, a user classification device, a cascaded user classification model, an electronic device, and a computer-readable storage medium, thereby improving the marketing conversion rate to a certain extent.

[0006] According to one aspect of the present disclosure, a user classification method is provided, the method comprising: training a first prediction model through first user data that has not been historically marketed, and training a second prediction model through second user data that has been historically marketed; determining combined data based on the first user data and the second user data; inputting the combined data into the trained first prediction model to obtain a first order probability, and inputting the combined data into the trained second prediction model to obtain a second order probability; determining an interaction variable based on the first order probability and the second order probability, and determining sample data of a third prediction model based on the interaction variable and the combined data; predicting the user data of the target user based on the trained first prediction model, the second prediction model and the third prediction model to obtain a prediction result, the prediction result including: types that do not order without marketing and order after marketing.

[0007] According to one aspect of the present disclosure, a cascaded user classification model is provided, which includes: a first prediction model, a second prediction model, and a third prediction model cascaded with the first prediction model and the second prediction model respectively; wherein the first prediction model is trained by first user data that has not been historically marketed, the second prediction model is trained by second user data that has been historically marketed, and sample data of the third prediction model is determined based on interactive variables and combination data; wherein the number of combinations is determined based on the first user data and the second user data, and the combination data is input into the trained first prediction model to obtain a first order probability, the combination data is input into the trained second prediction model to obtain a second order probability, and the interactive variable is determined based on the first order probability and the second order probability.

[0008] According to one aspect of the present disclosure, a user classification device is provided, which includes: a sub-model training module, configured to: train a first prediction model through first user data that has not been historically marketed, and train a second prediction model through second user data that has been historically marketed; a combination module, configured to: determine combination data based on the above-mentioned first user data and the above-mentioned second user data; a sub-model prediction module, configured to: input the above-mentioned combination data into the trained first prediction model to obtain a first order probability, and input the above-mentioned combination data into the trained second prediction model to obtain a second order probability; a sample determination module, configured to: determine an interaction variable based on the above-mentioned first order probability and the above-mentioned second order probability, and determine sample data of a third prediction model based on the above-mentioned interaction variable and the above-mentioned combination data; a classification module, configured to: predict the user data of the target user based on the trained first prediction model, the second prediction model and the third prediction model to obtain a prediction result, and the above-mentioned prediction result includes: types that do not order without marketing and order after marketing.

[0009] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the user classification method described in any of the above embodiments is implemented.

[0010] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the user classification method described in any of the above embodiments by executing the executable instructions.

[0011] The exemplary embodiments of the present disclosure may have some or all of the following beneficial effects:

[0012] In the scheme provided by an example implementation of the present disclosure, on the one hand, user data that has not been marketed historically is distinguished from user data that has been marketed historically, and is used to train order probability prediction models respectively. Furthermore, the interaction variables are determined based on the outputs of the above two models, and the sample data for training the third prediction model cascaded with the above two models is determined based on the interaction variables, so that the interactive influence between the data can be fully constructed, and the classification effect of the data can be enhanced to a certain extent. Then the output of the third prediction model can finely distinguish the types that will not be ordered without marketing and will be ordered after marketing, and then implement marketing for this type, so as to effectively improve the marketing conversion rate, and the precision marketing method can effectively save marketing resources.

[0013] On the other hand, the cascade machine learning model achieves multi-level predictions and incorporates the impact of marketing on user ordering behavior, with high prediction accuracy. At the same time, the model architecture meets complex business function requirements, can improve the learner's robustness to outliers and abnormal values, and has strong generalization capabilities.

[0014] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0016] Figure 1 A schematic diagram of user classification according to an embodiment of the present disclosure is shown.

[0017] Figure 2 The flowchart of the user classification method according to an embodiment of the present disclosure is schematically shown.

[0018] Figure 3 A diagram schematically shows a cascaded user classification model and its training process according to an embodiment of the present disclosure.

[0019] Figure 4 The flowchart schematically shows a data processing method according to an embodiment of the present disclosure.

[0020] Figure 5 The flowchart of the method for determining interaction variables according to an embodiment of the present disclosure is schematically shown.

[0021] Figure 6The flowchart of the method for determining sample data of the third prediction model according to an embodiment of the present disclosure is schematically shown.

[0022] Figure 7 The flowchart schematically shows a method for classifying target users based on a cascaded user classification model according to an embodiment of the present disclosure.

[0023] Figure 8 A schematic diagram of the structure of a user classification device to which an embodiment of the present disclosure can be applied is shown.

[0024] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0025] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more comprehensive and complete and will fully convey the concept of the example embodiments to those skilled in the art.

[0026] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0027] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0028] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0029] The user classification scheme provided by this technical solution can be used in the promotion process of marketing objects, where the marketing objects include goods, services, etc. For example, operators market their business projects (such as 4G to 5G, adding secondary cards, churn prediction, etc.). Specifically, different types of users can be obtained through this technical solution, and different marketing strategies can be adopted for different types of users. Through targeted marketing strategies, the probability of marketing success can be effectively improved.

[0030] In the existing related technologies for product marketing based on machine learning, in order to adapt to different business scenarios and factors such as channel sales products, different model algorithms in the field of machine learning are generally used, such as logistic regression, decision tree, neural network, etc. It should be noted that the related technologies generally use a single-level / single model to achieve user classification prediction, and its prediction accuracy is generally low, and the generalization ability needs to be enhanced.

[0031] The most important evaluation indicator for evaluating marketing effectiveness in related technologies is lift, and the lift is as follows:

[0032]

[0033]

[0034]

[0035] Among them, SE represents the success rate of planned marketing, S represents the number of customers with successful planned marketing, E represents the total number of customers with planning, NI represents the success rate of unplanned marketing, N represents the number of customers with successful unplanned marketing, I represents the total number of customers, IE represents the number of unplanned customers, and LIFT represents the degree of lift.

[0036] It can be seen that the current lift is a comparison of whether or not modeling planning is used in marketing activities. This is only the lift of the model effect. However, it should be noted that the lift of the above model cannot distinguish which customers are those who will actively order, that is, customers who will definitely buy, and which customers will only order after being persuaded by marketing, that is, customers who can be persuaded. In other words, the relevant technology cannot obtain customers with higher marketing value, that is, customers who can effectively improve the marketing conversion rate. In addition, there are types of customers who may order if they are not marketed, but will not order once they are marketed (such as by phone) (referred to as "do not disturb type", such as Figure 1 12 in ), and there are also types that will not order regardless of marketing (recorded as "absolutely not buy" type, such as Figure 1 11). For users corresponding to the above four marketing response types, a more refined classification of customers' real marketing responses can be obtained. The real response classification and targeted marketing strategies will be introduced in the following embodiments.

[0037] In view of the problems existing in the above solutions, this technical solution further subdivides user types by adding prediction dimensions. Figure 1 As shown, the prediction of whether the user is ordered by the historical marketing probability is predicted, where P marketing represents the probability of a user ordering after historical marketing, P unmarketing Indicates the probability of a user ordering without historical marketing. The following categories can be determined: if the probability of ordering without historical marketing is high and the probability of ordering with historical marketing is low, it is determined as the "do not disturb type" 12; if the probability of ordering without historical marketing is high and the probability of ordering with historical marketing is also high, it is determined as the "definitely will buy type" 14; if the probability of ordering without historical marketing is low and the probability of ordering with historical marketing is high, it is determined as the "persuadable type" 13; if the probability of ordering without historical marketing is low and the probability of ordering with historical marketing is also low, it is determined as the "definitely will not buy type" 11.

[0038] Exemplarily, the operator uses the cascaded user classification model provided by this technical solution based on the existing big data, and then finely classifies the users through the trained model. Then, different marketing strategies are launched for different types of users. For example, for users who do not order without marketing (that is, they will buy after being persuaded by marketing), targeted outbound calls, text messages, WeChat and other touch points are used to match target users, marketing channels and sales products, carry out marketing activities, and improve the marketing conversion rate. The precision marketing method can effectively save marketing resources.

[0039] Figure 2 The flowchart of the user classification method according to an embodiment of the present disclosure is schematically shown. Figure 2 , the embodiment shown in the figure includes:

[0040] Step S211, training a first prediction model using first user data that has not been marketed historically, and step S212, training a second prediction model using second user data that has been marketed historically;

[0041] Step S220, determining combined data according to the first user data and the second user data;

[0042] Step S231, inputting the combined data into the trained first prediction model to obtain a first order probability, and step S232, inputting the combined data into the trained second prediction model to obtain a second order probability;

[0043] Step S240, determining an interaction variable according to the first ordering probability and the second ordering probability, and determining sample data of a third prediction model according to the interaction variable and the combined data;

[0044] Step S250, predicting the user data of the target user based on the trained first prediction model, the second prediction model and the third prediction model to obtain prediction results, wherein the prediction results include: types that would not be ordered without marketing and would be ordered after marketing.

[0045] Figure 2 The provided solution can increase the impact of marketing on ordering behavior based on comparing the difference in probability responses of the two models, thereby further subdividing user types by adding prediction dimensions. Furthermore, precision marketing can be carried out for the classified users, which can effectively improve the marketing accuracy. At the same time, the precision marketing method can effectively save marketing resources. Compared with the related technologies that do not distinguish whether users are marketed or only use the user data of historical marketing, this technical solution can increase the impact of marketing on ordering behavior based on comparing the difference in probability responses of the two models, and effectively improve the marketing accuracy.

[0046] In addition, the cascaded machine learning model architecture meets complex business function requirements and can improve the robustness against outliers and abnormal values, thereby achieving stronger generalization capabilities than single or single-stage learning models.

[0047] In an exemplary embodiment, since the technical solution is implemented based on a cascade user classification model, Figure 3 A diagram schematically shows a cascaded user classification model and its training process according to an embodiment of the present disclosure.

[0048] The following combination Figure 3 right Figure 2 The specific implementation methods of each step of the illustrated embodiment are described as follows:

[0049] refer to Figure 3 The embodiment shown in the figure provides a cascaded user classification model training process including the following steps: a data collection stage 31, a data processing stage 32, a first prediction model determination stage 33, a second prediction model determination stage 34, a third prediction model sample determination stage 35, and a third prediction model determination stage 36.

[0050] In an exemplary embodiment, before executing step S211 / step S212, data for training the first prediction model or the second prediction model is first collected. The above historical marketing refers to marketing activities that have been carried out on the user, such as telephone marketing, SMS marketing, etc.

[0051] Specifically, Figure 3The implementation method of the data collection stage 31 can refer to step S410: extracting user portrait data that has not been historically marketed from the database in the form of a structured query language (SQL) script. Each user portrait data contains a user identifier, and a set of user portrait data corresponding to each user identifier is obtained.

[0052] For example, the data of 20,323 users who meet the conditions for adding a secondary card for outbound marketing are obtained from the data warehouse of the customer operation service center of xx Province in April 2019. The relevant fields in each group of user portrait data and the contents of each field are as follows:

[0053] User characteristics: basic natural characteristics, star rating, points, and online age;

[0054] Product attributes: subscription package, broadband under your name, secondary cards, number of secondary cards, etc.;

[0055] Behavioral characteristics: usage in the past three months, fees in the past three months, traffic overflow in the past three months, primary and secondary cards, whether the card is active, etc.;

[0056] Package attributes: package level, whether there is a contract, whether there is a subsidy, subsidy type, etc.;

[0057] Terminal characteristics: terminal type, whether it is 4G, whether it is 5G, brand, grade, replacement time, etc.;

[0058] Derived indicator characteristics: voice saturation, traffic saturation, etc.

[0059] In an exemplary embodiment, the implementation of the data processing stage 32 is as shown in step S420. After the above user portrait data is loaded, one or more of the following processes are performed on each group of the user portrait data:

[0060] Missing value processing: filling, mean filling, null value filling;

[0061] Data binning: continuous values, usage, package value;

[0062] Feature conversion processing: categorical variable encoding, one-hot encoding, frequency encoding, label encoding;

[0063] Data normalization: min-max normalization.

[0064] Exemplarily, the label of the user portrait data is determined according to whether the user has subscribed. Exemplarily, the target variable of the first user data is determined according to whether the user has subscribed, so that a wide table of data used for modeling can be obtained.

[0065] Exemplarily, given that there is likely to be a large correlation between the field features in the data wide table, in order to speed up the model training process, the present technical solution can also reduce the dimension of the features in the data wide table to reduce the computational workload (step S430). Exemplarily, the principal component analysis (PCA) method is used for dimensionality reduction to reduce the number of features used for model training, thereby effectively reducing the computational workload of model training. Among them, PCA transforms the coordinate space so that the information represented by the data is distributed as concentratedly as possible on a smaller number of dimensions, so that the coordinate dimension that occupies more information can be selected to achieve the purpose of data dimensionality reduction.

[0066] It should be noted that this technical solution distinguishes the above user portrait data according to whether the user has been marketed before. Figure 3 The data processing stage 32 shown in the figure obtains data generated by users who have not been marketed to in the past (referred to as "first user data") and uses it to train and test the first prediction model and data generated by users who have been marketed to in the past (referred to as "second user data").

[0067] For example, marketing indicates that the user has been marketed, and unmarketing indicates that the user has not been marketed. model indicates that modeling and identification are performed. Then, for each user in the entire user set, y = {y1, y2, ... y n} represents a tag combination, and its content is whether it is a response or not. X represents a feature set, and its content includes personal information, bill consumption information, communication record information, customer service information, terminal information, subscription service information, channel preference, data traffic usage, life trajectory information, etc. The mapping relationship between tags and features is as follows:

[0068] f classification :X→y i

[0069] Where i is a positive integer not greater than n. Figure 3 , the determination phase 33 of the first prediction model is performed through the first user data (corresponding to step S211), and the determination phase 34 of the second prediction model is performed through the second user data (corresponding to step S212). Specifically, the first prediction model for predicting the probability of ordering without historical marketing and the second prediction model for predicting the probability of ordering with historical marketing use the same type of model to calculate the probability of marketing recognition, so as to facilitate the measurement of the impact of whether marketing is applied to the same user based on the same scale system, thereby improving the prediction accuracy of the entire model.

[0070] Exemplarily, the technical solution trains two XGBoost (Extreme Gradient Boosting) models using the first user data and the second user data, respectively.

[0071] Taking the determination stage 33 of the first prediction model as an example, the first user data (historical user data that did not participate in marketing) can be divided into a training set and a test set. The XGBoost model is trained with the training set to mine the features in the historical user data that did not participate in marketing. As mentioned above, the target variable of the first prediction model is whether to order. The model will predict the probability P of whether users who did not participate in marketing recommendations will eventually buy the product. u The model process through the training set is the parameter adjustment process, which includes:

[0072] 1) Determine the initial value of the learning rate and the parameter tuning, and choose a higher learning rate. For example, the learning rate is 0.1. However, for different problems, the ideal learning rate sometimes fluctuates between 0.05 and 0.3. Select the ideal number of decision trees corresponding to this learning rate.

[0073] 2) max_depth and min_child_weight parameter tuning:

[0074] max_depth: The maximum depth of the tree. The larger the value, the larger the tree and the more complex the model. It can be used to prevent overfitting. The typical value is 3-10.

[0075] min_child_weight: The minimum weight sum of all observations in a subset. If the sample weight sum of the newly split node is less than min_child_weight, the split is stopped, which can be used to reduce overfitting.

[0076] 3) Gamma parameter tuning

[0077] Gamma: When splitting a node, the node will only be split if the loss function decreases by a value greater than or equal to gamma. The larger the gamma value, the more conservative the algorithm is and the less likely it is to overfit.

[0078] 4) Optimize subsample and colsample_bytree parameters

[0079] subsample: Construct the sampling rate of each tree for samples. If it is set to 0.5, XGBoost will randomly select half of the samples as training sets.

[0080] colsample_bytree: Column sampling rate, that is, feature sampling rate, set to 1.

[0081] 5) Regularization parameter alpha tuning

[0082] alpha: Add L1 regularization to reduce the complexity of the model, thereby improving the performance of the model. Adding this value will make the model more convergent.

[0083] 6) Reduce the learning rate and use more decision trees.

[0084] Continue to refer Figure 3 , for the process model obtained during the training process, the model model is evaluated. Specifically, the model evaluation process is performed through the above test set. Exemplarily, the AUC, F1 score and model improvement are used to evaluate the model prediction ability, where the specific implementation method of the test indicator being the model improvement can refer to Formula 1 to Formula 3. When the test indicators are AUC and F1 scores, the test process can specifically include:

[0085] 1) The area under the receiver operating characteristic curve (ROC) AUC (a model evaluation indicator, specifically used to evaluate the predictive value of the process model; it is the abbreviation of Area Under Curve) is used to evaluate the above iteratively optimized entity recognition model. Specifically:

[0086] The false positive rate FPR and true positive rate TPR are determined using the following formula distribution:

[0087] FPR=FP / (FP+TN) (4)

[0088] TPR=TP / (TP+FN) (5)

[0089] Among them, TP (true positive) is the number of positive classes that are still in the positive class after being judged by the model to be tested, TN (true negative) is the number of negative classes that are judged by the model to be tested for the positive class, FN (false negative) is the number of negative classes that are judged by the model to be tested for the negative class, and FP (false positive) is the number of positive classes that are judged by the model to be tested for the negative class.

[0090] Furthermore, the receiver operating characteristic curve (ROC curve) is plotted with FPR as the horizontal axis and TPR as the vertical axis. The ROC curve is the characteristic curve of each indicator obtained, which is used to show the relationship between each indicator, and the area under the ROC curve AUC is further calculated. The ROC curve is the characteristic curve of each indicator obtained, which is used to show the relationship between each indicator. AUC is the area under the ROC curve. The larger the AUC, the higher the predictive value of the process model.

[0091] 2) About F1 score

[0092]

[0093] Among them, precision represents the accuracy and recall represents the recall, which are determined according to the following formula:

[0094]

[0095]

[0096] In an exemplary embodiment, reference Figure 3 When judging whether the model evaluation result meets the preset requirements of the test index, if the model evaluation result does not meet the preset requirements of the test index, the model training is continued. When the model evaluation result meets the preset requirements of the test index, the first prediction model is obtained.

[0097] refer to Figure 3 The specific implementation of the determination stage 34 (corresponding to step S212) of executing the second prediction model through the above-mentioned second user data is the same as the specific implementation of the determination stage 33 through the first prediction model, and will not be repeated here.

[0098] In an exemplary embodiment, continue to refer to Figure 3 After obtaining the first prediction model and the second prediction model that meet the test index, the sample determination process 35 of the third prediction model is introduced below:

[0099] (I) Determine the combined data

[0100] Exemplarily, step S220 is performed: combined data is determined according to the first user data and the second user data. Figure 3, add variable M to the data obtained from the database. Specifically, if M=1, it means that the group of data is the user data that has participated in historical marketing, and if M=0, it means that the group of data is the user data that has not participated in historical marketing. Exemplarily, the data corresponding to each user in the user data that has not been historically marketed in stage 32 can be added to variable M with a value of 0, and the data corresponding to each user in the user data that has been historically marketed in stage 32 can be added to variable M with a value of 1.

[0101] Further, refer to Figure 3 , the variables formed by connecting the above two groups of variables are recorded as "combined data". Each group of user data in the "combined data" corresponds to a "real response classification" field, which specifically includes "whether marketing" and "whether ordering". In other words, the target variable of the combined data is the above "real response classification" field.

[0102] 2. Determining the interaction variables

[0103] Exemplarily, step S231 and step S232 are performed again: inputting the combined data into the trained first prediction model to obtain the first order probability, and inputting the combined data into the trained second prediction model to obtain the second order probability. Figure 3 , the above combined data is input into the second prediction model obtained by the above training through step (1), and the second order probability P is obtained. m The combined data is also input into the first prediction model obtained by the training through step (1) to obtain the first order probability P u Furthermore, the obtained P is obtained by step (2) / (2)' m With P u Summarize and according to the above first order probability P u and the above second ordering probability P m The interaction variable is determined (step S240).

[0104] Among them, reference Figure 5 The flowchart of the method for determining the interaction variable according to an embodiment of the present disclosure is schematically shown. In step S510, the first order probability and the second order probability are processed by a kernel function to obtain an interaction influence factor. In step S520, the combination of the first order probability, the second order probability and the interaction influence factor is determined as the interaction variable.

[0105] For example, the common influencing factor is determined by using a linear kernel function, and the probability P predicted by the above two XGBoost models can be m , P u (representing the order intention of users after marketing and the order intention of users without marketing, respectively) multiplied together as the common marketing influence factor Pm *P u . Further, the ternary feature {P m , P u , P m *P u} as the above interaction variable. Among them, the common marketing impact factor P m *P u The role is to consider P m , P u The composite correlation between m , P u Mapping from two-dimensional space to three-dimensional space can, to a certain extent, transform the linearly inseparable categories in the low-dimensional space into linearly separable ones. This can not only fully construct the interaction between data, but also enhance the classification effect of data to a certain extent.

[0106] It should be noted that the method of determining the above common influencing factors is not limited to the linear kernel function, and can also be determined by other kernel functions such as polynomial, radial basis function and Sigmod.

[0107] The implementation method of this technical solution uses kernel functions to construct interactive variables to achieve prediction of predictions, obtains interactive influence variables through first-level predictions (first order probability and second order probability), and adds interactive influence variables to second-level predictions (prediction of the third prediction model) to determine the fine classification of users. The introduction of kernel functions to construct interactive variables and the second-level prediction based on the first-level prediction are both conducive to improving the prediction of whether marketing has an impact on user orders, and thus to improving the accuracy of classification.

[0108] (III) Using the above combined data and the above interactive variables as sample data for the third prediction model

[0109] Exemplary, reference Figure 6 A flowchart schematically shows a method for determining sample data of a third prediction model according to an embodiment of the present disclosure, including:

[0110] In step S610, the combined data is cascade-extended by the interaction variables to obtain cascade-extended features. m , P u , P m *P u} as a derived feature, the above combined data is further expanded to obtain a cascade extended feature. In step S620, the target variable of the cascade extended feature is determined according to whether the user is marketed and whether the user is subscribed, and the sample data of the third prediction model is obtained. As mentioned above, the target variable of the sample data of the third prediction model remains the target variable of the above combined data unchanged, that is, each group of user data in the "sample number of the third prediction model" corresponds to a "real response classification" field, which specifically includes "whether marketing" and "whether ordering". In other words, the target variable of the sample number of the third prediction model is the above-mentioned "real response classification" field.

[0111] In this technical solution, "whether marketing" is added on the basis of "whether to order" to increase the dimension of the target variable, so that the analysis of the third prediction model, that is, the prediction perspective, is expanded to whether marketing and whether to order, which is conducive to the increase of the classification dimension of the third prediction model to obtain a classification close to reality. It can be seen that the present invention proposes a method that not only focuses on model improvement, but also incorporates the interactive influence between marketing and ordering behaviors, thereby achieving a fine classification of customers' real marketing responses.

[0112] In this technical solution, the prediction probabilities of the two XGBoost sub-models are constructed as cascade extension variables through kernel functions to complete the horizontal interaction of the sub-models. Further, the cascade extension variables and user attribute features are combined to generate interactive cascade combination features, which are used as sample data for the next-level prediction model (the third prediction model), thus realizing the vertical cascade between the two-level models after the interaction. Among them, the target variable of the sample data of the third prediction model is the above-mentioned "real response classification" field, and the training and testing of the third prediction model are completed according to this sample data set. Then the user data of the user to be predicted is loaded into the model after regular processing according to the above-mentioned processing process, and the prediction and output of the real response classification can be completed. Specifically, the target user is obtained by the output of the third prediction model as one of the following user classifications: the type that does not order without marketing and does not order after marketing, the type that does not order without marketing and orders after marketing, the type that does not order without marketing but does not order after marketing, and the type that does not order without marketing but does not order after marketing, and the type that does not order without marketing and orders after marketing.

[0113] Continue to refer Figure 3 After the number of samples of the third prediction model is determined, the third prediction model is trained and tested in the third prediction model determination stage 36. Exemplarily, the third prediction model adopts GBDT (Gradient Boosting Decision Tree). The training and testing process of the third prediction model is similar to Figure 3The training and testing process of the first prediction model is the same as that of the previous one, and will not be repeated here. This technical solution uses an interactive cascade machine learning model based on user data that has not been marketed to historically and data that has been marketed to historically as a control, thereby identifying the true response type of customers to marketing activities.

[0114] Referring to Table 1, the output of the third prediction model after training is a classification result that is close to the actual situation and is specifically divided into: the type that does not order without marketing and does not order after marketing (recorded as "definitely will not buy type") - category 0, the type that does not order without marketing and orders after marketing (recorded as "can be persuaded type") - category 1, the type that orders without marketing but does not order after marketing (recorded as "do not bother type") - category 2, the type that orders without marketing and orders after marketing (recorded as "definitely will buy type") - category 3.

[0115] Table 1

[0116]

[0117] Corresponding to the above four types of marketing response users in Table 1, a more detailed classification of customers’ real marketing responses can be obtained. The real response classification and targeted marketing strategies are as follows:

[0118] 1. For the type of users who will never buy: This type of users will never buy a service regardless of marketing or not. There is no need to invest marketing resources.

[0119] 2. For the persuadable type: If this type of users are not marketed to, they will not buy a service, but if they are marketed to, the success rate of buying a service will be greatly increased, so it is necessary to increase marketing to this type of people. This is also the customer group with the greatest marketing value.

[0120] 3. For the Do Not Disturb type: If this type of user is not marketed to, they will still buy a service, but if marketing is applied to them, it will be counterproductive. Existing customer churn warning models usually cannot predict this type of user, so if marketing is directly applied to this type of user, it will be counterproductive and will not help users successfully finalize the product.

[0121] 4. For users who will definitely buy: This type of users will buy a service regardless of marketing or not. You can strategically reduce marketing investment for this type of users.

[0122] Through the above embodiments, a cascaded user classification model that meets the test indicators is obtained. For example, refer to Figure 7, this technical solution can be used for any user (referred to as "target user" in this technical solution), and generally includes a comparison prediction stage 710 and a true response classification stage 720. The following introduces an embodiment of classifying the target user through the above cascaded user classification model:

[0123] (1) Obtain user data 71 corresponding to the target user, process the user data through the embodiment corresponding to the above data processing stage 32, and input the processed user data into the above first prediction model 72 to obtain the first order probability P corresponding to the target user. u ', and also input the processed user data into the second prediction model 73 to obtain the second order probability P corresponding to the target user m '.

[0124] (2) Using the kernel function method, the first order probability P is processed u ' and the second ordering probability P m ', get the interaction variable factor P m '*P u ', and then get the interaction variable (P u ',P m ',P m '*P u ').

[0125] (3), through the above interaction variable (P u ',P m ',P m '*P u ') Perform cascade expansion on the user data 71 of the target user to obtain the cascade expansion feature 74 corresponding to the target user.

[0126] (4) The above-mentioned cascade expansion features are input into the trained third prediction model 75, and combined with the impact of marketing on user orders in the model, the actual response classification of the target user is obtained through the output of the third prediction model, that is, one of the following user classifications: the type that does not order without marketing and does not order after marketing, the type that does not order without marketing and orders after marketing, the type that orders without marketing but does not order after marketing, and the type that orders without marketing and orders after marketing.

[0127] (5) Implement corresponding marketing strategies for the target users based on their type.

[0128] If the target user belongs to the user group with greater marketing influence, that is, the type that can be persuaded, if this type of user is not marketed to, he or she will not subscribe to a certain service by itself. However, if marketing is applied to him or her, the success rate of subscribing to a certain service will be greatly increased. Therefore, it is necessary to focus on increasing marketing to this user.

[0129] In another embodiment, the user data of 3,000 target users are obtained, and the real response classification of the 3,000 target users is predicted according to the above steps (1)-(5). The specific prediction results are shown in Table 2:

[0130] Table 2

[0131]

[0132] Refer to Table 2, where the number of customers who belong to the "persuadable type" is 1033, and outbound marketing is carried out for this group of customers. The marketing conversion rate of this type of customers is 36.1%, and for the "definitely not buying type" because they will not order, there is no need to carry out outbound marketing, and for the "do not bother type" and "definitely will buy type" will naturally convert or may naturally convert, so there is no need to carry out outbound marketing, which effectively saves marketing resources.

[0133] For example, the marketing business is "Enjoy free secondary card". After conducting outbound marketing for the above-mentioned "persuadable type" customers, the marketing results are shown in Table 3:

[0134] Table 3

[0135] Marketing services, such as free additional cards Marketing data Marketing data load 1033 Marketing Order Volume 373 Marketing conversion rate 36.1％

[0136] It can be seen that the marketing order quantity in Table 3 is 373. If the "persuadable type" is identified and classified by this technical solution and outbound marketing is carried out focusing on "persuadable type" users, the marketing conversion rate is (373 / 1033=36.1%).

[0137] Compare with Table 4 showing the marketing conversion rate (1895 / 20323=9.37%) for the same marketing service “Enjoy free secondary card”, in which the second prediction model is used alone to achieve the marketing conversion rate.

[0138] Table 4

[0139] Enjoy free additional card First prediction model The second prediction model User data load 2636 20323 Marketing Order Volume 48 1895 Marketing conversion rate 1.82％ 9.37％

[0140] It can be seen that this technical solution introduces the first prediction model and then introduces the influence of marketing on whether the user orders, so as to accurately classify the users and then carry out precision marketing, which is not only conducive to a significant increase in marketing conversion rate but also can save marketing resources.

[0141] Those skilled in the art will appreciate that all or part of the steps to implement the above embodiments are implemented as a computer program executed by a processor (including a CPU and a GPU). When the computer program is executed by the processor, the above functions defined by the above method provided by the present disclosure are performed. The program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk or an optical disk, etc.

[0142] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiment of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0143] The following is an introduction to the cascaded user classification model provided by this technical solution:

[0144] refer to Figure 3 , the cascaded user classification model includes: a first prediction model, a second prediction model, and a third prediction model cascaded with the first prediction model and the second prediction model respectively; wherein:

[0145] The first prediction model is trained by first user data that has not been marketed historically, the second prediction model is trained by second user data that has been marketed historically, and the sample data of the third prediction model is determined based on interactive variables and combined data.

[0146] Exemplarily, the above-mentioned number of combinations is determined based on the first user data and the second user data, and the combined data is input into the trained first prediction model to obtain the first order probability, the combined data is input into the trained second prediction model to obtain the second order probability, and the above-mentioned interaction variable is determined based on the first order probability and the second order probability.

[0147] On the one hand, this technical solution trains the same type of machine learning models (such as XGBoost model) for two groups of comparative marketing data (including: the first user data that has been historically marketed and the second user data that has not been historically marketed, and the target variable is whether to subscribe). The two prediction results and interaction influence information are combined through interactive cascading, and combined with user portrait features, and aggregated prediction is performed through the above-mentioned third prediction model (such as GBDT model). The interactive influence relationship between user characteristic attributes, marketing or not, and user ordering behavior is comprehensively considered to calculate the output The size of the influence of marketing activities on users, so as to better realize the real response classification and improve the accuracy of targeted marketing.

[0148] On the other hand, this technical solution obtains the real response marketing classification through interactive cascade machine learning prediction. The first-level prediction model outputs the marketing response probability, and constructs the interactive influence variables through the kernel function to generate interactive combination feature variables as the input of the second-level aggregation prediction model. It is an optimized version of cascade learning that meets complex business functions. Its characteristics are horizontal and vertical interactive cascade prediction, which integrates the impact of marketing on user ordering behavior. After iterative optimization of multi-level machine learning models, the prediction accuracy is very high and the generalization ability is stronger.

[0149] At the same time, the sophisticated architecture of the cascade machine learning model implements complex functions and realizes complex business functions. It not only has good accuracy and generalization performance, but also meets the performance and efficiency requirements of the machine learning model.

[0150] The specific details of the training process and the testing process of the prediction model in the above cascaded user classification model have been described in detail in the corresponding user classification method, so they will not be repeated here.

[0151] The following is an introduction to the user classification device provided by this technical solution:

[0152] A user classification device provided in this example implementation is applied to a node between a resource consumption end and a resource provision end. Figure 8 As shown, the user classification device 800 includes: a sub-model training module 801, a combination module 802, a sub-model prediction module 803, a sample determination module 804 and a classification module 805.

[0153] Among them, the above-mentioned sub-model training module 801 is configured to: train the first prediction model through the first user data that has not been historically marketed, and train the second prediction model through the second user data that has been historically marketed; the above-mentioned combination module 802 is configured to: determine the combination data according to the above-mentioned first user data and the above-mentioned second user data; the above-mentioned sub-model prediction module 803 is configured to: input the above-mentioned combination data into the trained first prediction model to obtain the first order probability, and input the above-mentioned combination data into the trained second prediction model to obtain the second order probability; the above-mentioned sample determination module 804 is configured to: determine the interaction variable according to the above-mentioned first order probability and the above-mentioned second order probability, and determine the sample data of the third prediction model according to the above-mentioned interaction variable and the above-mentioned combination data; and the above-mentioned classification module 805 is configured to: predict the user data of the target user based on the trained first prediction model, the second prediction model and the third prediction model to obtain the prediction results, and the above-mentioned prediction results include: the type that does not order without marketing and orders after marketing.

[0154] In an exemplary embodiment, based on the aforementioned scheme, the above prediction results also include: a type that does not order without marketing and does not order after marketing, a type that orders without marketing but does not order after marketing, and a type that orders without marketing and orders after marketing.

[0155] In an exemplary embodiment, based on the aforementioned solution, the sample determination module 804 includes an interaction variable determination unit 8041 .

[0156] Among them, the above-mentioned interaction variable determination unit 8041 is configured to: use the kernel function method to process the above-mentioned first order probability and the above-mentioned second order probability to obtain the interaction influence factor; and determine the combination of the above-mentioned first order probability, the above-mentioned second order probability and the above-mentioned interaction influence factor as the above-mentioned interaction variable.

[0157] In an exemplary embodiment, based on the above-mentioned solution, the sample determination module 804 includes a cascade expansion unit 8042 .

[0158] Among them, the above-mentioned cascade expansion unit 8042 is configured to: cascade expand the above-mentioned combined data through the above-mentioned interactive variables to obtain cascade expansion features; determine the target variable of the above-mentioned cascade expansion features according to whether the user is marketed and whether the user has subscribed, and obtain sample data of the third prediction model.

[0159] In an exemplary embodiment, based on the aforementioned scheme, the sub-model training module 801 is specifically configured as follows: obtaining user portrait data of multiple users who have not been historically marketed from a database to obtain the first user data; determining the target variable of the first user data according to whether the user has subscribed, obtaining sample data of the first prediction model, and training the first prediction model.

[0160] In an exemplary embodiment, based on the aforementioned scheme, the above-mentioned sub-model training module 801 is also specifically configured as follows: extracting user portrait data that has not been historically marketed from the database in the form of a structured query language script, wherein each user portrait data contains a user identifier, and obtaining a group of user portrait data corresponding to each user identifier, and each group of user portrait data includes one or more of the following information: user characteristics, product attributes, behavior characteristics, package attributes, terminal characteristics, and derived indicator characteristics; performing one or more of the following processing on each group of the above-mentioned user portrait data: missing value processing, data binning processing, feature conversion processing, and data standardization processing to obtain a wide data table; performing dimensionality reduction processing on the above-mentioned wide data table to obtain the first user data for training the above-mentioned first prediction model.

[0161] In an exemplary embodiment, based on the aforementioned solution, the first prediction model and the second prediction model adopt the same type of model.

[0162] The specific details of each module or unit in the above user classification device have been described in detail in the corresponding user classification method, so they will not be repeated here.

[0163] Fig. 9 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present invention is shown.

[0164] It should be noted that Fig. 9The computer system 900 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0165] like Fig. 9 As shown, the computer system 900 includes a processor 901, wherein the processor 901 may include: a graphics processing unit (GPU), a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 to a random access memory (RAM) 903. In RAM 903, various programs and data required for system operation are also stored. The processor (GPU / CPU) 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0166] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 905 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

[0167] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 811. When the computer program is executed by the processor (GPU / CPU) 901, various functions defined in the system of the present application are executed. In some embodiments, the computer system 900 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0168] It should be noted that the computer-readable medium shown in the embodiment of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0169] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0171] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0172] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0173] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0174] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0175] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A user classification method, characterized in that: The method comprises: The first prediction model is trained by first user data that has not been historically marketed, and the second prediction model is trained by second user data that has been historically marketed; determining combined data according to the first user data and the second user data; Inputting the combined data into a trained first prediction model to obtain a first order probability, and inputting the combined data into a trained second prediction model to obtain a second order probability; Determine an interaction variable according to the first order probability and the second order probability, and determine sample data of a third prediction model according to the interaction variable and the combined data; Predicting the user data of the target user based on the trained first prediction model, the second prediction model, and the third prediction model to obtain prediction results, wherein the prediction results include: types that would not order without marketing and would order after marketing; The step of determining an interaction variable according to the first order probability and the second order probability, and determining sample data of a third prediction model according to the interaction variable and the combined data includes: Obtaining an interaction influence factor according to the first ordering probability and the second ordering probability, and determining a combination of the first ordering probability, the second ordering probability and the interaction influence factor as the interaction variable; The combined data is cascade-expanded through the interactive variables to obtain cascade-expanded features; the target variable of the cascade-expanded features is determined according to whether the user is marketed and whether the user orders, to obtain sample data of the third prediction model.

2. The user classification method according to claim 1, characterized in that: The prediction results also include: The type that is not ordered without marketing and is not ordered after marketing, the type that is ordered without marketing but not ordered after marketing, and the type that is ordered without marketing and is ordered after marketing.

3. The user classification method according to claim 1, characterized in that: The method further comprises: The first ordering probability and the second ordering probability are processed by a kernel function to obtain an interaction influence factor.

4. The user classification method according to any one of claims 1 to 3, characterized in that: The first prediction model is trained by using the first user data that has not been historically marketed, including: Acquire user portrait data of multiple users that have not been marketed in the past from the database to obtain the first user data; According to whether the user has subscribed, the target variable of the first user data is determined, sample data of the first prediction model is obtained, and the first prediction model is trained.

5. The user classification method according to claim 4, characterized in that: Obtaining user portrait data of multiple users who have not been marketed in the past from the database to obtain the first user data includes: Extracting user portrait data that has not been marketed historically from a database in the form of a structured query language script, wherein each user portrait data includes a user identifier, and obtaining a set of user portrait data corresponding to each user identifier, wherein each set of user portrait data includes one or more of the following information: user characteristics, product attributes, behavior characteristics, package attributes, terminal characteristics, and derived indicator characteristics; Perform one or more of the following processing on each group of user portrait data: missing value processing, data binning processing, feature conversion processing and data standardization processing to obtain a data wide table; Dimensionality reduction is performed on the wide data table to obtain first user data for training the first prediction model.

6. The user classification method according to any one of claims 1 to 3, characterized in that: The first prediction model and the second prediction model adopt the same type of model.

7. A system of cascaded user classification models, characterized in that: The system comprises: A first prediction model, a second prediction model, and a third prediction model cascaded with the first prediction model and the second prediction model respectively; wherein, The first prediction model is trained by first user data that has not been marketed historically, the second prediction model is trained by second user data that has been marketed historically, and the sample data of the third prediction model is obtained by cascading and expanding the combined data through interactive variables to obtain cascading expansion features; the target variable of the cascading expansion feature is determined according to whether the user has been marketed and whether the user has subscribed; wherein, The number of combinations is determined according to the first user data and the second user data, The combined data is input into the trained first prediction model to obtain a first order probability, and the combined data is input into the trained second prediction model to obtain a second order probability. An interaction influence factor is obtained according to the first order probability and the second order probability, and the interaction variable is determined by combining the first order probability, the second order probability and the interaction influence factor.

8. A user classification device, characterized in that: The device comprises: The sub-model training module is configured to: train a first prediction model using first user data that has not been historically marketed, and train a second prediction model using second user data that has been historically marketed; a combining module, configured to: determine combined data according to the first user data and the second user data; The sub-model prediction module is configured to: input the combined data into the trained first prediction model to obtain a first order probability, and input the combined data into the trained second prediction model to obtain a second order probability; a sample determination module, configured to: determine an interaction variable according to the first order probability and the second order probability, and determine sample data of a third prediction model according to the interaction variable and the combined data; The classification module is configured to: predict the user data of the target user based on the trained first prediction model, the second prediction model and the third prediction model to obtain prediction results, wherein the prediction results include: types that would not order without marketing and would order after marketing; The step of determining an interaction variable according to the first order probability and the second order probability, and determining sample data of a third prediction model according to the interaction variable and the combined data includes: Obtaining an interaction influence factor according to the first ordering probability and the second ordering probability, and determining a combination of the first ordering probability, the second ordering probability and the interaction influence factor as the interaction variable; The combined data is cascade-expanded through the interactive variables to obtain cascade-expanded features; the target variable of the cascade-expanded features is determined according to whether the user is marketed and whether the user orders, to obtain sample data of the third prediction model.

9. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the user classification method described in any one of claims 1 to 6 by executing the executable instructions.

Images