A customer repurchase prediction method and device based on RF-LightGBM fusion model in non-contractual scenarios

Abstract

The present invention relates to a method and device for predicting customer repurchase based on an RF-LightGBM fusion model in a non-contract scenario. The method includes: obtaining historical data of users, preprocessing and feature engineering the data; using the preprocessed data as samples, balancing the sample set using the SMOTE-ENN method; optimizing the random forest algorithm and the LightGBM algorithm through the TPE optimization algorithm to perform hyperparameter optimization, and constructing a weak classifier; using each weak classifier to perform ensemble learning on the training samples to obtain a strong classifier, and obtaining the final result of repurchase prediction. The present invention analyzes the consumption data of the company's purchased customers, accurately predicts the repurchase behavior of existing customers, guides customer relationship management decisions and precision marketing strategies based on the repurchase behavior, improves marketing conversion rates, and reduces corporate operating costs.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for predicting customer repurchase based on an RF-LightGBM fusion model in a non-contractual scenario. Background Art

[0002] With the advent of the big data era, predicting consumers' future purchase intentions from massive consumer historical transaction data has become an important issue in enterprise management. Customer repurchase behavior prediction in non-contractual scenarios mainly refers to the prediction of the customer's next purchase behavior of the company's products when the company and the customer have not signed a purchase contract. Accurately predicting consumers with repeat purchase intentions can more accurately match customer needs through precision marketing, enhance the value of new consumers and convert them into loyal customers.

[0003] In the prior art, a Chinese invention patent (authorization announcement number CN109146533B) discloses an information push method and device, which specifically discloses obtaining at least two order information of a user for items of the same item category, determining the user's average daily consumption of items of the item category during an interval based on the purchase quantities in the at least two order information, and determining a push date for pushing item information associated with items of the item category to the user terminal of the user based on the average daily consumption and the purchase quantity corresponding to the latest order, thereby improving the effectiveness of information push. A Chinese invention patent (authorization announcement number CN108171530B) discloses a method and device for improving the average order value and repurchase rate, which: selects the historical marketing data of the target store, obtains the historical marketing activity effects, and obtains the initial estimated value of the marketing activity effect of the target store based on the historical marketing data and the historical marketing activity effects; constructs a threshold adjustment factor based on the ratio of the number of orders that meet the threshold and the number of orders that meet the average order value of the historical marketing activities of each store, and uses the threshold adjustment factor to calibrate the initial estimated value of the marketing activity effect of the target store to obtain the estimated value of the marketing activity effect of the target store, thereby solving the problem that the existing promotion activity effect evaluation technology cannot more accurately estimate the marketing activity effect according to the change of the threshold. Although the above-mentioned prior art realizes product recommendation and effect prediction based on historical data, it cannot accurately predict customer behavior.

[0004] Existing machine learning methods are widely used in the field of customer behavior prediction, but most of them focus on prediction in e-commerce scenarios. In the prior art, a Chinese invention application (publication number CN110956497A) discloses a method for predicting repeated purchase behavior of users on an e-commerce platform, including: obtaining user historical purchase behavior data, integrating a deep Catboost individual model, a double-layer attention BiGRU individual model, and a DeepGBM individual model, modeling the discrete purchase record values ​​and behavior sequence features in the user's historical purchase data, and improving the accuracy of the prediction results. The Chinese invention application (publication number CN108520469A) discloses a method for analyzing user repurchase behavior based on an e-commerce platform, which selects valid purchase records of users within a statistical period; performs data cleaning; labels each valid purchase record as a repeated purchase, a platform repeated purchase, or a insurance repeated purchase; counts the total number of purchasing users, the number of repeated purchasing users, the total number of purchasing users on each platform, the total number of repeated purchasing users on each platform, the total number of purchasing users of each insurance type, and the total number of repeated purchasing users of each insurance type; calculates the repeat purchase rate, platform repeat purchase rate, and insurance type repeat purchase rate within the statistical period. However, in e-commerce scenarios, customers' "implicit" feedback behaviors such as collections and likes can be retained, which is not available in broader non-contract scenarios. In addition, the current use of machine learning algorithms focuses on algorithm integration, ignoring the impact of data sets on prediction results. Generally, in a purchase scenario, the proportion of users who make repeated purchases is less than that of users who make single purchases, which leads to the problem of imbalanced data categories, often resulting in overfitting of the model and low prediction accuracy. Summary of the invention

[0005] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a customer repurchase prediction method and device based on the RF-LightGBM fusion model in a non-contract scenario. The present invention adopts the following technical solutions:

[0006] A customer repurchase prediction method based on the RF-LightGBM fusion model in a non-contract scenario includes the following steps:

[0007] Obtain the user's historical purchase record data, preprocess it, and extract features;

[0008] Use the sample balancing method to balance the data after feature extraction to obtain a balanced sample;

[0009] The training sample data is trained using the optimization algorithm, and the weak classifier is iteratively optimized in the specified weak classifier hyperparameter space;

[0010] By assigning the same weight to each weak classifier, a strong classifier is obtained through ensemble learning;

[0011] Use strong classifiers to make predictions and obtain the final results on product recommendations and repeat purchase behavior predictions;

[0012] According to the final result, the product information is pushed to the user's terminal device and / or the repurchase behavior prediction result is sent to the management system.

[0013] Further, the feature extraction includes:

[0014] Recent purchase time, purchase frequency, total purchase amount, relationship duration, and purchase interval.

[0015] Further, the sample balancing method includes:

[0016] First, the SMOTE oversampling method is used to generate minority class samples of the extracted features, and then the ENN (Edited KNN) method is used to judge the generated samples. If the predicted result is different from the actual category label, the sample is eliminated to obtain a balanced sample.

[0017] Further, the optimization algorithm includes:

[0018] The TPE (Tree-structured Parzen Estimator) tree-structured Parzen estimation optimization algorithm is used to optimize the model hyperparameters and perform model training under the optimal hyperparameters.

[0019] Furthermore, the weak analyzer includes a random forest RF (Random Forestsm) model and a Light GBM model. The output results of the weak analyzer are all classification probability values, and the mathematical expression is:

[0020]

[0021] Where N tree is the total number of decision trees, h i is the i-th decision tree, and P(x|y) represents the probability of predicting that sample x belongs to category y.

[0022] Furthermore, the integrated learning specifically includes:

[0023] The RF model and the Light GBM model are given the same weight, and the soft voting method is used to integrate them based on their prediction probabilities. The mathematical expression is:

[0024] P Soft Voting =(P RF +P LightGBM ) / 2

[0025]

[0026] Among them, P Soft Voting refers to the predicted probability of the soft voting fusion model, P RF ,P LightGBM They represent the prediction probabilities of the random forest and LightGBM models respectively, Result represents the prediction results of the fusion model, 1 represents a repeat purchase user, 0 represents a non-repurchase user, and threshold represents the classification threshold.

[0027] Furthermore, repurchase behavior prediction and repurchase probability prediction are used as guidance for product recommendations.

[0028] The present invention also includes a customer repurchase prediction device based on the RF-LightGBM fusion model in a non-contract scenario, including:

[0029] The acquisition module obtains the user's historical purchase record data, preprocesses it, and extracts features;

[0030] The balancing module uses the sample balancing method to balance the data after feature extraction to obtain balanced samples;

[0031] The optimization training module uses the optimization algorithm to train the training sample data and iteratively optimizes the weak classifier in the specified weak classifier hyperparameter space;

[0032] The ensemble learning module assigns the same weight to each weak classifier and performs ensemble learning to obtain a strong classifier.

[0033] The prediction module uses a strong classifier to make predictions and obtain the final results on product recommendations and repurchase behavior predictions;

[0034] The push module pushes product information to the user's terminal device according to the final result.

[0035] The present invention also includes an electronic device, comprising:

[0036] processor, and memory;

[0037] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to execute the customer repurchase prediction method based on the RF-LightGBM fusion model in any non-contractual scenario as described above.

[0038] The present invention also includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a customer repurchase prediction method based on the RF-LightGBM fusion model in any of the non-contractual scenarios described above.

[0039] The beneficial effects achieved by the present invention are as follows: by analyzing the purchase behavior records of existing users of the enterprise, accurately predicting the repurchase of existing users, and guiding the customer relationship management strategy and marketing strategy accordingly, improving the marketing conversion rate, and reducing related operating costs; based on customer purchase behavior data, accurately predicting the customer's repurchase behavior of goods, while meeting the actual effective needs of customers, it can reduce the company's communication costs; dynamically guiding the company's business strategy through data, driving decision-making with data, and assisting in achieving product marketing goals, ultimately achieving the goal of recommending suitable products to suitable users in an intelligent manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings. The following examples are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention. It should be pointed out that the following detailed descriptions are all exemplary and are intended to provide further explanation of the present application.

[0042] Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those generally understood by those of ordinary skill in the art to which this application belongs. It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0043] like Figure 1 As shown, this embodiment discloses a customer repurchase prediction method based on the RF-LightGBM fusion model in a non-contract scenario, including the following steps:

[0044] (1) Obtain the user's historical purchase record data, pre-process it, and extract features. The historical purchase record data is existing data. The extracted features include: the most recent purchase time (R), purchase frequency (F), total purchase amount (M), relationship duration (S), and purchase interval (T).

[0045] (2) Use the SMOTE-ENN method to perform sample balancing on the data after feature extraction to obtain the model training set. Multiple sampling with replacement is used for each category of the original sample set to form the test sample.

[0046] (3) The training sample data is trained using the TPE optimization algorithm, and the weak classifier is iteratively optimized in the specified weak classifier hyperparameter space.

[0047] (4) By assigning the same weight to each weak classifier, ensemble learning is performed to obtain a strong classifier, and the final results of product recommendation and repurchase behavior retest are obtained.

[0048] In this embodiment, the preprocessing step includes: for the convenience of computer processing and user labeling, converting character data into numeric data and converting numeric data into date data. The extracted features include the most recent purchase time (R), purchase frequency (F), total purchase amount (M), relationship duration (S), and purchase interval (T):

[0049] a) R: The time when the customer last consumed the product, in the following format:

[0050] R=T last_time -T plast_time

[0051] Where T last_time Indicates the end time of the reference time period, T plast_time Indicates the last order transaction time of the customer for the product within the reference time period.

[0052] b) F: The number of times a customer purchases this product during the observation period.

[0053] c) M: The total amount of money the customer spends on this product, in the following format:

[0054]

[0055] Where n represents the total number of times the customer consumes during the reference time period, and M represents the amount of the customer's single consumption.

[0056] d) S: refers to the interval between the first transaction and the last transaction of the customer within the reference time, in the following format:

[0057] S=T plast_time -T pfirst_time

[0058] Where T plast_time Indicates the last order transaction time of the customer for the product within the reference time period, T pfirst_time Indicates the time when the customer placed the first order for the product within the reference time period.

[0059] e) T: is the average transaction time interval of the customer in a certain period of time, in the following form:

[0060]

[0061] There are many methods for processing unbalanced samples after preprocessing and feature extraction. The present invention proposes to use the SMOTE-ENN method. The advantage of using the SMOTE-ENN method is that it has a good effect on the binary classification problem with only a small number of positive samples. After comparing different methods, the SMOTE-ENN method performs better. The steps of the SMOTE-ENN method are as follows:

[0062] (1)SMOTE method (Synthetic Minority Oversampling Technique):

[0063] Let A represent the minority class, and any X i ∈A, calculate the distance from it to all samples in the minority class sample set A using the Euclidean distance as the standard, and get X i k nearest neighbor samples, randomly select a sample from this nearest neighbor sample, which is X ij (j=1,2,...,n); in X i With X ij Perform random linear interpolation between (j=1,2,...,n) to construct new minority class samples Y j :

[0064] Y j =X i +rand(0,1)×(X ij -X i )

[0065] In the formula, rand(0,1) represents a random number in the interval (0,1).

[0066] (2) ENN method (Edited KNN)

[0067] Each sample in the dataset ND generated by the SMOTE method is predicted using the K nearest neighbor (K=5) method. If the predicted result is different from the actual category label, the sample is removed. The Euclidean distance is selected as the metric formula of the KNN algorithm, which is as follows:

[0068]

[0069] In the formula, x and y represent two different users respectively, and i represents the feature number.

[0070] The hyperparameter configuration space of the specified weak classifier is specified, and the TPE optimization algorithm is used on the sample set constructed by the SMOTE-ENN method to iteratively optimize the parameter space of the specified weak classifier. The optimization formula is:

[0071] x * = arg min x∈χ F(x)

[0072] Where F(x) represents the objective function of the weak learner; x * is the parameter when F(x) achieves the best result.

[0073] The TPE algorithm density is defined as:

[0074]

[0075] In the formula, l(x) is the observed value {x i The objective function F(x) of} is less than y * The density of g(x) is composed of the observed values ​​{x i The objective function F(x) of} is greater than or equal to y * The density composition of y * As the quantile γ of the observed value y, the expected improvement (EI) is obtained as:

[0076]

[0077] The output of the random forest model is the average of the probabilities of all decision trees, which is expressed mathematically as follows:

[0078]

[0079] Among them, N tree is the total number of decision trees, h i is the i-th decision tree, and P(x|y) represents the probability of predicting that sample x belongs to category y.

[0080] The LightGBM model also uses the above method to output classification probabilities.

[0081] The RF model and the Light GBM model are given the same weight, and the soft voting method is used to integrate them based on their prediction probabilities. The mathematical expression is:

[0082] P Soft Voting =(P RF +P LightGBM ) / 2

[0083]

[0084] Among them, P Soft Voting refers to the predicted probability of the soft voting fusion model, P RF ,P LightGBMRepresent the prediction probabilities of the random forest and LightGBM models respectively, Result represents the prediction result of the fusion model, 1 represents a repeat purchase user, and 0 represents a non-repeat purchase user. According to the test, the threshold of the present invention is set to 0.5, the prediction label is 1 when it is greater than 0.5, and the prediction label is 0 when it is less than 0.5, and the prediction matrix is ​​obtained.

[0085]

[0086] The above can realize the prediction of customer repurchase behavior.

[0087] According to the final result, the product information is pushed to the user's terminal device and / or the repurchase behavior prediction result is sent to the management system.

[0088] Performance metric of the present invention: The current algorithm uses precision P, recall R and F1 value as evaluation indicators. The above indicators are calculated by implementing the data preprocessing method of the present invention, and the evaluation indicators are calculated using the above-obtained label matrix. The calculation formula is:

[0089]

[0090]

[0091]

[0092] The present invention performs well in the multi-channel marketing process of enterprises in non-contractual scenarios. Taking supermarket telemarketing as an example, after the application of the system, the telemarketing conversion rate can be greatly improved, which can promote the generation of more transactions. For enterprises, it can achieve the effects of improving marketing orientation, increasing sales success rate, increasing order volume and transaction amount, and reducing personnel costs. The performance on the data set is as follows: (1) On the training set generated by SMOTE-ENN, the model prediction accuracy is 98.73%, the recall rate is 99.09%, and the F1 value is 0.9874; (2) On the validation set composed of real samples, the model prediction accuracy is 87.13%, the recall rate is 95.15%, and the F1 value is 0.8587; (3) The results are better than the prediction performance of RF and LightGBM single models.

[0093] The present invention improves the classic RFM model to extract user behavior features from the displayed feedback of customer historical purchase records to form a sample set, thereby solving the problem of the lack of a large amount of implicit feedback in non-contractual scenarios in the prior art; the present invention effectively solves the problem of unbalanced data categories in the data set in the prior art through the SMOTE-ENNN sample balancing method; the results of the implementation examples show that the method of the present invention has good prediction performance and practical application value.

Claims

1. A customer repurchase prediction method based on RF-LightGBM fusion model in non-contract scenarios, characterized by: The steps include: The user's historical purchase record data is obtained, preprocessed, and features are extracted. The extracted features include: the most recent purchase time (R), purchase frequency (F), total purchase amount (M), relationship duration (S), and purchase interval (T), where: Last Purchase Time (R): The time when the customer last purchased this product, in the following format: R=T last_time -T plast_time Where T lass_time Indicates the end time of the reference time period, T plast_time Indicates the time when the customer placed the last order for the product within the reference time period; Purchase frequency (F): the number of times a customer purchases this product during the observation period; Total purchase amount (M): The total amount of the customer's purchase of this product, in the following format: Where n represents the total number of times the customer consumes in the reference time period, M i Indicates the amount of the customer's i-th consumption; Relationship duration (S): refers to the interval between the first transaction and the last transaction of a customer within the reference time, in the following format: S=T plast_time -T pfirst_time Where T plast_time Indicates the last order transaction time of the customer for the product within the reference time period, T pfirst_time Indicates the time when the customer placed the first order for the product within the reference time period; Purchase interval (T): the average transaction time interval of a customer in a certain period of time, in the following form: The sample balancing method is used to balance the data after feature extraction to obtain balanced samples, specifically including: first using the SMOTE oversampling method to generate minority class samples of the extracted features, specifically including: let A represent the minority class, and take any X i ∈A, calculate the distance from it to all samples in the minority class sample set A using the Euclidean distance as the standard, and get X i k nearest neighbor samples, randomly select a sample from this nearest neighbor sample, which is X ij , where j = 1, 2, ..., n; in X i With X ij Perform random linear interpolation between them to construct new minority class samples Y j : Y j =X i +rand(0,1)×(X ij -X i ) In the formula, rand(0,1) represents a random number in the interval (0,1); Then use the Edited KNN method to judge the generated samples. If the predicted result is different from the actual category label, the sample is removed to obtain a balanced sample. Specifically, the K nearest neighbor method is used to predict each sample in the data set ND generated by the SMOTE method. If the predicted result is different from the actual category label, the sample is removed. The Euclidean distance is used as the measurement formula of the KNN algorithm, which is as follows: In the formula, x and y represent two different users, and i represents the feature number; The TPE tree-like Parzen estimation optimization algorithm is used to optimize the model hyperparameters. The training sample data is trained under the optimal hyperparameters, and the weak classifier is iteratively optimized in the specified weak classifier hyperparameter space. The optimization formula is: x * =arg min x∈χ F(x) Where F(x) represents the objective function of the weak learner; x * is the parameter when F(x) achieves the best result; The TPE algorithm density is defined as: In the formula, l(x) is the observed value {x i The objective function F(x) of} is less than y * The density of g(x) is composed of the observed values ​​{x i The objective function F(x) of} is greater than or equal to y * The density composition is expressed as y * As the quantile γ of the observed value y, the expected improvement in EI is: The same weights are given to the weak classifiers including the random forest RF model and the Light GBM model. The output results of the weak classifiers are all classification probability values. The mathematical expression is: Where N tree is the total number of decision trees, h i is the i-th decision tree, P(x|y) represents the probability of predicting that sample x belongs to category y; The soft voting method is used to integrate the prediction probabilities of each weak classifier, and its mathematical expression is: P soft Voting =(P RF +P LightGBM ) / 2 Among them, P SoftVoting refers to the predicted probability of the soft voting fusion model, P RF， P LightGBM They represent the prediction probabilities of the random forest and LightGBM models respectively. Result represents the prediction result of the fusion model. 1 represents a repeat purchase user, 0 represents a non-repurchase user, and threshold represents the classification threshold. A strong classifier is obtained through the above ensemble learning; Use strong classifiers to make predictions and obtain the final results on product recommendations and repeat purchase behavior predictions; According to the final result, push product information to the user's terminal device and / or send the repurchase behavior prediction result to the management system; Use repurchase behavior prediction and repurchase probability prediction as a guide for product recommendations.

2. A customer repurchase prediction device based on the RF-LightGBM fusion model in a non-contract scenario, the device is based on the customer repurchase prediction method based on the RF-LightGBM fusion model in a non-contract scenario according to claim 1, characterized in that: include: The acquisition module obtains the user's historical purchase record data, preprocesses it, and extracts features; The balancing module uses the sample balancing method to balance the data after feature extraction to obtain balanced samples; The optimization training module uses the optimization algorithm to train the training sample data and iteratively optimizes the weak classifier in the specified weak classifier hyperparameter space; The ensemble learning module assigns the same weight to each weak classifier and performs ensemble learning to obtain a strong classifier. The prediction module uses a strong classifier to make predictions and obtain the final results on product recommendations and repurchase behavior predictions; The push module pushes product information to the user's terminal device according to the final result.

3. An electronic device, characterized in that: including a processor and a memory; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to thereby execute the customer repurchase prediction method based on the RF-LightGBM fusion model in a non-contractual scenario as described in claim 1.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, it implements the customer repurchase prediction method based on the RF-LightGBM fusion model in the non-contractual scenario as described in claim 1.

Images