E-commerce user re-purchase behavior prediction method and system based on improved algorithm

Through the improved XGBoost algorithm and feature engineering, the problems of traditional models in data processing and complexity balance are solved, the accuracy and efficiency of e-commerce users' repurchase behavior prediction are improved, and more accurate user demand analysis and personalized marketing are achieved. , promotes user repurchase.

CN118941326AInactive Publication Date: 2024-11-12QINGDAO UNIV OF SCI & TECH

Patent Information

Application Number
CN202411112557.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The traditional e-commerce user repurchase behavior prediction model is difficult to balance data processing capabilities, model complexity and interpretability, resulting in low prediction accuracy and efficiency, and cannot effectively improve user satisfaction and repurchase rate.

Method used

Using the improved XGBoost algorithm, we optimize model parameters through data preprocessing, feature engineering and grid search, and build an efficient user repurchase behavior prediction model, combining cross-verification and feature selection to improve the model's prediction performance and generalization capabilities.

Benefits of technology

It significantly improves the accuracy and efficiency of user repurchase behavior prediction, achieves more accurate user demand analysis and personalized marketing, and improves user satisfaction and repurchase rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941326A_ABST
    Figure CN118941326A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of e-commerce, and discloses an e-commerce user re-purchase behavior prediction method and system based on an improved algorithm. Preprocessing the data; for three different subjects including different users, different merchants and different subjects to different merchants by different users, feature dimensions are increased by using field construction features, features related to re-purchase behaviors are obtained, and the features related to the re-purchase behaviors include different user features, different merchant features and interaction features between different users and different merchants; the selected characteristics related to the re-purchase behavior are utilized, the improved XGBoost algorithm is utilized as a prediction model, parameter selection is carried out through grid search, the prediction model is trained and evaluated, and an AUC score is obtained. The method aims at improving the accuracy and efficiency of user re-purchase behavior prediction by introducing and improving the advanced machine learning algorithm XGBoost, and the prediction efficiency of the user re-purchase behavior is improved. The method is expected to provide more accurate decision support for enterprises.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of e-commerce, and in particular relates to a method and system for predicting repurchase behavior of e-commerce users based on an improved algorithm. Background Art

[0002] Accurately predicting and identifying potential repeat purchasers has become the key to improving user stickiness and revenue. By leveraging big data and advanced algorithmic techniques, predictive models can be built to accurately identify which users are likely to become repeat purchasers of merchants or platforms. These models are based on historical behavior data of users on the platform, such as browsing history, purchase history, search history, and evaluation, and are trained and optimized through deep learning and machine learning algorithms. Once the predictive model is built, it can verify its feasibility in a relatively short period of time, and continuously update and iterate as new data is generated to improve the accuracy of predictions. In this way, merchants and platforms can formulate more accurate marketing strategies based on the prediction results, and focus resources on users who are most likely to repurchase, thereby improving marketing efficiency and reducing marketing costs.

[0003] The repurchase rate is an important indicator to measure consumers' loyalty to a brand or product. For e-commerce platforms and merchants, increasing the repurchase rate is crucial to improving user stickiness and long-term profits. There are two common statistical methods: one is the repurchase rate based on the number of repurchasers. The other is the repurchase rate based on the number of purchases.

[0004] XGBoost is an efficient gradient boosting decision tree algorithm. The goal of XGBoost is to build K regression trees so that the predicted value of the tree group is as close to the true value as possible. It has a wider range of applications, and by adding a regularization term to its objective function, the complexity of the model can be controlled. In addition, XGBoost also draws on the column sampling method of random forest, thereby reducing the variance and overfitting of the model.

[0005] The prior art also proposes a prediction method based on Bayesian optimization of XGBoost. Through three hyperparameter optimization methods such as data preprocessing, XGBoost training and Bayesian, the experiment shows that Bayesian optimization has the best effect and the model accuracy is improved by 4.84%. The prior art adopts the GA-XGBoost model, combined with Pearson analysis, to optimize the model parameters and accurately predict grain production. Simulation shows that the GA-XGBoost model has higher prediction accuracy, with an RMSE of 0.034, which is suitable for grain production prediction. The prior art constructs a personal auto finance default risk assessment model for S Bank based on the XGBoost model and particle swarm algorithm (PSO). By optimizing model parameters and expanding the indicator system, the accuracy of default risk assessment is improved. Empirical results show that the optimized model is better than other models, which provides help for improving the personal auto finance risk control system. The prior art combines GBDT and Xgboost models through integrated learning methods and feature engineering, and optimizes their parameters to improve the accuracy and generalization ability of coupon usage prediction. By integrating multiple models, experimental results show that the fusion model significantly improves the accuracy and generalization in predicting coupon usage.

[0006] Through the above analysis, the problems and defects of the existing technology are as follows: traditional prediction models are often limited by the balance between data processing capabilities, model complexity and interpretability, and the accuracy and efficiency of predicting user repurchase behavior are low, the prediction time is long, and it cannot effectively improve user satisfaction and repurchase rate. Summary of the invention

[0007] In order to overcome the problems existing in the related art, the disclosed embodiments of the present invention provide a method and system for predicting the repurchase behavior of e-commerce users based on an improved algorithm, and specifically discloses a method for predicting the repurchase behavior of e-commerce users based on an improved XGBoost algorithm.

[0008] The technical solution is as follows: a method for predicting repurchase behavior of e-commerce users based on an improved algorithm, comprising:

[0009] S1, data preprocessing: data cleaning, data segmentation, data type conversion, missing value processing, outlier processing and timestamp data processing for e-commerce users’ repurchase behavior;

[0010] S2, for different users, different merchants, and different users to different merchants, three different subjects, use the existing fields to construct features to increase feature dimensions, and obtain features related to repurchase behavior. Features related to repurchase behavior include different user features, different merchant features, and interaction features between different users and different merchants;

[0011] S3, using the selected features related to repurchase behavior, the improved XGBoost algorithm is used as the prediction model, and the prediction model is trained and evaluated through parameter selection through grid search, and the AUC score is obtained.

[0012] In step S1, data cleaning is used to remove duplicate, erroneous or irrelevant information;

[0013] Data partitioning includes: extracting features and labels from the original data set data, the feature set X is obtained by deleting the label column label from data, and the label set y is directly extracted from the label column label column of data; dividing the data set into training set, validation set and test set, using the train_test_split function to split the data set twice, including: dividing the original data set into 80% training set and 20% temporary sets X_temp and y_temp, and then dividing the temporary set into 50% validation sets X_valid and y_valid, and 50% test sets X_test and y_test;

[0014] The data type conversion includes: after dividing the data set, the label sets y_train, y_valid, and y_test contain string type labels, and these string labels are converted to integer labels using the defined convert_to_int function, and the label values ​​that are not 0.0 or 1.0 are converted to integers 0 or 1.

[0015] In step S1, missing value processing, gaps in the data are handled by interpolation or deletion methods;

[0016] Outlier processing, identifying and processing extreme values ​​that are significantly different from the overall data distribution;

[0017] The processing of timestamp data includes: using the pd.to_datetime function to convert the time_stamp column from a string to a datetime type. During the conversion process, the missing value of gender is replaced with 2, and the data type of several columns in the all_data DataFrame is converted. The info() method is used to view the detailed information of the converted DataFrame, including the column data type and the number of non-null values.

[0018] In step S2, the feature dimensions include user dimensions, merchant dimensions, and interaction dimensions between different users and different merchants;

[0019] User dimensions include the user's purchase history, browsing behavior, and preferences.

[0020] The merchant dimension includes the merchant’s product quality and sales time factors;

[0021] The interaction dimensions between different users and different merchants include the users' purchase frequency.

[0022] In step S3, the improved XGBoost algorithm is used as the prediction model, and the prediction model is trained and evaluated through grid search for parameter selection, and the AUC score is obtained, including:

[0023] (1) According to the data situation, select the following parameters: max_depth, learning_rate, n_estimators, min_child_weight, subsample, colsample_bytree;

[0024] (2) Use grid search to adjust parameters and find the best parameter combination;

[0025] (3) Initialize the XGBoost classifier object model and set the objective function to binary logistic regression and a fixed random seed;

[0026] (4) Use the GridSearchCV object grid_search to perform grid search; use the defined model as the estimator, param_grid as the parameter grid, use ROC AUC as the scoring indicator, and adopt 3-fold cross validation; search for the best parameter combination on the training data by calling grid_search.fit(X_train,y_train); retrain the prediction model using the best parameter combination and store it in the best_model object; fit the prediction model again on the training data, use best_model to predict the validation set and test set, and calculate the corresponding ROC AUC scores.

[0027] In step (2), the parameters include the maximum depth of the tree, the learning rate, the number of base learners, the minimum sum of sample weights required on the leaf nodes, the sample subset ratio and the feature subset ratio; the grid search is used to adjust the parameters and find the best parameter combination including:

[0028] (2.1) Define parameter grid;

[0029] (2.2) Select the scoring criteria: select roc_auc;

[0030] (2.3) Set cross validation: 3-fold cross validation: cv=3;

[0031] (2.4) Initialize the grid using GridSearchCV;

[0032] (2.5) Perform grid search, grid_search.fit(X_train,y_train);

[0033] (2.6) Evaluate the results and use best_params_ and best_score_ to output the best parameter combination and the best score;

[0034] print("Best parameters found:",grid_search.best_params_);

[0035] print("Best ROC AUC score found:",grid_search.best_score_);

[0036] (2.7) Train the prediction model using the optimal parameters.

[0037] In step (3), the initialization includes:

[0038] model=xgb.XGBClassifier(objective='binary:logistic',random_state=42)

[0039] Use logistic regression as the objective function, and random_state=42 to set the seed value of the random number generator to ensure consistent results each time the prediction model is trained using the same parameters.

[0040] In step (4), a grid search is performed using the GridSearchCV object grid_search, including:

[0041] (401) Instantiate GridSearchCV and pass in the necessary parameters:

[0042] (401-1) estimator: The model to be optimized defines the model to initialize the XGBoost classifier:

[0043] model=xgb.XGBClassifier(objective='binary:logistic',random_state=42)

[0044] (401-2) param_grid: contains the parameter name to be traversed and the corresponding value list, defined as a parameter grid;

[0045] (401-3) scoring is the scoring indicator used for model evaluation, which is roc_auc;

[0046] (401-4)cv: the number of cross-validation folds, set to 3;

[0047] (402) Fitting GridSearchCV: grid_search.fit(X_train,y_train); When the fit method is called, GridSearchCV will perform:

[0048] (402-1) Parameter grid traversal, traversing all parameter combinations defined in param_grid;

[0049] (402-2) cross validation;

[0050] (402-3) Model training: train the model using the current parameter combination on each training set;

[0051] (402-4) Performance evaluation: Evaluate the model performance on each training set and calculate the score using the scoring metric specified by scoring;

[0052] (402-4) Record the results;

[0053] (402-6) Select optimal parameters.

[0054] In step (2.3), a 3-fold cross validation is used, including:

[0055] (b) The dataset is first randomly divided into three subsets of equal size or as close to equal size as possible;

[0056] (b) For each parameter combination in the parameter grid: (b-1) perform 3 training and validation cycles; (b-2) in each cycle, select one of the subsets as the validation set, and the remaining two subsets are combined as the validation set; (b-3) train the model on the training set using the current parameter combination; (b-4) evaluate the model on the validation set and calculate the ROCAUC score;

[0057] (c) After each validation, the ROC AUC score of the current parameter combination is recorded;

[0058] (d) In this way, each parameter combination obtains a ROCAUC list with a length equal to the number of cross-validation folds;

[0059] (e) Calculate the average ROCAUC: For each parameter combination, calculate the average of its ROCAUC scores in all three validations;

[0060] In the calculation of the corresponding ROC AUC score, roc_auc_score is used to calculate the score, y_valid is the true label of the validation set, which is 0 or 1, and y_valid_pred is the probability that the validation set predicted by the model belongs to the positive class; specifically, it includes:

[0061] (A) Determine the validation set;

[0062] (B) Determine the test set;

[0063] (C) TRR is used to represent the proportion of positive instances correctly predicted by the model as positive, and the calculation formula is: TP / (TP+FN), where TP is the number of positive instances and FN is the number of false instances; FRR is used to represent the proportion of negative instances incorrectly predicted as positive by the model, and the calculation formula is: FP / (TN+FP), where FP is the number of false positive instances and TN is the number of true instances;

[0064] (D) Draw the ROC curve: Use different thresholds, with TRR as the ordinate and FRR as the abscissa, draw a series of points, and then connect them to draw the ROC curve;

[0065] (E) AUC score is the area under the ROC curve.

[0066] Another object of the present invention is to provide an e-commerce user repurchase behavior prediction system based on an improved algorithm, the system implements the e-commerce user repurchase behavior prediction method based on the improved algorithm, and the system comprises:

[0067] The data preprocessing module is used to clean the data, divide the data, convert the data type, process missing values, process outliers, and process timestamp data for the repurchase behavior of e-commerce users;

[0068] The feature acquisition module related to repurchase behavior is used to construct features using the existing fields to increase feature dimensions for three different subjects: different users, different merchants, and different users to different merchants, and obtain features related to repurchase behavior. The features related to repurchase behavior include different user features, different merchant features, and interaction features between different users and different merchants;

[0069] The improved XGBoost algorithm prediction module is used to utilize the selected features related to repurchase behavior, use the improved XGBoost algorithm as the prediction model, and train and evaluate the prediction model through parameter selection through grid search to obtain the AUC score.

[0070] Combining all the above technical solutions, the beneficial effects of the present invention are as follows: In the field of e-commerce, the prediction of user repurchase behavior is the key for enterprises to improve customer stickiness and optimize marketing strategies. However, traditional prediction models are often limited by the balance between data processing capabilities, model complexity and interpretability. The present invention aims to improve the accuracy and efficiency of user repurchase behavior prediction by introducing an advanced machine learning algorithm, XGBoost, and improving it, in order to provide enterprises with more accurate decision support.

[0071] The present invention first clarifies the research goal, that is, to use the improved XGBoost algorithm to build an effective user repurchase behavior prediction model. By collecting and organizing user behavior data of the e-commerce platform, including purchase history, browsing history, user attributes, etc., as the input features of the model. The improved XGBoost algorithm is used for feature selection and model training, and the model parameters are optimized by combining cross-validation and grid search. After experiments, it is found that the model based on the improved XGBoost shows significant advantages in predicting user repurchase behavior. The improved XGBoost model studied by the present invention has improved prediction accuracy while greatly reducing prediction time, and has better prediction performance and generalization ability in predicting e-commerce user repurchase behavior. The research results of the present invention have practical application value for e-commerce companies, and can help them accurately predict users' repurchase behavior, realize personalized marketing, and improve user satisfaction and repurchase rate.

[0072] The present invention can achieve precision marketing, and e-commerce platforms and merchants can more accurately understand the needs and preferences of online users. This in-depth analysis enables the platform to provide users with highly personalized product recommendations, coupon distribution and message push. Precision marketing not only improves the efficiency and effectiveness of marketing activities, but also enhances user experience and helps cultivate user loyalty.

[0073] The present invention can enhance user experience and loyalty. The user experience of e-commerce platforms is an important factor affecting user loyalty and word of mouth. E-commerce platforms and merchants can deeply analyze users' shopping preferences. For example, they can recommend similar items based on the browsing clicks and purchasing behaviors of some target users, or provide customized shopping experiences and promotions. These measures help improve user satisfaction and loyalty, and promote user repurchase and dissemination. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure;

[0075] Figure 1 is a flow chart of a method for predicting repurchase behavior of e-commerce users based on an improved algorithm provided by an embodiment of the present invention;

[0076] Figure 2 is a flow chart of a regression algorithm provided by an embodiment of the present invention;

[0077] Figure 3 is a flow chart of a binary classification algorithm provided by an embodiment of the present invention;

[0078] Figure 4 is a schematic diagram of an e-commerce user repurchase behavior prediction system based on an improved algorithm provided by an embodiment of the present invention;

[0079] In the figure: 1. Data preprocessing module; 2. Feature acquisition module related to repurchase behavior; 3. Improved XGBoost algorithm prediction module. DETAILED DESCRIPTION

[0080] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.

[0081] The innovation of the present invention is that it uses deep learning technology to comprehensively analyze user behavior data, thus realizing personalized and real-time repurchase prediction. The system continuously optimizes the prediction model through an adaptive learning mechanism to adapt to changes in user behavior while ensuring user privacy and data security.

[0082] Embodiment 1, as Figure 1 As shown, the e-commerce user repurchase behavior prediction method based on the improved algorithm provided in the embodiment of the present invention includes:

[0083] S1, data preprocessing: data cleaning, data segmentation, data type conversion, missing value processing, outlier processing and timestamp data processing for e-commerce users’ repurchase behavior;

[0084] S2, for different users, different merchants, and different users to different merchants, three different subjects, use the existing fields to construct features to increase feature dimensions, and obtain features related to repurchase behavior. Features related to repurchase behavior include different user features, different merchant features, and interaction features between different users and different merchants;

[0085] User characteristics: the total number of interactions performed by each user, the number of different products viewed by each user, the number of different product categories viewed by each user, the number of different merchants interacted with by each user, the number of different brands interacted with by each user, the time interval between the first and last interaction of each user, the number of products clicked by each user, the number of items added to the shopping cart by each user, the number of items purchased by each user, and the number of items added to favorites by each user.

[0086] Merchant characteristics: the number of interactions with the merchant, the unique number of different user_ids that interact with the merchant, the unique number of different item_ids that interact with the merchant, the unique number of different cat_ids that interact with the merchant, and the unique number of different brand_ids that interact with the merchant;

[0087] Characteristics of different users towards merchants: the number of times users interact with different merchants, the unique number of different item_ids that users interact with at different merchants, the unique number of different cat_ids that users interact with at different merchants, the unique number of different brand_ids that users interact with at different merchants, the unique number of action_type values ​​of 0 that users interact with at different merchants, the unique number of action_type values ​​of 1 that users interact with at different merchants, the unique number of action_type values ​​of 2 that users interact with at different merchants, the unique number of action_type values ​​of 3 that users interact with at different merchants;

[0088] S3, using the selected features related to repurchase behavior, the improved XGBoost algorithm is used as the prediction model, and the prediction model is trained and evaluated through parameter selection through grid search, and the AUC score is obtained.

[0089] Example 2, in order to further analyze in detail the e-commerce user repurchase behavior prediction method based on the improved algorithm of the present invention, the following is explained.

[0090] (a) Data analysis

[0091] 1. Specific process of data preprocessing.

[0092] Use the pd.concat method to merge the train_data and test_data DataFrames row by row into a new DataFrame all_data. Connect the all_dataDataFrame to the user_infoDataFrame using the user_id column they share as the connection condition.

[0093] pd.concat is a function in the pandas library in Python that is used to merge multiple DataFrame objects into one DataFrame by rows or columns. This is a common data processing technique that is widely used in the field of data science and analysis.

[0094] The following is a simple description of using pd.concat to merge the two DataFrames train_data and test_data into a new DataFrame all_data by row:

[0095] 1. Import the pandas library: import pandas as pd

[0096] 2. Prepare DataFrame: Make sure train_data and test_data are pandas DataFrame objects.

[0097] 3. Use pd.concat to merge DataFrame: Use the pd.concat function to merge two DataFrames row by row.

[0098] all_data=pd.concat([train_data,test_data],axis=0)

[0099] 4. Result: all_data will be a new DataFrame containing all the rows of train_data and test_data.

[0100] In the current scenario, in order to be consistent with the merchant ID field in the user log table, the seller_id field in the training set is renamed to merchant_id. This ensures that the name of the merchant ID field is consistent during subsequent data connection or integration, thus avoiding confusion and errors.

[0101] At the same time, since date fields are often used for time series analysis, periodic trend detection, or building date-related features, it is very useful to split the purchase date field into two separate fields, month and day. This allows for more flexibility in building features for months or days, such as monthly sales, mid-week purchase patterns, etc.

[0102] Checks if several specified columns in user_log contain missing values ​​(i.e. NaN values) and if so, fills these missing values ​​with 0. A loop is defined that iterates over a list of column names. Checks if the column contains NaN values. The .fillna(0) function is used to fill missing values ​​and replaces all NaN values ​​in the column with 0.

[0103] In the process of data processing and analysis, data type conversion and missing value processing are two crucial steps. For the columns of 'user_id', 'merchant_id', 'item_id', 'cat_id', and 'action_type' in the user behavior dataset, the accuracy of the data type is crucial for subsequent analysis and modeling. Therefore, the data types of these columns are converted from possible default types (such as object or float64) to int32, because user ID, merchant ID, product ID, category ID, and action type should usually be integer types.

[0104] Convert data type: Use astype() function to convert the data type of a particular column to int32.

[0105] df['user_id']=df['user_id'].astype('int32')

[0106] df["merchant_id']=df["merchant_id'].astype('int32')

[0107] df['item_id']=df['item_id'].astype('int32')

[0108] df['cat_id']=df['cat_id'].astype('int32')

[0109] df['action_type']=df['action_type'].astype('int32')

[0110] In terms of missing value processing, the possible NaN values ​​in the 'brand_id' column need to be properly handled. In order to avoid introducing misleading information, placeholders are chosen to replace these missing values.

[0111] 1. Import the pandas library: import pandas as pd

[0112] 2. Identify missing values: missing_values ​​= df.isna()

[0113] 3. Select a placeholder

[0114] 4. Determine a suitable placeholder to replace the missing value. This can be any value, but is usually a specific numeric value or string, such as -1, 0, 'missing', or np.nan.

[0115] 5. Replace missing values

[0116] Use the fillna() function to replace missing values ​​with the chosen placeholders

[0117] 6. df_filled = df.fillna(0)

[0118] It should be noted that using 0 as a placeholder may not be a good choice if 0 is a valid brand ID. In this case, it is recommended to use a dedicated placeholder (such as -1) to replace the NaN value to ensure data accuracy and reliability.

[0119] The processing of timestamp data is also important for user behavior analysis. In this example, the pd.to_datetime function is used to convert the 'time_stamp' column from a string to a datetime type, including:

[0120] 1. Import the Pandas library: import pandas as pd

[0121] 2. Read data

[0122] 3. Convert column type:

[0123] Convert the strings in the 'time_stamp' column to datetime type using the pd.to_datetime function.

[0124] You can call this function directly on a DataFrame and specify the column names:

[0125] df['time_stamp']=pd.to_datetime(df['time_stamp'])

[0126] This line of code will convert each string element of the 'time_stamp' column in df (your DataFrame variable) to a datetime object.

[0127] In order to perform time-related analysis and visualization. During the conversion process, you need to pay attention to the format of the timestamp. Replace the missing value of gender with 2, which means the gender is unknown. The data types of several columns in the all_data DataFrame are converted to optimize storage and improve the efficiency of subsequent data analysis or model training. The detailed information of the converted DataFrame is viewed using the info() method. In Python's pandas library, the info() method is used to provide summary information of the DataFrame, including the data type of the column, the number of non-null values, etc.

[0128] Following is a description of the steps to use the info() method to view detailed information about the transformed DataFrame:

[0129] 1. Import the pandas library: import pandas as pd

[0130] 2. Create or convert a DataFrame, assuming that all_data is the DataFrame previously merged by pd.concat.

[0131] 3. Use the info() method: Use the info() method to view detailed information of all_data. all_data.info()

[0132] 4. Output results: The info() method prints out the column name, data type, number of non-null values, and other information of the DataFrame. This helps to quickly understand the structure and data quality of the DataFrame.

[0133] 5. Possible output content:

[0134] Data columns: Displays the data column names in the DataFrame.

[0135] Non-null values: Displays the number of non-null values ​​in each column.

[0136] Dtypes: Displays the data type of each column.

[0137] Include the data type of the column, the number of non-null values, etc. to ensure that the conversion is correct and there are no unexpected missing values ​​or data type mismatches in the DataFrame. See Table 1 for the basic data.

[0138] Table 1 Data type conversion

[0139]

[0140] 2. Feature engineering.

[0141] 2.1 Construct features.

[0142] The basic features are shown in the table.

[0144] Table 2 Basic characteristics

[0145]

[0146] User characteristics are shown in Table 3.

[0147] Table 3 User characteristics

[0148]

[0149] The merchant characteristics are shown in Table 4.

[0150] Table 4 Merchant characteristics

[0151]

[0152]

[0153] The merchant characteristics of different users are shown in Table 5.

[0154] Table 5 Characteristics of merchants by different users

[0155]

[0156] 3. Data Modeling Methods

[0157] 3.1 Evaluation method

[0158] When training a model and making predictions, the goal is to accurately predict unknown data in practical applications. To prevent overfitting, the data set needs to be divided. There are several methods: leave-out method, cross-validation method, stratified cross-validation, and bootstrap method.

[0159] 3.2 Classifier Evaluation Indicators, ROC curves are obtained by continuously adjusting the decision threshold of the classifier to obtain a series of different binary classification results, and then the curve is drawn based on these results. The AUC value is the area under the ROC curve and the coordinate axis, and its value range is 0.5-1.

[0160] 4.XGBOOST algorithm.

[0161] 4.1XGBoost algorithm principle.

[0162] XGBoost is an optimization algorithm based on gradient boosting decision trees. Since its birth, XGBoost has received widespread attention and application in the fields of statistics, data mining and machine learning, especially showing significant performance advantages when dealing with large-scale data sets and complex models. XGBoost adopts a "divide and conquer" strategy to split large-scale data sets into small pieces, and uses parallelization technology to execute in parallel on multi-core CPUs or GPUs, greatly reducing training time. The implementation of this parallel computing mainly relies on the column-major storage data structure, which stores data by column instead of by row, so that when processing sparse data, missing values ​​can be skipped efficiently, memory access overhead can be reduced, and parallel computing is facilitated. Compared with traditional row-by-row storage, this not only improves the data reading speed, but also makes XGBoost still efficient when running on large-scale data sets. XGBoost introduces a second-order Taylor expansion to approximate the loss function, which not only makes the calculation of the loss function more accurate, but also reduces the number of iterations of model training. Through the optimized second-order Taylor series, XGBoost can more accurately estimate the gradient and Hessian matrix of the loss function, so as to update the model parameters more accurately at each iteration and improve the convergence speed of the model. Regularization is introduced to prevent overfitting and ensure the generalization ability of the model. It uses L1 and L2 regularization to constrain the complexity of the model. L1 regularization makes the weights of some tree nodes zero, achieving the effect of feature selection; while L2 regularization prevents overfitting caused by excessive weights by penalizing model weights. By balancing model complexity and fit, XGBoost can avoid the problem of overfitting of the model while ensuring prediction accuracy. In the model initialization stage, XGBoost usually builds a baseline model, and then updates a weak classifier each time through iteration to minimize the loss function. The design of weak classifiers usually adopts a tree structure, where each node represents a test of a feature value, and each leaf node is a predicted value. Through continuous iteration, XGBoost gradually optimizes the model so that each newly added decision tree contributes to reducing the overall prediction error. XGBoost also has a series of practical features, such as built-in cross-validation, missing value processing, and flexible programming interfaces, allowing users to easily adjust and optimize models in practical applications. It can exert its powerful power both in academic and industrial production. XGBoost achieves significant improvement in computational efficiency and reduction in overfitting risk while ensuring model prediction performance through parallel processing, efficient data structures, accurate loss function approximation, regularization, and a series of practical features. These basic principles lay a solid foundation for parameter tuning and comparison with other algorithms in subsequent chapters, and also provide theoretical support for the wide application of XGBoost in the field of machine learning.

[0163] 4.2 XGBoost Application Fields XGBoost has shown great potential in many application fields with its high efficiency and flexibility, covering multiple industries such as computer vision, natural language processing, bioinformatics, e-commerce recommendations, etc. In each field, XGBoost has become the tool of choice for data scientists and engineers with its excellent performance and ease of use.

[0164] 4.3 Parameter Tuning,In XGBoost, the main goal of parameter tuning is to find an optimal set of parameter settings that,ensure that the model exhibits excellent performance and good,generalization ability on the training data.,To achieve this goal, it is necessary to have a deep understanding of the,XGBoost parameters and use appropriate methods to tune them.

[0165] Parameters include:

[0166] n_estimators (the number of decision trees) determines the complexity of the model. More trees can provide more accurate predictions, but too high a number of trees may lead to overfitting.

[0167] max_depth (maximum depth of the tree), setting a lower max_depth can prevent overfitting, but a tree that is too shallow may not be enough to capture the complex structure and patterns in the data, affecting the predictive performance of the model. Therefore, careful selection is required to balance the complexity and generalization ability of the model.

[0168] The learning rate controls the magnitude of the model parameter update in each iteration. A smaller learning rate is conducive to convergence, while a larger learning rate may lead to faster convergence but may be unstable.

[0169] min_child_weight (minimum leaf node weight) controls the complexity of the tree. A larger value can prevent too few leaf nodes caused by splitting, thereby preventing overfitting.

[0170] subsample (random sampling rate) and colsample_bytree (column sampling rate) control the sample ratio and feature ratio used in each iteration, respectively, which helps prevent overfitting and increase model diversity.

[0171] reg_alpha (L1 regularization) and reg_lambda (L2 regularization) respectively control the model complexity and prevent overfitting by limiting feature weights.

[0172] Tuning methods usually include grid search and random search. Cross-validation divides the data set into training and validation sets, or uses k-fold cross-validation to more accurately evaluate the performance of the model on unseen data, thereby guiding parameter tuning. In practical applications, parameter tuning is often an iterative process. First, grid search or random search can be used to find preliminary optimization directions in a set of candidate parameters. Subsequently, based on the preliminary results, some key parameters may need to be adjusted more finely, or even more complex optimization algorithms may be used. Throughout the process, it is necessary to always pay attention to the model's performance on the training and validation sets.

[0173] 4.4 Algorithm derivation.

[0174] The biggest difference between XGBoost and GBDT is the definition of the objective function. The basic idea is the same, and both use the additive model and forward step-by-step algorithm to achieve the optimization process of learning.

[0175] The implementation process is as follows.

[0176] The regression algorithm process is as follows Figure 2 , the binary classification algorithm process is as follows Figure 3 .

[0177] 1. Initialization

[0178] Before building the first tree, we first need to initialize the model's predictions. Typically, this can be done by calculating the mean of the training data.

[0179] 2. Calculate first and second derivatives

[0180] For each sample, the first-order derivative of the loss function with respect to the current prediction value, gigi, and the second-order derivative, hihi, are calculated. These derivatives are used to indicate how to update the model to reduce the loss. For the squared loss function, the first-order derivative and the second-order derivative are:

[0181] 3. Build the first tree

[0182] Use the greedy algorithm to traverse all possible splitting points of each feature, calculate the gain of each splitting point, and select the splitting point with the largest gain as the splitting point of the current node.

[0183] 4. Build the first tree

[0184] Use the greedy algorithm to traverse all possible splitting points of each feature, calculate the gain of each splitting point, and select the splitting point with the largest gain as the splitting point of the current node.

[0185] 5. Iteratively build multiple trees

[0186] After building the first tree, use a similar method to iteratively build more trees. For each new tree:

[0187] Calculate updated first and second derivatives, taking into account the predictions of all previous trees

[0188] Using the updated derivative information, repeat the greedy algorithm in step 3 to build the next tree.

[0189] 4.5 Algorithm Characteristics

[0190] High performance: XGBoost uses a series of optimization techniques, such as parallel processing, cache optimization, and approximate algorithms, to make it faster to train on large-scale data sets.

[0191] Regularization: XGBoost provides regularization parameters to prevent overfitting, including controlling the complexity of the tree and using regularization terms to control the size of the weights. This enables XGBoost to automatically balance the model's fitting ability and generalization ability when building the model, thereby improving the model's generalization performance.

[0192] Customized loss function: XGBoost allows users to customize the loss function to suit different prediction problems. Feature importance evaluation: XGBoost can evaluate the importance of each feature based on the constructed decision tree model, thereby helping users with feature selection and feature engineering.

[0193] (ii) Model construction

[0194] Step 1: Prepare training set, validation set, and test set.

[0195] The present invention fully utilizes the Python programming language and its rich scientific computing libraries to carry out data analysis and machine learning. The core dependent tools include NumPy, which is used to efficiently process large multidimensional arrays and matrix operations; Pandas, which provides convenient data structures and data analysis functions; Matplotlib and Seaborn, which are used for data visualization to help intuitively understand data distribution and characteristics; sklearn (Scikit-learn), a comprehensive machine learning library that includes various classic algorithms; and the XGBoost gradient boosting framework.

[0196] In addition, the features (stored in X) and labels (stored in y) in the original dataset data are extracted. The feature set X is obtained by removing the label column ('label') from data, while the label set y is directly extracted from the 'label' column of data. The dataset is divided into training set, validation set and test set. The train_test_split function is used to split the dataset twice. First, the original dataset is divided into 80% training set and 20% temporary set (X_temp and y_temp). Then, the temporary set is divided into 50% validation set (X_valid and y_valid) and 50% test set (X_test and y_test).

[0197] After partitioning the dataset, notice that the label sets y_train, y_valid, and y_test still contain string-type labels. In order to meet the requirements of the machine learning algorithm, these string labels are converted to integer labels using the previously defined convert_to_int function. This step is implemented by a list comprehension, which iterates over each label in the label set and calls the convert_to_int function for conversion. If the value of the label is not '0.0' or '1.0', through this series of preprocessing steps, a dataset containing a training set, a validation set, and a test set is successfully prepared, where the feature set is of numeric type and the label set is of integer type.

[0198] Exemplarily, calling the convert_to_int function to perform the conversion includes:

[0199]

[0200] The label column data in the training data set is "0.0" or "0.1", so the defined convert_to_int is used to convert this column of data to integers 0 or 1.

[0201] Step 2: XGBoost modeling. The main parameters of XGBoost modeling are shown in Table 6.

[0202] Table 6 XGB parameter introduction diagram

[0203]

[0204]

[0205] (1) According to the data situation, select 6 parameters: max_depth, learning_rate, n_estimators, min_child_weight, subsample, colsample_bytree.

[0206] (2) Then adjust the parameters using grid search. The selection of model parameters is crucial to improving the performance of the model. For a powerful tool like the XGBoost classifier, the predictive ability of the model can be further improved by carefully adjusting its parameters. A dictionary named param_grid is defined, which contains a series of potential parameter values ​​for the XGBoost classifier. These parameters include the maximum depth of the tree (max_depth), the learning rate (learning_rate), the number of base learners (n_estimators), the minimum sum of sample weights required on leaf nodes (min_child_weight), the sample subset ratio (subsample), and the feature subset ratio (colsample_bytree). Each parameter contains a set of possible values, and the best parameter combination will be found through grid search, where the grid search searches for the best parameter setting by traversing all possible combinations in a given parameter grid, which is used for hyperparameter optimization. The method of finding the best parameter combination through grid search includes:

[0207] (2.1) Define parameter grid

[0208] param_grid = {

[0209] 'max_depth':[3,5,7,10],

[0210] 'learning_rate':[0.01,0.05,0.1],

[0211] 'n_estimators':[100,500,1000],

[0212] 'min_child_weight':[1,3,5],

[0213] 'subsample':[0.6,0.8,1.0],

[0214] 'colsample_bytree':[0.6,0.8,1.0],

[0215] }

[0216] (2.2) Select the scoring standard: Select roc_auc

[0217] (2.3) Set cross validation: 3-fold cross validation: cv = 3

[0218] (2.4) Initialize the grid using GridSearchCV:

[0219] grid_search=GridSearchCV(estimator=model,param_grid=param_grid,scoring='roc_auc',cv=3,verbose=2)

[0220] (2.5) Perform grid search:

[0221] grid_search.fit(X_train,y_train)

[0222] (2.6) Evaluation results:

[0223] Use best_params_ and best_score_ to output the best parameter combination and the best score

[0224] print("Best parameters found:",grid_search.best_params_)

[0225] print("Best ROC AUC score found:",grid_search.best_score_)

[0226] (2.7) Train the model using the optimal parameters.

[0227] (3) Next, an XGBoost classifier object model is initialized, and the objective function is set to binary logistic regression (binary:logistic) and a fixed random seed (random_state=42). This random seed ensures the reproducibility of the results during the parameter search process.

[0228] Exemplarily, the initialization includes:

[0229] model=xgb.XGBClassifier(objective='binary:logistic',random_state=42)

[0230] Use logistic regression as the objective function, and set random_state=42 to set the seed value of the random number generator to ensure that the results are consistent each time the model is trained with the same parameters.

[0231] (4) Then, use the GridSearchCV object grid_search to perform a grid search. Use the previously defined model as the estimator, param_grid as the parameter grid, use ROC AUC as the scoring metric (scoring='roc_auc'), and adopt 3-fold cross validation (cv=3). The setting of verbose=2 means that more detailed log information will be output during the search process. By calling grid_search.fit(X_train,y_train), search for the best parameter combination on the training data. This process will take some time because all possible parameter combinations need to be evaluated. Retrain the model using the best parameter combination and store it in the best_model object. By fitting this model again on the training data, then use best_model to predict the validation set and test set and calculate the corresponding ROC AUC scores.

[0232] Exemplarily, the GridSearchCV object grid_search is used to perform a grid search. This function helps to traverse the predefined hyperparameters and fit the model to the training set, so the best parameters can be selected from the listed hyperparameters, i.e., the param_grid dictionary. GridSearchCV tries all combinations of the values ​​passed in the dictionary and evaluates the model for each combination using a cross-validation method. After using GridSearchCV again, the accuracy of each hyperparameter combination is obtained to select the best combination.

[0233] Among them, grid search is used for parameter optimization:

[0234] grid_search=GridSearchCV(estimator=model,param_grid=param_grid,

[0235] scoring='roc_auc',cv=3,verbose=2);

[0236] grid_search.fit(X_train,y_train);

[0237] Specifically include:

[0238] (401) Instantiate GridSearchCV and pass in the necessary parameters:

[0239] (401-1) estimator: The model to be optimized. The model defined here is the initialized XGBoost classifier.

[0240] model=xgb.XGBClassifier(objective='binary:logistic',random_state=42).

[0241] (401-2) param_grid: Contains the parameter name to be traversed and the corresponding value list. Here it is defined as: #parameter grid

[0242] param_grid = {

[0243] 'max_depth':[3,5,7,10],

[0244] 'learning_rate':[0.01,0.05,0.1],

[0245] 'n_estimators':[100,500,1000],

[0246] 'min_child_weight':[1,3,5],

[0247] 'subsample':[0.6,0.8,1.0],

[0248] 'colsample_bytree':[0.6,0.8,1.0],

[0249] }.

[0250] (401-3) scoring is the scoring indicator used for model evaluation, here it is roc_auc.

[0251] (401-4)cv: the number of cross-validation folds, here it is 3.

[0252] (402) Fitting GridSearchCV: grid_search.fit(X_train,y_train). When the fit method is called, GridSearchCV will perform:

[0253] (402-1) Parameter grid traversal, traversing all parameter combinations defined in param_grid.

[0254] (402-2) Cross validation.

[0255] (402-3) Model training: Train the model using the current parameter combination on each training set.

[0256] (402-4) Performance evaluation: Evaluate the model performance on each training set and calculate the score using the scoring metric specified by scoring.

[0257] (402-5) Record the results.

[0258] (402-6) Select optimal parameters.

[0259] Exemplarily, using 3-fold cross validation (cv=3) includes:

[0260] (c) The dataset is first randomly divided into three subsets of equal size or as close to equal size as possible.

[0261] (b) For each parameter combination in the parameter grid: (b-1) Perform 3 training and validation cycles (i.e., cross-validation each time). (b-2) In each cycle, select one of the subsets as the validation set and the remaining two subsets as the validation set. (b-3) Train the model on the training set using the current parameter combination.

[0262] (b-4) Evaluate the model on the validation set and calculate the ROCAUC score.

[0263] Scoring and Recording:

[0264] (c) After each validation, the ROC AUC score of the current parameter combination is recorded;

[0265] (d) In this way, each parameter combination will get a ROCAUC list with a length equal to the number of cross-validation folds.

[0266] (e) Calculate the average ROCAUC: For each parameter combination, calculate the average of its ROCAUC scores in all 3 validations.

[0267] Exemplarily, retraining the model using the best parameter combination includes:

[0268] best_model=grid_search.best_estimator_;

[0269] best_model.fit(X_train,y_train);

[0270] grid_search.best_estimator_ obtains the model instance trained with the best parameters. Call the best_model.fit method to train the model using the data set X_train, y_train.

[0271] Exemplarily, fitting the model again on the training data includes:

[0272] best_model.fit(X_train,y_train) is to call this method to train the model again. Use the training set X_train and the target variable y_train to train and call the model's fit method.

[0273] Exemplarily, using best_model to predict the validation set and test set includes:

[0274] Use the trained model and prepare the data set (validation set X_valid) y_valid_pred = best_model.predict_proba(X_valid)[:,1] For classification problems, use probability prediction: use the predict_proba method. Here, the [:,1] slicing operation is used to select the second column from the returned data, that is, the predicted probability that the user will repurchase.

[0275] For example, in calculating the corresponding ROC AUC score, roc_auc_score is used to calculate the score, y_valid is the true label of the validation set, usually 0 or 1, and y_valid_pred is the probability that the validation set predicted by the model belongs to the positive class. Specifically, it includes:

[0276] (A) Validation set:

[0277] y_valid_pred=best_model.predict_proba(X_valid)[:,1];

[0278] auc_valid=roc_auc_score(y_valid,y_valid_pred);

[0279] (B) Test set:

[0280] y_test_pred=best_model.predict_proba(X_test)[:,1];

[0281] auc_test=roc_auc_score(y_test,y_test_pred);

[0282] (C) TRR represents the proportion of positive instances correctly predicted by the model as positive, and the calculation formula is: TP / (TP+FN), where TP is the number of positive instances and FN is the number of false instances. FRR represents the proportion of negative instances incorrectly predicted as positive by the model, and the calculation formula is: FP / (TN+FP), where FP is the number of false positive instances and TN is the number of true instances.

[0283] (D) Draw the ROC curve: Use different thresholds, with TRR as the vertical axis and FRR as the horizontal axis, draw a series of points, and then connect them to draw the ROC curve.

[0284] (E) The AUC score is the area under the ROC curve.

[0285] Embodiment 2, as Figure 4 As shown, the e-commerce user repurchase behavior prediction system based on the improved algorithm provided by the embodiment of the present invention includes:

[0286] Data preprocessing module 1 is used to clean data, divide data, convert data types, process missing values, process outliers, and process timestamp data for e-commerce users' repurchase behavior;

[0287] The feature acquisition module 2 related to repurchase behavior is used to construct features using the existing fields to increase feature dimensions for three different subjects: different users, different merchants, and different users to different merchants, and obtain features related to repurchase behavior. The features related to repurchase behavior include different user features, different merchant features, and interaction features between different users and different merchants;

[0288] The improved XGBoost algorithm prediction module 3 is used to utilize the selected features related to repurchase behavior, utilize the improved XGBoost algorithm as a prediction model, and train and evaluate the prediction model through parameter selection through grid search to obtain an AUC score.

[0289] To further illustrate the effects of the embodiments of the present invention, the following experiment was conducted. The output results are shown in Table 7.

[0290] Table 7 Parameter tuning values

[0291] Parameter name Parameter Value colsample_bytree 0.6 learning_rate 0.01 max_depth 7 min_child_weight 5 n_estimators 1000 subsample 0.6

[0292] Using the same method, we compare with other algorithms and tuning algorithms, such as

[0293] As shown in the table.

[0294] Table 7 Results comparison

[0295] method AUC value Tuning XGBoost 0.685 XGBoost 0.63 Logistic Regression 0.492 Decision Tree 0.519 Bayesian classification 0.59 Random Forest 0.631

[0296] This paper analyzes repurchase as a key indicator of e-commerce success. In order to improve the repurchase rate, companies need to deeply understand these factors and formulate corresponding strategies. In the e-commerce industry, the research on data mining algorithms has also made significant progress. At the same time, it also analyzes the research status of data mining algorithms in the e-commerce industry, providing theoretical support for subsequent research.

[0297] The present invention completes the data processing work. A series of data preprocessing operations are performed. These operations include data cleaning, which aims to remove duplicate, erroneous or irrelevant information; missing value processing, which handles the gaps in the data through methods such as interpolation or deletion; and outlier processing, which identifies and handles extreme values ​​that are significantly different from the overall data distribution.

[0298] The present invention performs feature engineering. Considering three different subjects, features are constructed using existing fields to increase feature dimensions, and a large number of features related to repurchase behavior are obtained. In the user dimension, multiple features such as user purchase history, browsing behavior, and preferences are considered. In the merchant dimension, factors such as the merchant's product quality and sales time are analyzed to evaluate the merchant's attractiveness to users. Special attention is paid to the interaction features between users and merchants, such as the user's purchase frequency. After feature engineering, a large number of features closely related to repurchase behavior are obtained, which provides strong data support for subsequent model training and prediction. These features cover different user features, different merchant features, and interaction features between different users and different merchants.

[0299] The present invention constructs an XGBoost prediction model using the selected features, and trains and evaluates the model through parameter selection by grid search. By comparing the performance of different models, the improved XGBoost algorithm as a prediction model obtains a good AUC score.

[0300] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for predicting repurchase behavior of e-commerce users based on an improved algorithm, characterized in that: The method includes: S1, data preprocessing: data cleaning, data segmentation, data type conversion, missing value processing, outlier processing and timestamp data processing for e-commerce users’ repurchase behavior; S2, for different users, different merchants, and different users to different merchants, three different subjects, use the existing fields to construct features to increase feature dimensions, and obtain features related to repurchase behavior. Features related to repurchase behavior include different user features, different merchant features, and interaction features between different users and different merchants; S3, using the selected features related to repurchase behavior, the improved XGBoost algorithm is used as the prediction model, and the prediction model is trained and evaluated through parameter selection through grid search, and the AUC score is obtained.

2. The e-commerce user repurchase behavior prediction method based on the improved algorithm according to claim 1 is characterized in that: In step S1, data cleaning is used to remove duplicate, erroneous or irrelevant information; Data partitioning includes: extracting features and labels from the original data set data, the feature set X is obtained by deleting the label column label from data, and the label set y is directly extracted from the label column label column of data; dividing the data set into training set, validation set and test set, using the train_test_split function to split the data set twice, including: dividing the original data set into 80% training set and 20% temporary sets X_temp and y_temp, and then dividing the temporary set into 50% validation sets X_valid and y_valid, and 50% test sets X_test and y_test; The data type conversion includes: after dividing the data set, the label sets y_train, y_valid, and y_test contain string type labels, and these string labels are converted to integer labels using the defined convert_to_int function, and the label values ​​that are not 0.0 or 1.0 are converted to integers 0 or 1.

3. The e-commerce user repurchase behavior prediction method based on the improved algorithm according to claim 1 is characterized in that: In step S1, missing value processing, gaps in the data are handled by interpolation or deletion methods; Outlier processing, identifying and processing extreme values ​​that are significantly different from the overall data distribution; The processing of timestamp data includes: using the pd.to_datetime function to convert the time_stamp column from a string to a datetime type. During the conversion process, the missing value of gender is replaced with 2, and the data type of several columns in the all_data DataFrame is converted. The info() method is used to view the detailed information of the converted DataFrame, including the column data type and the number of non-null values.

4. The e-commerce user repurchase behavior prediction method based on the improved algorithm according to claim 1 is characterized in that: In step S2, the feature dimensions include user dimensions, merchant dimensions, and interaction dimensions between different users and different merchants; User dimensions include the user's purchase history, browsing behavior, and preferences. The merchant dimension includes the merchant’s product quality and sales time factors; The interaction dimensions between different users and different merchants include the users' purchase frequency.

5. The e-commerce user repurchase behavior prediction method based on the improved algorithm according to claim 1 is characterized in that: In step S3, the improved XGBoost algorithm is used as the prediction model, and the prediction model is trained and evaluated through grid search for parameter selection, and the AUC score is obtained, including: (1) According to the data situation, select the following parameters: max_depth, learning_rate, n_estimators, min_child_weight, subsample, colsample_bytree; (2) Use grid search to adjust parameters and find the best parameter combination; (3) Initialize the XGBoost classifier object model and set the objective function to binary logistic regression and a fixed random seed; (4) Use the GridSearchCV object grid_search to perform grid search; use the defined model as the estimator, param_grid as the parameter grid, use ROC AUC as the scoring indicator, and adopt 3-fold cross validation; search for the best parameter combination on the training data by calling grid_search.fit(X_train,y_train); retrain the prediction model using the best parameter combination and store it in the best_model object; fit the prediction model again on the training data, use best_model to predict the validation set and test set, and calculate the corresponding ROC AUC scores.

6. The e-commerce user repurchase behavior prediction method based on the improved algorithm according to claim 5 is characterized in that: In step (2), the parameters include the maximum depth of the tree, the learning rate, the number of base learners, the minimum sum of sample weights required on the leaf nodes, the sample subset ratio and the feature subset ratio; Use grid search to adjust parameters and find the best parameter combination including: (2.1) Define parameter grid; (2.2) Select the scoring criteria: select roc_auc; (2.3) Set cross validation: 3-fold cross validation: cv=3; (2.4) Initialize the grid using GridSearchCV; (2.5) Perform grid search, grid_search.fit(X_train,y_train); (2.6) Evaluate the results and use best_params_ and best_score_ to output the best parameter combination and the best score; print("Best parameters found:",grid_search.best_params_); print("Best ROC AUC score found:",grid_search.best_score_); (2.7) Train the prediction model using the optimal parameters.

7. The method for predicting repurchase behavior of e-commerce users based on an improved algorithm according to claim 5, characterized in that: In step (3), the initialization includes: model=xgb.XGBClassifier(objective='binary:logistic',random_state=42) Use logistic regression as the objective function, and random_state=42 to set the seed value of the random number generator to ensure consistent results each time the prediction model is trained using the same parameters.

8. The method for predicting repurchase behavior of e-commerce users based on an improved algorithm according to claim 5, characterized in that: In step (4), a grid search is performed using the GridSearchCV object grid_search, including: (401) Instantiate GridSearchCV and pass in the necessary parameters: (401-1) estimator: The model to be optimized defines the model to initialize the XGBoost classifier: model=xgb.XGBClassifier(objective='binary:logistic',random_state=42) (401-2) param_grid: contains the parameter names to be traversed and the corresponding value list, defined as a parameter grid; (401-3) scoring is the scoring indicator used for model evaluation, which is roc_auc; (401-4)cv: the number of cross-validation folds, set to 3; (402) Fitting GridSearchCV: grid_search.fit(X_train,y_train); When the fit method is called, GridSearchCV will perform: (402-1) Parameter grid traversal, traversing all parameter combinations defined in param_grid; (402-2) cross validation; (402-3) Model training: train the model using the current parameter combination on each training set; (402-4) Performance evaluation: Evaluate the model performance on each training set and calculate the score using the scoring metric specified by scoring; (402-4) Record the results; (402-6) Select optimal parameters.

9. The method for predicting repurchase behavior of e-commerce users based on an improved algorithm according to claim 6, characterized in that: In step (2.3), a 3-fold cross validation is used, including: (a) The dataset is first randomly divided into three subsets of equal size or as close to equal size as possible; (b) For each parameter combination in the parameter grid: (b-1) perform 3 training and validation cycles; (b-2) in each cycle, select one of the subsets as the validation set, and the remaining two subsets are combined as the validation set; (b-3) train the model on the training set using the current parameter combination; (b-4) evaluate the model on the validation set and calculate the ROCAUC score; (c) After each validation, the ROC AUC score of the current parameter combination is recorded; (d) In this way, each parameter combination obtains a ROCAUC list with a length equal to the number of cross-validation folds; (e) Calculate the average ROCAUC: For each parameter combination, calculate the average of its ROCAUC scores in all three validations; In the calculation of the corresponding ROC AUC score, roc_auc_score is used to calculate the score, y_valid is the true label of the validation set, which is 0 or 1, and y_valid_pred is the probability that the validation set predicted by the model belongs to the positive class; specifically, it includes: (A) Determine the validation set; (B) Determine the test set; (C) TRR is used to represent the proportion of positive instances correctly predicted by the model as positive. The calculation formula is: TP / (TP+FN), where TP is the number of positive instances and FN is the number of false instances. FRR is used to represent the proportion of negative instances that are incorrectly predicted as positive by the model. The calculation formula is: FP / (TN+FP), where FP is the number of false positive instances and TN is the number of true instances. (D) Draw the ROC curve: Use different thresholds, with TRR as the ordinate and FRR as the abscissa, draw a series of points, and then connect them to draw the ROC curve; (E) AUC score is the area under the ROC curve.

10. An e-commerce user repurchase behavior prediction system based on an improved algorithm, characterized in that: The system implements the e-commerce user repurchase behavior prediction method based on the improved algorithm as described in any one of claims 1 to 9, and the system includes: Data preprocessing module (1), used for data cleaning, data segmentation, data type conversion, missing value processing, outlier processing and timestamp data processing of e-commerce users' repurchase behavior; The feature acquisition module (2) related to repurchase behavior is used to construct features using the existing fields to increase feature dimensions for three different subjects: different users, different merchants, and different users to different merchants, so as to obtain features related to repurchase behavior. The features related to repurchase behavior include features of different users, features of different merchants, and features of interactions between different users and different merchants. The improved XGBoost algorithm prediction module (3) is used to utilize the selected features related to repurchase behavior, use the improved XGBoost algorithm as a prediction model, and train and evaluate the prediction model through parameter selection through grid search to obtain an AUC score.

Citation Information

Patent Citations

  • E-commerce platform user repeated purchase behavior prediction method

    CN110956497A

Cited By

  • Metacosm data intelligent sharing platform and method

    CN119622832A

  • E-commerce user behavior deep insight analysis method and system based on big data artificial intelligence

    CN121937154A