A personalized recommendation method integrating user interest evolution and gradient boosting algorithm

Abstract

This invention discloses a personalized recommendation method that integrates user interest evolution and gradient boosting algorithms, belonging to the recommender system research field within machine learning. The products recommended by the model of this invention show a high consistency rate with products purchased by customers within 7 days after the training data is completed. In experiments, preprocessing and feature engineering of the raw data improve the model's ranking and prediction accuracy, providing a useful feature engineering method for this scenario. Several important features affecting the ranking results of the recommender system are identified, providing a reference for continuously improving the accuracy of algorithms and recommender systems. The method of this invention has wide applications, not only in personalized clothing recommendations but also in various other recommendation fields, such as music and book recommendations.

Description

Technical Field

[0001] This application falls under the research area of ​​recommender systems within the field of machine learning. Background Technology

[0002] Keyword Terminology Definitions:

[0003] Neural networks are mathematical or computational models that mimic the structure and function of biological neural networks, used to estimate or approximate functions. Neural networks consist of a large number of interconnected artificial neurons performing calculations. In most cases, artificial neural networks can change their internal structure based on external information, making them adaptive systems.

[0004] Gradient Boosting Decision Tree (GBDT) is a member of the boosting family of ensemble learning algorithms. It's an iterative decision tree algorithm consisting of multiple decision trees, with the conclusions of all trees summed to arrive at the final answer. From its inception, it was considered, along with SVM, to be an algorithm with strong generalization capabilities. In recent years, it has gained further attention due to its application in machine learning models for search ranking.

[0005] User Interest Evolution: In most non-search e-commerce scenarios, users do not express their current interests and preferences in real time. Therefore, designing models to capture users' dynamically changing interests is key to improving recommendation effectiveness.

[0006] The internet has transformed people's lifestyles, making life and learning more convenient. In recent years, with the widespread adoption of the internet and the rapid development of e-commerce, online shopping has become an indispensable part of life. Websites are flooded with product information, often leaving customers disoriented and unable to find what they want. Recommendation systems that categorize products can quickly and effectively help customers find their preferred items and potential buyers, thereby increasing sales and generating significant economic benefits. Simultaneously, they can save customers time and improve shopping efficiency.

[0007] Product data, transaction data, and customer data contain a wealth of untapped information that influences customer product choices. It's difficult to determine which factors are important based solely on subjective experience. Currently, machine learning algorithms are widely used in product recommendation and ranking research. Li et al. established a recommendation system based on sentiment analysis, analyzing customer reviews to recommend products of greatest interest to customers. Sun et al. transformed the product recommendation problem into a density-based clustering problem, demonstrating that this model can solve the problem to some extent.

[0008] The shortcomings of existing technology:

[0009] On the one hand, the aforementioned algorithms can only describe the impact of a few features on ranking and recommendation results. Faced with massive amounts of unlabeled data, the accuracy and efficiency of traditional machine learning algorithms are greatly reduced. On the other hand, the aforementioned algorithms cannot simulate the evolution of users' interests. Nowadays, users have more and more choices when shopping online, and their interests are constantly changing. Users may have multiple interests within a certain period of time, that is, user interests are constantly evolving and overlapping. Therefore, it is particularly important to express the dynamic changes of user interests more accurately and to capture users' long-term interests.

[0010] This paper uses historical transaction data from H&M provided by the Kaggle platform, as well as multimodal data on customers and products, as background. It employs a recommendation model that incorporates simulated user interest evolution and gradient boosting algorithms to make personalized recommendations for products that users will purchase in the coming week. Summary of the Invention

[0011] This invention addresses the shortcomings of existing technologies by providing a recommendation model that integrates simulated user interest evolution and gradient boosting algorithms. This method not only handles massive amounts of data but also accurately simulates the evolution of user interests. Specifically, for handling massive amounts of data, the successive proposals and applications of algorithms from the Boosting family, such as GBDT, XGBoost, and LightGBM, have made it possible to recommend and rank products from large datasets. The LightGBM algorithm incorporates a lambda gradient, making it more suitable for recommendation and ranking scenarios. For simulating the evolution of user interests, the attention mechanism, successful in the NLP field, can be utilized to filter out key elements from target product and user behavior data for downstream tasks.

[0012] The technical solution of this invention is a personalized recommendation method that integrates user interest evolution and gradient boosting algorithms, the method comprising:

[0013] Step 1: Retrieve user information, product information, and scenario information from the database;

[0014] User information includes attributes such as user ID, age, postal code, club member status, activity level, and news feed subscription status; product information includes attributes such as product ID, product code, product name, product category number, product category name, product color, production date, and production department, as well as visual information such as product images and textual information such as product descriptions; scenario information includes attributes such as user ID, product ID, transaction date, and transaction channel.

[0015] Step 2: Dataset partitioning;

[0016] Use the samples from the last week as the test set and the samples from the previous week as the training set;

[0017] Step 3: Construct a model of the evolution of user interests;

[0018] The model consists of a behavior sequence layer, an interest extraction layer, an interest evolution layer, and an MLP network.

[0019] Behavior sequence layer: used to convert the user's original ID-type behavior sequence over n days into an embedding behavior sequence;

[0020] Interest Extraction Layer: Used to extract interest from embedding data, using GRU units to extract interest.

[0021] u t =σ(W u i t +U u h t-1 +b u )

[0022] r t =σ(W r i t +U r h t-1 +b r )

[0023]

[0024]

[0025] Among them, u t r represents the output value of the update gate in the GRU at time t. t i represents the output value of the reset gate in the GRU at time t. t This represents the input of the GRU at time t. This represents the newly learned memory state of the GRU at time t, W represents the parameter weights of the GRU unit control input, U represents the parameter weights of the GRU controlling the hidden state at the previous time step, b represents the parameter bias, the superscripts u, r, and h represent the update gate, reset gate, and newly learned state, respectively, σ represents the sigmoid function, ° represents element-wise product, and i t It is the input of GRU, that is, the embedding vector of each behavior in the behavior sequence layer, representing the embedding vector of the user's t-th behavior, h t Then, it is the t-th hidden state of GRU. After passing through the GRU interest network, the user behavior vector b(t) is further abstracted to form the interest state vector h(t).

[0026] Interest Evolution Layer: Used to characterize the evolution of user interests, incorporating an attention mechanism, with an attention mechanism score.

[0027]

[0028] Among them, a t Let W represent the attention score at time t, and let W represent the parameter weights of the attention unit. a Let h represent the embedding vector of the target item, T represent the total number of time points, and h represent the number of time points. t This represents the output of the interest extraction layer at time t. It utilizes an AUGRU (GRU with Attentional Update gate) structure, a GRU based on an attention update gate, by adding an attention score to the original update gate structure. The specific form is as follows:

[0029]

[0030]

[0031] Among them, u ′ t It's AUGRU's original update gate. It is an attentional update gate designed for AUGRU, h ′ t This is the hidden state of AUGRU; the output h of the interest evolution layer. ′ t As input to the subsequent MLP network;

[0032] Step 4: Train a model that simulates the evolution of user interests;

[0033] The model of user interest evolution is trained using the dataset obtained in step 2;

[0034] Step 5: Construct a gradient boosting tree model for processing massive amounts of data;

[0035] Step 5.1: Initially, the model score for each product is 0, and N tree models are generated;

[0036] Step 5.2: For training each tree, traverse the pairs of items with different labels in the training dataset to obtain the Lambda value λ for each sample. i ;

[0037] The calculation method is as follows:

[0038]

[0039]

[0040] Where, λ i,j This represents the Lambda value when product i is listed before product j, |ΔZ ij | indicates the change in the MAP index caused by swapping the positions of items i and j in the list. i This represents the output score of the gradient boosting tree model for item i;

[0041] Step 5.3: Calculate λ i The corresponding derivative ω i This is used to solve for the leaf node values ​​using Newton's method later.

[0042] With all documents λ i A decision tree is trained using labels, and nodes are split by minimizing the sum of squared errors: for a selected feature, a value val is selected, all samples less than or equal to val are assigned to the left child node, and samples greater than val are assigned to the right child node; then the sum of squared errors of lambda is calculated for the left and right nodes respectively, and the sum is added together as the cost of this split, and then the (feature, val) pair with the minimum cost is selected as the current split point, finally generating a decision tree with L leaf nodes;

[0043] For the decision tree generated above, the Newton step is used to calculate the value of each leaf node, that is, for the document set that falls into the leaf node, the output value of the leaf node is calculated.

[0044] Step 5.4: Update the model by adding the currently learned decision tree to the existing LightGBM model and regularizing it using the learning rate;

[0045] Step 6: Train a gradient boosting tree model to process massive amounts of data;

[0046] The gradient boosting tree model is trained using the dataset obtained in step 2.

[0047] Step 7: Model fusion;

[0048] The data from the multidimensional matrix is ​​used as input and fed into the user interest evolution model and the gradient boosting tree model for training and learning. The product scores of the two models are obtained. The scores of the two models are linearly weighted to obtain the total score, and the final product recommendation list is obtained by sorting the products according to the total score.

[0049] Step 8: Obtain the prediction data;

[0050] The test set data is obtained from the database and preprocessed and feature-engineered to obtain the user data to be tested.

[0051] Step 9: Obtain the recommendation list through combined model prediction;

[0052] The output of the model after fusing the inputs to be tested is a prediction of the list of recommended products that the user will buy in the next week.

[0053] Compared with existing technologies, the advantages of this invention are as follows:

[0054] 1. A ranking model integrating DIEN and LightGBMRanker algorithms is proposed to recommend H&M Group products, thereby improving the customer shopping experience. Experimental results show that the products recommended by the model have a high consistency rate with the products purchased by customers within 7 days after the training data is completed.

[0055] 2. In the experiment, the ranking and prediction accuracy of the model were improved by preprocessing and feature engineering the raw data, providing a useful feature engineering method in this scenario.

[0056] 3. Several important features that affect the ranking results of recommendation systems are given, providing a certain reference for continuously improving the accuracy of algorithms and recommendation systems.

[0057] 4. The method of this invention has a wide range of applications, not only in personalized clothing recommendations, but also in various other recommendation fields, such as music recommendations and book recommendations. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the integration of the DIEN and LightGBMRanker modules in one embodiment;

[0059] Figure 2 Here is a histogram used by the DIEN module in one embodiment;

[0060] Figure 3 This is a graph showing the change in transaction volume over time in one embodiment;

[0061] Figure 4 Here is a graph showing the age distribution of customers in one embodiment;

[0062] Figure 5 In one embodiment, the DIEN module is shown.

[0063] Figure 6 Here is the pseudocode for the LightGBMRanker module for user behavior prediction in one embodiment;

[0064] Figure 7 This is a feature importance graph of the top ten features in a given instance. Detailed Implementation Plan

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] In one embodiment, such as Figure 1 As shown, a personalized recommendation process for users is provided by combining DIEN and LightGBMRanker models. The method includes the following steps:

[0067] Step 1: Data exploration, data preprocessing, and feature engineering;

[0068] The acquired dataset includes user information, product information, and scenario information.

[0069] User information includes seven attributes: user ID, profile number, whether the user is active, whether the user is a club member, whether the user receives company push notifications, age, and postal code.

[0070] Product information includes 24 attribute features such as product ID, product code, product name, product type number, product type name, and product group name, 1 image information feature such as product image, and 1 text information feature such as product description.

[0071] The scenario information includes four attribute features: user ID, product ID, transaction time, and transaction channel (online or offline), as well as a time feature.

[0072] Examine the size of the dataset and the meaning and distribution of each field. Without preconceived notions about which information might be useful, statistical analysis of the data is necessary. For example... Figure 3 As shown, there is a sudden increase in commodity transaction volume, and this trend appears periodically over time. Therefore, the influence of factors such as promotional activities and holidays should be considered. From Figure 4 It can be seen that the customers are mainly between 20 and 30 years old, which is different from the shopping habits of other age groups. Therefore, the age distribution difference should be reflected when feature engineering.

[0073] Preprocessing involves filling in missing data in the dataset with the global average and replacing outliers.

[0074] Feature engineering includes normalizing and binning continuous features, and performing one-hot encoding on discrete features.

[0075] Due to the large volume of data and the uncertainty about which data points are useful, feature engineering is necessary before the experiment. First, memory compression techniques are used, such as compressing the data into smaller floating-point types, to reduce the demand on computer resources and improve processing speed. Second, feature extraction is performed, mainly including temporal feature extraction, user feature extraction, and high-order feature combinations. Then, feature statistical analysis is conducted, such as identifying maximum, minimum, median, and correlation coefficients. Finally, algorithms such as RFECV and SFS are used to select features and remove unimportant ones.

[0076] Step 2: Split the dataset;

[0077] The model aims to predict the products to be purchased in the coming week. To avoid "information slippage" caused by including the "future" time window data to be predicted in the training set during the dataset partitioning process, the dataset is divided into training and validation sets according to time.

[0078] Each user's behavioral data is used as a sample input, resulting in a dataset spanning two years. The goal is to predict the list of items a user will purchase in the coming week. To avoid the "information loop" problem, the dataset will be split as follows:

[0079] The dataset was divided into four sets: data0 (validation set), data1, data2, and data3 (training set). In the validation set (data0), data from the last week's window was used as the target, and data from previous time windows was used as the valid data. In the training set (data1), data from the second-to-last week's window was used as the target, and data from previous time windows was used as the valid data. In the training set (data2), data from the third-to-last week's window was used as the target, and data from previous time windows was used as the valid data. In the training set (data3), data from the fourth-to-last week's window was used as the target, and data from previous time windows was used as the valid data.

[0080] Step 3: Construct a model simulating the evolution of user interests;

[0081] A DIEN module is constructed within a combined deep learning network. The DIEN model is an improvement upon the DIN model. It utilizes a behavior sequence layer to transform the original ID-type behavior sequence into an embedding behavior sequence. An interest extraction layer captures real-time interests based on user history sequences and proposes a loss function to supervise the learning of user interests at each step. An interest evolution layer captures the interest evolution process related to the target item, introducing an attention mechanism into the sequence structure to amplify the influence of related items during interest evolution. In essence, the model adds GRU and Attention to the classic Embedding & MLP model and modifies the loss function.

[0082] This paper uses the DIEN module as a benchmark model for capturing the migration curve of user shopping interests. The DIEN model mainly consists of three layers:

[0083] Behavioral sequence layer: such as Figure 5 As shown in the light blue layer, it is similar to a regular embedding layer and is responsible for converting the user's original ID-type behavior sequence over n days into an embedding behavior sequence.

[0084] Interest extraction layer: such as Figure 5 As shown in the light yellow layer, a sequence model composed of GRUs is used to simulate the user interest migration process and extract the user interest corresponding to each product node. The main goal of the Interest Extractor Layer is to extract interest from the embedding data, using GRU units to extract interest:

[0085] u t =σ(W u i t +U u h t-1 +b u )

[0086] r t =σ(W r i t +U r h t-1 +b r )

[0087]

[0088]

[0089] Where σ represents the sigmoid function, ° represents the element-wise product, and i tIt is the input of GRU, that is, the embedding vector of each behavior in the behavior sequence layer, representing the embedding vector of the user's t-th behavior, h t This is the t-th hidden state of GRU. After passing through the GRU interest network, the user behavior vector b(t) is further abstracted to form the interest state vector h(t).

[0090] Interest Evolution Layer: such as Figure 5 As shown in the light red layer, an AUGRU sequence model is used. An attention mechanism is added to the interest extraction layer to simulate the evolution of interests related to the current target product. The output of the last state of the interest evolution layer is the user's current interest vector. The main goal of the Interest Evolution Layer is to characterize the evolution of user interests. An attention mechanism is added, and the attention mechanism score is as follows:

[0091]

[0092] The AUGRU (GRU with Attentional Update gate) architecture, based on the attentional update gate, incorporates an attention score into the original update gate structure, as follows:

[0093]

[0094]

[0095] Among them, u ′ t It's AUGRU's original update gate. It is an attentional update gate designed for AUGRU, h ′ t This is the hidden state of AUGRU. The output h of the interest evolution layer. ′ t As input to the subsequent MLP network.

[0096] Step 4: Train the DIEN deep learning network using the training set, validate the trained combined deep learning network using the validation set, and adjust the network's hyperparameters until the predetermined conditions are met to obtain the trained user behavior prediction model.

[0097] Step 5: Construct a gradient boosting tree model for processing massive amounts of data;

[0098] The GBDT module in the combined deep learning network is constructed; the LightGBM model in the GBDT family is lighter and faster than the XGBoost model, and its accuracy is also guaranteed; the LightGBMRanker used in this invention adds lambda gradient to the LightGBM, making it more suitable for ranking and recommendation scenarios. It is a ListWise type LTR algorithm.

[0099] First, the continuous floating-point feature values ​​are discretized into integers, and a histogram of width dimension is constructed. When traversing the data, the discretized values ​​are used as indices to accumulate statistics in the histogram. After one data traversal, the histogram has accumulated the required statistics. Then, based on the discrete values ​​of the histogram, the optimal split point is found through traversal. Feature discretization has many advantages, such as convenient storage, faster computation, strong robustness, and more stable model. For this algorithm, the most direct advantages are as follows:

[0100] Smaller memory footprint: The algorithm not only does not require additional storage for pre-sorted results, but it can also save only the discretized feature values, which are generally sufficient to store using 12-bit integers, reducing memory consumption to 1 / 8 of the original. In other words, XGBoost requires 32-bit floating-point numbers to store feature values ​​and 12-bit integers to store indices, while LightGBM only needs 8 bits to store the histogram, effectively reducing memory usage to 1 / 8.

[0101] Less computational cost: The pre-sorting algorithm XGBoost needs to calculate the split gain once for each feature value traversed, while the histogram algorithm LightGBM only needs to calculate it k times (k can be considered a constant).

[0102] Goss is a sample sampling algorithm that can eliminate most samples with small gradients while maintaining the basic distribution of the data, thus reducing the amount of data while ensuring accuracy. EFB is a method to reduce feature dimensionality (dimensionality reduction technique) and improve computational efficiency by binding features. This method can selectively bind two features that are not completely mutually exclusive without affecting the final accuracy. Efficient parallel processing, including feature parallelism, data parallelism, and voting parallelism, further improves the running speed and reduces resource consumption.

[0103] This invention presents the LightGBMRanker module as a baseline model for gradient boosting trees used to process massive datasets. LightGBM can be seen as an improvement on the XGBoost algorithm, offering faster speed, lower resource consumption, and higher accuracy. It incorporates histogram-based decision trees, gradient-based one-side sampling (GOSS), explicit feature binding (EFB), categorical features, and support for efficient parallelism. The LightGBMRanker module of this invention adds lambda gradients to the LightGBM model, making it more suitable for ranking and recommendation scenarios.

[0104] First, discretize the continuous floating-point eigenvalues ​​into integers, and simultaneously construct a histogram of width , such as... Figure 2 As shown, continuous values ​​are binned. During data traversal, the discretized values ​​are used as indices to accumulate statistics in the histogram. After one data traversal, the histogram accumulates the required statistics. Then, based on the discrete values ​​of the histogram, the optimal split point is found through traversal. The Lambda gradient is defined using ranking metrics such as MAP and NDCG; this invention uses MAP as the experimental metric.

[0105]

[0106] Where U is the number of customers, n is the number of products recommended (ranked) for each customer, and m is the number of ground truth values ​​for each customer. P(k) represents the precision of the cutoff point k, and rel(k) is an indicator function that is 1 if the item of rank k is a relevant (correct) label, and 0 otherwise.

[0107] Consider an ordered pair of goods (i, j). To emphasize the importance of their order in the sorting, we introduce a swap index |ΔZ|. ij | indicates the change in the MAP index caused by swapping the positions of item i and item j in the list. i This represents the model's output score for item i.

[0108]

[0109] Next, we calculate the lambda gradient of product i as λ. i .

[0110]

[0111] After defining the lambda gradient, the loss function L is derived in reverse. ij .

[0112] L ij=log{1+exp(s i -s j )}·|ΔZ ij |

[0113] The optimization objective during model training iterations is the loss function L with added lambda gradient. ij . Figure 6 The specific steps for constructing the LightGBMRanker module using lambda gradients are explained in detail:

[0114] Initially, there was no decision tree model, so the model score for each item was 0;

[0115] For each tree, the algorithm iterates through pairs of products with different labels in the training dataset and calculates the change in metrics |ΔZ| caused by swapping the positions of these pairs. ij |and λ i,j This allows us to obtain the Lambda value λ for each document. i ;

[0116] Calculate the derivative ω for each λ. i This is used to solve for the leaf node values ​​using Newton's method later.

[0117] A decision tree is trained using the value λ of all documents as the label. Node splitting is achieved by minimizing the sum of squared errors: for a given feature, a value val is chosen, and all samples less than or equal to val are assigned to the left child node, while those greater than val are assigned to the right child node. The sum of squared errors of the lambda is then calculated for both the left and right nodes and added together as the cost of this split. The (feature, val) pair with the minimum cost is then selected as the current split point, ultimately generating a decision tree with L leaf nodes.

[0118] For the decision tree generated above, the Newton step is used to calculate the value of each leaf node, that is, for the document set that falls into the leaf node, the output value of the leaf node is calculated.

[0119] Update the model by adding the currently learned decision tree to the existing model and regularizing it using the learning rate.

[0120] Step 6: Train the LightGBMRanker tree model using the training set, validate the trained combined deep learning network using the validation set, and adjust the network's hyperparameters until the predetermined conditions are met to obtain the trained user behavior prediction model.

[0121] Step 7: Model fusion;

[0122] Model ensemble is performed, and the training parameters of the fused model are fine-tuned. The resulting recommendation lists are then linearly weighted and sorted according to the final score to obtain the final list of recommended items. Weighting formula:

[0123] Weighted=(LightGBMRanker+DIEN) / 2

[0124] Step 8: Obtain the behavior of multiple users over n days and preprocess it to obtain matrix data of the users to be tested.

[0125] Step 9: Input the matrix data of the user to be tested into the combined model to obtain the list of products the user will purchase in the following week. Input the data series to be tested into the DIEN model and the LightGBMRanker model to obtain the predicted scores of different products for the user. Then, linearly weight the predicted values ​​and sort them according to the final weighted scores to obtain the final recommended product list for the week.

[0126] The verification experiment was conducted on historical transaction data provided by H&M on the Kaggle platform, as well as multimodal data of customers and products. The performance of DIEN, LightGBM and the combined deep learning model of this invention, which integrates simulated user interest evolution and gradient boosting algorithms, was compared on the test set. Table 1 shows the MAP@12 index scores of the DIEN model, LightGBM model and the combined deep learning model of this invention on the training set and the test set. On the training set, the MAP@12 score of the DIEN model was 0.02256, the LightGBM model was 0.02321, the LightGBMRanker model was 0.02384, and the combined deep learning model of this invention was 0.02841, representing increases of 25.9%, 22.4%, and 19.1% compared to the former three. On the test set, the MAP@12 score of the DIEN model was 0.02239, the LightGBM model was 0.02298, the LightGBMRanker model was 0.02361, and the combined deep learning model of this invention was 0.0282, representing increases of 26.0%, 22.8%, and 19.5% compared to the former three. This indicates that the recommendation results of the combined deep learning network of this invention are closer to the customer's purchasing behavior and can more accurately recommend products that users will purchase in the coming week. Compared with DIEN, LightGBM, and LightGBMRanker algorithms, the prediction accuracy of the combined deep learning algorithm of this invention is significantly improved.

[0127] Finally, the top 10 most important features are given, such as Figure 7As shown in the feature importance graph, the more important features in this example are: article_id, product_code, department_no, colour_group_code, graphical_appearance_no, product_type_no, perceived_colour_master_id, perceived_colour_value_id, age, and active. This provides a certain reference for continuously improving the accuracy of algorithms and recommendation systems in the future.

[0128] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

[0129] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this description may be replaced by other equivalent or similar features unless otherwise specified. All features or steps in all methods or processes disclosed may be combined in any way, except for mutually exclusive features and steps.

[0130] Table 1 shows the experimental results of the combined deep learning model that integrates DIEN and LightGBMRankwe in the embodiments.

[0131] Model Training set MAP@12 Test set MAP@12 DIEN 0.02256 0.02239 LightGBM 0.02321 0.02298 LightGBMRanker 0.02384 0.02361 DIEN+LightGBMRanker 0.02841 0.02822

Claims

1. A personalized recommendation method that integrates user interest evolution and gradient boosting algorithms, the method comprising: Step 1: Retrieve user information, product information, and scenario information from the database; The user information includes: user ID, age, postal code, whether they are a club member, whether they are active, and whether they receive news pushes; the product information includes: product ID, product code, product name, product category number, product category name, product color, production date, production department, visual information such as product image, and text information such as product description; the scenario information includes: user ID, product ID, transaction date, and transaction channel. Step 2: Dataset partitioning; Use the samples from the last week as the test set and the samples from the previous week as the training set; Step 3: Construct a model of the evolution of user interests; The model consists of a behavior sequence layer, an interest extraction layer, an interest evolution layer, and an MLP network. Behavior sequence layer: used to convert the user's original ID-type behavior sequence over n days into an embedding behavior sequence; Interest Extraction Layer: Used to extract interest from embedding data, using GRU units to extract interest. ； ； ； ； in, This represents the output value of the update gate in the GRU at time t. This represents the output value of the reset gate in the GRU at time t. This represents the input of the GRU at time t. This represents the newly learned memory state of the GRU at time t. This indicates the weights of the parameters that control the GRU unit. This represents the weights of the parameters that control the hidden state in the previous time step in the GRU. This indicates parameter bias, with superscripts u, r, and h representing the update gate, reset gate, and new learning state, respectively. This represents the sigmoid function. Represents element-wise product, This is the input to the GRU, which is the embedding vector of each behavior in the behavior sequence layer, representing the embedding vector of the user's t-th behavior. This is the t-th hidden state of GRU, which, after passing through the GRU interest network, becomes the user behavior vector. This was further abstracted, forming an interest state vector. ; Interest Evolution Layer: Used to characterize the evolution of user interests, incorporating an attention mechanism, with an attention mechanism score. ； in, This represents the attention score at time t of the attention mechanism. The parameter weights represent the attention unit. Let T represent the embedding vector of the target item, and T represent the total number of time points. This represents the output of the interest extraction layer at time t. Using an AUGRU (GRU with Attentional Update gate) structure, an attention score is added to the original update gate structure, as shown below: ； ； in, It's AUGRU's original update gate. It is an attentional update gate designed for AUGRU. This is the hidden state of AUGRU; the output of the interest evolution layer. As input to the subsequent MLP network; Step 4: Train a model that simulates the evolution of user interests; The model of user interest evolution is trained using the dataset obtained in step 2; Step 5: Construct a gradient boosting tree model for processing massive amounts of data; Step 5.1: Initially, the model score for each product is 0, and N tree models are generated; Step 5.2: For training each tree, iterate through the pairs of items with different labels in the training dataset to obtain the Lambda value for each sample. ; The calculation method is as follows: ； ； in, This represents the Lambda value when product i is listed before product j. This indicates the change in the MAP index caused by swapping the positions of product i and product j in the list. This represents the output score of the gradient boosting tree model for item i; Step 5.3: Calculation corresponding derivative This is used to solve for the leaf node values ​​using Newton's method later. With all documents A decision tree is trained using labels, and nodes are split by minimizing the sum of squared errors: for a selected feature, a value val is selected, all samples less than or equal to val are assigned to the left child node, and samples greater than val are assigned to the right child node; then the sum of squared errors of lambda is calculated for the left and right nodes respectively, and the sum is added together as the cost of this split, and then the (feature, val) pair with the minimum cost is selected as the current split point, finally generating a decision tree with L leaf nodes; For the decision tree generated above, the Newton step is used to calculate the value of each leaf node, that is, for the document set that falls into the leaf node, the output value of the leaf node is calculated. Step 5.4: Update the model by adding the currently learned decision tree to the existing LightGBM model and regularizing it using the learning rate; Step 6: Train a gradient boosting tree model to process massive amounts of data; The gradient boosting tree model is trained using the dataset obtained in step 2; Step 7: Model fusion; The data from the multidimensional matrix is ​​used as input and fed into the user interest evolution model and the gradient boosting tree model for training and learning. The product scores of the two models are obtained. The scores of the two models are linearly weighted to obtain the total score, and the final product recommendation list is obtained by sorting the products according to the total score. Step 8: Obtain the prediction data; The test set data is obtained from the database and preprocessed and feature-engineered to obtain the user data to be tested. Step 9: Obtain the recommendation list through combined model prediction; The output of the model after fusing the inputs to be tested is a prediction of the list of recommended products that the user will buy in the next week.

Images