A Deep Learning-Based Multi-Objective Optimization Recommendation Method

Abstract

This invention belongs to the technical field of recommendation system ranking algorithms, and relates to a multi-objective optimization recommendation method based on deep learning. The method includes the following steps: establishing training and testing datasets and setting model learning objectives; introducing an embedding layer to address data feature sparsity; constructing high- and low-order feature interactions to predict CTR; introducing a gating network; introducing a multi-level expert network; establishing dependencies between objectives and constructing a weighted multi-objective loss. This invention demonstrates the benefits of promoting cooperation between objectives, preventing negative transfer and seesaw effects in multi-objective scenarios, and achieves substantial improvements in satisfaction and accuracy. It can be widely applied in various multi-objective recommendation scenarios. As people's demands for information accuracy and diversity continue to increase, recommendation systems can help users find items of interest and improve user experience and satisfaction.

Description

Technical Field

[0001] This invention belongs to the field of recommendation system ranking algorithm technology, and relates to a multi-objective optimization recommendation method based on deep learning. Background Technology

[0002] With the advent of the big data era and the rapid development of smart devices, personalized recommendations play a crucial role in various applications. Recommendation systems often use prediction models that focus solely on user clicks, failing to adequately consider post-click behavior. This leads to users becoming trapped in increasingly smaller interest networks, reducing user engagement and satisfaction, resulting in an unbalanced development of the recommendation ecosystem and decreased corporate profits. Therefore, applying multi-task learning to simultaneously simulate user satisfaction and engagement for multi-objective optimization has become a trend. For example, in video recommendations, considering only click-through rate might favor videos with clickbait titles or suggestive posters; considering only completion rate might favor shorter videos. These biases negatively impact user experience, leading to a decline in long-term business goals and hindering system development. Therefore, research has begun to explore incorporating multiple related but inconsistent objectives into a comprehensive model. Practical experience shows that multi-objective learning can improve recommendation performance. However, the predictions of commonly used multi-objective models are often highly sensitive to the relationships between objectives; therefore, studying and modeling the relationships between specific objectives is crucial.

[0003] Elkanky et al., recognizing the difficulty in acquiring user features in traditional content-based recommendation systems, extracted user features by analyzing browsing and search history, thus enriching user feature representation. They extended Deep Structured Semantic Models (DSSM) to propose a Multi-View Deep Neural Network (Multi-View DNN), which achieves item recommendation through semantic matching of user and item information entities. Cheng et al., utilizing multi-source heterogeneous data such as user features, contextual features, and item features, proposed a Wide & Deep Learning model, which possesses both high memory and generalization capabilities. Guo et al., based on Wide & Deep, combined with factorization machines and deep learning, proposed a Factorization-Machine based Neural Network (DeepFM) model for click-through rate prediction, employing factorization machines and deep neural networks to model low-level and high-level feature interactions respectively. Currently, many large-scale recommendation systems both domestically and internationally have adopted multi-task learning using deep neural network models. Researchers have reported that multi-objective models can improve predictions for all objectives by leveraging regularization and transfer learning. However, in practice, multi-objective models do not always outperform their corresponding single-objective models across all objectives. In fact, many deep learning-based multi-objective models are sensitive to factors such as differences in data distribution and relationships between objectives. Multi-objective recommendation often faces problems such as sparse sample data, selection bias implicit in user feedback, and the "seesaw" phenomenon. The inherent conflict arising from differences in objectives can impair the prediction of at least some objectives, especially when model parameters are widely shared across all objectives. Summary of the Invention

[0004] The purpose of this invention is to provide a deep learning-based multi-objective optimization method for personalized recommendations, in order to solve the problems of poor user experience in single-objective recommendations and the difficulty in jointly optimizing multi-objective recommendation models under complex relationships with large differences in objectives, which are also prone to conflicts.

[0005] To achieve the objectives of this invention, the technical solution adopted is as follows:

[0006] A deep learning-based multi-objective optimization recommendation method is provided, the method specifically including the following steps:

[0007] Step 1: Preprocess and classify the raw video recommendation data, establish training and testing datasets, and set model learning objectives;

[0008] Step 2: Introduce the Embedding layer to solve the data feature sparsity.

[0009] Step 3: Construct high- and low-order feature interactions to predict CTR.

[0010] Step 4: Introduce a separate gating network for each target.

[0011] Step 5: Introduce a multi-level expert network so that each target has an independent Expert while retaining the shared Expert.

[0012] Step 6: Establish the dependency relationship between targets and construct a weighted multi-objective loss.

[0013] Furthermore, the above Step 1 includes the following steps:

[0014] Step 1.1: Acquisition of original video recommendation data

[0015] Collect the user logs of users on the video website for 15 consecutive days. The data includes user features, video content features, and user historical behavior data, and all are desensitized to obtain data labels. <*

[0016] Step 1.2: Data preprocessing and classification

[0017] Analyze the proportion of each data label in the exposure scenario and the click scenario, and conduct statistical analysis on the video duration label among them. Set the threshold of the duration label ranging from 0 to 9 to 4, set the duration label greater than 4 as the effective duration, and the others as the invalid duration to obtain the training and test data sets.

[0018] Step 1.3: Set the model learning objectives

[0019] Select the click label and the video duration label in the data labels as the two objectives that the model will learn.

[0020] Furthermore, in the above Step 2, the Embedding layer is the embedding layer, which is used to embed the high-dimensional sparse one-hot data vector to obtain a low-dimensional dense Embedding vector. Then, each dense vector obtained after passing through the embedding layer is horizontally concatenated as the input of the model;

[0021] The output of the embedding layer is expressed as:

[0022] a

[0019] ,

[0023] ,

[0018] ,

[0022] ,

[0017] ,

[0021] ,

[0016] , (0) ,

[0020] , m , , , , , , i , , (0) , , = [e1, e2, …, e m

[0023] where e i is the embedding of the i-th field, and m is the number of fields. Then, a (0) ​The data is fed into a deep neural network, and the forward propagation process is as follows:

[0024] a (l+1) =σ(W (l) a (l) +b (l) )

[0025] Where l is the layer depth and σ is the activation function. (l) W (l) b (l) These are the output, model weights, and biases of the l-th layer, respectively. Then, a dense real-valued feature vector is generated.

[0026] Furthermore, step 3 above includes the following steps:

[0027] Step 3.1: Constructing high- and low-order feature interactions

[0028] First, factorization machine and deep neural network are used to model low-level and high-level feature interactions respectively. The FM part of DeepFM is kept unchanged, and the DNN part of DeepFM is replaced with a shared-bottom structure with hard parameter sharing, so as to obtain a multi-objective basic model that combines DeepFM and shared-bottom.

[0029] Step 3.2: Predict CTR

[0030] The FM subnetwork calculates the second-order cross-scores of sparse and dense features, and the deep subnetwork concatenates dense and continuous features into the network. The first-order and second-order scores of FM are concatenated with the last layer input of deep, and the result is obtained by passing the sigmoid function to obtain the estimated CTR.

[0031] Furthermore, in step 4 above, a separate gating network g is added for each target k. k The output of task k is:

[0032] y k =h k (f k (x))

[0033] The gating network learns different combinations of the experts network for its respective objectives and adaptively weights the output of the experts network.

[0034] Furthermore, in step 5, the gating network in the j-th extraction network of the k-th sub-target is defined as follows:

[0035] g k,j (x)=w k, j(g k,j-1 (x))S k,j (x)

[0036] Among them, w k,j The weight function of target k is used as g k,j-1 The input, S k,j It is the selection matrix of task k in the j-th extraction network.

[0037] Furthermore, step 6 above includes the following steps:

[0038] Step 6.1: Establish dependencies between objectives

[0039] The method of constructing the loss function by introducing ESMM uses two DNNs with shared underlying parameters to predict CTR and CVR respectively. The predicted CTCVR is obtained by multiplying the predicted CTR and CVR. The pCVR can be obtained by estimating the pCTR and pCTCVR of the separately trained models and using the relational expression.

[0040] Step 6.2: Construct a weighted multi-objective loss.

[0041] The CTR loss and CTCVR loss are weighted and summed to obtain a total loss. The model parameters are solved by minimizing the total loss.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] This invention first proposes a model construction method that utilizes factorization machines and deep learning to build high- and low-order feature interactions, aiding in multi-objective learning and eliminating sample sparsity. Then, a separate gating network is constructed for each objective, followed by the introduction of a shared expert network. The gating network and multi-level expert network, built using fully connected neural networks, learn the feature relationships of user behavior to avoid harmful parameter interference and achieve more effective knowledge transfer between complex and related objectives, eliminating the "seesaw" phenomenon. Finally, the connection between objectives is established using an ESMM-constructed loss function, which is then used to optimize the model, resulting in a more accurate fit to various conversion rates.

[0044] Experiments have demonstrated that this invention exhibits greater advantages in improving the shared learning efficiency of target groups of different sizes. This indicates that the invention demonstrates the benefits of promoting cooperation among targets, preventing negative transfer and seesaw effects in multi-target scenarios, and achieves substantial improvements in satisfaction and accuracy metrics.

[0045] The method of this invention can be widely applied in the field of recommender systems, especially in multi-objective recommender systems across various scenarios, and has broad application prospects. As people's demands for information accuracy and diversity continue to increase, this invention can help users find items of interest through recommender systems, thereby improving user experience and satisfaction. Attached Figure Description

[0046] Figure 1 It is a recommendation framework based on deep learning;

[0047] Figure 2 This is a model framework diagram for constructing high- and low-order feature interactions according to the method described in this invention;

[0048] Figure 3 This is a model framework diagram of the method described in this invention, which incorporates a gating network.

[0049] Figure 4 It is the internal structure of the gating network;

[0050] Figure 5 This is a model framework diagram of the method described in this invention, which incorporates a multi-level expert network.

[0051] Figure 6 This is the model framework for establishing connections between targets in the method described in this invention. Detailed Implementation

[0052] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0053] This invention improves upon the Shared-Bottom model by proposing the use of high- and low-order feature interactions in model construction to aid multi-objective learning. Then, a separate gating network is introduced for each objective, followed by a shared expert network to address the issue of large differences between objectives. Finally, an ESMM-based loss function is introduced for optimization, resulting in a more accurate fit to various conversion rates. This invention improves the prediction accuracy of current multi-objective recommendation algorithms and reduces the seesaw effect in multi-objective modeling.

[0054] This invention proposes a multi-objective optimization recommendation method based on deep learning, see [link to relevant documentation]. Figure 1 The framework for a deep learning-based recommendation system includes the following steps:

[0055] Step 1: Preprocess and classify the raw video recommendation data, establish training and testing datasets, and set the model learning objectives.

[0056] Step 1.1: Obtaining raw video recommendation data

[0057] User logs from video websites over 15 consecutive days were collected. The data included user characteristics, video content characteristics, and user historical behavior data, all of which were anonymized to obtain data tags.

[0058] The data tags mentioned can include user dimensions, video dimensions, and user historical behavior data. User dimension information includes user ID, age group, gender, province / city, city level, and device type. Video dimension information includes video ID, video year, video month, video rating, and video duration. User historical behavior data includes user ID, video ID, whether it was played, shared, favorited, commented, viewing time, playback tags, and viewing date.

[0059] Step 1.2: Perform data preprocessing and classification

[0060] We analyzed the proportion of each data tag in exposure and click scenarios, and performed statistical analysis on the video duration tag, setting a threshold of 4 for duration tags ranging from 0 to 9. Using binary classification, duration tags greater than 4 were treated as positive samples and assigned a value of 1 (valid duration), while others were treated as negative samples and assigned a value of 0 (invalid duration). The data tags were then segmented using a fixed time threshold, filtering out data that did not contain message information, resulting in training and testing datasets.

[0061] Step 1.3: Set model learning objectives

[0062] Among all the objectives, is_watch (whether it plays) and watch_label (playback duration) are two important ones. These two objectives will be used as the targets that the model will learn, and will be used to model the key metrics of playback clicks and viewing time, respectively.

[0063] Step 2: Introduce an Embedding layer to address data feature sparsity.

[0064] The Embedding layer is used to embed high-dimensional sparse one-hot vectors to obtain low-dimensional dense Embedding vectors. Then, each dense vector obtained after passing through the embedding layer is concatenated horizontally and used as the input of the model.

[0065] The output of the embedding layer is represented as:

[0066] a (0) =[e1,e2,…,e m ]

[0067] Where e i It is the embedding of the i-th field, and m is the number of fields. Then, a (0) The data is fed into a deep neural network, and the forward propagation process is as follows:

[0068] a (l+1) =σ(W (l) a (l) +b (l) )

[0069] Where l is the layer depth and σ is the activation function. (l) W (l) b (l) These are the output, model weights, and biases of the l-th layer, respectively. Then, a dense real-valued feature vector is generated.

[0070] Based on the above description, in this embodiment, it is assumed that the dataset used for training consists of n instances (x, y), where x is an m-field data record, typically involving a pair of users and items, and y∈{0,1} is an association label indicating user click behavior (y=1 indicates that the user clicked the item, otherwise y=0). It may include categorical and continuous fields, each categorical field represented as a one-hot encoded vector, and each continuous field represented as the value itself, or a discretized one-hot encoded vector. Then, each instance is transformed into (x, y), where x = [x...]. field1 ,x field2 ,…,x fieldj ,x fieldn ] is a d-dimensional vector, x fieldj Let X be the vector representation of the j-th field of X, where X is high-dimensional and extremely sparse.

[0071] Step 3: Construct high- and low-order feature interactions to predict CTR:

[0072] Step 3.1: Constructing high- and low-order feature interactions

[0073] First, factorization machine (FDEM) and deep neural network (DNN) are used to model low-level and high-level feature interactions, primarily targeting clicks and viewing time. The FM part of DeepFM is retained, while the DNN part is replaced with a shared-bottom structure with hard parameter sharing, resulting in a multi-objective base model combining DeepFM and Shared-bottom. The model's output formula is:

[0074]

[0075] Where H represents the number of hidden layers.

[0076] Step 3.2: Predict CTR

[0077] See Figure 2 The FM subnetwork on the left calculates the second-order cross-score between sparse and dense features, while the deep subnetwork on the right concatenates dense and continuous features as input. Finally, the first and second-order scores of the FM subnetwork are concatenated with the input of the last layer of the deep subnetwork, and then passed through a sigmoid function to obtain the predicted value. The model prediction results are as follows:

[0078]

[0079] in, To predict CTR, y FM For the output of the FM module, y deep The formulas for the output of the deep module are as follows:

[0080]

[0081] y deepk =h k (f(x))

[0082] The output f(x) of the shared hidden layer is input into their respective tower networks (sub-networks) h. k Finally, each task k yields an output y. deepk .

[0083] Step 4: Introduce a separate gating network for each target

[0084] See Figure 3 To address the challenges of multi-objective learning where the relationships between sub-objectives are not very strong in real-world applications, this step adds a separate gating network g for each objective k. k The output of task k is:

[0085] y k =h k (f k (x))

[0086] The gating network learns different combinations of expert networks for its respective target, adaptively weighting the output of the expert networks. For the target, the output of its corresponding expert network is:

[0087]

[0088] Among them, f i (i = 1, ..., n) is a network of n experts, and g is a gated network that combines the results of the experts' networks.

[0089] See Figure 4 The gating mechanism is internally implemented using identical multilayer perceptrons with ReLU activation. The gating network is a simple linear transformation of the input followed by a softmax layer. The input vector and the output vectors of each expert network are fed into the gating network. The input vector first passes through an MLP, and the final softmax layer yields the weights of each expert. The output of the gating network is the weights of all the expert networks.

[0090] g k (x) = softmax(W)gk x)

[0091] Among them, W gk ∈R n×d It is a trainable matrix, where n is the number of experts networks and d is the feature dimension.

[0092] Step 5: Introduce a multi-level expert network to ensure that each objective has an independent expert while retaining shared experts.

[0093] See Figure 5 By introducing a shared expert network, the gating network in the j-th extraction network for the k-th sub-target is defined as follows:

[0094] g k,j (x)=w k,j (g k,j-1 (x))S k,j (x)

[0095] Among them, w k,j The weight function of target k is used as g k,j-1 The input, S k,j This is the selection matrix for task k in the j-th extraction network. After calculating all the gated networks and the experts network, the final output of the k-th sub-objective is obtained as follows:

[0096] y k (x)=t k (g k,N (x))

[0097] Step 6: Establish dependencies between objectives and construct a weighted multi-objective loss.

[0098] Step 6.1: Establish dependencies between objectives

[0099] See Figure 6This step introduces the ESMM method for constructing the loss function, using two DNNs with shared underlying parameters to predict CTR and CVR respectively. The predicted CTCVR is obtained by multiplying the predicted CTR and CVR. The pCVR of the separately trained models pCTR and pCTCVR are estimated separately, and pCVR can be obtained through the relational expression. The three related targets jointly train the classifier, which can utilize the sequence model of the data and pass information to each other, ensuring physical meaning. The predicted CTR and click label are used to calculate the loss for fitting CTR, and the predicted CTR × CVR and label are used to calculate the loss for fitting CTCVR. CVR is merely an intermediate variable used to construct the predicted CTCVR and does not fit any target. Because the prediction of CTCVR is decomposed into the prediction of CTR and the prediction of CVR, the two different negative samples can be well distinguished when predicting CTCVR. The transformation relationship of the three targets is shown below:

[0100]

[0101] Step 6.2: Construct a weighted multi-objective loss.

[0102] Finally, the CTR loss and CTCVR loss are weighted and summed to obtain a total loss. The model parameters are solved by minimizing the total loss. The expression for the total loss function is as follows:

[0103]

[0104] Where L1 and L2 are the loss functions for fitting CTR and CTCVR, respectively, both being binary cross-entropy; y i For clickable category labels; z i The category label is the playback duration (1 for playback duration exceeding the threshold, 0 otherwise), P ctr (x i ,θ) is a prediction of CTR; P cvr (x i ,θ) is the prediction of CVR; θ is the model parameter; w1 and w2 are the weights of the two losses, respectively, and N is the total number of samples.

[0105] The AUC (Area Under Curve) metric, widely used in recommender systems, is used to evaluate the performance of this method. AUC, as a numerical value, can intuitively evaluate the classifier's performance; the higher the value, the better the ranking model's performance. The calculation formula is as follows:

[0106]

[0107] The experimental results are as follows:

[0108] Table 1: Prediction performance of this method compared with other classic models on the dataset.

[0109]

[0110] The prediction performance of different models on the video dataset is shown in the table, with the best scores highlighted in bold. Table 1 shows that this model significantly outperforms single-objective models and classic multi-objective models in terms of conversion objectives. Due to the complex correlation between click and duration objectives, a seesaw effect is clearly observed in the results: some models improve the click objective but harm the duration objective, while others improve the duration objective but harm the click objective. Compared to typical and widely used multi-objective models MMoE and ESMM, this model shows a much greater improvement in the duration objective and further improvements in the click objective. Finally, this model converges at a similar speed and achieves significant progress on the aforementioned models with one of the AUC values. The above description is only for the purpose of helping to understand the method and core ideas of this invention. For those skilled in the art, variations may occur in specific implementations based on the ideas of this invention; therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A multi-objective optimization recommendation method based on deep learning, characterized in that: The method specifically includes the following steps: Step 1: Preprocess and classify the raw video recommendation data, establish training and testing datasets, and set model learning objectives; Step 2: Introduce an embedding layer to address data feature sparsity; Step 3: Construct high- and low-order feature interactions to predict CTR; Step 4: Introduce a separate gating network for each target; Step 5: Introduce a multi-level expert network so that each objective has an independent expert, while retaining shared experts; Step 6: Establish dependencies between objectives and construct a weighted multi-objective loss; In step 2, the embedding layer is used to embed high-dimensional sparse one-hot data vectors to obtain low-dimensional dense embedding vectors. Then, each dense vector obtained after the embedding layer is concatenated horizontally as the input to the model. The output of the embedding layer is represented as follows: a (0) =[e1,e2,…,e m ] Where e i It is the embedding of the i-th field, m is the number of fields, and then, a (0) The data is fed into a deep neural network, and the forward propagation process is as follows: a (l+1) =σ(W (l) a (l) +b (l) ) Where l is the layer depth, σ is the activation function, and a (l) W (l) b (l) These are the output of the l-th layer, the model weights, and the bias, respectively. Then, a dense real-valued feature vector is generated. Step 3 includes the following steps: Step 3.1: Constructing high- and low-order feature interactions First, factorization machine and deep neural network are used to model low-level and high-level feature interactions respectively. The FM part of DeepFM is kept unchanged, and the DNN part of DeepFM is replaced with a shared-bottom structure with hard parameter sharing, so as to obtain a multi-objective basic model that combines DeepFM and shared-bottom. Step 3.2: Predict CTR The FM subnetwork calculates the second-order cross-scores of sparse and dense features, and the deep subnetwork concatenates dense and continuous features into the network. The first-order and second-order scores of FM are concatenated with the last layer input of deep, and the result is obtained by passing the sigmoid function to obtain the estimated CTR.

2. The multi-objective optimization recommendation method based on deep learning according to claim 1, characterized in that: Step 1 includes the following steps: Step 1.1: Acquisition of raw video recommendation data. Collect user logs from video websites for 15 consecutive days. The data includes user characteristics, video content characteristics, and user historical behavior data, all of which have been anonymized to obtain data tags. Step 1.2: Data preprocessing and classification analysis of the proportion of each data tag in the exposure scenario and click scenario, and statistical analysis of the video duration tag. The duration tag in the range of 0-9 is set to a threshold of 4. The duration tag greater than 4 is set as the valid duration, and the others are set as invalid duration, thus obtaining the training and test datasets. Step 1.3: Set the model learning objectives. Select the click label and video duration label from the data labels as the two objectives that the model will learn.

3. The multi-objective optimization recommendation method based on deep learning according to claim 2, characterized in that: In step 4, a separate gating network g is added for each target k. k The output of task k is: y k =h k (f k (x)) The gating network learns different combinations of the experts network for its respective objectives and adaptively weights the output of the experts network.

4. The multi-objective optimization recommendation method based on deep learning according to claim 3, characterized in that: In step 5, the definition of the gating network in the j-th extraction network of the k-th sub-target is: g k,j (x)=w k,j (g k,j-1 (x))S k,j (x) Among them, w k,j The weight function of target k is used as g k,j-1 The input, S k,j It is the selection matrix of task k in the j-th extraction network.

5. The deep learning-based multi-objective optimization recommendation method according to claim 4, characterized in that: Step 6 includes the following steps: Step 6.1: Establish dependencies between objectives The method of constructing loss function by introducing ESMM is used. Two DNNs with shared underlying parameters are used to predict CTR and CVR respectively. The predicted value of CTCVR is obtained by multiplying the predicted values ​​of CTR and CVR. The pCTR and pCTCVR of the separately trained models are estimated respectively, and pCVR is obtained by relational expression. Step 6.2: Construct a weighted multi-objective loss. The CTR loss and CTCVR loss are weighted and summed to obtain a total loss. The model parameters are solved by minimizing the total loss.

Images