A multi-task learning based multi-behavior recommendation method and system
By combining multi-task learning with a multi-channel embedding layer and a user behavior attention layer, the problems of data sparsity and imbalance in recommendation models are solved, the impact of noise is reduced, and the accuracy and generalization ability of recommendation models are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA UNIV OF PETROLEUM (EAST CHINA)
- Filing Date
- 2024-08-15
- Publication Date
- 2026-07-31
AI Technical Summary
Existing recommendation models face problems of data sparsity and imbalanced data distribution, resulting in poor recommendation performance, and implicit feedback data noise also affects model performance.
A multi-task learning approach is adopted, combining a multi-channel embedding layer, a user behavior attention layer, and a multi-task learning module. By learning the influence weights between behaviors and users through the attention mechanism, the impact of data distribution imbalance and noise is reduced, and multi-behavioral interaction features are integrated.
By using multi-task learning, the interest bias caused by imbalanced data distribution is reduced, the accuracy and generalization ability of the recommendation model are improved, and the impact of noise on the recommendation results is reduced.
Smart Images
Figure CN119003874B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of product recommendation technology, specifically to a multi-behavior recommendation method and system based on multi-task learning. Background Technology
[0002] Recommender systems are important tools for meeting personalized needs. Traditional recommendation models typically utilize only one type of user-item interaction data, facing severe data sparsity problems and failing to capture users' true preferences, resulting in poor recommendation performance. In real-world scenarios, users exhibit multiple behaviors. Platforms can collect other frequently occurring user behaviors, such as browsing, clicking, and saving, and leverage these diverse behaviors to help predict future user interactions with target behaviors.
[0003] Multi-behavior recommendation can effectively alleviate the data sparsity problem by enriching user interests and preferences, thereby improving recommendation performance. However, while multi-behavior data can alleviate the problem of target behavior data sparsity, it suffers from a severe imbalance in distribution. Different users and behaviors are highly imbalanced, with only a small subset of users and behaviors covering most interactions. The amount of interaction data varies significantly across different user behaviors, making the final user preferences more susceptible to the influence of behaviors with more interaction data (such as browsing behavior). This data imbalance causes user representations to be optimized towards behaviors with larger data volumes for prediction. Moreover, most user feedback obtained by current recommendation models is implicit. Some studies have shown a discrepancy between implicit user feedback data and their preferences. For example, accidental clicks or purchases followed by negative reviews or returns can be mistakenly identified as positive samples by the model. Such interactions between users and items they are not interested in are a major source of noise in the dataset, which affects model performance. Summary of the Invention
[0004] The purpose of this invention is to provide a multi-behavior recommendation method and system based on multi-task learning, so as to reduce the impact of imbalanced data distribution and noise in interactions on the recommendation model.
[0005] To achieve the above objectives, a first aspect of the present invention provides a multi-behavior recommendation method based on multi-task learning, comprising:
[0006] Acquire multi-behavioral interaction data of user items;
[0007] Input user and item multi-behavior interaction data into the recommendation model, and output the predicted scores of user and item under specific behaviors;
[0008] Items are categorized into user-recommended items based on predicted values;
[0009] The recommendation model includes a multi-channel embedding layer, a user behavior attention layer, and a multi-task learning module. The multi-channel embedding layer generates user embedding vectors and item embedding vectors based on the interaction data between users and items. The user behavior attention layer updates the user embedding vectors and item embedding vectors based on an attention mechanism. The multi-task learning module performs multi-task learning on the updated user embedding vectors and item embedding vectors to obtain prediction scores for users and items under specific behaviors.
[0010] A second aspect of the present invention provides a multi-behavior recommendation system based on multi-task learning, comprising:
[0011] The data acquisition module is configured to acquire multi-behavioral interaction data of user items;
[0012] The prediction module is configured to input multi-behavioral interaction data of users and items into the recommendation model and output the predicted scores of users and items under specific behaviors.
[0013] The recommendation module is configured to categorize user-recommended items based on predicted values.
[0014] The recommendation model includes a multi-channel embedding layer, a user behavior attention layer, and a multi-task learning module. The multi-channel embedding layer generates user embedding vectors and item embedding vectors based on the interaction data between users and items. The user behavior attention layer updates the user embedding vectors and item embedding vectors based on an attention mechanism. The multi-task learning module performs multi-task learning on the updated user embedding vectors and item embedding vectors to obtain prediction scores for users and items under specific behaviors.
[0015] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to perform the steps of the multi-behavior recommendation method based on multi-task learning described above.
[0016] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps of the aforementioned multi-behavior recommendation method based on multi-task learning.
[0017] In a fifth aspect, the present invention provides a computer program product comprising a computer program / instructions which, when executed by a processor, implement the steps of the multi-behavior recommendation method based on multi-task learning described above.
[0018] Compared with the prior art, the beneficial effects of the present invention are:
[0019] This invention provides a multi-behavior recommendation method and system based on multi-task learning. By learning the influence weights between behaviors and between different users through an attention mechanism, the influence of interest bias caused by data imbalance is reduced. By combining multi-task learning with a multi-behavior architecture, multi-behavior interaction features are perceived from both user and item perspectives. By performing multi-task learning on multi-behavior interaction data, multiple behaviors are learned simultaneously to obtain a more generalized feature representation, which averages out the different noise patterns between different behaviors and reduces the impact of data noise on the recommendation results.
[0020] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.
[0022] Figure 1 This is a flowchart of the method of the present invention;
[0023] Figure 2 This is a schematic diagram of the model of the present invention. Detailed Implementation
[0024] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0025] Example 1
[0026] This implementation example Figures 1-2 As shown, a multi-behavior recommendation method based on multi-task learning is provided, including:
[0027] S1. Obtain multi-behavioral interaction data of user items;
[0028] S2. Input the user-item multi-behavior interaction data into the recommendation model and output the predicted scores of the user and items under specific behaviors. The recommendation model includes a multi-channel embedding layer, a user behavior attention layer, and a multi-task learning module. The multi-channel embedding layer generates user embedding vectors and item embedding vectors based on the interaction data between the user and items. The user behavior attention layer updates the user embedding vectors and item embedding vectors based on an attention mechanism. The multi-task learning module performs multi-task learning on the updated user embedding vectors and item embedding vectors to obtain the predicted scores of the user and items under specific behaviors.
[0029] S3. Based on the predicted values, items are categorized into user-recommended items.
[0030] Learning the importance of different users and behaviors is a crucial step in addressing imbalanced data distribution. Existing multi-behavior models mostly use attention mechanisms to learn the relative importance of different behaviors, which doesn't take a comprehensive approach. Different behaviors have different noise patterns, and learning user features for only one behavior inevitably introduces errors. Learning user features for multiple behaviors better averages out noise patterns, resulting in better embedding representations. Therefore, this embodiment perceives multi-behavior interaction features from both user influence weights and behavior influence weights, and integrates these features into the embedding propagation process to obtain more accurate user and item embeddings. Specifically, firstly, an attention mechanism is used to learn the weights between different behaviors and different users, enhancing the expressive power of node embeddings. Then, the output embedding vector is used as input to a multi-task learning module, employing a multi-objective loss optimization strategy to optimize the model, ultimately obtaining accurate recommendation results.
[0031] First, user and item embeddings are initialized using a shared embedding layer. Then, to preserve behavior type features, the model uses a behavior-aware messaging paradigm to generate propagation information between user and item pairs, retaining the highest-level propagation information to construct a semantic encoder to update user and item embeddings. Next, to avoid user interest bias, an attention mechanism is used to learn weights between different users and behaviors, leveraging a multi-attention head mechanism to enhance the attention layer and enable attention relationship learning across multiple representation subspaces. After obtaining calibrated behavior and item embedding vectors, a two-layer feedforward network is used to estimate personalized user behavior weights. After obtaining personalized preference vectors describing users and items, these are used as input to a multi-task module, fed into the underlying network and mapped to multiple gating networks. The underlying network is divided into multiple expert modules shared across all tasks, reducing the impact of data noise. Because different expert modules may contain different user preferences or item attributes, a weighted average of multiple experts is obtained through corresponding gating mechanisms, and then input into the task-specific tower network for final prediction across multiple tasks.
[0032] (1) Problem Description
[0033] Assume the model is based on recommendations from an e-commerce platform, with a user set U = {u1, ..., u...} i ,…,u I}, the set of items V = {v1, ..., v j ,…,v J User-item multi-behavior interaction matrix set Where I, J, and K represent the number of users, items, and behavior categories, respectively. This indicates that user u and item v interact under action K, otherwise they do not. Where Y = {Y1, Y2, ..., Y} K-1} represents auxiliary behavior, Y K Indicates the target behavior.
[0034] Input: User multi-behavior interaction matrix set Y.
[0035] Output: The probability that user u and item i will interact under action k.
[0036] We define the user-item interaction graph G. u =(U,V,E) k ), where U and V represent user and item nodes respectively, and E represents the set of edges for all user-item interaction categories (such as purchase, add to cart, etc.). k This represents the Kth type of user interaction with items.
[0037] (2) Multichannel embedding layer
[0038] First, the input layer uses a shared embedding layer to transform the one-hot vectors of user and item IDs into low-dimensional embeddings. Given a user-item pair (u,v), its initial embedding is:
[0039] x i =P Τ ·p i ,y j =Q Τ ·q j
[0040] in, and One-hot vectors representing user and item IDs. and These are the user embedding matrix and the item embedding matrix, respectively, where d is the embedding dimension.
[0041] First, based on a multi-behavior interaction graph, a behavior-aware messaging paradigm is used to generate propagation information for user and item pairs:
[0042]
[0043]
[0044] Where λ(·) represents the behavior-aware encoding function, used to capture specific types of behavioral context information during message construction, thereby preserving user u i and items v j The semantics of the interaction behavior type. and The propagation information of users and items in layer (l+1) is preserved, where the number of convolutional layers in the graph convolution ranges from [1,L]. The embedding dimension of the output of the λ(·) function is d, preserving features of k behavior types.
[0045] Then, a behavioral semantic encoder is constructed on a multichannel neural framework to learn customized representations of user-item interactions for individual types. Our multichannel embedding layer uses external representation units, where each unit corresponds to a dimension of a behavioral semantic. We combine this with a behavior-aware attention mechanism to fuse semantic information learned from different channels. For each behavior type k, this module is first equipped with a context transition layer to update the input user / item embedding in layer l. and
[0046]
[0047]
[0048] in, It is the m-th channel transformation, corresponding to M potential projection subspaces, which reflect different aspects of different behaviors in the context of common behavior. Representing user u i Neighboring nodes under k behaviors The weight. This represents a context transition specific to the type of learning behavior. Additionally, and These are the transformation parameter and the bias parameter, respectively, used to calculate the weights, and δ(·) represents ReLU activation.
[0049] A similar semantic encoder was used to learn the item v j and its connected neighboring user nodes Embedding between Finally, we can obtain user and item embeddings with specific behavioral semantics. and
[0050] (3) User behavior attention layer
[0051] To avoid the user preferences learned by the model being influenced by irrelevant information in auxiliary behaviors with many interaction records, we use an attention mechanism to learn the importance of different behaviors and the importance of different behavior types to users.
[0052] After obtaining the embedding representations for different behavior types, we first create three transformation matrices (Q... h ,K h V hThe input representation is projected onto three latent dimensions, and the dependency scores between different behavioral messages are calculated in a dot product manner.
[0053]
[0054]
[0055] in, The relevant weights are learned between the k-th and k′-th behavior categories, and are derived from the intermediate scores by the softmax function. The calculation yielded the result.
[0056] Based on the characteristics of the multi-head mechanism, this module is endowed with the ability to learn attentional relationships in multiple representation subspaces. In this case, we associate each head-specific attention component with a set of weight matrices, and based on head-specific relevance scores, we recalibrate specific types of behavioral information by connecting multiple attention heads.
[0057]
[0058] Where || denotes vector concatenation, It propagates to the target node u under k behavior types. i The recalibration information. To preserve the original, specific type of behavioral information and prevent the gradient vanishing problem, we use a residual connection scheme to perform element-wise addition between the original information and the recalibrated, specific type of message. That is... It is an improved representation of a specific type of behavior that preserves information about the interdependencies between cross-type behaviors.
[0059] For different users, different behaviors have varying degrees of impact on the target behavior. Therefore, it's also necessary to learn the contribution of different types of behavior to the target behavior to distinguish the impact of different behavior types on users. We use a two-layer feedforward network for weight estimation:
[0060]
[0061]
[0062] in, and These are the transformation vector and bias vector for weight calculation, respectively. Subscripts 1 and 2 indicate two layers of feedforward network, d′ is the dimension of the intermediate hidden layer, and δ is the ReLU activation function. k It is the intermediate attention value. It is the final influence weight of interaction type k calculated using the softmax function.
[0063]
[0064]
[0065] We update the embeddings of specific behavior types using the calculated weights until we obtain the embedding vector for the Lth layer. The final result is a user embedding vector that integrates inter-behavior weights and user behavior weights. Similarly, the item's embedding vector is also obtained. Since graph convolution is calculated layer by layer, from layer 1 to layer L, the output of layer L is the output of the graph convolution module. In other words...
[0066] (4) Multi-task learning module
[0067] After obtaining the personalized preference vectors describing users and items, we then use a multi-task learning approach to reduce the noise impact of auxiliary behaviors. Since different behaviors have different noise patterns, learning multiple tasks simultaneously yields a more generalized representation. Learning only one behavior carries a certain risk of error; however, learning multiple tasks simultaneously averages the impact of noise, resulting in a better model representation and a smoother final output.
[0068] We leverage the soft-sharing mechanism of parameters in multi-task learning to reduce the impact of noise from auxiliary behaviors. Similar to the MMoE model, not all tasks share a single underlying network. The underlying network is divided into multiple experts, each being a fully connected layer with a ReLU activation function. Each task then has its own gating network. The model input is mapped to multiple expert layers shared across all tasks, and also to multiple gates. Since these different expert modules may contain different user preferences or item attributes, it is necessary to combine these experts with different weights to make a final prediction for each task.
[0069]
[0070] Where (||) represents the vector concatenation operation. and For the characteristic transformation matrix and the bias matrix, The attention vector is obtained by using a corresponding gating mechanism to obtain a weighted average of multiple experts, which is then input into the tower network corresponding to the task.
[0071] The final prediction score for the final task is calculated as follows:
[0072]
[0073]
[0074] in, This represents the dot product operation. This means replacing the dot product of users and projects with the expert's operation. The predicted score for behavior k is represented by a score. Representing vectors The j-th element, h k (·) represents the tower function, which is mainly used for averaging operations.
[0075] (5) Target Loss Optimization
[0076] The predicted value is scored based on the obtained behavior k. Multi-task learning is performed using Bayesian Personalized Ranking (BPR) loss, which can be expressed as:
[0077]
[0078] in, Denotes the training dataset, where This represents the positive user-item interactions observed under k different behaviors. This represents the unobserved user-item interaction under behavior k, and Θ represents the set of all model parameters. This represents the score of the predicted value for each behavior, trained from positive user-item interactions. σ represents the predicted score for each behavior trained on unobserved user-item interactions, where σ is a sigmoid function and λ is the L2 regularization coefficient of Θ to prevent overfitting.
[0079] Example 2
[0080] This embodiment provides a multi-behavior recommendation system based on multi-task learning, including:
[0081] The data acquisition module is configured to acquire multi-behavioral interaction data of user items;
[0082] The prediction module is configured to input multi-behavioral interaction data of users and items into the recommendation model and output the predicted scores of users and items under specific behaviors.
[0083] The recommendation module is configured to categorize user-recommended items based on predicted values.
[0084] The recommendation model includes a multi-channel embedding layer, a user behavior attention layer, and a multi-task learning module. The multi-channel embedding layer generates user embedding vectors and item embedding vectors based on the interaction data between users and items. The user behavior attention layer updates the user embedding vectors and item embedding vectors based on an attention mechanism. The multi-task learning module performs multi-task learning on the updated user embedding vectors and item embedding vectors to obtain prediction scores for users and items under specific behaviors.
[0085] It should be noted that the modules in this embodiment correspond to the steps of the method in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.
[0086] Example 3
[0087] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to complete the steps of the method in Embodiment 1.
[0088] Example 4
[0089] This embodiment provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the steps of the method in Embodiment 1.
[0090] Example 5
[0091] This embodiment provides a computer program product, including a computer program / instruction, characterized in that the computer program / instruction, when executed by a processor, implements the steps of the method in Embodiment 1.
[0092] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A multi-behavior recommendation method based on multi-task learning, characterized in that, include: Acquire multi-behavioral interaction data of user items; Input user and item multi-behavior interaction data into the recommendation model, and output the predicted scores of user and item under specific behaviors; Items are categorized into user-recommended items based on predicted values; The recommendation model includes a multi-channel embedding layer, a user behavior attention layer, and a multi-task learning module. The multi-channel embedding layer generates user embedding vectors and item embedding vectors based on the interaction data between users and items. The user behavior attention layer updates the user embedding vectors and item embedding vectors based on an attention mechanism. The multi-task learning module performs multi-task learning on the updated user embedding vectors and item embedding vectors to obtain the predicted scores of users and items under specific behaviors. The update of user embedding vectors and item embedding vectors based on the attention mechanism includes: Based on the attention mechanism, obtain the relevant weights between different behaviors; Based on the relevant weights, obtain recalibration information; Perform element-wise addition between the recalibration information and the user embedding vector or item embedding vector to obtain an improved behavior representation; The final impact weights are calculated based on the improved behavioral representation. Update the user embedding vector and item embedding vector using the final impact weights; The step of performing multi-task learning on the updated user embedding vectors and item embedding vectors to obtain prediction scores for users and items under specific behaviors includes: The updated user embedding vector and item embedding vector are mapped to multiple expert layers and multiple gates shared by all tasks, and experts with different weights are combined to obtain attention vectors. By inputting the attention vector into the tower network, the predicted score for a specific behavior is obtained.
2. The multi-task learning based multi-behavior recommendation method of claim 1, wherein, The recommendation model also includes a target loss optimization module, used to optimize the recommendation model based on the training dataset, where the loss function L is: in, This represents the training dataset. This represents the positive user-item interactions observed under k different behaviors. This indicates user-item interactions that were not observed under action k. Represents the set of all model parameters. This represents the score of the predicted value for each behavior, trained from positive user-item interactions. This represents the predicted score for each behavior trained on previously unobserved user-item interactions. It is a sigmoid function. yes of Regularization coefficient.
3. The multi-behavior recommendation method based on multi-task learning as described in claim 1, characterized in that, A two-layer feedforward network is used to estimate the weights and calculate the final impact weights.
4. The multi-behavior recommendation method based on multi-task learning as described in claim 1, characterized in that, The method for calculating the predicted score under a specific behavior is as follows: in, This represents the dot product operation. This means replacing the dot product of users and projects with the expert's operation. The predicted score for behavior k is represented by a score. Representing vectors The j-th element, For tower functions, and This represents the updated user embedding vector and item embedding vector.
5. A multi-action recommendation system based on multi-task learning, characterized in that, include: The data acquisition module is configured to acquire multi-behavioral interaction data of user items; The prediction module is configured to input multi-behavioral interaction data of users and items into the recommendation model and output the predicted scores of users and items under specific behaviors. The recommendation module is configured to categorize user-recommended items based on predicted values. The recommendation model includes a multi-channel embedding layer, a user behavior attention layer, and a multi-task learning module. The multi-channel embedding layer generates user embedding vectors and item embedding vectors based on the interaction data between users and items. The user behavior attention layer updates the user embedding vectors and item embedding vectors based on an attention mechanism. The multi-task learning module performs multi-task learning on the updated user embedding vectors and item embedding vectors to obtain the predicted scores of users and items under specific behaviors. The update of user embedding vectors and item embedding vectors based on the attention mechanism includes: Based on the attention mechanism, obtain the relevant weights between different behaviors; Based on the relevant weights, obtain recalibration information; Perform element-wise addition between the recalibration information and the user embedding vector or item embedding vector to obtain an improved behavior representation; The final impact weights are calculated based on the improved behavioral representation. Update the user embedding vector and item embedding vector using the final impact weights; The step of performing multi-task learning on the updated user embedding vectors and item embedding vectors to obtain prediction scores for users and items under specific behaviors includes: The updated user embedding vector and item embedding vector are mapped to multiple expert layers and multiple gates shared by all tasks, and experts with different weights are combined to obtain attention vectors. By inputting the attention vector into the tower network, the predicted score for a specific behavior is obtained.
6. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to perform the steps of the method according to any one of claims 1-4.
7. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the steps of the method according to any one of claims 1-4.
8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1-4.