A weighted graph convolutional network rating prediction recommendation method with multi-behavior enhanced information

Abstract

The present invention relates to the fields of recommendation systems and graph convolutional neural networks, and specifically, to a weighted graph convolutional network rating prediction recommendation method with multi-behavior enhanced information. The traditional recommendation system model mainly focuses on a single type of user-item interaction behavior, which leads to cold start or data sparsity problems. The present invention proposes a weighted graph convolutional network rating prediction recommendation model with multi-behavior enhanced information to capture the implicit relationship between different types of behaviors under message passing and behavioral type-aware collaborative signals with high-order relationships. Self-learning weights are combined to distinguish different behavior intensities to obtain the final representation of users and items for prediction. At the same time, an embedding propagation layer between items is designed to model the similarities between items, which reflects the various semantics of different behaviors.

Description

Technical Field

[0001] The present invention relates to the fields of recommendation systems and graph convolutional neural networks, and in particular to a weighted graph convolutional network rating prediction and recommendation method with multi-behavior enhanced information. Background Art

[0002] In recent years, as the information age has developed into the big data age, the scale of data on the Internet has exploded, causing information overload to become an increasingly serious problem in our lives. The recommendation system is an effective solution to alleviate the problem of information overload and facilitate users to seek information. It can also increase the traffic and revenue of service providers. Therefore, efficient and accurate personalized recommendation systems have become a hot topic in academia and industry. At the same time, the research on graph neural networks (GNNs) has gradually emerged. Since most of the information in the recommendation system has a graph structure, GNNs have advantages in graph representation learning, so they have been widely used in recommendation systems.

[0003] Traditional recommendation models mainly focus on a single type of user-item interaction behavior, which may lead to cold start or data sparsity problems. In real online platforms, user behaviors are multi-type in nature, involving heterogeneous relationships between users and items, such as likes, favorites, and add to carts. Different user-item interaction patterns may express different semantic information. These heterogeneous information provides valuable user preference signals and helps build fine-grained recommendation systems.

[0004] In other words, the recommendation system should be able to use other types of behaviors: auxiliary behaviors, to help predict the user's future interactions on the target behavior. This is multi-behavior recommendation. Currently, the recommendation system does not have this capability. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a weighted graph convolutional network rating prediction and recommendation method with multi-behavior enhanced information, which comprehensively considers various types of user-item interaction behaviors and captures the intensity of different behaviors by adding weights.

[0006] In order to achieve the above object, the specific technical solution adopted by the present invention is as follows:

[0007] A weighted graph convolutional network rating prediction and recommendation method for multi-behavior enhanced information adopts a weighted graph convolutional network rating prediction and recommendation model for multi-behavior enhanced information. The model includes an embedding module, a user-item message passing module, an item relevance perception module and a final prediction module, and specifically includes the following steps:

[0008] Step 1: Establish the problem: Establish a unified heterogeneous graph to model the problem under study and clarify the input and output of the model;

[0009] Step 2: Initial Embedding Layer: Using Embedding Vectors and To describe a user and an item, where d is the embedding dimension, the user embedding vector and the item embedding vector can be respectively expressed as the embedding matrix and To indicate, use To describe a type of interaction pattern between users and items;

[0010] Step 3, User Embedding Propagation Layer: The embedding propagation layer in the model is built on the message passing architecture of GCN to capture CF signals and the graph structure of heterogeneous interactions between users and projects. The basic idea of ​​GCN is to learn the representation of nodes by smoothing the features on the graph. In the model, the representation of users is modeled by accumulating incoming messages from all heterogeneous interaction projects. Rewriting the formula, the relationship between adjacent nodes is Perform embedding combination to model relational user-item interactions, while assigning a propagation weight α to each behavior relation in order to incorporate the importance and sparsity of the behavior;

[0011] Step 4: Project embedding and propagation layer: This is the same as the user embedding and propagation layer, except that the weight α is not assigned to the behavior relationship. Because the characteristics of the project are static, it is assumed that different users contribute the same amount to the same project content regardless of the behavior type, so different behaviors here do not have an impact coefficient on the project.

[0012] Step 5, Item Relevance Propagation Layer: Multi-type behavior semantics are further proposed. There should be some common features or special relationships between the interactive items under each behavior type. For example, when users click and view products on an e-commerce website, the co-clicked / viewed products may be interchangeable (such as different types of mobile phones); when it comes to purchases, these co-purchased products may be complementary (such as iPhones and AirPods). When the amount of data is sparse, the relationship between items may enhance the ability to learn item embeddings.

[0013] Step 6, multi-task prediction: After using L layers of propagation, multiple representations of user u, item v, and relation r are obtained. The representations obtained from different layers emphasize the information transmitted from different jump layers. Therefore, they are further combined to obtain the final representation, establish the probability formula for the target user u to perform the kth action on item v, and calculate the correlation score between the target item v and the items that user u has interacted with under a specific action k. The two are combined to make a prediction score, and the model is trained using the loss function.

[0014] The step 1 specifically includes:

[0015] The input interaction data is represented by an undirected graph G = (V, E), where the nodes are composed of user nodes u∈U and item nodes v∈V. The edges in E contain different user-item interaction edges with different behaviors. r k ∈N r , N r is the set of all behavior types that occur between users and items. There will be an edge Establish.

[0016] Assuming the number of behaviors is T, the interaction matrix Indicates whether the user interacts with the item under behavior k. All behavior matrices can be expressed as {Y 1 ,Y 2 ,....,Y T}. Specifically, the interaction matrix It is in binary form, and each entry has a value of 1 or 0. , it means that user u and item v interact under behavior k.

[0017] Model input: user-item interaction data under T behaviors, {Y 1 ,Y 2 ,....,Y T}.

[0018] Model output: A recommendation model that estimates the probability that user u will interact with item v under the kth behavior, i.e., the target behavior.

[0019] The step 2 specifically includes:

[0020] The user embedding vector and item embedding vector can be represented by embedding matrices U and V respectively:

[0021]

[0022]

[0023]

[0024] Using one-hot encoding as input to describe a user, an item, and a behavior relationship, the embeddings in the matrix U, matrix V, and matrix E can be used as initialization features of users, items, and behavior types, and can be regarded as input features of each user, item, and relationship in the graph neural network framework.

[0025] The step 3 specifically includes:

[0026] The graph convolution operation is defined as follows, through which the model performs message passing. There are two main operations in each convolution layer: message passing and message aggregation.

[0027]

[0028]

[0029]

[0030] in, represents the representation of user u after embedding the propagation layer at layer L, W (l) ∈R d'×d is a trainable weight matrix for extracting useful information in propagation, and d' is the transformation size. u and N v The neighbor node sets of u and v respectively; use symmetric normalization terms to avoid the embedding size increase with graph convolution operations. ⊙ is the element-wise product, incorporating relation and neighbor item embeddings into the formula. The activation function δ is LeakyReLU.

[0031] Assign a weight to each behavior, such as behavior k In order to integrate the importance and sparsity of behaviors, the propagation weight is defined for the specific behavior k of user u as in, w m The model is trained by itself. is the number of actions k performed by the user, is the total number of user interactions.

[0032] The relation is embedded in the transformation formula, is a layer-specific parameter that projects all relations into the same embedding space as the nodes and allows them to be used in the next GCN layer.

[0033] The step 4 specifically includes:

[0034] The graph convolution operation is defined as follows. There are two main operations in each convolution layer: message passing and message aggregation.

[0035]

[0036]

[0037] in, represents the representation of user v after embedding the propagation layer at layer L, W (l) ∈R d'×d is a trainable weight matrix for extracting useful information in propagation, and d' is the transformation size. u and N v The neighbor node sets of u and v respectively; use symmetric normalization terms to avoid the embedding size increase with graph convolution operations. ⊙ is the element-wise product, incorporating the relationship and neighbor user embeddings into the formula. The activation function δ is LeakyReLU.

[0038] The relation is embedded in the transformation formula, is a layer-specific parameter that projects all relations into the same embedding space as the nodes and allows them to be used in the next GCN layer.

[0039] The step 5 specifically includes:

[0040] Designed an item relevance propagation layer to capture behavioral semantics

[0041]

[0042] in, is a set of items that interact with the user under behavior k together with item v. is the encoding matrix with row k. The aggregation function here is a simple average function, and is equal to the initial embedding vector of the corresponding item v.

[0043] The step 6 specifically includes:

[0044]

[0045]

[0046]

[0047]

[0048] The probability that user u performs the kth action on item v is estimated as:

[0049]

[0050] And calculate the correlation score y2(u, v) between the target item v and the item interacted by user u under a specific behavior k.

[0051]

[0052] The two together are used as the prediction score:

[0053] y(k) u,v =λ·y1(k) u,v +(1-λ)·y2(k) u,v (13)

[0054] You can try different λ in subsequent experiments to examine their impact.

[0055] Loss function for training:

[0056] loss=∑ (u,v,j)∈O -lnσ((y(k) u,v -y(k) u,j ))+β·||Θ|| 2 (14)

[0057] Among them, O={(u, v, j)|(u, v)∈R + ,(u,j)∈R -} represents the paired target behavior training dataset, R + represents the observed target behavior, R - represents the unobserved target behavior, σ(·) is the sigmoid function, Θ represents all trainable parameters, and β is the L2 normalization coefficient, which controls the L2 normalization strength to prevent overfitting.

[0058] Beneficial effects: The present invention captures the implicit relationship between different types of behaviors under message passing and behavioral type-aware collaborative signals with high-order relationships. The self-learning weights are combined to distinguish different behavior intensities to obtain the final representation of users and items for prediction. At the same time, an embedding propagation layer between items is designed to model the similarity between items, which reflects the various semantics of different behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is an example diagram of various types of user items in the present invention.

[0060] Figure 2 It is the user-item heterogeneous graph in the present invention.

[0061] Figure 3 This is a diagram of the user message transmission module in the present invention.

[0062] Figure 4 This is a diagram of the project message transmission module in the present invention.

[0063] Figure 5 This is a diagram of the project relevance perception module in the present invention. DETAILED DESCRIPTION

[0064] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0065] like Figure 1 As shown in the figure, it is an example diagram of multiple types of user items. It can be seen that in real e-commerce scenarios, users can click to view products, add products to favorites, and purchase products. These heterogeneous behaviors help build fine-grained recommendation systems. Recommendation systems should be able to use other types of behaviors: auxiliary behaviors, to help predict users' future interactions on target behaviors. This is multi-behavior recommendation.

[0066] A weighted graph convolutional network rating prediction recommendation model with multi-behavior enhanced information includes four modules: an embedding module, a user-item message passing module, an item relevance perception module, and a final prediction module. The weighted graph convolutional network rating prediction recommendation method with multi-behavior enhanced information using the model specifically includes the following steps:

[0067] Step 1: Create a problem such as Figure 2 , a unified heterogeneous graph is established to model the problem under study. If users u and v interact under behavior k, there will be an edge Establish. Clarify the input and output of the model.

[0068] Step 2: Initial embedding layer, the user embedding vector and the item embedding vector can be represented by the embedding matrices U and V respectively:

[0069]

[0070]

[0071]

[0072] The embeddings in the matrix U, matrix V, and matrix E can be used as the initial features of users, items, and behavior types, which will be optimized. In the framework, the embeddings are refined by propagating the embeddings on the user-item interaction graph. This provides more effective embeddings for recommendation tasks because the refinement step explicitly injects collaborative signals into the embeddings.

[0073] Step 3, Step 4: User (item) embedding propagation layer. Intuitively, the items that users interact with provide direct evidence of user preferences. Similarly, users who operate an item can be regarded as features of the item, which is used to measure the collaborative similarity of two items.

[0074] The embedding propagation layer in the model is built on the message passing architecture of GCN to capture the CF signal and the graph structure of heterogeneous user-item interactions. The basic idea of ​​GCN is to learn the representation of nodes by smoothing the features on the graph. In the model, the representation of users (items) is modeled by accumulating incoming messages from all heterogeneous interacting items (users). Embedding propagation is performed between connected users and items, forming a process with two main operations: message passing and message aggregation.

[0075] Figure 2 In the example, user u1 interacts with v1 in behavior r1 and interacts with v2 in behavior r2. Using formula (1), we can write e v1 and The interaction between E v2 and The interaction between Encoded into the message being transmitted, this combination operator is used to jointly embed the representation and relations of the item nodes for multi-relation prediction. The principle of embedding the representation and relations of the user nodes is the same as above.

[0076] In formula (2) is the behavior importance weight of behavior k, which is the same for all users, is the count of actions k performed by user u, which varies from user to user. Specifically, The behavior ratio of the large value Behaviors with smaller values ​​are more important. is the final propagation weight of user u’s k behavior, is learned by the model, so the importance of each behavior can be learned automatically without the need to assign importance to each behavior.

[0077] More embedding propagation layers are stacked to explore high-order connectivity information. Such high-order connections are crucial for encoding collaborative signals to estimate the relevance between users and items. By stacking L embedding propagation layers, a user (and an item) is able to receive messages propagated from its L-hop neighbors. In the L-th step, the representation of user u is recursively expressed as formula (1), is the item representation generated by the previous message passing step, memorizing the messages from its (L-1)-hop neighbors. It further contributes to the representation of user u at level L. Similarly, the representation of items on level L can be obtained.

[0078] Step 5: Figure 5 , the embedding of item j that shares common behavior with v1 under behavior r2 Aggregate to get the next layer of embedding Through this mechanism, the features of items that share common behaviors with item v1 will be aggregated into the embedding of v1, which facilitates better embedding learning and final prediction.

[0079] Step 6: After propagation through L layers, multiple representations of user u are obtained: Multiple representations of item v: Multiple representations of relation r: The representations obtained from different layers emphasize the information passed from different jump layers. For example, the first layer enforces smoothing between users and their interacted items, the second layer smoothes the users connected to the interacted items, and higher layers capture higher-order relationships. Therefore, they are further combined to obtain the final representation: as shown in Equations (7)(8)(9) (10), where each embedding layer sets a simple uniform weight This results in good general performance.

[0080] In order to predict the possibility of multiple behaviors of users on items, the learned representation of each behavior is predicted as an independent prediction layer. A simple inner product interaction function is used here to calculate the score y1. As described in step 5, there may be behavioral semantics in multi-behavior recommendation. The common behavior of users on two items may mean the correlation between the two items, and the correlation between items may affect the user's behavior (buy or not buy). Therefore, a scoring mechanism is deployed here to mine the behavioral semantics and calculate the correlation score y2 between the target item v and the items that user u interacted with under behavior k. By adding these two scores, the final score not only extracts the collaborative filtering signal between users and items, but also takes into account the behavioral semantics. You can try using different λ in subsequent experiments to test its impact.

[0081] Loss function for training:

[0082] loss=∑ (u,v,j)∈O -lnσ((y(k) u,v -y(k) u,j ))+β·||Θ|| 2 (14)

[0083] Among them, O={(u,v,j)|(u,v)∈R + ,(u,j)∈R -} represents the paired target behavior training dataset, R + represents the observed target behavior, R - represents the unobserved target behavior, σ(·) is the sigmoid function, Θ represents all trainable parameters, and β is the L2 normalization coefficient, which controls the L2 normalization strength to prevent overfitting.

[0084] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A weighted graph convolutional network rating prediction and recommendation method with multi-behavior enhanced information, characterized in that: A weighted graph convolutional network rating prediction recommendation model using multi-behavior enhanced information is proposed. The model includes an embedding module, a user-item message passing module, an item relevance perception module, and a final prediction module. Specifically, the following steps are included: Step 1: Establish the problem: Establish a unified heterogeneous graph to model the problem under study and clarify the input and output of the model; Step 2: Initial Embedding Layer: Using Embedding Vectors and To describe a user and an item, where d is the embedding dimension, the user embedding vector and the item embedding vector are respectively embedded in the embedding matrix and To indicate, use To describe a class of interaction patterns between users and items; Step 3: User embedding propagation layer: relationships between adjacent nodes Perform embedding combination to model relational user-item interactions, while assigning a propagation weight α to each behavioral relation to incorporate both the importance and sparsity of behaviors; Step 4: Project Embedding Propagation Layer: Relationships between Adjacent Nodes Perform embedding composition to model relational user-item interactions; Step 5, project relevance propagation layer: further propose multi-type behavior semantics; Step 6, multi-task prediction: the representations obtained from different layers emphasize the information passed from different skip layers; The step 1 specifically includes: The input interaction data is represented by an undirected graph G = (V, E), where the nodes are composed of user nodes u∈U and project nodes v∈V, and the edges in E contain different user-project interaction edges with different behaviors. r k ∈N r , N r is the set of all behavior types that occur between users and items. When creating an entry Assume the number of behaviors is T, the interaction matrix Indicates whether the user interacts with the item under behavior k. All behavior matrices are represented as {Y 1 ,Y 2 ,....,Y T }; The step 2 specifically includes: The user embedding vector and item embedding vector are represented by embedding matrices U and V respectively: Use one-hot encoding as input to describe a user, an item, and a behavior relationship, and the embeddings in the matrix U, matrix V, and matrix E are used as initialization features for users, items, and behavior types; The step 3 specifically includes: The graph convolution operation is defined as follows, through which the model performs message passing. There are two main operations in each convolution layer, message passing and message aggregation: in, represents the representation of user u after embedding the propagation layer at layer L, W (l) ∈R d’×d is a trainable weight matrix for extracting useful information in propagation, d' is the transformation size, N u and N v The neighbor node sets of u and v are respectively, using symmetric normalization terms To avoid the embedding scale that increases with graph convolution operations, ⊙ is the element-wise product, the relationship and neighbor item embeddings are incorporated into the formula, the activation function δ is LeakyReLU, a weight is assigned to each behavior, and the weight of behavior k is set to In order to integrate the importance and sparsity of behaviors, the propagation weight is defined for the specific behavior k of user u as in, w m The model is trained by itself. is the number of actions k operated by the user, is the total number of user interactions, and the relationship is embedded in the transformation formula. is a layer-specific parameter that projects all relations into the same embedding space as nodes and allows them to be used in the next GCN layer; The step 4 specifically includes: The graph convolution operation is defined as follows. There are two main operations in each convolution layer: message passing and message aggregation: in, represents the representation of user v after embedding the propagation layer at layer L, W (l) ∈R d’×d is a trainable weight matrix for extracting useful information in propagation, d' is the transformation size, N u and N v The neighbor node sets of u and v respectively; use symmetric normalization terms To avoid the embedding scale that increases with graph convolution operations, ⊙ is the element-wise product, the relationship and neighbor user embeddings are incorporated into the formula, the activation function δ is LeakyReLU, and the relationship embedding conversion formula is: is a layer-specific parameter that projects all relations into the same embedding space as nodes and allows them to be used in the next GCN layer; The step 5 specifically includes: Designed an item relevance propagation layer to capture behavioral semantics in, is a set of items that interact with the user under behavior k together with item v. is the encoding matrix of row k, where the aggregation function is a simple average function, and is equal to the initial embedding vector of the corresponding item v; The step 6 specifically includes: The probability that user u performs the kth action on item v is estimated as: And calculate the relevance score y2(u, v) between the target item v and the item interacted by user u under a specific behavior k: The two together are used as the prediction score: y(k) u,v =λ·y1(k) u,v +(1-λ)·y2(k) u,v (13)。

Images