A diversified recommendation method based on graph contrastive learning

Abstract

Currently, graph computing, especially graph neural networks, has experienced rapid development and widespread application. As researchers continue to study recommender systems, they are no longer satisfied with simply pursuing accuracy and have begun to explore other directions. Research on the diversity of recommender systems is gradually becoming richer. This invention proposes a diversified recommender method based on graph contrastive learning. By modifying a graph convolutional neural network (GCN), it utilizes graph contrastive learning to perform diversified recommendations. Specifically, this patent applies a maximum entropy-based neighbor selection module to the GCN neighbor aggregation stage to select diverse neighbor subsets for aggregation; designs a layered attention module to mitigate oversmoothing problems using high-order neighbor information; utilizes a class-balanced loss function to learn long-tail items; and designs a graph contrastive learning method to alleviate data sparsity problems.

Description

Technical Field

[0001] This invention designs a diverse recommendation domain, and in particular, designs a diverse recommendation based on graph comparison learning. Background Technology

[0002] Recommendation systems are essential tools for web applications, helping users navigate vast amounts of online information. These systems provide personalized recommendations for items a user might be interested in, such as products on online retail platforms and posts on social networking sites and video sharing platforms. One of the most common methods for generating these recommendations is collaborative filtering (CF), where the system uses the preferences of similar users or items to recommend new items to a given user.

[0003] Collaborative filtering (CF) models traditionally rely on matrix factorization (MF) to learn latent user and item embeddings from interaction data. However, with the rise of graph neural networks (GNNs), there is growing interest in using these models to propagate information along the user-item interaction graph and learn more complex representations of user-item interactions. PinSage, NGCF, and LightGCN are examples of GNN-based CF models that have shown promising results in personalized recommendation. These models use graph convolutional networks (GCNs) to propagate embeddings across the user-item interaction graph, allowing them to capture higher-order interactions between users and items that other alternative CF models cannot. In particular, PinSage and NGCF use multi-layer GCNs to capture both local and global information about the user-item interaction graph, while LightGCN simplifies the message passing process by omitting nonlinear transformers and using only a simple weighted sum of neighboring embeddings.

[0004] Graph-based collaborative filtering (CF) models are becoming increasingly popular in recommender systems. However, these models face challenges that current technologies have not fully addressed. One such challenge is data noise, which can arise from various factors, such as users clicking on irrelevant products due to over-recommendation of popular items. Directly aggregating information from all interaction edges in the user-item interaction graph can lead to inaccurate user representations, and multi-hop embedding propagation can exacerbate the noise effect. Therefore, existing graph-based CF models may fail to accurately capture user interests and generate inaccurate recommendations. Furthermore, the sparsity and skewed distribution of recommender data negatively impact effective user-item interaction modeling. Consequently, current methods may encounter the problem of scarce user data, where high-quality training signals may be limited.

[0005] Meanwhile, a well-designed recommender system should be evaluated from multiple perspectives, such as diversity. Accuracy only reflects the correctness of recommendations; a purely accuracy-oriented approach may lead to the echo chamber effect, trapping users in a small subset of familiar items while preventing them from exploring the vast majority of others. To break the echo chamber, diversity in recommender systems is receiving increasing attention. Research shows that as the number of users increases, using diverse recommender systems can effectively increase the average time users spend. The goal of diverse recommendations is to increase the differentiation between recommended items to capture different user interests; however, simply optimizing diversity often leads to a decrease in accuracy. The accuracy-diversity dilemma reflects this trade-off. Therefore, diverse recommender systems aim to increase diversity at the lowest possible accuracy cost. Summary of the Invention

[0006] This invention proposes a diversified recommendation method based on graph contrastive learning. By modifying a Graph Convolutional Neural Network (GCN), it utilizes graph contrastive learning for diversified recommendations. Specifically, this method incorporates a maximum entropy-based neighbor selection module into the GCN neighbor aggregation stage to achieve diversified neighbor subset selection; a layered attention module is designed to mitigate oversmoothing issues using high-order neighbor information; a class-balanced loss function is used to learn long-tail items; and a graph contrastive learning method is designed to alleviate data sparsity problems. This method better addresses the lack of diversity in recommendation systems, achieving a balance between recommendation accuracy and diversity.

[0007] The present invention provides a diversified recommendation method based on graph contrastive learning, comprising the following steps:

[0008] S1: Diverse graph convolutional networks;

[0009] S2: View Builder;

[0010] S3: Loss reweighting;

[0011] S4: Model training.

[0012] Furthermore, the specific steps of S1 are as follows:

[0013] S11: To encode the interaction patterns between users and items, following a common collaborative filtering paradigm, they are embedded into a d-dimensional latent space. Specifically, a latent space of size R is generated for user u and item i respectively. d Embedded vector e u and e i Define the embedding matrix E u ∈R |U|×d and E i ∈R |I|×dThe embeddings for users and items are represented respectively. To propagate the embeddings, a simplified graph convolutional network used in LightGCN is employed, which is defined as shown in the following equation.

[0014]

[0015] in and These represent the embedding representations of user u and item i at layer l, respectively. This is a normalization term in LightGCN, used to avoid excessively large embedding representation values ​​due to multiple aggregation operations; S u and S i It is the set of neighbors selected by the maximum entropy-based neighbor selection algorithm introduced in S12 for user u and item i;

[0016] S12: To generate diverse neighbor subsets during the neighbor aggregation phase, a neighbor selection algorithm based on maximum entropy is designed. The concept of "entropy" was first proposed in 1856 to describe the degree of disorder in a system. The entropy of a closed system increases over time, changing from order to disorder. In 1948, Shannon introduced entropy into information theory to represent the uncertainty of information sources. The information entropy is rewritten for recommender systems as shown in the following equation:

[0017]

[0018] Where S u and S i These are the neighbor subsets selected by user u and item i, respectively. The size of the set is constrained by the hyperparameter k. u' and i' are the unselected neighbors in the neighbor set. sim(e,e') is the cosine similarity function, used to measure the similarity between two vectors. The target maximum entropy function to be optimized is shown in the following formula.

[0019]

[0020] Where N u and N i Let represent the sets of all neighbors of user u and item i, respectively; maximizing the maximum entropy function is an NP-hard problem under cardinality constraints, but it can be bounded by a lower bound of . A greedy algorithm is used to approximate the solution; taking user u as an example, this greedy algorithm starts from the empty set. Initially, each time we add an i∈N term that maximizes marginal revenue. u \S u The specific formula is shown below;

[0021] S u ∪i * →S u

[0022]

[0023] Where i * This represents the neighbors selected at each step; after k steps of greedy neighbor selection, a diverse subset of neighbors for each user is obtained, and this subset is used for subsequent aggregation operations;

[0024] S13: To leverage user-item high-order neighbor representations and mitigate node oversmoothing issues, a layered attention mechanism is designed to integrate multiple embedding propagation layers. For each user or item, the model obtains L sets of embedding vectors through L layers of GNN. Layered attention aims to learn [e (1) ,e (2) ,...,e (L) The Readout function of [] obtains the final node embedding vector, as shown in the following formula:

[0025]

[0026] Where a (l) These are the attention weights of the l-th layer; the calculation is shown in the following formula;

[0027]

[0028] in These are the parameters for attention calculation. Layered attention mechanisms can learn different GNN layer weights to optimize the loss function; the inner product between the final embeddings of user u and item i is used to predict the user's preference for the item.

[0029]

[0030] Furthermore, the specific steps of S2 are as follows:

[0031] S21: Two-View GCL Paradigm. Existing Graph Comparison Learning (GCL) methods typically use the same method to generate two views, which can lead to overly similar views and limit the effectiveness of graph comparison learning. Based on the existing self-supervised collaborative filtering paradigm, this approach uses node self-discrimination to generate positive and negative pairs. Specifically, nodes in different views are considered positive pairs, such as ((e...). u ',e u ")|u∈U), different nodes in different views are considered negative pairs, such as ((e u ',e v ")|u,v∈U,u≠v); Formally, the contrastive loss function is as shown in the following equation, which can maximize the consistency of positive pairs while minimizing the consistency of negative pairs;

[0032]

[0033] The hyperparameter τ is the temperature parameter in the softmax function; similarly, the project loss can be obtained. Combining these two types of losses, we can obtain the objective function for the self-supervised task.

[0034] S22: Edge-DropOut (ED) is a commonly used graph augmentation-based view generation method; random edge deactivation means that edges in the graph are dropped with probability ρ; the graph generated by ED can be modeled as:

[0035] G ED =(V,M"⊙ε)

[0036] Where M"∈{0,1} |ε| It is a mask vector of size |ε| that acts on the edge set to generate a subgraph; not all edges can help in node representation, so this enhancement is expected to capture useful patterns of local node structure and further enhance the robustness of the model against noisy interactions;

[0037] S23: Graph generation models as view generators. Recently emerging learning-based graph generation models offer a promising solution for view generators. They employ variational graph autoencoders (VGAE) as the generative model, combining the concept of variational autoencoders with graph generation. Compared to GAE, VGAE incorporates KL divergence to reduce the risk of overfitting, thereby generating more diverse graphs by increasing uncertainty. Furthermore, VGAE is relatively easier to train and faster than other currently popular generative models (such as generative adversarial networks and diffusion models).

[0038] The model uses a multi-layered diversified GCN as an encoder to obtain the graph embedding; two MLPs are used to fit the mean and standard deviation of the graph embedding respectively; another MLP is used as a decoder to decode the input mean and standard deviation with Gaussian noise to generate a new graph; the loss of VGAE is defined as follows;

[0039]

[0040] in This refers to the KL divergence between node embeddings distributed in a standard Gaussian distribution; It is the cross-entropy loss, which quantifies the dissimilarity between the generated graph and the original graph; It is the BPR loss calculated based on the generated graph; ε' represents the edge set of the generated graph.

[0041] Furthermore, the specific steps of S3 are as follows:

[0042] S31: The number of items in each category of the dataset is highly unbalanced and follows a power-law distribution; a few categories contain most of the items, while most categories have only a limited number of items; training the model by directly optimizing the average loss of all samples will make training for long-tail categories more difficult; the model reweights the sample loss according to the category of the sample during training; if the item belongs to the popular category, the model will relatively reduce its weight; if the item belongs to the long-tail category, the model will relatively increase its weight; this model uses class balance loss to reweight the sample (u,i) according to the effective number of items in each category, and the calculation formula is shown below;

[0043]

[0044] Where β is a hyperparameter that determines the weight, and a larger β will further reduce the weight of the popular category; Map(i) is the mapping between items and categories.

[0045] Furthermore, the specific steps of S4 are as follows:

[0046] S41: Get e u and e i Next, the relevance scores of user u and item i are obtained by calculating the dot product of the two vectors; for each positive sample pair (u,i), a negative sample pair (u,j) is randomly selected to calculate the Bayesian personalization loss; in order to increase the diversity of recommendations, the model reweights the loss to increase the importance of long-tail items; the loss function is shown in the following formula;

[0047]

[0048]

[0049] in The BPR loss is calculated from (u,i) positive sample pairs and (u,j) negative sample pairs, Θ represents all the parameters of the model, and λ1, λ2 and λ3 are the hyperparameters of graph generation loss, contrast loss hyperparameter and L2 regularization parameter, respectively. Attached Figure Description

[0050] Figure 1 This is a flowchart of the method;

[0051] Figure 2 This is a diagram of the VGAE model structure. Detailed Implementation

[0052] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to examples and accompanying drawings.

[0053] As attached Figure 1As shown, a diversified recommendation method based on graph contrastive learning includes the following steps:

[0054] Step 1: Diverse Graph Convolutional Networks

[0055] To encode the interaction patterns between users and items, following a common collaborative filtering paradigm, they are embedded into a d-dimensional latent space. Specifically, a latent space of size R is generated for user u and item i respectively. d Embedded vector e u and e i Define the embedding matrix E u ∈R |U|×d and E i ∈R |I|×d Let represent the embeddings of users and projects, respectively. To propagate the embeddings, a simplified graph convolutional network used in LightGCN is employed, which is defined as shown in the following equation.

[0056]

[0057] in and These represent the embedding representations of user u and item i at layer l, respectively. This is a normalization term in LightGCN, used to avoid excessively large embedding representation values ​​due to multiple aggregation operations; S u and S i It is the set of neighbors of user u and item i selected by the maximum entropy-based neighbor selection algorithm introduced in S12.

[0058] To generate diverse neighbor subsets during the neighbor aggregation phase, a neighbor selection algorithm based on maximum entropy is designed. The concept of "entropy," first proposed in 1856, describes the degree of disorder in a system; the entropy of a closed system increases over time, evolving from order to disorder. In 1948, Shannon introduced entropy into information theory to represent the uncertainty of information sources. The information entropy is rewritten for recommender systems as follows:

[0059]

[0060] Where S u and S i Let be the subsets of neighbors selected by user u and item i, respectively, with the size of the subsets constrained by the hyperparameter k. Let u' and i' be the unselected neighbors in the neighbor set. sim(e,e') is the cosine similarity function used to measure the similarity between two vectors. The target maximum entropy function to be optimized is shown in the following equation.

[0061]

[0062] Where Nu and N i Let represent the sets of all neighbors for user u and item i, respectively. Maximizing the maximum entropy function is an NP-hard problem under cardinality constraints, but it can be bounded by a lower bound of . A greedy algorithm is used to approximate the solution. Taking user u as an example, this greedy algorithm starts from the empty set... Initially, each time we add an i∈N term that maximizes marginal revenue. u \S u The specific formula is shown below.

[0063] S u ∪i * →S u

[0064]

[0065] Where i * This represents the neighbors selected at each step. After k steps of greedy neighbor selection, a diverse subset of neighbors is obtained for each user, and this subset is used for subsequent aggregation operations.

[0066] To leverage user-item high-order neighbor representations and mitigate node oversmoothing, a layered attention mechanism is designed to integrate multiple embedding propagation layers. For each user or item, the model obtains L sets of embedding vectors through L layers of a GNN. The layered attention aims to learn [e (1) ,e (2) ,...,e (L) The Readout function of [] obtains the final node embedding vector, as shown in the following formula:

[0067]

[0068] Where a (l) These are the attention weights for the l-th layer. The calculation is shown in the following formula.

[0069]

[0070] in These are the parameters for attention calculation; the layer attention mechanism can learn different GNN layer weights to optimize the loss function. The inner product between the final embeddings of user u and item i is used to predict the user's preference for the item.

[0071]

[0072] Step 2: View Generator

[0073] The dual-view GCL paradigm addresses the limitations of existing graph contrastive learning (GCL) methods, which typically use the same method to generate two views. This approach can lead to overly similar generated views, limiting the effectiveness of graph contrastive learning. Based on the existing self-supervised collaborative filtering paradigm, this paradigm utilizes node self-discrimination to generate positive and negative pairs. Specifically, nodes in different views are considered positive pairs, such as ((e...). u ',e u ")|u∈U), different nodes in different views are considered negative pairs, such as ((e u ',e v ")|u,v∈U,u≠v). Formally, the contrastive loss function, as shown in the following equation, maximizes the consistency of positive pairs while minimizing the consistency of negative pairs.

[0074]

[0075] The hyperparameter τ is the temperature parameter in the softmax function. Similarly, the project loss can be obtained. Combining these two types of losses, we can obtain the objective function for the self-supervised task.

[0076] Edge-DropOut (ED) is a commonly used graph augmentation-based view generation method that randomly drops edges on user-project interaction graphs. Random edge deactivation means that edges in the graph are dropped with probability ρ. The graph generated by ED can be modeled as follows:

[0077] G ED =(V,M"⊙ε)

[0078] Where M"∈{0,1} |ε| It is a mask vector of size |ε| that acts on the edge set to generate a subgraph. Not all edges contribute to node representation, so this enhancement is expected to capture useful patterns in the local structure of nodes and further improve the robustness of the model to noisy interactions.

[0079] As a view generator, the recently emerging learning-based graph generation model offers a promising solution for view generators. It employs variational graph autoencoders (VGAE) as the generative model, which combines the concept of variational autoencoders with graph generation. Compared with GAE, VGAE incorporates KL divergence to reduce the risk of overfitting, thereby generating more diverse graphs by increasing uncertainty. In addition, VGAE is relatively easier to train and faster than other currently popular generative models (such as generative adversarial networks and diffusion models).

[0080] As attached Figure 2As shown, the model uses a multi-layered diversified GCN as the encoder to obtain the graph embedding. Two MLPs are used to fit the mean and standard deviation of the graph embedding, respectively. Another MLP is used as the decoder to decode the input mean and standard deviation with Gaussian noise, generating a new graph. The loss of VGAE is defined as follows.

[0081]

[0082] in It refers to the KL divergence between the node embedding distribution and the standard Gaussian distribution. It is the cross-entropy loss, which quantifies the dissimilarity between the generated graph and the original graph. This is the BPR loss calculated based on the generated graph. ε' represents the edge set of the generated graph.

[0083] Step 3: Loss Reweighting

[0084] The dataset exhibits a highly imbalanced number of items per category, following a power-law distribution. A minority of categories contain the majority of items, while most categories have only a limited number of items. Training the model by directly optimizing the average loss across all samples would make training for long-tail categories even more difficult. During training, the model reweights the sample loss based on the sample's category. If an item belongs to a popular category, the model will relatively decrease its weight; if an item belongs to a long-tail category, the model will relatively increase its weight. This model uses a class-balanced loss to reweight samples (u,i) based on the number of valid items in each category, as shown in the following formula.

[0085]

[0086] Where β is a hyperparameter that determines the weight, and a larger β will further reduce the weight of the popular category; Map(i) is the mapping between items and categories.

[0087] Step 4: Model Training

[0088] Get e u and e i Next, the relevance score between user u and item i is obtained by calculating the dot product of the two vectors. For each positive sample pair (u,i), a negative sample pair (u,j) is randomly selected to calculate the Bayesian personalization loss. To increase the diversity of recommendations, the model reweights the loss to increase the importance of long-tail items. The loss function is shown in the following equation.

[0089]

[0090] in The BPR loss is calculated from (u,i) positive sample pairs and (u,j) negative sample pairs, Θ represents all the parameters of the model, and λ1, λ2 and λ3 are the hyperparameters of graph generation loss, contrast loss hyperparameter and L2 regularization parameter, respectively.

Claims

1. A diversified recommendation method based on graph contrastive learning, characterized in that... Includes the following steps: S1: Diverse graph convolutional networks; The specific steps in step S1 are as follows: S11: Generate a size of R for user u and project i respectively. d Embedded vector e u and e i Define the embedding matrix E u ∈R |U|×d and E i ∈R |I|×d The embeddings for users and items are represented separately; a simplified graph convolutional network used in LightGCN is employed. S12: Rewrite the information entropy for use in the recommender system; S13: For each user or project, the model can obtain L sets of embedding vectors through L layers of GNN; layer attention aims to learn [e (1) ,e (2) ,...,e (L) The Readout function of [] obtains the final node embedding vector; the inner product between the final embeddings of user u and item i is used to predict the user's preference for the item; S2: View Builder; The specific steps in step S2 are as follows: S21: Based on the existing self-supervised collaborative filtering paradigm, node self-discrimination is used to generate positive and negative pairs; the same node in different views is considered a positive pair ((e u ',e u ")|u∈U), different nodes in different views are considered negative pairs ((e u ',e v ")|u,v∈U,u≠v); The contrastive loss function maximizes the consistency of positive pairs while minimizing the consistency of negative pairs. S22: A view generation method that uses randomly discarded edges on a user-project interaction graph; random edge deactivation means that edges in the graph will be discarded with probability ρ. S23: The variational graph autoencoder (VGAE) is used as the generative model, which combines the concept of variational autoencoders with graph generation. The model uses a multi-layer diversified GCN as the encoder to obtain the graph embedding. Two MLPs are used to fit the mean and standard deviation of the graph embedding respectively. Another MLP is used as the decoder to decode the mean and standard deviation of the input with Gaussian noise to generate a new graph. S3: Loss reweighting; The specific steps in step S3 are as follows: S31: During training, the model reweights the sample loss based on the sample category; if the item belongs to the popular category, the model will relatively reduce its weight; if the item belongs to the long-tail category, the model will relatively increase its weight. S4: Model training; S41: The specific steps in step S4 are: to obtain e u and e i Then, the relevant scores for user u and item i are obtained by calculating the dot product of the two vectors; For each positive sample pair (u,i), a negative sample pair (u,j) is randomly selected to calculate the Bayesian personalized loss.

2. The diversified recommendation method based on graph contrastive learning according to claim 1, characterized in that... The specific steps in step S1 are as follows: S11: The simplified graph convolutional network is defined as shown in the following equation; in and These represent the embedding representations of user u and item i at layer l, respectively. This is a normalization term in LightGCN, used to avoid excessively large embedding representation values ​​due to multiple aggregation operations; S u and S i It is the set of neighbors of user u and item i selected by the maximum entropy neighbor selection algorithm; S12: Rewrite the information entropy for the recommendation system, as shown in the following equation: Where S u and S i These are the neighbor subsets selected by user u and item i, respectively. The size of the set is constrained by the hyperparameter k. u' and i' are the unselected neighbors in the neighbor set. sim(e,e') is the cosine similarity function, used to measure the similarity between two vectors. The target maximum entropy function to be optimized is shown in the following formula. Where N u and N i Let u and i represent the sets of all neighbors of user u and item i, respectively; maximizing the maximum entropy function is an NP-hard problem under cardinality constraints, with a lower bound of 1. A greedy algorithm is used to approximate the solution; this greedy algorithm starts from the empty set. Initially, each time we add an i∈N term that maximizes marginal revenue. u \S u The specific formula is as follows: S u ∪i * →S u Where i * This represents the neighbors selected at each step; after k steps of greedy neighbor selection, a diverse subset of neighbors for each user is obtained, and this subset is used for subsequent aggregation operations; S13: The final node embedding vector is shown in the following formula: Where a (l) The attention weights for the l-th layer are calculated as follows: in These are the parameters for attention calculation; the layer attention mechanism learns different GNN layer weights to optimize the loss function. The inner product between the final embeddings of user u and item i is used to predict the user's preference for the item:

3. The diversified recommendation method based on graph contrastive learning according to claim 2, characterized in that... The specific steps in step S2 are as follows: S21: The contrastive loss function is shown in the following formula: Where the hyperparameter τ is the temperature parameter in the softmax function; the project loss is obtained. Combining contrastive loss and item loss, the objective function for the self-supervised task is obtained: S22: The ED-generated graph model is as follows: G ED =(V,M"⊙ε) Where M"∈{0,1} |ε| It is a mask vector of size |ε| that acts on the edge set to generate a subgraph; S23: The loss of VGAE is defined as follows: For each positive sample pair (u,i), a negative sample pair (u,j) is randomly selected. This refers to the KL divergence between node embeddings distributed in a standard Gaussian distribution; It is the cross-entropy loss, which quantifies the dissimilarity between the generated graph and the original graph; It is the BPR loss calculated based on the generated graph; ε' represents the edge set of the generated graph.

4. The diversified recommendation method based on graph contrastive learning according to claim 3, characterized in that... The specific steps in step S3 are as follows: S31: The model uses class-balanced loss to reweight the samples (u,i) based on the number of valid items in each class. The calculation formula is shown below: Where β is a hyperparameter that determines the weight, and a larger β will further reduce the weight of the popular category; Map(i) is the mapping between items and categories.

5. A diversified recommendation method based on graph contrastive learning according to claim 4, characterized in that... The specific steps in step S4 are as follows: S41: The Bayesian personalized loss function is shown in the following formula: in The BPR loss is calculated from the (u,i) positive sample pairs and (u,j) negative sample pairs, Θ represents all the parameters of the model, and λ1, λ2 and λ3 are the hyperparameters of graph generation loss, contrastive loss and L2 regularization, respectively.

Images