Contrastive learning graph recommendation method based on noise enhancement

Abstract

The application discloses a contrast learning graph recommendation method based on noise enhancement. The method comprises the following steps: 1, in the embedding module, the user-item interaction relationship is used to construct a user-item bipartite graph and initialize the feature representation of the user-item; 2, in the user-item message passing module, the implicit relationship between the user-item under the message passing and the perception cooperation signal with high-order relationship are captured; 3, in the graph contrast module, the graph is enhanced by using noise, and the node distribution is more uniform by contrasting positive and negative samples; 4, in the prediction module, the final prediction value is obtained by point multiplication of the final representation of each user and item. The application can alleviate the popularity bias and the data sparsity problem, improve the recommendation accuracy, and realize good personalized recommendation effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of recommendation systems, contrastive learning and graph convolutional neural networks, in particular to a simple graph enhanced graph convolutional network score prediction recommendation method. BACKGROUND

[0002] In recent years, with the development of the information age to the big data era, the data size on the Internet today is growing explosively, leading to information overload becoming an increasingly serious problem in our lives. The recommendation system is an effective solution to alleviate the problem of information overload and facilitate users to seek information, and can also increase the traffic and revenue of service providers. Therefore, an efficient and accurate personalized recommendation system has become a focus of attention in academia and industry. At the same time, the research of graph neural network (GNN) is gradually rising, and since most of the information in the recommendation system has a graph structure, GNN has an advantage in graph representation learning, so it has been widely used in recommendation systems.

[0003] Traditional methods are mainly based on collaborative filtering and content filtering techniques, which rely on user historical behavior and item feature information. However, these methods perform poorly in the face of popularity bias and data sparsity. In addition, traditional methods often cannot fully utilize the rich relationships between items and implicit interactions between users. Therefore, a more effective recommendation system method is needed to improve the quality and accuracy of personalized recommendations. Since contrastive learning can extract general features from a large amount of unlabeled data and regularize the representation in a self-supervised manner, and data annotation is not required in contrastive learning, it is an excellent method to solve the problem of data sparsity in recommendation systems. By introducing the contrastive learning method, the recommendation system can better utilize the differences between items and users, improve the effect of personalized recommendation, and alleviate the popularity bias.

[0004] In other words, the recommendation system should have the ability to handle the cold start problem and data sparsity, but the current traditional method performs poorly in the face of popularity bias and data sparsity. SUMMARY

[0005] To solve the above problems, the present application proposes a contrastive learning graph recommendation method based on noise enhancement, which introduces contrastive learning and graph neural network technology to solve the cold start and data sparsity problems in the recommendation system.

[0006] In order to achieve the above purpose, the present application provides a contrastive learning graph recommendation method based on noise enhancement, comprising the following steps:

[0007] Step 1, establishing a problem: a unified user-item bipartite graph is established to model the problem under study, and the input and output of the model are defined, wherein the user is represented by u, the item is represented by i, and when the user u and the item i interact, an edge (u, i) is established;

[0008] Step 2, initial embedding layer: an embedding vector u (0) ∈R d and i (0) ∈R d are used to describe a user and an item, wherein d is the embedding dimension, and the user embedding vector and the item embedding vector are represented by embedding matrices U and I respectively; and ;

[0009] Step 3, embedding propagation layer of the user: the embedding of adjacent nodes is combined to model the user-item interaction;

[0010] Step 4, embedding propagation layer of the item: the embedding of adjacent nodes is combined to model the user-item interaction;

[0011] Step 5, message propagation and graph comparison: message propagation is performed on the initial user embedding vector and the item embedding vector, and different noises are added to the initial data during the message propagation to obtain positive and negative representations of the initial data;

[0012] Step 6, prediction: after L-layer propagation, different layer user-item representations are obtained, the different layer user-item representations are combined to obtain the final representation, and the user-item final representation is used for inner product to obtain the interaction probability of the user to the item, and finally the model is trained by using a loss function.

[0013] As a further improvement of the application, in step 1, the input interaction data is represented by an undirected graph G=(V, E), wherein the node V is composed of a user node u∈U and an item node i∈I, and the edge E includes the edge (u, i) established by the user and the item interaction.

[0014] As a further improvement of the application, in step 2, the user embedding vector and the item embedding vector are represented by embedding matrices U and I respectively, wherein:

[0015]

[0016]

[0017] The one-hot encoding is used as input to describe a user and an item, and the embedding in the matrix U and the matrix I is used as the initialization feature of the user and the item.

[0018] As a further improvement of the present application, in the step 3:

[0019] The graph convolution operation is defined as follows, through which the model aggregates messages,

[0020]

[0021] where, represents the representation of item i after the L-th embedding propagation layer, N u ,N i represent the neighbor sets of users and items respectively; the symmetric normalization term is used to avoid the increase of embedding size caused by the graph convolution operation.

[0022] As a further improvement of the present application, in the step 4:

[0023] The graph convolution operation is defined as follows, through which the model aggregates messages,

[0024]

[0025] where, represents the representation of user u after the L-th embedding propagation layer, N u and N i are the neighbor node sets of users and items respectively; the symmetric normalization term is used to avoid the increase of embedding size caused by the graph convolution operation.

[0026] As a further improvement of the present application, the step 5 is specifically:

[0027] First, construct the noise to be added, formally, given a node i and its representation e i in the d-dimensional embedding space, the noise can be constructed according to the following formula:

[0028] Δ ′ = ω ⊙ sign(z i ), Δ'' = - ω ⊙ sign(z i ), ω ∈ R d ~ U(0, 1)

[0029] where the noise vectors Δ' i , Δ'' i satisfy ‖Δ‖2 = ∈, ∈ is a very small constant, this order constraint makes Δ numerically equal to a point on the hypersphere with radius ∈, each layer after adding noise propagates according to the following formula:

[0030]

[0031]

[0032] As a further improvement of the application, the step 6 is specifically:

[0033] The representations of each layer after propagation are averaged to obtain the final representation:

[0034]

[0035] Wherein, E (0) ∈R |N|*d is the node embedding to be learned, E is the predicted final representation, |N| is the number of nodes, L is the number of propagation layers, is a normalized undirected adjacency matrix without self-connection.

[0036] The recommendation task and the contrast task are jointly trained, and BPR loss and InfoNCE loss are used for training, respectively, and the inner product of the obtained final representation of each user-item is taken to obtain the probability of their interaction:

[0037]

[0038]

[0039] L=L rec +λL cl

[0040] Wherein, σ is the sigmoid function, e u ,e i are the user representation and the item representation interacted with, e j is the representation of a randomly selected item from the item that has not interacted with, z', z'' are the positive representation and negative representation learned after adding noise, τ is the temperature parameter for controlling the smoothness of the model, λ is the weight coefficient of the contrast loss, and the InfoNCE loss promotes the consistency between z i ′,z i ″

[0041] ″′

[0042] , while minimizing the consistency between z i ,z j .

[0043] Compared with the prior art, the application has the following technical effects:

[0044] The present application captures the implicit relationship between user items in message passing, adopts a contrast method based on noise enhancement, adjusts the uniformity of representation distribution in a more direct way by adding different directed random noise in the representation of original data, enhances the recommendation effect, and solves the popularity bias and data sparsity problem in the recommendation system. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 It is a flow chart of the noise-enhanced contrast learning graph recommendation method of the present application.

[0046] Figure 2 It is Figure 1 The overall framework diagram of the recommendation method model is shown. DETAILED DESCRIPTION

[0047] In order to facilitate those skilled in the art to understand and implement the present application, the present application will be further described in detail below in conjunction with examples, and it should be understood that the examples described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0048] Figure 1 It is a noise-enhanced contrast learning graph recommendation model, which includes four modules: embedding module, user item message passing module, graph contrast learning module and final prediction module, and the recommendation method using the model specifically includes the following steps:

[0049] Step 1: Establish the problem, establish a unified user-item bipartite graph to model the problem studied, if user u and item i have interaction, there will be an edge (u, i) established. The input and output of the model are determined;

[0050] Step 2: Initial embedding layer, user embedding vector and item embedding vector can be represented by embedding matrix U and I respectively:

[0051]

[0052]

[0053] The embedding in matrix U and matrix I can be used as the initial feature of user and item, which will be optimized, and in the framework, the embedding is refined by propagating the embedding on the user-item interaction graph. This provides more effective embedding for the recommendation task, because the refinement step explicitly injects collaborative signals into the embedding;

[0054] Steps 3 and 4: User (item) embedding propagation layer, intuitively, the items interacted by a user provide direct evidence of user preference, similarly, the users operating an item can be regarded as the features of the item, which are used to measure the collaborative similarity of two items. The message propagation is carried out by the following formula:

[0055]

[0056]

[0057] The embedding propagation layers in the model are built on the message passing architecture of LightGCN, which is used to capture the CF signals and the graph structure of user-item heterogeneous interactions. The basic idea of LightGCN is to learn the representation of nodes by smoothing the features on the graph. In the model, the representation of a user (item) is modeled by accumulating incoming messages from all the heterogeneous interaction items (users). It only contains the most important component of neighborhood aggregation for collaborative filtering. Specifically, LightGCN learns user and item embeddings by linearly propagating them on the user-item interaction graph and uses the weighted sum of the learned embeddings at all layers as the final embedding;

[0058] Stacking more embedding propagation layers to explore high-order connectivity information. This high-order connection is crucial for encoding the collaborative signal to estimate the relevance between users and items. By stacking L embedding propagation layers, a user (and an item) can receive messages propagated from its L-hop neighbors. In the Lth step, the representation of user u is recursively represented as formula (1), is the representation of items generated by the previous message passing step, which remembers the messages from its (L-1) hop neighbors. It further helps the representation of user u at the Lth layer. Similarly, the representation of items at the Lth layer can be obtained;

[0059] Step 5: Construct two different noise added in the step of message aggregation, so that the positive and negative representations of the original data are obtained, and by comparing the consistency of the two versions, the representation of users and items can be better learned, and the generalization contained therein can be discovered, which helps better embedding learning and final prediction.

[0060] First, construct the noise to be added. Formally, given a node i and its representation e i in the d-dimensional embedding space, the noise can be constructed according to the following formula:

[0061] Δ' = ω ⊙ sign(z i ), Δ'' = -ω ⊙ sign(z i ), ω ∈ R d ~ U(0, 1)

[0062] where the noise vectors Δ' i , Δ" i satisfy ‖Δ‖2 = ∈, ∈ is a very small constant, and this order constraint makes Δ numerically equal to a point on the hypersphere with radius ∈. Each layer after adding noise propagates according to the following formula:

[0063]

[0064]

[0065] After propagation through L layers, we obtain multiple representations of user u: Multiple representations of item i: The representations obtained from different layers emphasize the information passed from different hop layers. For example, the first layer enforces smoothness on users and their interacted items, the second layer smooths the users connected to the interacted items, and higher layers capture higher-order relationships. Therefore, we further combine them to obtain the final representation form: here, a simple uniform weight is set for each embedding layer This can achieve good general performance.

[0066] Joint training of recommendation task and contrastive task: After propagation, we obtain the representations of users u and items i, as well as positive representations and negative representations with added noise. The representations obtained from different layers emphasize the information passed from different hop layers. Therefore, we further combine them to obtain the final representation form, and jointly train the recommendation task and the contrastive learning task, and finally predict the probability of their interaction by calculating the inner product of users and items:

[0067]

[0068]

[0069] L = L rec + λL cl .

[0070] where σ is the sigmoid function, e u ,e i are the representations of users and items interacted with them, respectively, e j is the representation of a randomly selected item that has not interacted with it. z ′ ,z″ are positive and negative representations learned after adding noise, and τ is a temperature parameter that controls the smoothness of the model

[0071] ″′

[0072] number, and λ is the weight coefficient of the contrastive loss. The InfoNCE loss promotes the consistency between z i ,z i and z

[0073] ″′

[0074] , while minimizing the consistency between z i ,zj consistency between them.

[0075] To sum up, the noise enhancement-based contrast learning graph recommendation method of the present application can alleviate the popularity bias and data sparsity problem, improve the recommendation accuracy, and achieve good personalized recommendation effect.

[0076] The above description is only the preferred embodiments of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A contrastive learning graph recommendation method based on noise enhancement, characterized in that, Includes the following steps: Step 1: Problem Formulation: Establish a unified user-item bipartite graph to model the problem under study, and clarify the input and output of the model. Here, the user is represented by u, and the item is represented by i. When user u and item i interact, an edge (u, i) will be established. Step 2, Initial Embedding Layer: Using embedding vector u (0) ∈R d and i (0) ∈R d Let d be the embedding dimension, and let d be the embedding vector of the user and the embedding vector of the item, respectively, described by the embedding matrix. and To indicate; Step 3, User Embedding Propagation Layer: Embedding and combining items from adjacent nodes to model user-item interactions; graph convolution operation is defined as follows, and the model uses it for message aggregation. in, N represents the representation of item i obtained after embedding the propagation layer at layer L. u N i Represent the neighbor sets of users and projects, respectively; use symmetric normalization terms. To avoid the increase in embedding size caused by graph convolution operations; Step 4, Project Embedding and Propagation Layer: This layer embeds and combines users from adjacent nodes to model user-project interactions; the graph convolution operation is defined as follows, and the model uses it for message aggregation. in, N represents the representation of user u obtained after embedding the propagation layer at layer L. u and N i These are the sets of neighboring nodes for users and projects, respectively; using symmetric normalization terms. To avoid the increase in embedding size caused by graph convolution operations; Step 5, Message Propagation and Graph Comparison: Perform message propagation on the initial user embedding vector and item embedding vector, adding different noises during the message propagation process to obtain positive and negative representations of the initial data. Specifically, this includes: first, constructing the noise to be added; formally, given a node i and its representation e in the d-dimensional embedding space... i The noise is constructed according to the following formula: Δ′=ω⊙sign(z i ),Δ″=-ω⊙sign(z i ),ω∈R d ~U(0,1) Where the noise vector Δ′ i ,Δ″ i The constraint that ||Δ||² = ∈, where ∈ is a very small constant, makes Δ numerically equal to a point on a hypersphere with radius ∈. Each layer after adding noise propagates according to the following formula: Step 6, Prediction: After L layers of propagation, user item representations at different layers are obtained. These representations are combined to obtain the final representation. The inner product of these final representations is then performed to obtain the user's interaction probability with each item. Finally, the model is trained using a loss function. Specifically, this includes averaging the representations after each layer of propagation to obtain the final representation. Among them, E (0) ∈R |N|*d Here, E is the node embedding to be learned, E is the final predicted representation, |N| is the number of nodes, and L is the number of propagation layers. It is a normalized undirected adjacency matrix without self-connections; The recommendation and comparison tasks are jointly trained using BPR loss and InfoNCE loss respectively. The inner product is then performed on the final representation of each user item to obtain the probability of their interaction. L=L rec +λL cl Where σ is the sigmoid function, e u ,e i These are the user representation and the object representation that the user interacts with, e j Z is the representation of a randomly selected item from those with no interaction. Z′ and Z″ are the positive and negative representations learned after adding noise. τ is the temperature parameter controlling the smoothness of the model, and λ is the weighting coefficient of the contrastive loss. The InfoNCE loss promotes the positive and negative representations of each other. i ,z″ i Consistency between them, while minimizing the z′ of mutually negative samples. i ,z″ j Consistency between them.

2. The contrastive learning graph recommendation method according to claim 1, characterized in that, In step 1: the input interaction data is represented by an undirected graph G = (V, E), where nodes are composed of user nodes u ∈ U and project nodes i ∈ I, and the edges in E include the edges (u, i) established by the interaction between the user and the project.

3. The contrastive learning graph recommendation method according to claim 1, characterized in that, In step 2: the user embedding vector and the item embedding vector are represented by embedding matrices U and I, respectively, where: One-hot encoding is used as input to describe a user and an item, and the embeddings in matrices U and I are used as the initial features of the user and the item.

Images