Multi-behavior graph recommendation method based on comparative learning

Abstract

The invention discloses a multi-behavior graph recommendation method based on comparative learning. The recommendation accuracy can be improved by fully utilizing auxiliary behaviors. The method belongs to the field of artificial intelligence, data mining and recommendation systems. The overall flow chart is as shown in figure 1; the method comprises the following steps: firstly, generating different behavior embedding of a user and a project by using a LightGCN model, and synthesizing the embedding of each behavior type through a self-attention mechanism; and then performing three comparative learning tasks, namely behavior consistency comparative learning, behavior difference comparative learning and data enhancement denoising comparative learning. And finally, combining the comparative learning loss with the BPR loss of the target behavior, and optimizing the model to improve the recommendation performance. Compared with the prior art, the method has the advantages that the auxiliary behaviors are fully utilized, the difference between the behaviors is not concerned, the generality of the user reflected between the behaviors is more concerned, meanwhile, the influence caused by noise interaction can be effectively removed, and the method has wide application prospects in recommendation systems such as e-commerce and the like.

Description

Technical Field

[0001] The invention relates to a multi-behavior graph recommendation method based on contrastive learning, and belongs to the fields of artificial intelligence, data mining and recommendation systems. Background Art

[0002] With the vigorous development of emerging technologies such as the Internet, data traffic represented by online shopping, short videos, and news headlines has exploded. We have entered an era of information overload. How to extract effective and user-interesting content from these massive amounts of information has attracted the attention of researchers. Recommendation systems are an effective way to solve information overload. By mining user interest preferences from a large amount of data and effectively identifying information, the results are pushed to users in the form of personalized lists. For example, e-commerce systems such as Tmall, Taobao, and JD.com use recommendation algorithms to quickly recommend content that users may be interested in. This has brought great convenience to people's lives and has changed people's lifestyles to a certain extent.

[0003] Traditional recommendation systems mainly make recommendations based on historical behavior data of users. These historical behavior data can be divided into two categories: explicit feedback and implicit feedback. Explicit feedback refers to the behavior of users directly expressing likes or dislikes, such as purchases and ratings, while implicit feedback refers to the records left by users in the process of achieving target behaviors, such as browsing and clicking. In recommendation systems, the most widely recognized and applied method is collaborative filtering. The current mainstream recommendation algorithms based on matrix decomposition are MF and BPRMF. With the rise of deep learning, collaborative filtering methods have evolved from matrix decomposition models to models based on deep learning, such as NCF and Wide&Deep. Recommendation models based on deep learning can better learn the representation of users and items and capture more complex interactive relationships between them. Since most of the data in the recommendation system can be regarded as graph structure data, such as users and projects can be regarded as nodes, and the nodes are related to each other and correspond to several edges. The model based on graph convolutional network adopts the method of aggregated neighborhood embedding to propagate and iterate. By stacking convolutional layers layer by layer, each node can obtain the information of higher-order neighbor nodes and capture high-order relationships, so as to more effectively learn relevant feature attributes and further improve the recommendation effect. Most of the above system filtering models are based on a single behavior, usually modeling the target behavior (for example, purchasing behavior on an e-commerce platform). However, in real-world recommendation systems, target behaviors are usually very sparse, which leads to the problem of sparsity of supervisory signals. In reality, before users make a target behavior, they usually have a series of other behaviors (such as browsing or clicking, as well as collecting and adding to shopping carts, etc.), and there may be certain connections between these behaviors. Most people browse dozens of times more items than they buy. A large number of auxiliary behaviors outside of these target behaviors also contain some valuable user preference information. Their interactions are usually richer than target behaviors, which can be used to alleviate the problem of data sparsity and help predict the final recommendation results.

[0004] Some recent studies have considered multi-behavior based recommendation methods, including MBGCN and MBGMN. Some methods simply apply the recommendation method of a single behavior to multiple behaviors, or design new sampling strategies; some methods use attention to capture the relationship between and within behaviors, or enrich the embedding of users and items. Although the existing methods are effective, there are still some limitations in the recommendation field: one is the sparsity of the target supervision signal. Although researchers have tried their best to use other types of auxiliary behaviors to alleviate this problem, how to effectively model the coarse-grained commonalities between behaviors needs to be improved. The single behavior representation of a user reflects the user's preference for the corresponding behavior, and these behaviors also share certain commonalities to reflect the user himself. Take the e-commerce field as an example. Before purchasing, users often browse and click on similar products in large quantities, and also compare the products that are difficult to choose before adding them to the shopping cart. Therefore, these behaviors are related to a certain extent, reflecting the user's preferences and having commonalities. Considering the commonalities between these behaviors can help prediction and better solve the problem of sparse target data. However, existing methods model more differences between behaviors and less commonalities. Second, the introduction of auxiliary behavioral data will inevitably introduce noisy noise interactions. For example, if a user mistakenly clicks on an item that he is not interested in, these noise data may affect the final recommendation accuracy. Therefore, how to improve the robustness to noise interactions remains to be studied. Summary of the invention

[0005] Aiming at the problem that existing methods do not pay enough attention to the coarse-grained commonalities and fine-grained differences between behaviors and the problem of dealing with the influence of noise data, the present invention proposes a multi-behavior graph recommendation method based on contrastive learning.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A multi-behavior graph recommendation method based on contrastive learning includes the following steps:

[0008] Step S1, download a public multi-behavior dataset, pre-process it to obtain a multi-behavior dataset, and divide the multi-behavior dataset into a training set and a test set.

[0009] Step S2: Use the graph neural network to train the training set, adjust the trainable parameters to optimize the comprehensive loss of the multi-behavior graph neural network, and obtain the prediction result, namely:

[0010] Step S2.1: Multi-behavior information embedding learning.

[0011] The LightGCN model, a simplified graph neural network model, is used to obtain the embedding representation of users and items through the propagation of different behavior nodes. The embedding representation of users and items is initialized as and LightGCN updates the embedding by propagating through K layers:

[0012]

[0013]

[0014] In formula (1)(2), Represents the normalization coefficient, N u represents the set of items interacted with by user u, N i represents the set of users who interact with item i; (k) represents the encoding operation performed on the kth layer of the neural network. After propagation through the K layers of embedding, the final embedding of users and items is expressed as:

[0015]

[0016]

[0017] To address the data sparsity problem that may exist in single-behavior representation, multiple behavior embeddings are aggregated. The self-attention mechanism is used to dynamically assign weights according to task requirements, so as to better capture the heterogeneous information between different behavior types and obtain a comprehensive embedding representation for subsequent contrastive learning tasks:

[0018]

[0019]

[0020] In formula (5) and (6), e u represents the comprehensive embedding of the user under all behavior types, α b is the self-attention weight, q is the query vector, and similar operations can be applied to the aggregation of items.

[0021] Step S2.2: Behavior consistency contrast learning.

[0022] Influenced by the contrastive learning strategy, the present invention uses multiple views to construct different representations and uses them as auxiliary signals to alleviate the sparsity of the target behavior data. Since the multiple behaviors of a user reflect their overall characteristics together, the different behavior representations of the user are regarded as user-level information enhancement while capturing the common features between behaviors. Specifically, select users u, v, where u, v∈U and u≠v. The embeddings of user u under different behavior types are and is regarded as a positive sample pair, and the embedding of users u and v under any behavior type and It is regarded as a negative sample pair. Based on InfoNCE, the following contrast loss function is defined to achieve this:

[0023]

[0024] In formula (7), sim(·, ·) represents the similarity measurement function, and τ is the parameter that controls the smoothness of the softmax function. By minimizing this loss function, the model can effectively capture the common characteristics of users under different behavior types, thereby generating more consistent user representations.

[0025] Step S2.3: Behavioral Difference Contrastive Learning

[0026] The goal of behavior difference contrastive learning is to capture the fine-grained differences between different behaviors. Especially in application scenarios such as e-commerce, it is crucial to distinguish between high-priority behaviors (such as purchases) and auxiliary behaviors (such as clicks or browsing). Specifically, we first define the behavior priority in the multi-behavior recommendation task, the items of user u under the target behavior For user u, the embedding representations under behavior type b are First, get it close to the target project Secondly, hard negative samples close to auxiliary behaviors Finally, with random negative samples Keeping the distinction clear, the following loss function is defined for this purpose:

[0027]

[0028] In formula (8), Represents user u and target behavior The similarity between Represents user u and auxiliary behavior The similarity between Represents user u and negative sample behavior The similarity between them is , log(σ(·)) is the log-sigmoid function, which is used to measure the difference between two similarities. In order to further improve the model’s ability to capture subtle differences between different behavior types, the loss weight parameter β is set here. u Designed to dynamically adjust:

[0029]

[0030] In (9), α is a scaling factor that controls the amplitude of dynamic adjustment. Through this dynamic weight adjustment mechanism, the model can adaptively balance the similarity difference between the target behavior and the auxiliary behavior, so that the fine-grained differences between different behavior types can be better captured during the training process. By minimizing the loss function, the contrastive learning can capture the subtle differences between different behavior types, thereby achieving a more accurate understanding of the target behavior preference.

[0031] S2.4. Data enhancement denoising contrastive learning.

[0032] Although the method of using auxiliary behaviors as additional information to target sparse supervision signals can enhance the recommendation effect, the introduction of a large amount of auxiliary information will also introduce noise information, such as accidental clicks. These interactions cannot accurately reflect the user's interests, and these noises will be amplified as the number difference between auxiliary behaviors and target behaviors in the data set is amplified. The present invention adopts two data enhancement strategies, node dropping and edge perturbation, to generate different views of interaction data by changing the graph structure, thereby improving the robustness of the model to noise.

[0033] Specifically, the present invention generates two enhanced views G′ and G″ by performing node discarding and edge perturbation on the original user-item interaction graph G. In these views, node embeddings are propagated and learned through the LightGCN module. On these views, node discarding aims to perceive the more important information nodes in embedding propagation and reduce the model's dependence on some noise nodes, while edge perturbation improves the model's ability to resist noise interactions by capturing local structures in the view that are helpful for node embedding learning.

[0034] For the enhanced user node and item node embedding representations, the embeddings of user u on views G′ and G″ are e′ respectively. u and e″ u In order to take into account the similarity between different views, e′ u and e″ u is regarded as a positive sample pair, and the embedding e′ of different users u and v in any view is u and e″ v is regarded as a negative sample pair, thereby maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs. The denoising contrast learning loss function is:

[0035]

[0036] Among them, sim(·) represents the similarity metric function, and τ is the temperature hyperparameter, which is used to control the smoothness of the softmax function. By minimizing the contrastive learning loss function, the interference of noise data on the model learning process can be effectively reduced, and the model's generalization ability of user behavior can be enhanced, thereby generating a more accurate and robust user embedding representation.

[0037] S2.5 Joint Optimization Module

[0038] Integrating steps 2.2-2.4, the present invention combines the loss functions of multi-behavior consistency contrast learning, behavior difference contrast learning, and data enhancement denoising contrast learning to form the final loss function of the model:

[0039] L total=λ0L bpr +λ1L common +λ2L differ +λ3L denoise (11)

[0040] Among them, λ is a hyperparameter used to adjust the weights of different contrastive learning losses in the total loss function. By optimizing this comprehensive loss function, the model can not only learn the commonalities and differences of users under different behavior types, but also effectively filter noise and improve recommendation performance.

[0041] Technical Effects

[0042] Compared with the existing methods, the multi-behavior graph recommendation method based on contrastive learning proposed in this invention has the following advantages and effects:

[0043] 1. Based on the data characteristics of the recommendation field, effectively use auxiliary behaviors to improve the accuracy of recommendations

[0044] 2. Through multiple comparative learning strategies, we can better learn user preferences from both commonalities and differences, effectively reduce noise interference, and make recommendation results more reliable.

[0045] 3. It has broad application prospects in tasks such as product recommendation, news recommendation and job recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Flowchart of the multi-behavior graph recommendation method based on contrastive learning.

[0047] Figure 2 It is a model framework diagram of the present invention. DETAILED DESCRIPTION

[0048] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific examples.

[0049] See also Figure 1 and Figure 2 , a multi-behavior graph recommendation method based on contrastive learning, comprising the following steps:

[0050] Step S1, download a public multi-behavior dataset, pre-process it to obtain a multi-behavior dataset, and divide the multi-behavior dataset into a training set and a test set.

[0051] In this example, the multi-behavior data is mainly historical behavior data obtained from the existing e-commerce system website. The behavior types include purchase, favorite, add to cart, click, browse, etc. These multi-behavior interaction data are preprocessed, and users who have interacted with the items for a short time are filtered. The users and items are re-indexed, and the total number of users and items is counted for subsequent processing. The leave-one-out method is used when dividing the training set and the test set. All items under all behaviors of each user are used for training, and only the items under the last behavior are used for testing. This method is particularly effective on limited data sets because it can make full use of all historical behaviors of each user to train the model, thereby improving the generalization ability of the model.

[0052] The present invention defines the set of users and items as U = {u1, u2, ..., u M}, I = {i1, i2, ..., i N}, where M and N represent the number of users and items respectively. Assuming that users in the system have B behaviors, use W = [W 1 , W 2 , ..., W b , ..., W B ] represents a list of interaction matrices sorted in the defined order, where W b represents the interaction matrix of the b-th type of behavior, W B represents the interaction matrix of the target behavior, and B is the number of behaviors. In particular, if user u and item i have an interaction under behavior type b, then Otherwise = 0. The unified graph is defined as G = (V, E), which is a bipartite graph under all types of behavioral interactions. Where V = U∪I, represents the entire set of nodes involved in users and items, and the edge set E represents the observed interactions. The entire graph can be split into individual behavioral graphs {G1, G2, ..., G}, such as G b =(V b , E b ), is the interaction matrix W under the b-th behavior b Constructed bipartite graph. V b is a vertex set consisting of user nodes u∈U and item nodes i∈I, E b Representation graph G b The interaction edge between users and items in . Then there is an edge between the user node and the item node.

[0053] Step S2: The LightGCN model and self-attention mechanism are used to aggregate the multi-behavior information of users and projects into embedded representations, and multiple contrastive learning strategies (behavior consistency, behavior difference, denoising contrastive learning) are jointly optimized to improve the recommendation performance of the model and the understanding of user behavior. Finally, the trainable parameters are adjusted to optimize the comprehensive loss of the multi-behavior graph neural network to obtain the prediction result, namely:

[0054] Step S2.1, multi-behavior embedding learning: Use the LightGCN model to generate embeddings of users and items under different behaviors, and perform weighted aggregation of embeddings of different types of behaviors through the self-attention mechanism to capture the heterogeneous information between behaviors.

[0055] Initialize the embedding representation of users and items as and LightGCN updates the embeddings through K layers of propagation: the final embeddings of users and items are represented as The self-attention mechanism is used for aggregation, and weights are dynamically assigned according to task requirements to better capture the heterogeneous information between different behavior types and obtain a comprehensive embedding representation. u and e i .

[0056] In this example, the graph neural network that integrates multiple user behaviors uses a three-layer LightGCN.

[0057] Step S2.2, behavior consistency contrastive learning: minimize the distance between the embedding representations of the same user under different behavior types and maximize the embedding difference between different users to capture the coarse-grained common features of users in different behaviors:

[0058] For user u, the embedding representations under behavior types b and b′ are and The present invention minimizes the distance between the embedding representations of the same user under different behavior types through contrastive learning. When defining the positive and negative sample pairs, select users u, v, where u, v∈U and u≠v. and is regarded as a positive sample pair, and the embedding of users u and v under any behavior type and are considered as negative sample pairs.

[0059]

[0060] In (7), sim(·, ·) represents the similarity measurement function, and τ is the parameter that controls the smoothness of the softmax function. By minimizing this loss function, the model can effectively capture the common characteristics of users under different behavior types, thereby generating more consistent user representations.

[0061] S2.3. Behavior Difference Contrastive Learning: By minimizing the similarity difference between the target behavior and auxiliary behaviors and random negative sample behaviors, and dynamically adjusting the loss weight parameters, it can capture the fine-grained differences between users in different behavior types.

[0062] First, define the behavior priority in the multi-behavior recommendation task, the items of user u under the target behavior For user u, the embedding representations under behavior type b are It first approaches the target project Secondly, hard negative samples close to auxiliary behaviors Finally, with random negative samples Keep the distinction clear. The loss function of difference contrastive learning is as follows:

[0063] middle, Represents user u and target behavior The similarity between Represents user u and auxiliary behavior The similarity between Represents user u and negative sample behavior The similarity between them is , log(σ(·)) is the log-sigmoid function, which is used to measure the difference between two similarities. In order to further improve the model’s ability to capture subtle differences between different behavior types, the loss weight parameter β is u Designed to dynamically adjust:

[0064]

[0065] In (9), α is a scaling factor used to control the amplitude of dynamic adjustment.

[0066] Step S2.4, data augmentation denoising contrastive learning: Different views are generated through node dropping and edge perturbation strategies to enhance the model’s robustness to noise, and robust user embedding representations are generated by minimizing the contrast loss between positive and negative sample pairs.

[0067] By dropping nodes and perturbing edges on the original user-item interaction graph G, two enhanced views G′ and G″ are generated. In these views, node embeddings are propagated and learned through the LightGCN module. The embeddings of user u in views G′ and G″ are e′ respectively.u and e″ u In order to take into account the similarity between different views, e′ u and e″ u is regarded as a positive sample pair, and the embedding e′ of different users u and v in any view is u and e″ v is regarded as a negative sample pair, thereby maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs. The contrastive learning loss function is defined as follows:

[0068]

[0069] In (10), sim(·) represents the similarity metric function, and τ is the temperature hyperparameter used to control the smoothness of the softmax function. By minimizing the contrastive learning loss function, we can effectively reduce the interference of noisy data on the model learning process and enhance the model's generalization ability to user behavior, thereby generating more accurate and robust user embedding representations.

[0070] Step S2.5, joint optimization: Combine the loss functions of behavior consistency contrast learning, behavior difference contrast learning, and data enhancement denoising contrast learning with the BPR loss of the target behavior to construct the final comprehensive loss function, and improve the recommendation performance of the model by optimizing the loss function.

[0071] In this example, the model is trained using the gradient descent method, and the above loss function is calculated to update the model parameters until the loss function converges or the maximum number of training times is reached and the optimal recommendation model is obtained to predict the probability of user interaction with items. The top-n recommendation hit rate (HR@N) and normalized discounted cumulative gain (NDCG@N) are used as evaluation indicators. Higher HR and NDCG scores represent better performance of the model. Among them, HR@N represents the proportion of users with recommended hits to the total number of users. As long as the items in the test set appear in the user's top-n recommendation list, NDCG@N is used to evaluate the recommendation performance by combining the relevance scores and arrangement order of the items in the top-n recommendation list, focusing on the ranking of the items in the test set in the user's top-n recommendation list.

[0072] It should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. A multi-behavior graph recommendation method based on contrastive learning, characterized in that: The steps include: S1. Download a public multi-behavior dataset, pre-process it to obtain a multi-behavior dataset, and divide the multi-behavior dataset into a training set and a test set; S2. The LightGCN model and self-attention mechanism are used to aggregate the multi-behavior information of users and projects by embedding them, and a variety of contrastive learning strategies (behavior consistency, behavior difference, denoising contrastive learning) are jointly optimized to improve the recommendation performance of the model and the understanding of user behavior. Finally, the trainable parameters are adjusted to optimize the comprehensive loss of the multi-behavior graph neural network to obtain the prediction results, namely: S2.

1. Multi-behavior embedding learning: Use the LightGCN model to generate embeddings of users and items under different behaviors, and perform weighted aggregation of embeddings of different types of behaviors through the self-attention mechanism to capture the heterogeneous information between behaviors; S2.2, behavior consistency contrastive learning: minimize the distance between the embedding representations of the same user under different behavior types and maximize the embedding difference between different users to capture the coarse-grained common features of users in different behaviors; S2.3, Behavior Difference Contrastive Learning: By minimizing the similarity difference between the target behavior and auxiliary behaviors and random negative sample behaviors, and dynamically adjusting the loss weight parameters, it can capture the fine-grained differences between users in different behavior types; S2.4, Data Augmentation Denoising Contrastive Learning: Generate different views through node dropout and edge perturbation strategies to enhance the model’s robustness to noise, and generate robust user embedding representations by minimizing the contrast loss between positive and negative sample pairs: S2.5, Joint optimization: Combine the loss functions of behavior consistency contrast learning, behavior difference contrast learning, and data enhancement denoising contrast learning with the BPR loss of the target behavior to construct the final comprehensive loss function, and improve the recommendation performance of the model by optimizing the loss function.

2. The multi-behavior graph recommendation method based on contrastive learning as claimed in claim 1, characterized in that: The self-attention mechanism in step S2.1 dynamically assigns weights according to task requirements and weights different behavior embeddings by query vectors to obtain the final comprehensive embedding representation of users and items.

3. The multi-behavior graph recommendation method based on contrastive learning as claimed in claim 1, characterized in that: The behavior consistency contrastive learning in step S2.2 minimizes the difference between the embedding representations of users under different behavior types through the InfoNCE loss function, and maximizes the difference in embedding representations between different users, thereby enhancing the model's ability to capture common features of users.

4. The multi-behavior graph recommendation method based on contrastive learning as claimed in claim 1, characterized in that: The behavior difference contrastive learning in step S2.3 aims to improve the model's understanding of the user's target behavior preferences by defining behavior priorities and using dynamically adjusted loss weight parameters to control the similarity differences between different behavior types.

5. The multi-behavior graph recommendation method based on contrastive learning as claimed in claim 1, characterized in that: The data enhancement denoising contrastive learning in step S2.4 reduces the model's dependence on noisy nodes through a node discarding strategy, and captures local structures that facilitate node embedding learning through an edge perturbation strategy, thereby improving the model's robustness to noise.

6. The multi-behavior graph recommendation method based on contrastive learning as claimed in claim 1, characterized in that: The comprehensive loss function in step S2.5 combines the losses of multiple contrastive learning tasks with the BPR loss of the target behavior, and uses hyperparameters to control the weight ratio of different losses, balancing the impact of the model on behavioral commonalities, differences, and noise removal, and ultimately improving the overall performance of the recommendation system.

Images