A pseudo-relevant knowledge graph construction method in multi-behavior recommendation

Abstract

Since there are pseudo-relevant interactions in the multi-behavior interaction of a user, a pseudo-relevant knowledge graph construction method in multi-behavior recommendation is disclosed. The main role of the pseudo-relevant knowledge graph is to capture the fine-grained difference between the static characteristics of the item and the dynamic intention of the user, help the model model the pseudo-relevance between items under multi-behavior, and thus supervise and dissociate the pseudo-relevant interactions implied in the multi-behavior interaction of the user, so as to improve the behavior representation ability and recommendation accuracy of the model. Specifically, due to the dynamic and multi-aspect user intention, it is difficult to directly measure the pseudo-relevant relationship between the user and the item in the multi-behavior, so we turn to the pseudo-relevance between items to indirectly measure the pseudo-relevant relationship between the user intention and the item characteristics, and thus construct the pseudo-relevant knowledge graph. Introducing the pseudo-relevant knowledge graph on the basis model of the existing multi-behavior recommendation, and through the disentanglement contrast learning, the pseudo-relevant interaction can be disentangled from the multi-behavior interaction of the user, and then the real semantics is transferred from the auxiliary behavior to the target behavior through the inter-behavior contrast learning, so as to alleviate the sparse supervision signal in the target behavior and improve the behavior representation ability and recommendation accuracy of the model.

Description

Technical Field:

[0001] The present invention relates to the technical fields of graph neural networks, multi-behavior recommendation systems, knowledge graphs, pseudo-correlation representation learning, etc., and specifically designs a method for constructing pseudo-correlation knowledge graphs in multi-behavior recommendation. Background technology:

[0002] With the widespread adoption of the internet and social media platforms, the speed of information generation and dissemination has far surpassed any previous stage in history. While the rapid development of the internet has greatly facilitated people's daily lives, it has also inevitably led to information overload, leaving users often overwhelmed by the vast amount of content. In this situation, users often struggle to efficiently filter out useful information within a limited timeframe, while businesses also face the challenge of effectively attracting users' attention and accurately grasping their needs in a highly competitive market. Therefore, recommender systems, as a core technology for addressing information overload, analyze users' historical data, interaction records, and behavioral trajectories to deeply understand their interests and preferences, thereby predicting content or products they may be interested in. As such, recommender systems are now widely used in e-commerce platforms, online video sites, and social media, helping users quickly find content of interest. This not only effectively reduces information overload, but also improves user experience and enhances the platform's market competitiveness.

[0003] Among various recommendation technologies, graph convolutional neural networks (GCNNs) have been widely used in single-action user-item interaction recommendations due to their ability to incorporate high-order connectivity information into collaborative filtering and integrate multi-hop neighborhoods into node representation learning. Traditional collaborative filtering techniques typically assume that users have only a single interaction with an item. However, in real-world recommendation scenarios, user interactions are not limited to purchases but also include page browsing, adding items to carts, and liking items. These behaviors provide a rich source of information for learning user preferences. Therefore, multi-action recommendation systems propose treating these non-purchase behaviors as auxiliary behaviors to provide additional information support for the model, helping it better model and predict the user's target behavior (i.e., purchase behavior).

[0004] To further leverage the behavioral relationships between users and items contained in multi-behavioral information, knowledge graphs, as a structured representation, have received increasing attention. By constructing users, items, and their attributes into a network, knowledge graphs can effectively represent the relationships between users and items, as well as the potential semantic associations between items. Compared to traditional recommendation technologies, knowledge graphs can more intuitively and finely depict the multi-level connections behind the interactions between users and different items. They are not limited to direct interactive behaviors, but can also introduce implicit associations between items. These semantic relationships help the model understand the user's real needs at a higher level. Therefore, many studies have combined knowledge graphs with recommendation systems, showing great potential in multi-behavioral recommendation scenarios. They can not only explore the fine-grained differences between users and items, but also improve the accuracy and interpretability of recommendations, providing users with a more precise and personalized experience. Summary of the invention:

[0005] 1. Technical Problems Solved

[0006] In real-world user interaction scenarios, since users have different behavioral intentions in different behaviors, the inconsistency between these dynamic user intentions and static item features will lead to the emergence of pseudo-correlated user interactions. Although recent multi-behavior recommendation methods have demonstrated their effectiveness in modeling auxiliary behaviors and applying them to target behavior recommendations, due to the dynamic and multifaceted user intentions in multi-behavior interactions, they have difficulty distinguishing whether the interactions between user intentions and item features are truly relevant or pseudo-correlated interactions during the auxiliary behavior modeling process. However, ignoring the existence of these pseudo-correlated interactions in recommendation scenarios may undermine the model's ability to learn user target intentions.

[0007] In response to the above-mentioned shortcomings of the traditional multi-behavior recommendation model, the present invention proposes a pseudo-correlation knowledge graph construction method. Based on the original user-item multi-behavior interaction graph, a pseudo-correlation-oriented knowledge graph is constructed to capture the fine-grained differences between the static features of the items and the dynamic intentions of the users. This pseudo-correlation knowledge graph can help the model model the pseudo-correlation between items under multiple behaviors, thereby helping the multi-behavior recommendation model supervise and dissociate the pseudo-correlation interactions implicit in the user's multi-behavior interactions, thereby improving the model's behavior representation ability and recommendation accuracy.

[0008] 2. Technical Solution

[0009] Step 1, obtain a public multi-behavior recommendation dataset (such as the Retailrocket dataset, the Tmall dataset, etc.), which should contain multiple interaction behaviors of users on the website, such as page browsing, adding to the shopping cart, purchasing, etc. Data cleaning is performed on the multi-behavior recommendation dataset, thereby deleting some users with less website interaction times, and renumbering the remaining users and items. When dividing the dataset, the leave-one-out evaluation method is used, that is, the last interaction record of each user is taken as the test set, and the remaining records are taken as the training set. After processing, output the training set and test set files.

[0010] Step 2, first, import the multi-behavior training set and calculate the pseudo correlation score of the item by the cumulative item difference between the user multi-behavior interactions, and sample the head entity set according to the pseudo correlation score of the item.

[0011] Step 3, by calculating the difference set and the union set of the auxiliary behavior and the target behavior interaction graph in the training set, the item matrix of the items that the user has interacted with under the auxiliary behavior but has not purchased is obtained. Secondly, for each item in the head entity set, find all users who have browsed the item and extract the item matrix that the user has interacted with under the auxiliary behavior but has not purchased. Through this item matrix, the cumulative number of items that these users have not purchased is counted. Thus, select several items with the most cumulative number as the tail entity set (also known as the pseudo correlation entity set) corresponding to each item in the head entity set. The auxiliary behavior is the relationship corresponding to the connection between the head entity and the tail entity in the pseudo correlation knowledge graph.

[0012] Step 4, according to the head entity set, the tail entity set and the relationship obtained in steps 2 and 3, the pseudo correlation knowledge graph between items is constructed in a one-to-one correspondence.

[0013] Step 5, import the generated pseudo correlation knowledge graph, use the knowledge encoder to learn and dynamically weight the strength of the pseudo correlation relationship between items, and learn the final pseudo correlation knowledge graph embedding through back propagation.

[0014] Step 6, after generating the final pseudo correlation knowledge graph embedding, import the preprocessed multi-behavior dataset. Use graph convolution technology to perform information propagation under specific behavior, and generate multi-behavior embedding of users and items.

[0015] Step 7, apply the pseudo correlation knowledge graph embedding obtained in step 4 to the disentangled contrastive learning method to disentangle the pseudo correlation interactions implied in the multi-behavior embedding. In addition, in order to maximize the use of the disentangled auxiliary behavior user intent, the behavior-to-behavior contrastive learning is used to transfer the truly relevant semantics in the auxiliary behavior to the target behavior.

[0016] Step 8: Train the model based on the two contrastive learning tasks obtained in Step 7. This ensures that the model can disentangle the pseudo-correlation interactions that exist in multi-behavior recommendation scenarios and fully preserve the target behavior intentions contained in the user's interactive behaviors.

[0017] 3. Beneficial Effects

[0018] Compared with the existing technology, the present invention has the following advantages:

[0019] 1. Pseudo-correlation-guided knowledge graphs can perceive that users have different behavioral intentions under different behaviors, thereby capturing the fine-grained differences between the static features of the project and the dynamic intentions of the user.

[0020] 2. Pseudo-correlation knowledge graphs in multi-behavior scenarios can model multi-behavior interaction information. Compared to knowledge graphs in single-behavior scenarios, pseudo-correlation knowledge graphs in multi-behavior scenarios can capture both the coarse-grained commonalities and fine-grained differences between different behaviors, thereby accurately capturing pseudo-correlation relationships between items.

[0021] 3. The knowledge encoder can model the irrelevant representations between items under multiple behaviors based on the pseudo-correlation knowledge graph, which can help the multi-behavior recommendation model identify the pseudo-correlation interactions contained in the user's multi-behavior interactions.

[0022] 4. Adding a pseudo-correlation-oriented knowledge graph to the existing multi-behavior recommendation model can enable the model to gain the ability to disentangle the pseudo-correlations in multi-behavior recommendations, thereby retaining the real relevant semantic information as much as possible and improving the model's behavior representation ability. Description of the drawings:

[0023] Figure 1 Overall flow chart of the pseudo-correlation knowledge graph construction method

[0024] Figure 2 Model framework diagram of the specific application of the present invention in multi-behavior recommendation Specific implementation method:

[0025] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to specific examples and the accompanying drawings.

[0026] This invention recommends items based on user multi-behavior interaction data. The overall process is as follows: Figure 1 The specific steps include:

[0027] Step 1: Download the publicly available Tmall, IJCAI, and Retail multi-behavior recommendation datasets. These datasets contain historical interaction information for four user behaviors: browsing, adding to cart, adding to favorites, and purchasing. Preprocess the multi-behavior datasets to filter out users with too few interactions and renumber the users and items in the dataset. Finally, a time-based leave-one-out evaluation method is used to generate the corresponding training and test set files. Specifically, the test set contains only the user's last interaction item, while the remaining interaction data is used for training. Each processed dataset generates five text files: one containing historical interaction information for the user under four different behaviors (click.txt, cart.txt, favorite.txt, and buy.txt) and one test file (test.txt). In each of these four interaction files, each line consists of a user ID and the ID of the item the user interacted with under that behavior. In the test file, the first value in each line represents the user ID, and the second value represents the ID of the item the user last purchased.

[0028] Step 2. In the multi-action recommendation scenario, user intent is revealed by the items they have previously interacted with. Due to the dynamic and multifaceted user intent, it is intractable to directly measure the pseudo-correlation between users and items in auxiliary actions. Therefore, we calculate the pseudo-correlation between items by calculating the cumulative item difference between the user's auxiliary actions and the target action. Formally, we first measure the pseudo-correlation score by the user interaction item difference between the target action K and the auxiliary action a:

[0029]

[0030] in and Represent the set of items that user u interacts with the target behavior K and the auxiliary behavior a respectively. After obtaining the pseudo-relevance score ρ(i,a) of each item i under behavior a, we use the following formula to sample items as the head entity set corresponding to the relation a in the pseudo-relevance knowledge graph:

[0031]

[0032] Among them, β is the threshold to control the spurious correlation score, is the set of items in the training set. Then, we define the head entity under relation a using the following equation The corresponding pseudo-related entity set (ie, tail entity set):

[0033] A=R a -R a ∩R K ,

[0034]

[0035] Among them, top(·,N) is the top-N function, R a and R K They represent the user-item interaction matrix corresponding to the auxiliary behavior a and the target behavior K respectively. u ,: represents the row vector corresponding to user u in the interaction matrix A. Therefore, the final pseudo-related knowledge graph is obtained by the union of the pseudo-related knowledge graphs corresponding to all auxiliary behaviors a:

[0036] Step 3. Import the generated pseudo-correlation knowledge graph. In order to accurately capture the correlation between items, we use the distance translation knowledge encoder TransR and the semantic matching knowledge encoder TATEC to model the pseudo-correlation between items from multiple perspectives. Among them, TransR can learn the entity representation in the relational space, so that the head (tail) entity reflects the pseudo-correlation features of its corresponding tail (head) entity in the pseudo-correlation knowledge graph under different relations. Similarly, TATEC can capture the potential semantics of the entity, thereby modeling the pseudo-correlation between entities. Therefore, for the triple (h, r, t), the two knowledge encoders TransR and TATEC are defined as follows:

[0037]

[0038] fT ATEC (h,r,t)=e h T M r e t +e h T e r +e t T e r +e h T De t ,

[0039] in is the projection matrix corresponding to the relation r, which can project the entity from the entity space to the corresponding relation space. And D is a diagonal matrix shared by all relations.

[0040] Step 4: The model performs back propagation based on the pseudo-related knowledge graph constructed in step 2 and the two knowledge encoders in step 3 to learn the irrelevance representation between items, thereby obtaining the pseudo-related knowledge graph embedding on the item side (e h ,e r ,e t ).

[0041] Step 5: To integrate the existing pseudo-correlation-oriented knowledge graph into the existing multi-behavior recommendation system, we chose to use the most classic model in the multi-behavior recommendation system as the system's backbone. Therefore, after importing the multi-behavior data, we used LightGCN, a graph encoder widely used in recommendation systems, to perform behavior-specific embedding propagation. The embedding aggregation for a specific behavior can be expressed as:

[0042]

[0043] in and They represent the improved embedding representations of user u and item i at layer l under behavior k. represents the set of interaction items observed by user u in behavior k, represents the set of users who have interacted with item i in behavior k. After L layers of graph convolution, we use the readout function to generate multi-behavior embeddings for prediction under behavior k (here we take the user side as an example, the same applies to the item side):

[0044]

[0045] Among them, f readout Common designs include: using only the last layer of convolutional embeddings, concatenating the convolutional embeddings of all layers, and taking the weighted sum of the convolutional embeddings of all layers.

[0046] Step 6. To disentangle the pseudo-correlation interactions in multi-behavior scenarios, we use mutual information to quantify and minimize the correlation between multi-behavior embeddings and pseudo-correlation knowledge graph embeddings. Specifically, we use the contrastive logarithmic ratio upper bound (CLUB) to model the irrelevance between multi-behavior embeddings and pseudo-correlation knowledge graph embeddings. Therefore, the user representation of each behavior is obtained by LightGCN. and project representation After that, we set CLUB is used to separate pseudo-correlated interactions, thereby improving the item embedding representation of the target behavior K:

[0047]

[0048] Among them, q φ (·|·) is a neural network that fits the conditional distribution p(·|·). Secondly, given that tail entities are specific to interaction relationships in pseudo-related knowledge graphs, we use the following formula to aggregate tail entity representations e t And its corresponding relationship representation e r As a relation-specific representation of the tail entity r,t :

[0049] er,t =LeakyReLU(W(e r +e t )),

[0050] Among them, LeakyReLU is a nonlinear activation function, and W is a trainable parameter matrix used to extract information useful for graph propagation. Similarly, we apply CLUB to the tail entity set To learn multi-behavior item embedding representations to separate pseudo-correlated interactions under auxiliary behavior a:

[0051]

[0052] Combining the disentangled representation of CLUB in the head entity set and the tail entity set, the final disentangled contrast learning objective function is expressed as:

[0053] After disentangling the pseudo-correlation interactions existing in the multi-behavior recommendation scenario, the semantic information in the auxiliary behaviors that is consistent with the target intent is fully preserved. Therefore, in order to enrich the sparse supervision signals in the target behavior, we perform inter-behavior contrastive learning to transfer the semantics of the truly relevant auxiliary behaviors to the target behavior. A widely used idea is to treat the same node embeddings from different behaviors as positive pairs and different node embeddings from different behaviors as negative pairs. Therefore, the project-side optimization objective is expressed as:

[0054]

[0055] Among them, τ is a hyperparameter, which is expressed as a temperature coefficient in softmax, and s(·) is the cosine similarity function.

[0056] Step 7: To predict user preferences, we still use the Bayesian Personalized Ranking (BPR) loss commonly used in recommendation systems as the loss of the main supervision task to encourage the probability estimate of the user's observed interaction to be higher than the probability estimate of its unobserved interaction. The BPR loss formula under behavior k is as follows:

[0057]

[0058] Here, σ is a nonlinear activation function. is the inner product of the final representation of the user and item under k behaviors. The overall BPR loss is the sum of the BPR losses of each behavior: Therefore, the final overall loss consists of the main supervision task loss, the disentanglement contrastive learning loss, and the inter-behavior contrastive learning loss, which is specifically expressed as:

[0059] Step 8. We use the optimizer to optimize the model parameters based on the gradient calculated in Step 7, ensuring that the model can disentangle the pseudo-correlation interactions existing in multi-behavior recommendation scenarios and fully retain the target behavior intentions contained in the user's interactive behaviors, thereby allowing the model to better model and predict the user's target behavior preferences.

[0060] It should be noted that although the embodiments of the present invention described above are illustrative, they are not intended to limit the present invention. Therefore, the present invention is not limited to the above-mentioned specific embodiments. Without departing from the principles of the present invention, any other embodiments obtained by those skilled in the art under the guidance of the present invention are deemed to be within the protection of the present invention.

Claims

1. A pseudo-correlation knowledge graph construction method for multi-behavior recommendation, its features include The steps are as follows: Step 1: Download a publicly available multi-behavior recommendation dataset from the internet. This dataset includes various user interactions on the website. Clean the dataset to remove users with fewer interactions with the website and renumber the remaining users and items. Use a leave-one-out approach to partition the dataset, using the last interaction record for each user as the test set and the remaining records as the training set. Step 2: First, import the multi-behavior training set and calculate the pseudo-relevance score of the item through the cumulative item difference between the user's multi-behavior interactions. The head entity set is sampled based on the pseudo-relevance score of the item; secondly, the matrix of items that the user has interacted with but not purchased under the auxiliary behavior is calculated through the interaction graph of the auxiliary behavior and the target behavior in the training set; for each item in the head entity set, find all users who have browsed the item and extract the matrix of items that they have interacted with but not purchased under the auxiliary behavior; use this item matrix to count the cumulative number of items that these users have not purchased, and then select the items with the largest cumulative number as the tail entity set corresponding to each item in the head entity set, where the auxiliary behavior is the relationship corresponding to the connection between the head entity and the tail entity in the pseudo-relevance knowledge graph; Step 3: Based on the head entity set, tail entity set and their relationships obtained in steps 2 and 3, a pseudo-correlation knowledge graph between projects is constructed in a one-to-one correspondence manner; Step 4: Import the generated pseudo-correlation knowledge graph, use the knowledge encoder to learn and dynamically weight the strength of the pseudo-correlation relationship between items, and obtain the final pseudo-correlation knowledge graph embedding through back-propagation learning; Step 5: After generating the final pseudo-correlation knowledge graph embedding, import the preprocessed multi-behavior dataset; Utilize graph convolution technology to propagate information under specific behaviors and generate multi-behavior embeddings of users and items; Step 6: Apply the pseudo-correlation knowledge graph embedding obtained in step 4 to the disentanglement contrastive learning method to disentangle the pseudo-correlation interactions implicit in the multi-behavior embedding; In addition, in order to maximize the utilization of user intentions in the disentangled auxiliary behaviors, inter-behavior contrastive learning is used to transfer the truly relevant semantics in the auxiliary behaviors to the target behavior; Step 7: Train the model based on the two contrastive learning tasks obtained in step 6 to ensure that the model can disentangle the pseudo-correlation interactions existing in the multi-behavior recommendation scenario and fully retain the target behavior intentions contained in the user's interactive behavior, so that the model can better model and predict the user's target behavior preferences.

2. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 1, wherein: In step 2, the pseudo-correlation score between the target behavior K and the auxiliary behavior a for item i is defined as follows: in and They represent the item sets of user u interacting with the target behavior K and the auxiliary behavior a respectively.

3. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 2, wherein: In step 2, the head entity set corresponding to relation a in the pseudo-related knowledge graph is represented as follows: Among them, β is the threshold to control the spurious correlation score, is the set of items in the training set.

4. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 3, wherein: In step 2, the head entity under relation a The corresponding pseudo-related entity set is represented as follows: A=R a -R a ∩R K , Among them, top(·,N) is the top-N function, R a and R K They represent the user-item interaction matrix corresponding to the auxiliary behavior a and the target behavior K, respectively. u,: Represents the row vector corresponding to user u in the interaction matrix A.

5. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 1, wherein: In step 3, the TransR knowledge encoder and TATEC knowledge encoder are defined as follows: in, is the projection matrix corresponding to the relation r, which can project the entity from the entity space to the corresponding relation space, and D is a diagonal matrix shared by all relations.

6. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 5, wherein: In step 6, applied to the head entity set The disentanglement contrastive learning loss is expressed as follows: Among them, q φ (·|·) is a neural network that fits the conditional distribution p(·|·).

7. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 5, wherein: In step 6, the tail entity-specific relation representation e r,t Defined as: have been r,t =LeakyReLU(W(e r +e t )), Among them, LeakyReLU is a nonlinear activation function, and W is a trainable parameter matrix used to extract information useful for graph propagation.

8. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 7, wherein: In step 6, applied to the tail entity set Disentangled Contrastive Learning Loss It is expressed as follows:

9. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 1, wherein: In step 6, behavior contrast learning is expressed as: where τ is a hyperparameter, expressed as a temperature coefficient in softmax, and s(·) is the cosine similarity function.

10. The method for constructing a pseudo-correlation knowledge graph in multi-behavior recommendation according to claim 1, wherein In step 7, the final overall loss is expressed as:

Images