A knowledge graph recommendation method based on preference transfer

Abstract

This invention belongs to the field of recommender system technology and provides a knowledge graph recommendation method based on preference transfer. The attention embedding propagation layer mainly consists of an attention embedding propagation layer, a preference transfer layer, and a prediction layer. The attention embedding propagation layer recursively propagates the embeddings of a node's neighbors on the collaborative knowledge graph to refine the node's embeddings, and learns the weight of each neighbor during the propagation process through a knowledge-aware attention mechanism. Finally, it aggregates user and item representations from all layers. The preference transfer layer identifies users belonging to the cold-start category based on its set threshold, and then performs secondary propagation on the neighbor users of the cold-start user in the user logical interaction space based on a graph convolutional network architecture, using a normalized intersection-union ratio (IUU) to represent the weight of each neighbor user. The prediction layer outputs the predicted interaction probability based on the final user and item representations.

Description

Technical Field

[0001] This invention belongs to the field of recommendation system technology, and in particular relates to a knowledge graph recommendation method based on preference transfer. Background Technology

[0002] With the rapid development of the internet, online content and services have exploded. While this information overload brings convenience to users, extracting useful information from this deluge has become a challenge. Recommendation systems emerged to address this need. Today, recommendation systems have become an indispensable component of web applications that provide content services, offering users high-quality content recommendations. The core of a recommendation system is predicting whether a user will interact with an item—such as browsing, clicking, purchasing, or other forms of interaction—to satisfy the user's personalized interests.

[0003] In recent years, knowledge graphs have played a crucial role in the development of recommender systems. On the one hand, knowledge graphs can alleviate the cold start problem in collaborative filtering-based recommender systems from a data perspective; on the other hand, knowledge graph-based recommender systems can provide more diverse, interpretable, and accurate recommendation results. The core of applying knowledge graphs to recommender systems is to integrate the edge information from the knowledge graph into the latent vector representations of users and items, making them more representative and relevant. To address this challenge, existing knowledge graph-based recommender systems can be categorized into three types: embedding-based, path-based, and propagation-based.

[0004] In embedding-based methods, Knowledge Graph Embedding (KGE) methods are used for pre-training in the knowledge graph to obtain entity embeddings in a specific semantic space, which are then integrated into the recommendation framework. For example, CKE proposed by Zhang et al. learns entity embeddings in the knowledge graph through TransR to obtain the embeddings of items aligned with it. DKN proposed by Wang et al. treats entity embeddings and word embeddings as different channels. It first learns entity embedding representations through TransD, then extracts entities from the news text, and finally obtains the news representation by combining the entity embedding representations with the original word embedding representations. KTUP proposed by Cao et al. learns a translation-based model through TransH.

[0005] In path-based approaches, we explore various connection patterns between users and items based on knowledge graphs, providing additional guidance for recommendations. Meta-path-based and meta-graph-based recommendations treat knowledge graphs as directed heterogeneous information networks and extract their latent features based on meta-paths or meta-graphs to represent the relationships between users and items.

[0006] In propagation-based methods, information is propagated iteratively across the entire knowledge graph to build a model of the knowledge graph and align its optimization objective with that of the recommendation system, thereby improving recommendation performance. RippleNet, proposed by Wang et al., is a memory-network-like model that propagates latent user preferences across the knowledge graph, updating user representations using the embeddings of entity nodes. KGCN, also proposed by Wang et al., samples some of the neighboring nodes of each entity in the knowledge graph and then obtains item embeddings through a graph convolutional network. KGAT, proposed by Wang et al., is a model that enriches entity embeddings through information propagation on a collaborative knowledge graph. It is first pre-trained using TransR and then applies an attention-based graph network method to propagate information forward and backward across the knowledge graph, iteratively obtaining user and item representations.

[0007] Existing knowledge graph-based recommendation methods typically treat all users indiscriminately during the modeling process. This leads to a situation where cold-start users still suffer from insufficient representational information compared to users with richer interaction data. Furthermore, existing knowledge graph-based recommendation methods focus only on mining the association signals between users and items in the knowledge graph, often neglecting the importance of collaboration signals between users. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, this invention provides a knowledge graph recommendation method based on preference transfer. This method explores collaborative signals between users by constructing a user logical interaction space, and enriches the representation of cold-start users by using preference transfer, thereby improving the model's recommendation performance for cold-start users and ultimately enhancing the overall performance of the model.

[0009] To achieve the above objectives, this application proposes a knowledge graph recommendation method based on preference transfer, including:

[0010] We use TransR to pre-train the collaborative knowledge graph (a joint knowledge graph and user-item interaction graph) to obtain initial embedding representations of users and items containing the structural information of the collaborative knowledge graph;

[0011] Information is disseminated on collaborative knowledge graphs, and domain representations of head entities are obtained through modeling based on a relation-based attention mechanism.

[0012] Use the GCN aggregator to aggregate the entity's representation and its domain representation to update the new representation of the obtained entity;

[0013] Iteratively propagate through L layers to obtain L+1 representations of users and projects respectively;

[0014] The L+1 representations of users and projects are connected to a single vector through an inter-layer aggregation mechanism;

[0015] Furthermore, a logical interaction space is constructed through a collaborative knowledge graph, specifically as follows:

[0016] Different users serve as nodes, and the intersection-union ratio between the project sets that different users interact with on the collaborative knowledge graph serves as the association between users, thereby constructing the user logical interaction space;

[0017] A threshold K1 is set, and users whose interactive item sets are smaller than K1 are considered cold start users, thus distinguishing cold start users from users with more interactive content.

[0018] By performing preference transfer in the user's logical interaction space, the representation of cold-start users is enriched, specifically as follows:

[0019] Information is propagated to the first-order neighbors of the cold-start user in the user's logical interaction space. The neighborhood representation of the cold-start user is obtained by modeling through an attention mechanism based on intersection-union ratio. An additive aggregator is used to aggregate the representation of the cold-start user and its partial neighborhood representations to update the new representation of the cold-start user.

[0020] Furthermore, the predicted matching score is obtained from the final user embedding representation and item embedding representation, specifically:

[0021] User Embedded Representation and project embedding representation Perform the inner product to predict its matching score.

[0022] The knowledge-aware recommendation task is optimized using paired BPR loss, specifically as follows:

[0023] It assumes that users’ historical interactions reveal more user preferences and should be assigned higher predictive scores than unobserved interactions;

[0024] Furthermore, the target loss function is obtained by combining the TransR loss function and the knowledge-aware recommendation task loss function.

[0025] The beneficial technical effects of this invention are as follows: Experiments were conducted on the Last-FM and Amazon-Book datasets. In the Top-K recommendation scenario, this method significantly improves upon existing state-of-the-art baseline methods. This is attributed to the method's ability to explicitly explore higher-order connectivity in collaborative knowledge graphs. More importantly, this method effectively mines collaborative signals between users in the user logical interaction space, thereby enriching the embedded representation of cold-start users through preference transfer, thus improving the recommendation performance for cold-start users and ultimately enhancing the overall performance of the model. Attached Figure Description

[0026] Figure 1 This is a flowchart of a knowledge graph recommendation method based on preference transfer in one embodiment;

[0027] Figure 2 This is an example diagram of the user logic interaction space in the embodiment. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and embodiments;

[0029] The overall process diagram of the present invention is as follows: Figure 1 As shown:

[0030] This invention comprises three components: an attention embedding propagation layer, a preference transfer layer, and a prediction layer.

[0031] First, the attention embedding propagation layer recursively propagates the embeddings of the neighbors of a node on the collaborative knowledge graph to refine the node embeddings. It also learns the weight of each neighbor during the propagation process through a knowledge-aware attention mechanism, and finally aggregates user and item representations from all layers.

[0032] The second layer is the preference migration layer. It finds users who belong to the cold start by setting a threshold, and then performs secondary propagation on the neighbor users of the cold start users in the user logical interaction space, and uses the normalized intersection-union ratio to represent the weight of each neighbor user.

[0033] The third layer, the prediction layer, outputs the predicted interaction probabilities based on the final user and item representations.

[0034] A knowledge graph recommendation method based on preference transfer includes the following steps:

[0035] Step 1: Perform attention embedding propagation on the collaborative knowledge graph;

[0036] Attention embedding propagation consists of four parts: pre-training, information propagation, information aggregation, and higher-order propagation.

[0037] Step 1.1: Pre-training;

[0038] TransR was applied to pre-train the collaborative knowledge graph, mapping the entire graph to a relation space; simultaneously, initial embedding representations of users and items containing structural information were obtained; more specifically, TransR ensured that the triples (h,r,t) on the collaborative knowledge graph satisfy:

[0039]

[0040] Among them, e h ,e t ∈Rd and e r ∈R k , respectively, are the embeddings of h, t, and r; d and k are the embedding dimensions of entities and relations, respectively; and These are the projections of the head entity h and the tail entity t in the relation space, respectively.

[0041] The scoring function of TransR is defined as follows:

[0042]

[0043] in W r ∈R d×k It is a relation matrix that projects entities from a d-dimensional entity space to a k-dimensional relation space;

[0044] TransR encourages the difference between positive and negative triples through a pairwise ranking loss, the loss function of which is shown below:

[0045] L KG =∑-lnσ(s) neg -s pos )

[0046] One of the negative triples is obtained by randomly replacing the corresponding collaborative knowledge graph G. c The head or tail entity in a positive triple (h, r, t) on the knowledge graph and user-item interaction graph is constructed; σ is the sigmoid function; s neg It is a negative triple, s pos It is a positive triplet;

[0047] Step 1.2: Information dissemination;

[0048] Information is propagated on a collaborative knowledge graph to obtain the neighborhood representation of an entity; consider an entity h, N h ={(h,r,t)|(h,r,t)∈G c} represents the set of triples associated with entity h when h is the head entity; N was calculated h A linear combination of these terms is used to obtain the neighborhood representation of h:

[0049]

[0050] Where π(h,r,t) represents the attention weights of different neighboring entities to the central entity h during information propagation, defined as follows:

[0051]

[0052] Where cos_sim represents cosine similarity; using and Modeling attention using cosine similarity between nodes can fully preserve the structural information of the subgraph centered at h, allowing neighbors closer to h in the relation space to receive more attention.

[0053] Step 1.3: Information Aggregation;

[0054] Aggregate the representation of h, e, using the GCN aggregator. h and its domain indicates To update and obtain a new representation of h, defined as follows:

[0055]

[0056] Where σ is the LeakyReLU activation function; W is a trainable transformation matrix used to extract useful propagation information; and b is the bias.

[0057] Step 1.4: Higher-order propagation;

[0058] Further exploration of higher-order connectivity information involves collecting information propagated from l-hop neighbors; specifically, the embedding representation of an entity is recursively expressed as:

[0059]

[0060] Where l represents the distance from the central entity h; Defined as follows:

[0061]

[0062] The pre-trained entity representation as e (0) Participate in the initial information propagation iteration; after L iterations, obtain the representation sequence of user node u. and the representation sequence of project node v Considering the connectivity information at different orders, a layer aggregation mechanism is used to connect the representation of each step into a single vector, as shown below:

[0063]

[0064] Where || is the concatenation operator;

[0065] Step 2: Perform preference migration within the user's logical interaction space;

[0066] Based on the architecture of graph convolutional networks, secondary information propagation is performed along the user's logical interaction space (UI) to enrich the representation of cold-start users.

[0067] A threshold K1 is determined based on the distribution of the training data. Users whose number of user-item interactions is below the threshold are called users with limited information, i.e., cold start users.

[0068] To avoid introducing more noise, preference transfer is only applied to cold-start users, and when propagating information, only the 1-hop neighbors of the cold-start user in the user's logical interaction space are considered.

[0069] Considering a cold start user u cold , To represent the interaction between u in the user logic interaction space cold The set of associated triples. It should be noted that triples in the user logic interaction space are undirected.

[0070] Where IoU is related to user u cold Interactive project sets and user u * The intersection-union ratio between interacting item sets is used to represent the degree of association between them, as shown below:

[0071]

[0072] IoU takes into account the size of each itemset, thus it can adequately represent the relationship between two users; for example... Figure 2 As shown: User u1 interacts with the item set {v1, v2}, and user u3 interacts with the item set {v2, v3}. The correlation between u1 and u3 can be calculated as follows: This indicates that user u1's preferences are different from user u3's preferences. The similarity, from another perspective, illustrates that user u1 has The probability of being interested in v3 is high, and the same user u3 has... The probability that they will be interested in v1;

[0073] By setting the receptive field for cold-start users, the K2 neighboring users with the strongest correlation to the cold-start user are selected, thereby avoiding the introduction of more noise, as shown below:

[0074]

[0075] in, top_K2(IoU,u r This indicates that only samples with the same frequency as u are selected when sampling the neighborhood. cold The top K2 neighbors with the strongest correlation;

[0076] u is evaluated using the softmax function. cold Normalize the IoU of all neighbors in the field:

[0077]

[0078] Further calculations The linear combination of u yields cold Neighborhood representation:

[0079]

[0080] in The user u obtained in step 1 r The representation of;

[0081] Finally, the cold start user u cold The representation of and its neighborhood representation Perform additive aggregation to obtain u cold The final representation after preference transfer is as follows:

[0082]

[0083] in The user u obtained in step 1 is still the same as the user u obtained in step 1. cold The representation of;

[0084] Step 3: Recommendation and prediction;

[0085] Based on the final user representation and project representation The inner product between them is used to predict the match score between user u and item v, as shown below:

[0086]

[0087] Step 4: Optimization;

[0088] The knowledge-aware recommendation task is optimized using pairwise BPR loss; specifically, it assumes that a user's historical interactions reveal more user preferences and should be assigned a higher prediction score than unobserved interactions, as defined below:

[0089]

[0090] Among them, O={(u,v s ,v p )|(u,v s )∈O + ,(u,v p )∈O -} represents the training set; O + Indicates the user u's historical (positive) interactions; O - σ represents the unobserved (negative) interaction of user u; σ is the sigmoid function;

[0091] United L KG and L BPR The final target loss function is obtained as follows:

[0092]

[0093] Where Θ is the model parameter set; It is an L2 regularization term in the λ parameterization set to prevent the model from overfitting.

[0094] The method of this invention was extensively experimented with on two real-world datasets, Amazon-Book and Last-FM. In the Top-K recommendation scenario, two widely used evaluation metrics, recall@K and ndcg@K, were used to evaluate the recommended set. The Top-K recommendation results based on the two datasets are shown in Table 1. The experimental results show that the method of this invention consistently outperforms the state-of-the-art baseline on both datasets. More specifically, on the Amazon-Book dataset, the method of this invention achieves relative improvements of 6.38% and 7.46% respectively compared to the best-performing baseline for recall@20 and ndcg@20 evaluation metrics, and on the Last-FM dataset, it achieves relative improvements of 3.10% and 4.00% respectively compared to the best-performing baseline for recall@20 and ndcg@20 evaluation metrics.

[0095] Table 1 shows the recall@20 and ndcg@20 results for Top-K recommendation scenarios based on two datasets.

[0096]

Claims

1. A knowledge graph recommendation method based on preference transfer, characterized in that, Specifically, the following steps are included: Step 1: Perform attention embedding propagation on the collaborative knowledge graph. Attention embedding propagation includes four parts: pre-training, information propagation, information aggregation, and higher-order propagation. Step 1.1: Pre-training TransR is applied to pre-train the collaborative knowledge graph, mapping the entire graph to a relation space and obtaining initial embeddings of users and items containing structural information. More specifically, TransR ensures that the triples (h,r,t) on the collaborative knowledge graph satisfy: Among them, e h ,e t ∈R d and e r ∈R k , respectively, are the embeddings of h, t, and r; d and k are the embedding dimensions of entities and relations, respectively; and These are the projections of the head entity h and the tail entity t in the relation space, respectively. The scoring function of TransR is defined as follows: in, W r ∈R d×k It is a relation matrix that projects entities from a d-dimensional entity space to a k-dimensional relation space; TransR encourages the difference between positive and negative triples through a pairwise ranking loss, the loss function of which is shown below: L KG =∑-lnσ(s neg -s pos ) One of the negative triples is obtained by randomly replacing the corresponding collaborative knowledge graph G. c The head or tail entity in a positive triple (h,r,t) is constructed; the collaborative knowledge graph G is constructed. c It is composed of a knowledge graph and a user-item interaction graph; σ is the sigmoid function; s neg It is a negative triple, s oos It is a positive triplet; Step 1.2: Information Dissemination Information is propagated on a collaborative knowledge graph to obtain the neighborhood representation of an entity; consider an entity h, N h ={(h,r,t)|(h,r,t)∈G c } represents the set of triples associated with entity h when h is the head entity; by calculating N h A linear combination of these terms is used to obtain the neighborhood representation of h: Where π(h,r,t) represents the attention weights of different neighboring entities to the central entity h during information propagation, defined as follows: Where cos_sim represents cosine similarity; Step 1.3: Information Aggregation Aggregate the representation of h, e, using the GCN aggregator. h and its domain indicates To update and obtain a new representation of h, defined as follows: Where σ is the LeakyReLU activation function; W is a trainable transformation matrix used to extract useful propagation information; and b is the bias. Step 1.4: Higher-order propagation Further exploration of higher-order connectivity information involves collecting information propagated from l-hop neighbors; specifically, the embedding representation of an entity is recursively expressed as: Where l represents the distance from the central entity h; Defined as follows: The pre-trained entity representation as e (0) It participates in the initial information propagation iteration; after L iterations, it obtains the representation sequence of user node u. and the representation sequence of project node v Considering the connectivity information at different orders, a layer aggregation mechanism is used to connect the representation of each step into a single vector, as shown below: Where || is the concatenation operator; Step 2: Perform preference migration within the user's logical interaction space Based on the architecture of graph convolutional networks, secondary information propagation is performed along the user's logical interaction space (UI) to enrich the representation of cold-start users. A threshold K1 is determined based on the distribution of the training data. Users whose number of user-item interactions is below the threshold are called users with limited information, i.e., cold start users. To avoid introducing more noise, preference transfer is only applied to cold-start users, and when propagating information, only the 1-hop neighbors of the cold-start user in the user's logical interaction space are considered. Considering a cold start user u cold , To represent the interaction between u in the user logic interaction space cold The set of associated triples. It should be noted that triples in the user logic interaction space are undirected. Where IoU is related to user u cold Interactive project sets and user u * The intersection-union ratio between interacting item sets is used to represent the degree of association between them, as shown below: By setting the receptive field for cold-start users, the K2 neighboring users with the strongest correlation to the cold-start user are selected, thereby avoiding the introduction of more noise, as shown below: in, top_K2(IoU,u r This indicates that only samples with the same frequency as u are selected when sampling the neighborhood. cold The top K2 neighbors with the strongest correlation; u is evaluated using the softmax function. cold Normalize the IoU of all neighbors in the field: Further calculations The linear combination of u yields cold Neighborhood representation: in, The user u obtained in step 1 r The representation of; Finally, the cold start user u cold The representation of and its neighborhood representation Perform additive aggregation to obtain u cold The final representation after preference transfer is as follows: in The user u obtained in step 1 is still the same as the user u obtained in step 1. cold The representation of; Step 3: Recommendation and Prediction Based on the final user representation and project representation The inner product between them is used to predict the match score between user u and item v, as shown below: Step 4: Optimization The knowledge-aware recommendation task is optimized using pairwise BPR loss; specifically, it assumes that a user's historical interactions reveal more user preferences and should be assigned a higher prediction score than unobserved interactions, as defined below: Among them, O={(u,v s ,v p )|(u,v s )∈O + ,(u,v p )∈O - } represents the training set; O + Indicates the user u's historical interactions; O - σ represents the unobserved interactions of user u; σ is the sigmoid function. United L KG and L BPR The final target loss function is obtained as follows: Where Θ is the model parameter set; It is an L2 regularization term in the λ parameterization set to prevent the model from overfitting.

Images