An item cold start recommendation method based on a knowledge graph and meta learning

Abstract

The application discloses an item cold start recommendation method based on a knowledge graph and meta learning, first, according to a user-item bipartite graph, user and item initialization embeddings are generated, and a meta embedding is constructed according to a self-attention encoder; secondly, in first-order propagation, for the problem of incomplete cold start of items, a meta aggregator is used to aggregate target items and neighbors to reconstruct the embedding of the items; finally, the probability of the user taking the target item is obtained through inner product, and the first N items are taken as the final recommendation result. The application combines the advantages of the knowledge graph and meta learning, integrates them into one, and more perfectly solves the item recommendation cold start problem, not only reduces noise and improves the recommendation result in the cold start scene through meta learning, but also explores hidden factors of user interaction with these items through the method of combining the knowledge graph, and digs out the potential interest of the user.

Description

Technical Field

[0001] This invention relates to the field of recommendation technology, and in particular to a cold-start recommendation method for items based on knowledge graphs and meta-learning. Background Technology

[0002] In recent years, with the development of the Internet, a large amount of information has spread rapidly on the Internet, making information overload a concern. Recommender systems (RS), as an important means to solve the problem of information overload, have been widely used in various fields.

[0003] Traditional recommender systems can generally be divided into three types of recommendation methods: content-based recommendation, collaborative filtering (CF) recommendation, and hybrid recommendation, with collaborative filtering being the most commonly used. However, CF relies too heavily on user-item interactions, making it prone to the cold start problem when filtering content that a target user might be interested in from massive amounts of data. The cold start problem can be categorized into user cold start, item cold start, and system cold start, depending on the missing information. User cold start refers to new users with little interaction, item cold start refers to new items lacking user ratings, and system cold start occurs in the early stages of a system with few users and minimal user activity. In special cases, new items with no interaction or new users without any ratings are generally referred to as absolutely cold start items or absolutely cold start users. Because the cold start problem makes it difficult to train accurate recommender models, it hinders information filtering in recommender systems and restricts their future development.

[0004] In recent years, with the rapid development of deep learning, researchers have realized that deep learning techniques, such as graph neural networks, can be used to solve the cold start problem in recommender systems. Deep learning recommends items similar to those liked by users based on their historical behavior, iteratively aggregating higher-order information to alleviate the cold start problem. However, it has the following problems: on the one hand, because there are many hidden factors when users interact with these items, it cannot effectively uncover users' potential interests or reflect their preferences; on the other hand, it encounters a lot of noise during the propagation of higher-order information, and the neighbors of the target cold-start node may also be cold-start nodes, thus affecting the embedding quality of the target node.

[0005] Ancillary information (social networks, multimedia, knowledge graphs, or contextual information, etc.) can effectively utilize user or item information and their interrelationships to compensate for insufficient data, thereby solving the cold start problem. The most commonly used ancillary information is the knowledge graph. Essentially, a knowledge graph is a semantic network that reveals the relationships between entities. It not only solves or alleviates the cold start problem but also provides more accurate personalized recommendations, while also offering interpretability and successfully reflecting user preferences. There are three main methods for recommendation based on knowledge graphs: first, embedding-based methods, which typically use information directly from the knowledge graph to enrich the representation of items or users; second, path-based methods, which construct a user-item bipartite graph and utilize the connectivity patterns of entities in the knowledge graph for recommendations; and third, joint methods, which propose a unified approach that combines the semantic representation of entities and relationships with path connectivity information to better utilize the information in the knowledge graph, refining entity representations through the idea of ​​embedding propagation and the use of paths. While these knowledge graph methods improve recommendation performance by supplementing the embedding of cold-start items and users, they overlook the noise problem caused by cold-start nodes' neighbors during high-order information propagation. This noise affects the embedding quality of the target node, as well as issues such as overfitting and popularity bias.

[0006] Besides the knowledge graph-based solutions mentioned above, some researchers have proposed a novel approach: meta-learning-based recommendation to address the cold start problem. This approach consists of metric-based recommendation and model-based recommendation. Meta-learning methods can effectively eliminate noise, improve the embedding quality of target cold start nodes, and alleviate overfitting. However, they do not always obtain high-order interaction information, thus failing to effectively uncover users' latent interests and reflect their preferences. Summary of the Invention

[0007] The present invention aims to solve the aforementioned technical problems existing in the prior art by providing a cold-start recommendation method for items based on knowledge graphs and meta-learning.

[0008] The technical solution of this invention is: a cold-start recommendation method for items based on knowledge graphs and meta-learning, characterized by the following steps:

[0009] Step 1. Construct a collaborative relationship graph and obtain the initial embeddings of items and users through BPRMF training, which will serve as the real embeddings;

[0010] Step 2. Pre-train the self-attention encoder m to obtain the item metadata embeddings;

[0011] Step 3. Item nodes are embedded into first-order neighbors through meta-aggregator f, and the final item embedding is obtained after higher-order propagation;

[0012] Step 4. The user node obtains the final user embedding through higher-order propagation;

[0013] Step 5. Obtain the probability of the user choosing the target item through the inner product, and take the top N items as the final recommendation result.

[0014] Step 1 is as follows:

[0015] Acquiring Users - Items (Part Two) and knowledge graph ,according to and Constructing a collaborative relationship graph ;

[0016] The two-part diagram middle, Indicates user With items There was interaction. Represents a set of users. Represents a set of items; the knowledge graph middle, Represents a set of entities. Denotes a set of relations, where This represents the total number of users. Indicates the total quantity of items. Represents the total number of entities. This represents the total number of relationships; TransR is used in the collaborative relationship graph G to learn the embeddings of each entity node and relationship node, and the training relationships are as follows:

[0017] ;

[0018] in, These represent the embedding of head entity, relation entity, and tail entity, respectively. Based on relationships The transformation matrix;

[0019] Finally, the item embeddings are obtained through BPRMF training and used as the true embeddings. ;

[0020] Step 2 is as follows:

[0021] First, a self-attention encoder m is trained, and the first-order neighborhood of the target item node is optimized using the self-attention mechanism. The specific formula is as follows:

[0022] ;

[0023] This represents the first-order neighborhood of the target cold-start item. It is a first-order neighbor of a cold-start node that uses self-attention weighted cold-start; SELF-ATTENTION is a self-attention mechanism.

[0024] Then, by using the first-order neighbors of the trained target cold-start item, the average item meta-embedding is obtained. :

[0025] ;

[0026] Step 3 is as follows:

[0027] First, the meta-aggregator f aggregates the item node itself, its first-order neighbors, and additional meta-embeds;

[0028] ;

[0029] Then, the item nodes are aggregated through higher-order propagation, as shown in the GCN formula below:

[0030] ;

[0031] This indicates the embedding of the target item of the previous level. It is the neighborhood of the target item of the previous order, and LeakyReLU is the activation function;

[0032] ;

[0033] Then, based on the embedding of items generated in each layer, the final embedding of the generated items is pieced together. ;

[0034] ;

[0035] in Here, L is the concatenation function, and L is the number of propagation layers.

[0036] Step 4 is as follows:

[0037] GCN is used to aggregate the user and its neighborhood, as shown in the following formula:

[0038] ;

[0039] Based on the user embeddings generated at each layer, the final user embeddings are then concatenated to generate the final user embedding. :

[0040] ;

[0041] Step 5 calculates the target user by using the inner product of the target user and the item's final embedding. u Take all non-interacted items i probability ;

[0042]

[0043] probability Arranged from largest to smallest, for target users u When recommending personalized items, the top N items with the highest probabilities are selected to generate a recommendation list for the user.

[0044] This invention first generates initial embeddings for users and items based on the user-item bipartite graph, and constructs meta-embeddednesses using a self-attention encoder. Secondly, during first-order propagation, to address the incomplete cold start problem for items, a meta-aggregator is used to aggregate target items and their neighbors to reconstruct item embeddings: item nodes aggregate first-order neighbor embeddings through the meta-aggregator f, and the final item embedding is obtained after higher-order propagation; user nodes obtain their final embeddings through higher-order propagation. Finally, the probability of a user choosing the target item is obtained through the inner product, and the top N items are taken as the final recommendation result. This not only solves the cold start item problem but also the absolute cold start item problem, making the recommendation results more accurate. In the absence of user interaction, items can be recommended to users with the same preference for similar items, and connections are established with users who share the same preference for the item through a collaboration graph formed by the knowledge graph and the user-item bipartite graph, systematically solving the cold start problem in recommendation systems. This invention combines the strengths of knowledge graphs and meta-learning, integrating them into a more comprehensive solution to the cold start problem in item recommendation. It not only reduces noise and improves recommendation results in cold start scenarios through meta-learning, but also explores hidden factors in user interactions with these items at a higher level by combining knowledge graph methods, thereby uncovering users' potential interests. Attached Figure Description

[0045] Figure 1 This is a flowchart of an embodiment of the present invention.

[0046] Figure 2 This is an item-entity matrix diagram according to an embodiment of the present invention.

[0047] Figure 3 This is a comparison chart of the precision of the embodiments of the present invention with other algorithms.

[0048] Figure 4 This is a comparison chart of the embodiments of the present invention with other F1 algorithms.

[0049] Figure 5 This is a comparison chart of the embodiments of the present invention with other algorithms such as NDCG.

[0050] Figure 6 This is a diagram showing the results of an absolute cold start test according to an embodiment of the present invention. Detailed Implementation

[0051] This invention provides a cold-start item recommendation method based on knowledge graphs and meta-learning. Figure 1 As shown, proceed as follows:

[0052] Step 1. Construct a collaborative relationship graph, and obtain the initial embeddings of items and users through BPRMF training, which will serve as the real embeddings:

[0053] Acquiring Users - Items (Part Two) and knowledge graph ,according to and Constructing a collaborative relationship graph ;

[0054] The two-part diagram middle, Indicates user With items There was interaction. Represents a set of users. Represents a set of items; the knowledge graph middle, Represents a set of entities. Denotes a set of relations, where This represents the total number of users. Indicates the total quantity of items. Represents the total number of entities. This represents the total number of relationships; TransR is used in the collaborative relationship graph G to learn the embeddings of each entity node and relationship node, and the training relationships are as follows:

[0055] ;

[0056] in, These represent the embedding of head entity, relation entity, and tail entity, respectively. Based on relationships The transformation matrix;

[0057] Finally, the item embeddings are obtained through BPRMF training and used as the true embeddings. ;

[0058] Where the user set U = {user, user 2, user 3, user 4, user 5}, the item set I = {item 1, item 2, item 3, item 4, item 5, target item 1, target item 2}, construct the user-item bipartite graph G1;

[0059] (1)

[0060] This refers to the interactions that have been observed. These are the interactions that have already been observed.

[0061] The generated adjacency matrix is ​​shown in Table 1.

[0062] Table 1

[0063]

[0064] As shown in Table 1:

[0065] User 1's interactive items are {item 2, item 3, item 4};

[0066] User 2's interactive items are {item 2, item 3, item 4, item 5, target item 1};

[0067] User 3's interactive items are {item 1, item 3, item 4, item 5};

[0068] User 4's interactive items are {item 1, item 3, item 4};

[0069] User 5's interactive items are {item 1, item 2, item 5};

[0070] By introducing a knowledge graph, an item-entity relationship table is constructed based on the entity relationships in the knowledge graph, as shown in Table 2:

[0071] Table 2

[0072]

[0073] Generate a collaborative knowledge graph G2, construct triples (head, relation, tail), and use TransR, a heterogeneous network embedding method, to learn each embedded entity and relation.

[0074] The set of triplets is {entity1, role, item2};

[0075] {Entity 1, Role, Item 4};

[0076] {Entity 1, playing the role of target item 2};

[0077] {Entity 2, Role-playing, Item 1};

[0078] {Entity 2, Role-playing, Item 3};

[0079] {Entity 2, role-playing, target item 1};

[0080] {Entity 3, Director, Item 3};

[0081] {Entity 3, Director, Item 4};

[0082] {Entity 3, Director, Target Item 1}; ......

[0084] Construct a collaborative knowledge graph G2, such as... Figure 2 As shown, Entity 1 can be understood as , and It is an embedding of h, r, t, and the formula is as follows:

[0085] (2)

[0086] It is the transformation matrix of relation R.

[0087] In order to construct good triples, the triples are optimized according to the principle of BPR loss;

[0088] (3)

[0089] In formula (3) , It is achieved by randomly replacing the effective triples of the entity structure. It is an activation function.

[0090] Step 2. Pre-train the self-attention encoder m to obtain the item metadata embeddings;

[0091] First, a self-attention encoder m is trained, and the first-order neighborhood of the target item node is optimized using the self-attention mechanism. The specific formula is as follows:

[0092] ;

[0093] This represents the first-order neighborhood of the target cold-start item. It is a first-order neighbor of a cold-start node that uses self-attention weighted cold-start; SELF-ATTENTION is a self-attention mechanism.

[0094] Then, by using the first-order neighbors of the trained target cold-start item, the average item meta-embedding is obtained. :

[0095] ;

[0096] Taking target cold-start item 1 as an example, the item embedding generated by the BPRMF model is used as the actual value of target cold-start item 1. .

[0097] First, the neighbors of target item 1 are weighted using a self-attention mechanism, represented as follows:

[0098] Target item 1 {User 2, User 7, Entity 5, Entity 7};

[0099]

[0100] In equation (4) This represents the first-order neighborhood of the target item 1 after weighting.

[0101]

[0102] Average generation of meta-embedding for target item 1 As an additional embedding of target cold start item 1;

[0103] Step 3. Item nodes are embedded into first-order neighbors through meta-aggregator f, and the final item embedding is obtained after higher-order propagation;

[0104] First, the meta-aggregator f aggregates the item node itself, its first-order neighbors, and additional meta-embeds;

[0105] ;

[0106] Then, the item nodes are aggregated through higher-order propagation, as shown in the GCN formula below:

[0107] ;

[0108] This indicates the embedding of the target item of the previous level. It is the neighborhood of the target item of the previous order, and LeakyReLU is the activation function;

[0109] ;

[0110] Then, based on the embedding of items generated in each layer, the final embedding of the generated items is pieced together. ;

[0111] ;

[0112] in Here, L is the concatenation function, and L is the number of propagation layers.

[0113] First, a meta-learner f is proposed to enhance the neighbor embeddings of target item 1:

[0114] (6)

[0115] Then, higher-order information is aggregated using GCN to obtain the embedding of target item 1:

[0116] (7)

[0117] It represents the first l Embedding of target item 1 It is the neighborhood of target item 1;

[0118] Higher-order propagation is represented as User 2 → Item 3 → Entity 2 → Target Item 1;

[0119] Assuming we want to calculate the embedding GCN aggregation method for each layer of target item 1 and user 1, which is used for user 1 who has not selected target item 1, the following formula is used:

[0120] (8)

[0121] Generate the embedding of the final target item 1.

[0122] (9)

[0123] These are the final embeddings of target item 1, where This is a concatenation function;

[0124] Step 4. The user node obtains the final user embedding through higher-order propagation;

[0125] GCN is used to aggregate users themselves and their neighborhoods.

[0126] (10)

[0127] The embedding of end user 1 is generated by splicing.

[0128] (11)

[0129] User 1 was ultimately embedded.

[0130] Step 5. Obtain the probability of the user choosing the target item through the inner product, and take the top N items as the final recommendation result.

[0131] The probability that target user 1 will take all uninteracted cold-start items 1 is calculated by using the inner product of the target user and the final embedding of the item. :

[0132] (12)

[0133] The initialization vectors for User 1 and Target Item 1 are represented as (0.2, 0.25, 0, 0.2, 0, 0, 0.34) and (0, 0.25, 0.25, 0, 0.288, 0.224, 0.25) respectively. After calculating the dot product using formula (12), the probability of User 1 using Target Item 1 is normalized as follows:

[0134]

[0135] Similarly, we can calculate the probability that user 1 uses other uninteracted items:

[0136]

[0137]

[0138]

[0139] Based on probability, user 1's top N recommendations are: .

[0140] Similarly, the same applies to the target item 2 for absolute cold start, thus effectively solving the item cold start problem.

[0141] Experimental verification of the method of the present invention is carried out, specifically including:

[0142] 1) Prepare a standard dataset

[0143] This invention uses the Yelp-2018 dataset, which contains 1,185,068 interaction messages, 45,919 users, and 45,538 items. Each user has at least 20 interactions. The knowledge graph is obtained from the free base. It mainly maps items in Amazon books to entities through title matching. When an item matches an entity more than ten times, the entity is left to form a triple. The final knowledge graph consists of 42 relations, a total of 90,961 entities, and 1,853,704 triples.

[0144] 2) Evaluation Indicators

[0145] One of the important components of recommendation algorithm experiments is calculating the evaluation metrics for recommendation quality. The evaluation metrics used in this invention are as follows:

[0146] Precision is a metric that refers to the prediction result. It represents the probability that a sample predicted as positive is actually positive among all samples predicted as positive. The formula is as follows:

[0147] (13)

[0148] Equation (13) Let N represent the set of N items recommended to user u in the training set. This represents the list of user behaviors in the test set. N represents the number of actions performed on the user. U A recommendation set of n items.

[0149] Precision and recall, also known as precision and recall rate, are applicable in cases of imbalanced samples. They are contradictory; to maximize one, the other must be minimized. To find a balance point that maximizes both metrics, the F1 score is used, as shown in the formula below:

[0150] (14)

[0151] (15)

[0152] NDCG (Normalized Discount Cumulative Gain) is a metric used to measure ranking quality. The specific formula is shown below:

[0153] (16)

[0154] Indicates to users U Recommended n The recommended set of items serves a normalization function.

[0155] 3) Conduct experiments on standard datasets.

[0156] This invention compares the performance of the proposed algorithm with other related algorithms:

[0157] BPRMF (Bayesian Personalized Ranking from Implicit Feedback): A Bayesian sorting matrix factorization algorithm.

[0158] CFKG (Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation): A learning algorithm for embedding heterogeneous knowledge graphs for explainable recommendation.

[0159] CKE (Collaborative Knowledge Base Embedding for Recommender Systems): An embedding-based collaborative knowledge recommendation algorithm.

[0160] GCMC (Graph Convolutional Matrix Completion): A graph convolutional matrix completion algorithm.

[0161] Our model: A recommendation algorithm that solves the cold start problem for items by using knowledge graph-enhanced meta-learning techniques.

[0162] To simulate a cold start environment, two hundred users were selected: 100 for the training set and 100 for the test set. The items they interacted with were limited to a maximum of 5 (K items) times the number of users. 5).

[0163] To make the experimental results appear more comprehensive, we take five cases in TOPN, namely N=(20,40,60,80,100), to examine the superiority of the present invention.

[0164] Depend on Figure 3 As can be seen, although the precision accuracy of this invention decreases as the number of recommendations increases, the precision of this invention is higher than that of other algorithms in most cases, achieving a significant advantage. This indicates that this invention can effectively solve the cold start problem of items.

[0165] Figure 4 The F1 score in the medium-low frequency evaluation index decreased with the increase of the number of recommendations in the cold start environment, but the score of this invention was higher than that of other algorithm models, especially when N=20, the score of 80 was much higher than that of other algorithms, which fully demonstrates the effectiveness of this invention for cold start of items.

[0166] from Figure 5 It can be seen that the evaluation index NDCG of this invention is better than other models overall, indicating that it has achieved good results in cold start ranking of items.

[0167] To verify that this invention can solve the absolute cold start problem of items, K=0 was taken as the absolute cold start environment for items, and the results are as follows. Figure 6 As shown, the present invention has good performance, demonstrating that it can solve the problem of absolute cold start of items.

Claims

1. A cold-start recommendation method for items based on knowledge graphs and meta-learning, characterized in that... Follow these steps: Step 1. Construct a collaborative relationship graph and obtain the initial embeddings of items and users through BPRMF training, which will serve as the real embeddings; Step 2. Pre-train the self-attention encoder m to obtain the item metadata embeddings; Step 3. Item nodes are embedded into first-order neighbors through meta-aggregator f, and the final item embedding is obtained after higher-order propagation; Step 4. The user node obtains the final user embedding through higher-order propagation; Step 5. Obtain the probability of the user choosing the target item through the inner product, and take the top N items as the final recommendation result; Step 1 is as follows: Acquiring Users - Items (Part Two) and knowledge graph ,according to and Constructing a collaborative relationship graph ; The two-part diagram middle, Indicates user With items There was interaction. Represents a set of users. Represents a set of items; the knowledge graph middle, Represents a set of entities. Denotes a set of relations, where This represents the total number of users. Indicates the total quantity of items. Represents the total number of entities. This represents the total number of relationships; TransR is used in the collaborative relationship graph G to learn the embeddings of each entity node and relationship node, and the training relationships are as follows: ； in, These represent the embedding of head entity, relation entity, and tail entity, respectively. Based on relationships The transformation matrix; Finally, the item embeddings are obtained through BPRMF training and used as the real embeddings. ; Step 2 is as follows: First, a self-attention encoder m is trained, and the first-order neighborhood of the target item node is optimized using the self-attention mechanism. The specific formula is as follows: ； This represents the first-order neighborhood of the target cold-start item. It is a first-order neighbor of a cold-start node that uses self-attention weighted cold-start; SELF-ATTENTION is a self-attention mechanism. Then, by using the first-order neighbors of the trained target cold-start item, the average item meta-embedding is obtained. : ； Step 3 is as follows: First, the meta-aggregator f aggregates the item node itself, its first-order neighbors, and additional meta-embeds; ； Then, the item nodes are aggregated through higher-order propagation, as shown in the GCN formula below: ； This indicates the embedding of the target item of the previous level. It is the neighborhood of the target item of the previous order, and LeakyReLU is the activation function; ； Then, based on the embedding of items generated in each layer, the final embedding of the generated items is pieced together. ; ； in Here, L is the concatenation function, and L is the number of propagation layers. Step 4 is as follows: GCN is used to aggregate the user and its neighborhood, as shown in the following formula: ； Based on the user embeddings generated at each layer, the final user embeddings are then concatenated to generate the final user embedding. : ； Step 5 calculates the target user by using the inner product of the target user and the item's final embedding. u Take all non-interacted items i probability ; ； probability Arranged from largest to smallest, for target users u When recommending personalized items, the top N items with the highest probabilities are selected to generate a recommendation list for the user.

Images