Cold-start recommendation method integrating knowledge graph and relation-aware contrastive learning

Abstract

The present invention discloses a cold-start recommendation method that integrates knowledge graph and relationship-aware contrastive learning. First, a collaborative relationship graph is constructed based on the acquired user-item bipartite graph and knowledge graph. Then, node dropout or relationship dropout is used for data enhancement to generate two subgraphs of the collaborative relationship graph. An adaptive neighbor aggregator is used for item nodes to allocate different attention according to the type of their neighbor nodes and aggregate neighbor information to obtain a high-order representation. The representations of each order are concatenated to obtain the final item embedding. A user node obtains a high-order representation by aggregating its own information and neighbor information. The representations of each order are concatenated to obtain the final user embedding. Contrastive learning is used to train the node embeddings of the two subgraphs to improve the quality of the final node embedding. The inner product of the user embedding and the item embedding is performed to predict the recommendation score. Finally, the n uninteracted items with the highest click probability are recommended to the user, which can effectively solve the cold-start problem of users and items.

Description

Technical Field

[0001] The present invention relates to the field of recommendation technology, and in particular to a cold start recommendation method that integrates knowledge graph and relationship-aware comparative learning. Background Art

[0002] In recent years, with the rapid development of computer technology, vast amounts of information have been rapidly disseminated across the internet, making data overload a major problem for researchers. Recommender systems, as an important means of addressing this data overload, are now widely used in various fields. Traditional recommendation methods can generally be categorized into three types: content-based recommendation, collaborative filtering-based recommendation, and hybrid recommendation. The most commonly used is the collaborative filtering algorithm (CF).

[0003] Although the CF collaborative filtering algorithm is widely used and highly effective, it relies heavily on interactions between users and items, making it prone to cold-start problems when recommender systems filter through massive amounts of data to identify content that may be of interest to target users. This not only makes information screening difficult for recommender systems but also hinders their future development. New users or new items are often more susceptible to cold-start problems. Cold-start problems can be categorized as user cold-start, item cold-start, and system cold-start, depending on the type of information missing. Specifically, user cold-start refers to new users with little interaction, while item cold-start refers to new items lacking user ratings. System cold-start occurs in the early stages of a system when there are few users and little user activity. Common algorithms such as collaborative filtering that rely on extensive user activity cannot effectively train accurate recommendation models. New items with no interaction or new users without any ratings are generally referred to as absolute cold-start items or users. As shown in Table 1, ? indicates no observed interaction between the user and the item, while 1 indicates observed interaction between the user and the item (such as clicks, purchases, or browsing). I1 and I2 represent partially cold-start items, while U1 and U5 represent cold-start users.

[0004] Table 1

[0005]

[0006] With the further development of deep learning, some researchers have realized that deep learning can be used to solve the cold start problem of recommendation systems. For example, graph neural network technology can iteratively aggregate high-level information, which can effectively alleviate the cold start problem. However, simply recommending items similar to those a user likes based on historical user behavior does not produce good results. On the one hand, this is because there are many hidden factors in user interactions with these items, so it cannot effectively tap into users' potential interests and reflect their preferences. On the other hand, the propagation of high-level information is subject to a lot of noise. The neighbors of the target cold start node may also be cold start nodes, which affects the embedding quality of the target node. Summary of the Invention

[0007] In order to overcome the defects of the existing technology, the technical problem solved by the present invention is to provide a cold start recommendation method that integrates knowledge graph and relationship-aware comparative learning.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] Step 1. Obtain the user-item bipartite graph G1 = {(u,i)|u∈U,i∈I} and the knowledge graph G2 = {(h,r,t)|h,t∈ε,r∈R}, and construct the collaborative relationship graph g = {(h,r,t)|h,t∈ε′,r∈R′} based on G1 and G2;

[0010] In the bipartite graph G1, (u,i) indicates that user u has interacted with item i, U = {u1,u2,…,u |U|} represents the user set, I={i1,i2,…,i |I|} represents a set of items. In the knowledge graph G2, ε={e1,e2,…,e |ε|} represents an entity set, R = {r1, r2, ..., r |R|} represents a set of relationships; where ε′=ε∪U, R′=R∪{interactions}, |U| represents the total number of users, |I| represents the total number of items, |ε| represents the total number of entities, and |R| represents the total number of relationships. TransR is used to learn the embedding of each entity node and relationship node in the collaborative relationship graph g, and the training relationship is as follows:

[0011]

[0012] Among them, e h ,e r ,e t Respectively represent the head entity, relation entity, and tail entity embedding, W r is the transformation matrix based on the relation r;

[0013] Step 2: Perform data augmentation on the collaborative relationship graph using node dropout or relationship dropout to generate two subgraphs s1 and s2 of the collaborative relationship graph:

[0014] s ε (g)=(M1⊙ε′,R′) (2)

[0015] s R (g)=(ε′,M2⊙R′) (3)

[0016] Formula (2) and formula (3) correspond to the node drop-out method and the relationship drop-out method, respectively. M1 and M2 represent the masking vectors of the two drop-out methods, respectively. Select one of the methods and execute it twice to obtain two subgraphs of the two collaborative relationship graphs, which are represented as s1 and s2 respectively.

[0017] Step 3: Use an adaptive neighbor aggregator to allocate different attentions to the item nodes according to the types of their neighbor nodes, aggregate the neighbor information, obtain a high-order representation, and concatenate the high-order representations to obtain the final item embedding;

[0018] use and represent the entity neighbors and user neighbors of item i respectively, Represents all neighbor information of item i, which is calculated as follows:

[0019]

[0020] Among them, W k ,W r Represent the attention weights of entity neighbors and user neighbors respectively. After obtaining the neighbor information of the item, the high-level representation of the item is obtained based on the low-level representation of the item itself and the neighbor information. The specific method is as follows:

[0021]

[0022] Among them, W1, W2 are trainable transformation matrices, LeakyReLU is the activation function; concatenate the various order representations of the items to obtain the final item embedding:

[0023]

[0024] Among them, L represents the highest order;

[0025] Step 4: The user node obtains a high-level representation by aggregating its own information and neighbor information, and concatenates the high-level representations to obtain the final user embedding; use N u represents the neighbor set of user u, Represents the neighbor information of user u, which is calculated as follows:

[0026]

[0027] Among them, W o is a trainable weight. Since the only neighbors of a user node are item nodes, this weight can be expressed as the importance of item i to user u. Similar to item nodes, the high-order representation of user nodes is also obtained based on their low-order representation and neighbor information:

[0028]

[0029] Finally, concatenate the user's various order representations to get the final user embedding:

[0030]

[0031] Among them, L represents the highest order;

[0032] Step 5: Use contrastive learning to train the node embeddings of the two subgraphs to improve the quality of the final node embeddings;

[0033] Through step 2, we obtain two subgraphs s1 and s2 of the collaborative relationship graph g. Through steps 3 and 4 on these two subgraphs, we can generate two embeddings for each node x. Use the following method to train to improve node embedding quality:

[0034]

[0035] Among them, sim(.) calculates the similarity between two embeddings, and τ represents a hyperparameter represented by temperature in softmax;

[0036] Step 6: Use formula (11) to embed the user and item embedding Perform dot product calculation to obtain the probability of user u clicking on item i:

[0037]

[0038] After the calculation is completed, the n uninteracted items with the highest click probability are recommended to the user.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1) By using contrastive learning and knowledge graphs as auxiliary tasks for the recommendation system, contrastive learning supplements the observed user-item interactions. Furthermore, through the knowledge graph and high-order propagation, users' potential interests are explored while also adding item information, thus solving the cold start problem for both users and items.

[0041] 2) Data augmentation using two methods: node dropout and relationship dropout. Relationship dropout, in particular, masks one knowledge graph relationship at each layer, helping to combat the noise introduced by node neighbors. This allows for identifying relationships that influence users without significantly impacting the node structure, reducing noise and achieving better user embedding.

[0042] 3) Through the relationship perception module, the two neighborhoods of the item node (including user interaction information and entity information) are better aggregated to obtain high-quality item embeddings and alleviate the item cold start problem;

[0043] 4) Compared with other models, the experimental training recommendation of the present invention on public datasets has obvious effects and can effectively solve the cold start problem of users and items. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flow chart of the model in the present invention;

[0045] Figure 2 is the collaborative knowledge graph in the present invention;

[0046] Figure 3 It is the node dropout graph in the present invention;

[0047] Figure 4 It is the relation dropout graph in the present invention. DETAILED DESCRIPTION

[0048] The present invention proposes a cold start recommendation method that integrates knowledge graph and relationship-aware comparative learning. In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below.

[0049] The flow chart of the cold start recommendation method that integrates knowledge graph and relation-aware comparative learning is as follows: Figure 1 shown.

[0050] According to step 1, obtain the user-item bipartite graph g1 = {(u,i)|u∈U,i∈I}, where the user set U = {user, user2, user3, user4, user5, user6, user7}, and the item set I = {movie1, item2, item3, item4, item5, target item1, target item2}. Generate the adjacency matrix as shown in Table 2:

[0051] Table 2

[0052]

[0053] From Table 2, we can see that

[0054] User 1's interactive items are {item 2, item 3, item 4};

[0055] User 2's interactive items are {item 2, item 3, item 4, item 5, target item 1};

[0056] User 3's interactive items are {item 1, item 3, item 4, item 5};

[0057] User 4's interactive items are {item 1, item 3, item 4};

[0058] User 5's interactive items are {item 1, item 2, item 5};

[0059] User 6's interactive items are {item 1, item 2, item 3, item 5};

[0060] User 7's interactive items are {item 3, item 4, item 5, target item 1};

[0061] Introducing the knowledge graph g2 = {(h, r, t) | h, t∈ε, r∈R}, the item-entity relationship table is constructed according to the collaborative knowledge graph g = {(h, r, t) | h, t∈ε′, r∈R′} as shown in Table 3:

[0062] Table 3

[0063]

[0064] Construct a triple (head, relation, tail),

[0065] The triple set is {entity 1, song, item 2};

[0066] {entity1,singing,item4};

[0067] {entity1, singing, targetitem2};

[0068] {entity2, composition, item1};

[0069] {entity2, composition, item3};

[0070] {entity2, composer, target item1};

[0071] {entity3, type, item3};

[0072] {entity3, type, item4};

[0073] {entity3, type, targetitem1}; ......

[0075] TransR is used to learn the embedding of each entity node and relationship node in the collaborative relationship graph g. The training relationship is as follows:

[0076]

[0077] Among them, e h ,e r ,e t Respectively represent the head entity, relation entity, and tail entity embedding, W ris the transformation matrix based on the relation r.

[0078] Step 2: Use node drop out ( Figure 3 shown) or the relationship drop out( Figure 4 As shown in the figure, data enhancement is performed to generate two subgraphs s1 and s2 of the collaborative relationship graph:

[0079] s ε (g)=(M1⊙ε′,R′) (2)

[0080] s R (g)=(ε′,M2⊙R′) (3)

[0081] Formula (2) and Formula (3) correspond to the node drop-out method and the relationship drop-out method, respectively. M1 and M2 represent the masking vectors of the two drop-out methods, respectively. Selecting one of the methods and executing it twice will yield two subgraphs of the two collaborative relationship graphs, denoted as s1 and s2, respectively.

[0082] Step 3: Apply an adaptive neighbor aggregator to target item 1, allocating different attention based on the type of its neighbor nodes. This aggregates neighbor information to obtain a high-level representation, which is then concatenated to obtain the final item embedding. When target item 1 appears, the cold-start item can more efficiently capture collaborative and knowledge-based feature representations, making it easier to adapt to cold-start environments.

[0083] use and represent the entity neighbors and user neighbors of item i respectively, Represents all neighbor information of target item 1, which is calculated as follows:

[0084]

[0085] Among them, W k ,W r Represent the attention weights of entity neighbors and user neighbors respectively. After obtaining the neighbor information of the item, the high-level representation of the item is obtained based on the low-level representation of the item itself and the neighbor information. The specific method is as follows:

[0086]

[0087]

[0088] Where W1 and W2 are trainable transformation matrices and LeakyReLU is the activation function. Concatenate the representations of each order of the item to get the final embedding of the target item 1:

[0089]

[0090] Among them, L represents the highest order.

[0091] Step 4. User 1 node obtains a high-level representation by aggregating its own information and neighbor information, and concatenates the high-level representations to obtain the final user embedding;

[0092] Using N u represents the neighbor set of user u, Indicates the neighbor information of user u1, which is calculated as follows:

[0093]

[0094] Among them, W o is a trainable weight. Since the only neighbors of the user node are item nodes, this weight can be expressed as the importance of item i to user u1. Similar to the item node, the high-order representation of the user node is also obtained based on its low-order representation and neighbor information:

[0095]

[0096] Finally, concatenate the various order representations of user 1 to get the final embedding of user 1:

[0097]

[0098] Among them, L represents the highest order.

[0099] Step 5: Use contrastive learning to train the node embeddings of the two subgraphs to improve the final node embedding quality. Taking target item 1 as an example, through step 2, we obtain two subgraphs s1 and s2 of the collaborative relationship graph g. Two embeddings are generated for target item 1 on these two subgraphs through steps 3 and 4 respectively. Use the following method to train to improve node embedding quality:

[0100]

[0101] Step 6. Finally, the probability of target user 1 taking all uninteracted cold start items 1 is calculated by the inner product of the final embedding of user 1 and target item 1.

[0102]

[0103] The initialization vectors of user 1 and target item 1 are expressed as (0.2, 0.25, 0, 0.2, 0, 0, 0.34) and (0, 0.25, 0.25, 0, 0.288, 0.224, 0.25) respectively. After the dot product calculation of formula (11), the probability of user 1 adopting target item 1 is normalized to:

[0104]

[0105] Similarly, the present invention can calculate the interaction probability of user 1 using other non-interacted items

[0106]

[0107]

[0108]

[0109]

[0110]

[0111] If the first two items are recommended for user 1, then items i1 and i4 will be recommended to user 1.

Claims

1. A cold-start recommendation method that integrates knowledge graph and relation-aware contrastive learning, characterized by: The steps include: Step 1. Obtain the user-item bipartite graph G1 = {(u,i)|u∈U,i∈I} and the knowledge graph G2 = {(h,r,t)|h,t∈ε,r∈R}, and construct the collaborative relationship graph g = {(h,r,t)|h,t∈ε′,r∈R′} based on G1 and G2; In the bipartite graph G1, (u,i) indicates that user u has interacted with item i, U = {u1,u2,…,u |U| } represents the user set, I={i1,i2,…,i |I| } represents a set of items. In the knowledge graph G2, ε={e1,e2,…,e |ε| } represents an entity set, R = {r1, r2, ..., r |R| } represents a set of relationships; where ε′=ε∪U, R′=R∪{interactions}, |U| represents the total number of users, |I| represents the total number of items, |ε| represents the total number of entities, and |R| represents the total number of relationships. TransR is used to learn the embedding of each entity node and relationship node in the collaborative relationship graph g, and the training relationship is as follows: Among them, e h ,e r ,e t Respectively represent the head entity, relation entity, and tail entity embedding, W r is the transformation matrix based on the relation r; Step 2. Perform data augmentation on the collaborative relationship graph using node dropout or relationship dropout to generate two subgraphs s1 and s2 of the collaborative relationship graph: s ε (g)=(M1⊙ε′,R′) (2) s R (g)=(ε′,M2⊙R′) (3) Formula (2) and formula (3) correspond to the node drop-out method and the relationship drop-out method, respectively. M1 and M2 represent the masking vectors of the two drop-out methods, respectively. Select one of the methods and execute it twice to obtain two subgraphs of the two collaborative relationship graphs, which are represented as s1 and s2 respectively. Step 3. Use an adaptive neighbor aggregator on the item node to allocate different attention according to the type of its neighbor nodes, aggregate the neighbor information, obtain a high-order representation, and concatenate the high-order representations to obtain the final item embedding; use and represent the entity neighbors and user neighbors of item i respectively, Represents all neighbor information of item i, which is calculated as follows: Among them, W k ,W r Represent the attention weights of entity neighbors and user neighbors respectively. After obtaining the neighbor information of the item, the high-level representation of the item is obtained based on the low-level representation of the item itself and the neighbor information. The specific method is as follows: Among them, W1, W2 are trainable transformation matrices, LeakyReLU is the activation function; concatenate the various order representations of the items to obtain the final item embedding: Among them, L represents the highest order; Step 4. The user node obtains a high-level representation by aggregating its own information and neighbor information, and concatenates the high-level representations to obtain the final user embedding; Using N u represents the neighbor set of user u, Represents the neighbor information of user u, which is calculated as follows: Among them, W o is a trainable weight. Since the only neighbors of a user node are item nodes, this weight can be expressed as the importance of item i to user u. Similar to item nodes, the high-order representation of user nodes is also obtained based on their low-order representation and neighbor information: Finally, concatenate the user's various order representations to get the final user embedding: Among them, L represents the highest order; Step 5. Use contrastive learning to train node embeddings of the two subgraphs to improve the quality of the final node embeddings; Through step 2, we obtain two subgraphs s1 and s2 of the collaborative relationship graph g. Through steps 3 and 4 on these two subgraphs, we can generate two embeddings for each node x. Use the following method to train to improve node embedding quality: Among them, sim(.) calculates the similarity between two embeddings, and τ represents a hyperparameter represented by temperature in softmax; Step 6. Use formula (11) to embed the user and item embedding Perform dot product calculation to obtain the probability of user u clicking on item i: After the calculation is completed, the n uninteracted items with the highest click probability are recommended to the user.

Images