Recommendation method based on knowledge graph and attention mechanism

Abstract

The invention relates to a recommendation method based on a knowledge graph and an attention mechanism, and belongs to the field of personalized recommendation. The method comprises the following steps: defining a knowledge graph perception recommendation problem, and establishing a user and project interaction matrix and a knowledge graph as input; setting seed sets of users and projects in a knowledge graph propagation layer, propagating the seed sets along adjacent entities, and capturing knowledge-based high-order interaction information of the users and the projects; learning a corresponding potential relation representation for each relation through an attention mechanism in the knowledge perception attention embedding layer to obtain a potentially associated entity set; processing the entity set through an aggregator to obtain a corresponding aggregation vector, and predicting a preference score of the user to the item according to the aggregation vector; and designing a loss function to train the process, and performing user preference recommendation after training is completed. According to the method, the potential relationship between the user and the article can be more accurately captured, and more accurate and personalized recommendation service is provided for the user.

Description

Technical Field

[0001] The present invention belongs to the field of personalized recommendation and relates to a recommendation method based on knowledge graph and attention mechanism. Background Art

[0002] With the continuous advancement of Internet technology, people can easily access a large amount of online content, covering a variety of fields such as music, movies, and commodities. However, the amount of information on the Internet is also growing, and the phenomenon of information overload is becoming more and more common, making it difficult for users to quickly and accurately find items that match their personal interests. In order to solve this problem, recommendation systems came into being, aiming to alleviate information overload and meet users' personalized needs.

[0003] The traditional recommendation technology is collaborative filtering, which provides personalized recommendations to users by analyzing the similarities between users or items. However, the recommendation performance is limited when faced with cold start problems and data sparsity problems. In order to overcome the cold start problem and data sparsity problem of collaborative filtering, researchers have adopted the method of adding auxiliary information to improve the recommendation performance by introducing additional information or features. Among them, knowledge graphs have received increasing attention because they can provide richer item information and association information between items.

[0004] As a kind of auxiliary information, knowledge graph contains rich semantic features, which helps to further obtain high-order neighborhood information between entities and enrich entity representation, thereby solving the data sparsity problem and improving recommendation accuracy. The topological structure information in the knowledge graph is closely related to the information in the user-item interaction matrix in the recommendation algorithm. The combination of the two brings deeper data association and makes up for the sparsity of the original data. The potential characteristics of users and items can be mined through the topological structure of the knowledge graph and effective recommendation algorithms.

[0005] By combining with knowledge graphs, recommendation systems can more accurately simulate user preferences and alleviate data sparsity. Compared with recommendation methods that do not incorporate knowledge graphs, introducing knowledge graphs into recommendations has the following benefits: (1) Knowledge graphs contain rich semantic relationships, which connect attributes and can improve the accuracy of recommendation systems and more accurately discover the similarities between attributes. (2) Knowledge graphs themselves contain rich structural information, which can be expanded through different categories to improve the diversity of recommendation systems. (3) The rich structural information in the knowledge graph, in which the user's historical interaction matrix is ​​associated with candidate items to be recommended, enhances the interpretability of recommendations. (4) When new users or new items are added to the knowledge graph, they may be associated with previous users or items through certain attributes, which helps alleviate the cold start problem of recommendations.

[0006] Despite the above advantages, it is still challenging to combine knowledge graphs in recommendation systems due to the high dimensionality and heterogeneity of knowledge graphs. The core work of using knowledge graphs for recommendation systems is how to enrich the latent vector representations of users and items with the heterogeneous information in the knowledge graph. Based on this core work, the recommendation systems based on knowledge graphs can be roughly divided into the following three categories: embedding-based methods, path-based methods, and propagation-based methods.

[0007] (1) Embedding-based methods preprocess the knowledge graph through Knowledge Graph Embedding (KGE), embed the entities and relationships contained in the graph into a low-dimensional space, and map them into representation vectors. For example, Collaborative Knowledge Base Embedding (CKE) learns project embeddings with the participation of KG through TransR. The deep knowledge-aware network (DKN) regards entity embedding and word embedding as different channels, and uses KG to generate news embeddings for recommendation through Trans D. However, the KGE method is more suitable for in-graph applications such as knowledge graph completion and knowledge graph prediction rather than recommendation. Therefore, the learned entity embeddings are not sufficient for project recommendation. Moreover, due to the lack of explicit modeling capabilities, embedding-based knowledge graph recommendation cannot guarantee that the dependencies of long-distance nodes can be captured, and it is difficult to explain the high-order semantic relationships between entities for recommendation.

[0008] (2) Path-based algorithms provide some preference information for entity recommendation by extracting entity paths rich in high-order node information in the knowledge graph. For example, Personalized Entity Recommendation (PERKG) regards it as a heterogeneous information network and extracts latent features based on meta-paths / graphs along different types of relationship paths / graphs to represent the connectivity between users and items. Path-based algorithms make full use of the network structure in the knowledge graph. However, path-based algorithms are severely limited by manually designed meta-paths, which is a challenging task in complex knowledge graphs and it is difficult to design the optimal path.

[0009] (3) The propagation-based method is to iteratively propagate information on the entire knowledge graph, combined with the aggregation mechanism of the graph neural network to obtain information in high-order fields. It can effectively alleviate the defects of the embedding-based and path-based methods, but there is still some room for improvement, such as insufficient in-depth information mining in the knowledge graph, inability to fully utilize the entity information and relationship information in the knowledge graph, and inability to simultaneously obtain high-order user features and high-order item features, enriching the user representation while also enriching the item representation. The existing algorithm only fully mines the information on one end, that is, only considers the association between the user end or the item end and the entity in the knowledge graph, lacking the full utilization of the knowledge graph information. At the same time, when aggregating neighborhood information, the existing work ignores the importance of the relationship vector and does not extract the potential relationship hidden deep in the entity set. Therefore, it is a challenge to explore the semantic association between user-side features and item-side features and entities in the knowledge graph and to mine hidden information in the user-item historical interaction records.

[0010] RippleNet is a ripple network. The author uses the historical interaction matrix of users and projects as seed nodes in the knowledge graph to spread the interests of users in the knowledge graph, just like ripples on the water surface, and has achieved very good results. However, RippleNet only performs high-level modeling on user features, enriches the representation of user features, but does not consider high-level modeling of project features. At the same time, it also ignores the potential relationship between entity sets when spreading user interests.

[0011] The Knowledge Graph Convolutional Networks (KGCN) uses the core idea of ​​graph neural networks to aggregate domain information to enrich item feature representations. It captures user preferences for different relationships in aggregated domain information, which shows that neighbor information can greatly improve recommendation tasks. It should be noted that both methods consider the influence of users on relationship weights, but ignore the potential relationship between entities, and the project embedding is insufficient. At the same time, only the association between the project side and the entities in the knowledge graph is considered, ignoring the association between the user side and the entities in the knowledge graph, lacking full use of knowledge graph information.

[0012] The Knowledge Graph Attention Network (KGAT) proposed a collaborative knowledge graph (CKG), which combines the user-item bipartite graph (UIG) and the knowledge graph (KG), and recursively propagates on the CKG through GCN to supplement the entity embedding. One of the shortcomings of KGAT is that users are regarded as the same nodes as entities in CKG, which makes it difficult to handle new users. Once a new interactive user appears, KGAT needs to rebuild CKG and retrain the entire model to recommend for him, and its computational cost is unacceptable to the industry. In addition, KGAT assumes that the interaction items in UIG and the associated entities in KG should be regarded as homogeneous nodes, but in fact they are in different latent spaces. In addition, since the types of items that users interact with are usually very wide, the high-order interactions in UIG may have completely different meanings from the latent semantic representations of the original items, which will introduce noise in the embedding.

[0013] The Collaborative Knowledge-aware Attentive Network (CKAN) combines collaborative filtering with knowledge graphs to propose collaborative signals, explicitly encodes user-item information and propagates it in the knowledge graph, and designs a knowledge-aware attention mechanism to obtain the importance of different neighbors. CKAN simultaneously performs high-level modeling of user features and item features, enriching the feature representation of users and items, but does not pay attention to the impact of potential relationships between entities on modeling user and item features.

[0014] The Knowledge Graph-based Intent Network (KGIN) can identify users' potential intentions and the relationship between users and items at a fine-grained level to achieve better recommendation effects, but it ignores the impact of potential relationships on recommendations, resulting in noisy learned user intentions.

[0015] Most existing research works only mine information on one side of the knowledge graph, and only consider the association between the user side or the project side and the entities in the knowledge graph. They lack full utilization of knowledge graph information and ignore the importance of potential relationships for personalized recommendations. Summary of the invention

[0016] In view of this, the object of the present invention is to provide a recommendation method based on knowledge graph and attention mechanism.

[0017] In order to achieve the above object, the present invention provides the following technical solutions:

[0018] A recommendation method based on knowledge graph and attention mechanism, the recommendation method comprises the following steps:

[0019] S1. Define the knowledge graph-aware recommendation problem, establish the user-item interaction matrix and the knowledge graph as input;

[0020] S2. Set the seed sets of users and items in the knowledge graph propagation layer, and propagate the seed sets along adjacent entities to capture the knowledge-based high-order interaction information of users and items;

[0021] S3, in the knowledge-aware attention embedding layer, the corresponding potential relationship representation is learned for each relationship through the attention mechanism to obtain a set of potentially related entities;

[0022] S4, processing the entity set through the aggregator to obtain the corresponding user aggregation vector and project aggregation vector, and predicting the user's preference score for the project based on the aggregation vector;

[0023] S5. Design a loss function to train the above process, and use it for user preference recommendation after training.

[0024] Further, in step S1, a user set of M users is set in the recommendation scenario. A project collection with N projects Then define the user-item interaction matrix Y∈R according to the user's implicit feedback M*N , which is expressed as:

[0025]

[0026] Among them, y uv =1 means that user u has historical interaction with item v, otherwise y uv =1 means that user u has no historical interaction with item v;

[0027] Create a knowledge graph G as input:

[0028] G={(h,r,t)|h,t∈E,r∈R}

[0029] Among them, (h, r, t) represents a large number of entity triples, h∈E, r∈R, t∈E represent the head, relationship and tail of the knowledge triple respectively, E and R represent the entity and relationship sets in the knowledge graph respectively;

[0030] Set A = {(v, e) | v∈V, e∈E} to indicate the alignment relationship between items and entities, where (v, e) means that item v can be aligned with entity e in the knowledge graph;

[0031] For a given user-item interaction matrix Y and knowledge graph G, establish a prediction function:

[0032] y uv =F(u,v|Θ,G)

[0033] Among them, y uv represents the predicted probability, and Θ represents the model parameters of the function F.

[0034] Further, in step S2, the following steps are implemented in the knowledge graph propagation layer:

[0035] S21. Define the user seed set of user u and the project seed set of project v:

[0036]

[0037] in, represents the user seed set, represents the project seed set;

[0038] S22. The seed set is propagated along the links in the knowledge graph to obtain the extended entity set and triples with different distances from the initial entity set, where the entity set of user u and item v is represented as:

[0039]

[0040] Where k represents the distance from the initial entity set, and the subscript symbol o is a unified placeholder for the symbol u or v;

[0041] Then the kth triplet set of user u and item v is expressed as:

[0042]

[0043] Where H represents the number of hops for transmission.

[0044] Further, in step S3, the following steps are performed in the knowledge-aware attention embedding layer:

[0045] S31. Based on the attention mechanism, for the query Q and key K, the relation relation and the tail entity node tail are used to aggregate information, and the value V is the relation itself, then:

[0046]

[0047] S32. Establish attention embedding of potential relationships between entities α i :

[0048] α i =π1(Q i ,K i ,V i )

[0049] in, π1(Q i ,K i ,V i ) is a network for generating potential correlation attention:

[0050] π1(Q i ,K i ,V i )=V i softmax(Q i ,K i )

[0051]

[0052] Among them, softmax(·) is the normalization function;

[0053] S33, using the optimized triples to learn the user or item's association vector α i The preference degree is to find the most valuable information in the entity set of each hop. i It is expressed as:

[0054] e i =π2(α i ,o)

[0055] Among them, α i is the association vector, o is the placeholder for user u or item v; π2(α i ,o) is the aggregation network, which is expressed as:

[0056] π2(α i ,o)=softmax(α i ,o)t i

[0057] Among them, t i is the embedding of the tail entity in the triple;

[0058] S34. Obtain the representation of the k-th layer triple of user u or item v:

[0059]

[0060] where subscript and superscript o are consistent placeholders for the symbols u or v, Is a collection The number of triples in ;

[0061] S35. Establish a set of potential associations between user u and item v:

[0062]

[0063] in, and Represent the seed sets added for users and items respectively.

[0064] Furthermore, in step S4, the entity representations in each hop set are aggregated by the aggregator when the user-item representation entity set is diffused, that is, multiple representations in the entity set of user u and item v are aggregated into a single vector representation, and then the inner product is used to predict the user's preference score for the item:

[0065]

[0066] in, represents the predicted preference score of user u for item v, ε u represents the user's aggregation vector, ε v An aggregate vector representing items.

[0067] Furthermore, the aggregator includes at least a summing aggregator, a concat aggregator, and a pooling aggregator. The summing aggregator sums the representation vectors of the multi-hop set before performing nonlinear transformation; the concat aggregator connects the representation vectors in the representation set in series and performs nonlinear transformation; the pooling aggregator takes the maximum value among multiple vectors and then performs nonlinear transformation; wherein,

[0068] The operation process of the sum aggregator is expressed as:

[0069]

[0070] Among them, σ is the nonlinear function Sigmoid, W and b are trainable weights and biases;

[0071] The operation process of the Concat aggregator is expressed as:

[0072]

[0073] in ∣∣ is the splicing operation;

[0074] The operation process of the Pooling aggregator is expressed as:

[0075]

[0076] Among them, pool max (·) is a pooling aggregator that can select the maximum value in the result set.

[0077] Further, in step S5, a loss function is set to train the above process, wherein the loss function is expressed as:

[0078]

[0079] in, is the cross entropy loss, represents the set of positive user-item pairs, represents the set of negative user-item pairs; is a parameter set; E and R are embedding tables of all entities and relations,

[0080] is the L2 regularization term parameterized by λ.

[0081] The beneficial effects of the present invention are:

[0082] The present invention proposes a new knowledge graph recommendation model KGER to solve the data sparsity and cold start problems of traditional recommendations. KGER makes recommendations by capturing the potential relationships between entities and exploring the different weights of users and projects on potential relationships. Specifically, KGER first iteratively propagates in the knowledge graph to obtain more knowledge. Through deep propagation based on knowledge, it successfully captures the high-order interaction information of users and items based on knowledge, effectively improving the model's ability to represent potential vectors of users and items; based on the self-attention mechanism, it explores the potential relationships between knowledge and constructs a knowledge-aware attention module to distinguish the contributions of different neighbors. The present invention takes into account the problem that has been ignored in previous knowledge graph recommendation systems, that is, the relationship diversity between different entities for the connection between the potential associations of entity sets, and designs a new relationship-aware personalized attention module to mine the potential associations between entities and diffuse the learning of the most valuable information for users or projects in the knowledge graph. Relationship awareness explores the potential associations of entities. By assigning weights to the diversity of relationships between different entities, the potential associations between entities can be detected.

[0083] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0085] Figure 1 It is a schematic diagram of the overall framework flow of the recommendation method based on knowledge graph and attention mechanism of the present invention;

[0086] Figure 2 Schematic diagram of the comparison results between the KGER model of the present invention and all baseline models. DETAILED DESCRIPTION

[0087] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0088] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0089] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0090] See also Figure 1-2 , which is a recommendation method based on knowledge graph and attention mechanism.

[0091] Example

[0092] Since it can effectively solve the sparsity and cold start problems of collaborative filtering, knowledge graphs have been widely studied and applied as auxiliary information in the field of recommendation systems. However, most existing research methods mainly focus on how to extract high-order user features or high-order item features to enrich the representation of users and items, while ignoring the importance of potential relationships hidden in entity sets for aggregating neighborhood information. This embodiment provides a recommendation method based on knowledge graphs and attention mechanisms. Specifically, a new method called Knowledge Graph for Exploring Relations (KGER) is proposed, which explores potential relationships between entities in a new way to capture implicit information hidden in the graph, and proposes a natural way to aggregate user high-order features and item high-order features based on potential relationships. Specifically, KGER captures the potential relationships hidden between entity sets through a self-attention network mechanism, designs a new knowledge-aware attention module to obtain the different importance of user and item for potential relationships to aggregate the features of entity neighborhoods, and automatically obtains high-order information in the knowledge graph. KGER captures more hidden information hidden in the knowledge graph to improve the representation of high-order user features and high-order item features. This example uses two real public datasets to conduct experiments and shows that KGER outperforms several current state-of-the-art baseline models in the scenarios of click-through rate prediction and top-k recommendation. Figure 1 As shown in the figure, the recommendation method based on knowledge graph and attention mechanism includes at least the following steps:

[0093] S1. Define the knowledge graph-aware recommendation problem, establish the user-item interaction matrix and the knowledge graph as input;

[0094] S2. Set the seed sets of users and items in the knowledge graph propagation layer, and propagate the seed sets along adjacent entities to capture the knowledge-based high-order interaction information of users and items;

[0095] S3, in the knowledge-aware attention embedding layer, the corresponding potential relationship representation is learned for each relationship through the attention mechanism to obtain a set of potentially related entities;

[0096] S4, processing the entity set through the aggregator to obtain the corresponding user aggregation vector and project aggregation vector, and predicting the user's preference score for the project based on the aggregation vector;

[0097] S5. Design a loss function to train the above process, and use it for user preference recommendation after training.

[0098] In step S1 of this embodiment, it is assumed that in a typical recommendation scenario, there is a user set of M users. A project collection with N projects According to the interaction history between users and items, the user-item interaction matrix Y can be constructed. User-item interaction matrix Y∈R M*N Defined by implicit feedback from users:

[0099]

[0100] Among them, y uv =1 means that user u has historical interaction with item v, otherwise y uv = 0. Since the interactions between users and items in the dataset used in this embodiment are all explicit feedback, it is necessary to convert the explicit feedback into implicit feedback, that is, to set specific scoring thresholds for different datasets and convert the scoring datasets into binary classification datasets.

[0101] In addition to the interaction matrix Y, a knowledge graph G corresponding to the recommendation scenario is also required as input:

[0102] G={(h,r,t)|h,t∈E,r∈R}

[0103] Among them, (h, r, t) represents a large number of entity triples, h∈E, r∈R, t∈E represent the head, relationship and tail of the knowledge triple respectively, and E and R represent the entity and relationship sets in the knowledge graph respectively. For example, the triple (Guo Fan, director, "The Wandering Earth 2") states the fact that Guo Fan is the director of the movie "The Wandering Earth 2". It should be noted that the item set V is included in the entity set E.

[0104] In addition, the set A = {(v, e)|v∈V, e∈E} is used to clarify the alignment relationship between items and entities, where (v, e) means that item v can be aligned with entity e in the knowledge graph.

[0105] The final task of recommendation is to give the user u whether he prefers item v for the item v that he has not interacted with. For a given user-item interaction matrix Y and knowledge graph G, the goal is to learn a prediction function:

[0106] y uv =F(u,v|Θ,G)

[0107] Among them, y uv represents the predicted probability, and Θ represents the model parameters of the function F.

[0108] In step S2 of this embodiment, a knowledge graph propagation layer is established. In the knowledge graph propagation layer, the seed set contains two information encodings. For the user seed set, the user's historical interest set is used as the user seed set. For the project seed set, since it has been aligned with the entity node in the knowledge graph, the seed node of project v is itself. The two seed sets with the same encoding simultaneously propagate high-order information by diffusion in the knowledge graph.

[0109] In the knowledge graph propagation layer, a seed set of users and items is set and propagated in the knowledge graph in a certain pattern to obtain a multi-hop set.

[0110] The knowledge graph carries a variety of entities and complex connections between entities, providing a potential perspective for exploring user interests and aggregating entity information. However, for a certain user or project, not all entities have the same importance, and there are many irrelevant entities (called noise).

[0111] Therefore, in order to filter noise, the present invention proposes to merge information of the seed set along the relationship, so that each merged entity is related to the seed set.

[0112] For the seed set of user u, the alignment items of the user's historical interest interaction set in the knowledge graph are used as the initial set to be propagated in the knowledge graph. The user seed set of user u is defined as follows:

[0113]

[0114] At the same time, the project seed set of project v is its own alignment item in the knowledge graph. The project seed set of project v is defined as follows:

[0115]

[0116] In the knowledge graph, adjacent entities often have strong correlations. By propagating along the links in the knowledge graph, we can obtain the extended entity set and triplets with different distances from the initial entity set, which can effectively expand the potential vector representation of users and items.

[0117] The entity set definition for user u and item v is recursively expressed as:

[0118]

[0119] Where k represents the distance from the initial entity set, and the subscript symbol o is a uniform placeholder for the symbol u or v. Given the definition of the entity set, the kth triple set of user u and item v is defined as follows:

[0120]

[0121] H represents the number of hops for transmission.

[0122] It is very meaningful to use the knowledge graph as side information to build the model of the method of the present invention, because the neighbor entities can be regarded as an intuitive extension of user preferences and item features. The initial entity set obtained by the user-item interaction matrix is ​​similar to the source of sound waves, which propagates layer by layer in the medium of the knowledge graph from near to far. Through knowledge-based deep propagation, the high-order interaction information of users and items based on knowledge is successfully captured, effectively improving the model's ability to represent the potential vectors of users and items.

[0123] In step S3 of this embodiment, a new knowledge-aware attention mechanism is proposed to learn how to select more valuable information in each propagation of the entity set. There are potential associations between entities, that is, the importance of each relationship between different entity sets will show great differences. Different users and projects should also have different levels of attention when facing relationships.

[0124] The difference between the user seed set and the project seed set is that a user set contains multiple entity nodes, corresponding to the historical interaction records of the user, but a project set only corresponds to one node.

[0125] Taking users as an example, the user seed set is used as the initial entity set to propagate user interests in the knowledge graph. Considering that different users should have different levels of attention to different relationships, a simple approach is to directly perform the inner product of the user u vector and the relationship relation vector to obtain the attention score. However, this approach ignores the potential associations between entities. To this end, this embodiment designs and proposes an attention network that can capture the potential associations between entities and learn a potential representation a for each relationship.

[0126] In general, the attention mechanism can be described as mapping a query and a set of key-value pairs to an output. The output is computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility function of the query and the corresponding key.

[0127] When exploring potential relationships between entities, the connection between the tail entity and the relationship is taken into consideration. For the pair Q and K, this embodiment uses the relationship relation and the tail entity node tail to perform information aggregation.

[0128] Because when considering the properties of a relationship node, the tail node it points to must be considered, the same relationship will exhibit different properties when it is in different node spaces.

[0129] V is the relation itself. As shown below:

[0130]

[0131]

[0132] Learning captures a weight for the relationship between each entity set to automatically explore the importance of different relationships between different entity sets. Since (h, r, t) is the i-th triple in the k-th layer triple set, a potential relationship attention embedding α between entities is constructed i as follows:

[0133] α i =π1(Q i ,K i ,V i )

[0134] in, π1(Q i ,K i ,V i ) is a network for generating potential correlation attention:

[0135] π1(Q i ,K i ,V i )=V i softmax(Q i ,K i )

[0136]

[0137] Among them, softmax(·) is the normalization function; Q i ,K i After calculation, the weights and V are obtained through normalization. i Multiplying them together gives us the vector representation after exploring the potential association, which is the weighted relation.

[0138] Correlation vector α i It can capture the potential association representation between entity sets. By exploring the potential association between entity sets, more attention is paid to more important relations and lower weights are given to insignificant relations, effectively reducing the noise between entity sets.

[0139] However, the correlation vector α i It cannot be used directly for recommendation. Capturing the potential association between entities is only the first step. The more critical work is to use the optimized triples to learn the association vector α of users or items. i The preference degree of each hop is used to find the most valuable information in the entity set of each hop. i As shown below:

[0140] e i=π2(α i ,o)

[0141] Among them, α i is the association vector, and o is the placeholder for user u or item v. i ,o) is an aggregated network.

[0142] π2(α i ,o)=softmax(α i ,o)t i

[0143] Among them, t i is the embedding of the tail entity in the triple.

[0144] This embodiment takes the first hop as an example:

[0145]

[0146] in, is the set of level 1 triples of users or items.

[0147] User u or item v and the associated vector α i After the inner product, normalization is performed to obtain the weight, which is multiplied by the tail node in the triple, and finally the entity set used for aggregation at each hop is obtained.

[0148] Finally, the representation of the k-th layer triple of user u or item v is obtained:

[0149]

[0150] where subscript and superscript o are consistent placeholders for the symbols u or v, Is a collection The number of triples in .

[0151] It should be noted that the seed set, as the source point of propagation and diffusion in the knowledge graph, is closest to the original representation, so the seed set has a strong connection with the user or item. Accordingly, the representation of the seed set is added to the user and item:

[0152]

[0153] In the knowledge-aware attention embedding layer, a set of entities for obtaining potential associations is constructed for user u and item v as follows:

[0154]

[0155] In step S4 of this embodiment, by exploring the potential associations in the entity set when users and items propagate in the knowledge graph, this embodiment can fully explore the impact of high-order relationships on recommendations. This embodiment proposes three aggregators to aggregate the entity representations in each hop set when the user-item representation entity set diffuses, that is, to aggregate multiple representations in the entity set of user u and item v into a single vector representation.

[0156] The first aggregator is the sum aggregator, which sums the representation vectors of the multi-hop set before performing the nonlinear transformation:

[0157]

[0158] Among them, σ is the nonlinear function Sigmoid, W and b are trainable weights and biases. The other two aggregators use the same symbols and nonlinear functions.

[0159] The second aggregator is the Concat aggregator, which concatenates the representation vectors in the representation set and applies the following nonlinear transformation:

[0160]

[0161] in ∣∣ is a concatenation operation. Using the Concat aggregator requires ensuring and The representation quantity in is the same.

[0162] The third aggregator is the Pooling aggregator, which takes the maximum value among multiple vectors and then performs a nonlinear transformation:

[0163]

[0164] Among them, pool max (·) indicates a pooling aggregator, which selects the maximum value in the result set. Pooling aggregators are not considered. and For the problem of whether the number of representations is the same, only the maximum value is selected for processing.

[0165] Using Epsilon u Represents the user's aggregate vector. Similarly, the aggregate vector of item v uses ε v To represent it, the inner product is finally used to predict the user's preference score for the item:

[0166]

[0167] in, represents the predicted preference score of user u for item v.

[0168] In step S5 of this embodiment, in order to balance the number of positive and negative samples and ensure the effect of model training, this embodiment extracts the same number of negative samples as positive samples for each user, and then designs a loss function for training. The loss function is expressed as:

[0169]

[0170] in, is the cross entropy loss, represents the set of positive user-item pairs, Represents the set of negative user-item pairs. is a parameter set, E and R are embedding tables of all entities and relations, is the L2 regularization term parameterized by λ.

[0171] After the training is completed, this embodiment also conducts experimental verification. This embodiment analyzes and evaluates the proposed model in three real scenarios.

[0172] Dataset description: In order to evaluate the effectiveness of the KGER model, this embodiment uses two public datasets for evaluation in two scenarios: music and books.

[0173] Last.FM: A large-scale dataset provided by the last.fm online music system, including music listeners’ listening history, music tags, artist information, etc., which is often used for music recommendation. The tracks are identified as items by the last.fm online music system.

[0174] Book-Crossing is collected from the book-crossing community, where users rate books on a scale of 0-10. Since both LastFM and Book-Crossing datasets are explicit feedback, they need to be converted into implicit feedback in the model of this invention, where 1 represents a positive sample and 0 represents a negative sample. If there is interaction between the user and the item, it is a positive sample, which is set to 1.

[0175] For constructing negative samples, this embodiment randomly selects the same number of items as positive samples from the set of items that have not interacted with the user to form negative samples. The constructed data set is shown in Table 1:

[0176] Table 1

[0177] Last.FM Book-Crossing Number of users 1872 17860 Number of projects 3846 14967 Number of interactions 42346 139746 Number of entities 9366 77903 Number of relationships 60 25 Number of triplets 15518 151500

[0178] Baseline Model:

[0179] CKE is a typical embedding-based model that combines the CF module with the structural, textual, and visual knowledge embedding of items in a unified Bayesian framework. In this embodiment, the CF module plus a structured knowledge module is used as the implementation.

[0180] PER is a typical path-based method that regards the knowledge graph as a heterogeneous information network and extracts meta-path-based latent features to represent the connectivity between users and items along different types of relational paths.

[0181] RippleNet is a typical propagation-based model, similar to water ripples, spreading from the center to the surrounding areas. It uses user historical interactions as user interest seed sets to propagate preferences in the knowledge graph to enrich user representations.

[0182] KGCN is a typical propagation-based model that preferentially aggregates domain information of entity nodes in the knowledge graph, extends the non-spectral GCN method to the knowledge graph, and enhances project representation by iteratively aggregating domain information in the knowledge graph.

[0183] KGNN-LS is a propagation-based model that transforms heterogeneous knowledge graphs into user-specific weighted graphs and computes personalized item embeddings in graph neural networks, uses label smoothing regularization to identify important relationships of users, and provides edge weight regularization to reduce inductive bias.

[0184] KGAT is a propagation-based model that proposes to combine the user-item mutual graph and the knowledge graph into one graph, using the TransR algorithm and the attention mechanism to optimize the embedding of nodes by recursively propagating the embedding of nodes from the embeddings of neighboring nodes.

[0185] CKAN is a propagation-based model and a collaborative knowledge-aware attention network that adopts a heterogeneous propagation method to encode two kinds of information and uses a knowledge-aware attention mechanism to distinguish the contributions of different knowledge neighbors.

[0186] RKGCN uses knowledge graphs to simultaneously improve the representation of users and items, and then uses the enhanced representations to capture the dynamic interests of users when encountering different items, thereby making more accurate recommendations. By using knowledge graphs to enrich the representation of users and items, RKGCN is able to provide more personalized and relevant recommendations.

[0187] Experimental setup: In the experiment of this embodiment, each data set is divided into a training set, an evaluation set, and a test set in a ratio of 6:2:2, and the application scenarios select CTR and top-k recommendation scenarios. This embodiment uses the model learned from the training set to predict the probability of each interaction in the test set. For CTR prediction, AUC and F1 are used to evaluate the performance of the model; for top-k recommendations, Recall@K is used for performance evaluation. For all baseline models, this embodiment conducts experiments according to the optimal data they can provide or according to the optimal parameters given, that is, all baseline models are in the optimal state.

[0188] Experimental results: This example gives the comparison results of KGER and all baseline models. The results of all methods in CTR prediction and top-K recommendation are shown in Table 2 and Table 3, respectively. Figure 2 shown.

[0189] Table 2

[0190]

[0191]

[0192] Intuitively, KGER achieves good results in both recommendation scenarios, and we can observe that:

[0193] (1) In both scenarios on both datasets, KGER performs best among all methods. Specifically, as shown in Table 2, KGER’s AUC and F1 on the Last.FM dataset are 1.19% and 0.65% higher than the best baseline, respectively; on the Book-Crossing dataset, KGER’s AUC and F1 are 2.68% and 1.95% higher than the best baseline, respectively. Figure 2 As shown in Figure 2, KGER achieves more significant improvements in Recall@K.

[0194] (2) The performance of all models on the Book-Crossing dataset is worse than that on the Last.FM dataset. The main reason is that the data in the Book-Crossing dataset is relatively sparse. For KGER, data sparsity means that there are fewer types of relationships and there is a lack of sufficient data to mine more accurate potential relationships.

[0195] (3) In the top-k recommendation scenario, when k becomes larger, KGER performs better. The excellent performance of KGER verifies the effectiveness of exploring potential relationships in improving recommendation performance.

[0196] (4) From the experimental results, we can see that in the two recommendation scenarios and in the two data sets, the performance of the embedding-based method and the path-based method is obviously worse than that of the propagation-based method. This verifies the effectiveness of aggregating high-order information based on associations in the knowledge graph in improving recommendation performance.

[0197] (5) The path-based method (PER) performs worse than other methods because it is difficult to define the optimal meta-path in practice. The embedding-based method (CKE) performs poorly, which may be due to the lack of visual and textual data.

[0198] (6) Experimental results show that the propagation-based methods have a good performance in recommendation, but other methods ignore the importance of potential relationships and ignore the fact that users and items should have different attention weights for different relationships when aggregating high-order neighborhood features. The experimental results of this embodiment verify the advancement and effectiveness of the KGER model.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A recommendation method based on knowledge graph and attention mechanism, characterized by: The recommended method comprises the following steps: S1. Define the knowledge graph-aware recommendation problem, establish the user-item interaction matrix and the knowledge graph as input; S2. Set the seed sets of users and items in the knowledge graph propagation layer, and propagate the seed sets along adjacent entities to capture the knowledge-based high-order interaction information of users and items; S3, in the knowledge-aware attention embedding layer, the corresponding potential relationship representation is learned for each relationship through the attention mechanism to obtain a set of potentially related entities; S4, processing the entity set through the aggregator to obtain the corresponding user aggregation vector and project aggregation vector, and predicting the user's preference score for the project based on the aggregation vector; S5. Design a loss function to train the above process, and use it for user preference recommendation after training.

2. A recommendation method based on knowledge graph and attention mechanism according to claim 1, characterized in that: In step S1, a user set of M users is set in the recommendation scenario. A project collection with N projects Then define the user-item interaction matrix Y∈R according to the user's implicit feedback M*N , which is expressed as: Among them, y uv =1 means that user u has historical interaction with item v, otherwise y uv =1 means that user u has no historical interaction with item v; Create a knowledge graph G as input: G={(h,r,t)|h,t∈E,r∈R} Among them, (h, r, t) represents a large number of entity triples, h∈E, r∈R, t∈E represent the head, relationship and tail of the knowledge triple respectively, E and R represent the entity and relationship sets in the knowledge graph respectively; Set A = {(v, e) | v∈V, e∈E} to indicate the alignment relationship between items and entities, where (v, e) means that item v can be aligned with entity e in the knowledge graph; For a given user-item interaction matrix Y and knowledge graph G, establish a prediction function: yes uv =F(u,v|Θ,G) Among them, y uv represents the predicted probability, and Θ represents the model parameters of the function F.

3. A recommendation method based on knowledge graph and attention mechanism according to claim 2, characterized in that: In step S2, the following steps are implemented in the knowledge graph propagation layer: S21. Define the user seed set of user u and the project seed set of project v: in, represents the user seed set, represents the project seed set; S22. The seed set is propagated along the links in the knowledge graph to obtain the extended entity set and triples with different distances from the initial entity set, where the entity set of user u and item v is represented as: Where k represents the distance from the initial entity set, and the subscript symbol o is a unified placeholder for the symbol u or v; Then the kth triplet set of user u and item v is expressed as: Where H represents the number of hops for transmission.

4. The recommendation method based on knowledge graph and attention mechanism according to claim 3, characterized in that: In step S3, the following steps are performed in the knowledge-aware attention embedding layer: S31. Based on the attention mechanism, for the query Q and key k, the relation relation and the tail entity node tail are used to aggregate information, and the value V is the relation itself, then: S32. Establish attention embedding of potential relationships between entities α i : α i =π1(Q i ,K i ,V i ) in, π1(Q i ,K i ,V i ) is a network for generating potential correlation attention: π1(Q i ,K i ,V i )=V i softmax(Q i ,K i ) Among them, softmax(·) is the normalization function; S33, using the optimized triples to learn the user or item's association vector α i The preference degree is to find the most valuable information in the entity set of each hop. i It is expressed as: e i =π2(a i ,o) Among them, α i is the association vector, o is the placeholder for user u or item v; π2(α i ,o) is the aggregation network, which is expressed as: π2(a i ,o)=softmax(α i ,o)t i Among them, t i is the embedding of the tail entity in the triple; S34. Obtain the representation of the k-th layer triple of user u or item v: where subscript and superscript o are consistent placeholders for the symbols u or v, Is a collection The number of triples in ; S35. Establish a set of potential associations between user u and item v: in, and Represent the seed sets added for users and items respectively.

5. A recommendation method based on knowledge graph and attention mechanism according to claim 4, characterized in that: In step S4, the entity representations in each hop set are aggregated by the aggregator when the user-item representation entity set is diffused, that is, multiple representations in the entity set of user u and item v are aggregated into a single vector representation, and then the inner product is used to predict the user's preference score for the item: in, represents the predicted preference score of user u for item v, ε u represents the user's aggregation vector, ε v An aggregate vector representing items.

6. The recommendation method based on knowledge graph and attention mechanism according to claim 5, characterized in that: The aggregator includes at least a summing aggregator, a concat aggregator and a pooling aggregator. The summing aggregator sums the representation vectors of the multi-hop set before performing nonlinear transformation; the concat aggregator connects the representation vectors in the representation set in series and performs nonlinear transformation; the pooling aggregator takes the maximum value among multiple vectors and then performs nonlinear transformation; wherein, The operation process of the sum aggregator is expressed as: Among them, σ is the nonlinear function Sigmoid, W and b are trainable weights and biases; The operation process of the Concat aggregator is expressed as: in ∣∣ is the splicing operation; The operation process of the Pooling aggregator is expressed as: Among them, pool max (·) is a pooling aggregator that can select the maximum value in the result set.

7. The recommendation method based on knowledge graph and attention mechanism according to claim 6, characterized in that: In step S5, a loss function is set to train the above process, where the loss function is expressed as: in, is the cross entropy loss, represents the set of positive user-item pairs, represents the set of negative user-item pairs; is a parameter set; E and R are embedding tables of all entities and relations, is the L2 regularization term parameterized by λ.

Images