Recommendation method based on knowledge graph and attention mechanism

Abstract

The application relates to a recommendation method based on a knowledge graph and an attention mechanism and belongs to the field of personalized recommendation. The method comprises the following steps: defining a knowledge graph perception recommendation problem, establishing a user and item interaction matrix and a knowledge graph as input; setting a seed set of users and items in a knowledge graph propagation layer, propagating the seed set along adjacent entities, and capturing knowledge-based high-order interaction information of the users and items; learning a corresponding latent relationship representation for each relationship through an attention mechanism in a knowledge perception attention embedding layer to obtain a latent associated entity set; processing the entity set through an aggregator to obtain a corresponding aggregation vector, and predicting a preference score of a user for an item according to the aggregation vector; and designing a loss function to train the above process, which is used for user preference recommendation after the training is completed. The application can more accurately capture the latent relationship between the user and the item, and provide more accurate and personalized recommendation services for the user.

Description

Technical Field

[0001] This invention belongs to the field of personalized recommendation and relates to a recommendation method based on knowledge graphs and attention mechanisms. Background Technology

[0002] With the continuous advancement of internet technology, people can easily access a vast amount of online content, covering various fields such as music, movies, and products. However, the amount of information on the internet is also constantly increasing, and information overload is becoming increasingly common, making it difficult for users to quickly and accurately find items that match their personal interests. To solve this problem, recommendation systems have emerged, aiming to alleviate information overload and meet users' personalized needs.

[0003] Traditional recommendation techniques, such as collaborative filtering, provide personalized recommendations by analyzing the similarity between users or items. However, their performance is limited when faced with the cold start problem and data sparsity. To overcome these issues, researchers have adopted methods that add auxiliary information, introducing additional information or features to improve recommendation performance. Among these approaches, knowledge graphs are receiving increasing attention due to their ability to provide richer information about items and the relationships between them.

[0004] Knowledge graphs, as a form of auxiliary information, contain rich semantic features, which help to further obtain higher-order neighborhood information between entities, enrich entity representations, and thus solve the data sparsity problem to improve recommendation accuracy. The topological structure information in knowledge graphs is closely related to the information in the user-item interaction matrix in recommendation algorithms. The combination of the two brings deeper data associations, making up for the original data sparsity. Through the topological structure of knowledge graphs and effective recommendation algorithms, the potential features of users and items can be mined.

[0005] By incorporating knowledge graphs, recommendation systems can more accurately simulate user preferences and alleviate data sparsity. Compared with recommendation methods that do not incorporate knowledge graphs, introducing knowledge graphs into recommendations has the following advantages: (1) Knowledge graphs contain rich semantic relationships, which can connect attributes and improve the accuracy of recommendation systems, enabling more accurate discovery of similarity between attributes. (2) Knowledge graphs themselves contain rich structural information, which can be expanded through different categories to improve the diversity of recommendation systems. (3) The rich structural information in knowledge graphs, including the association of candidate items to be recommended in the user's historical interaction matrix, enhances the interpretability of recommendations. (4) When new users or items are added to the knowledge graph, they may be associated with previous users or items through certain attributes, which helps to alleviate the cold start problem of recommendations.

[0006] Despite the advantages mentioned above, integrating knowledge graphs into recommender systems remains challenging due to their high dimensionality and heterogeneity. The core work of applying knowledge graphs to recommender systems lies in enriching the latent vector representations of users and items using the heterogeneous information within the knowledge graph. Based on this core work, knowledge graph-based recommender systems can be broadly categorized into three types: embedding-based methods, path-based methods, and propagation-based methods.

[0007] (1) Embedding-based methods preprocess knowledge graphs using Knowledge Graph Embedding (KGE), embedding entities and relationships within the graph into a low-dimensional space and mapping them to representation vectors. For example, Collaborative Knowledge Base Embedding (CKE) learns item embeddings with the participation of KG through TransR. Deep knowledge-aware networks (DKN) treat entity embeddings and word embeddings as different channels, using TransD to generate news embeddings for recommendations using KG. However, KGE methods are more suitable for in-graph applications such as knowledge graph completion and prediction than for recommendations. Therefore, the learned entity embeddings are insufficient for item recommendations. Furthermore, due to the lack of explicit modeling capabilities, embedding-based knowledge graph recommendations cannot guarantee the capture of long-distance node dependencies and struggle to interpret high-order semantic relationships between entities for recommendation purposes.

[0008] (2) Path-based algorithms provide preference information for entity recommendations by extracting entity paths rich in high-order node information from knowledge graphs. For example, Personalized Entity Recommendation (PERKG) treats a heterogeneous information network and extracts latent features based on meta-paths / graph primitives along different types of relationship paths / graphs to represent the connectivity between users and items. Path-based algorithms make full use of the network structure in knowledge graphs. However, path-based algorithms are severely limited by manually designed meta-paths, which is a challenging task in complex knowledge graphs and makes it difficult to design optimal paths.

[0009] (3) Propagation-based methods, which iteratively propagate information across the entire knowledge graph and combine it with the aggregation mechanism of graph neural networks to obtain information from higher-order domains, can effectively alleviate the shortcomings of embedding-based and path-based methods. However, there are still some areas for improvement. For example, the information mining in the knowledge graph is not deep enough, and the entity and relation information in the knowledge graph cannot be fully utilized. Furthermore, it is impossible to simultaneously obtain higher-order user features and higher-order item features, enriching both user and item representations. Existing algorithms only fully mine information from one end, that is, they only consider the association between the user end or item end and entities in the knowledge graph, lacking full utilization of the knowledge graph information. At the same time, existing work ignores the importance of relation vectors when aggregating neighborhood information and fails to extract the potential relationships hidden deep within the entity set. Therefore, exploring the semantic association between user-end features and item-end features and entities in the knowledge graph, as well as mining the hidden information in user-item historical interaction records, remains a challenge.

[0010] RippleNet, or Ripple Network, uses the historical interaction matrix of users and items as seed nodes to spread user interests across the knowledge graph, much like ripples on water. This approach has achieved excellent results. However, RippleNet only performs high-order modeling of user features, enriching their representation, but it doesn't consider high-order modeling of item features. Furthermore, it neglects the potential relationships between entity sets when spreading user interests.

[0011] Knowledge Graph Convolutional Networks (KGCNs) utilize the core idea of ​​graph neural networks (GCNs) to aggregate domain information to enrich item feature representations. By capturing user preferences for different relationships through this aggregation, it demonstrates that neighbor information can significantly improve recommendation tasks. However, it's important to note that while both methods consider the influence of users on relationship weights, they neglect potential relationships between entities, resulting in insufficient item embedding. Furthermore, they only consider the association between the item and entities in the knowledge graph, ignoring the association between the user and entities in the knowledge graph, thus failing to fully utilize the information within the knowledge graph.

[0012] Knowledge Graph Attention Network (KGAT) proposes a Collaborative Knowledge Graph (CKG), which combines a User-Item Bipartite Graph (UIG) and a Knowledge Graph (KG), and recursively propagates across the CKG via a Generative Network (GCN) to supplement entity embeddings. One drawback of KGAT is that it treats users as the same nodes as entities in the CKG, making it difficult to handle new users. Once a new interacting user appears, KGAT needs to rebuild the CKG and retrain the entire model to recommend to them, a computational cost that is unacceptable in industry. Furthermore, KGAT assumes that interaction items in the UIG and associated entities in the KG should be treated as homogeneous nodes, when in reality they reside in different latent spaces. Additionally, since the types of items interacted with by users are often very broad, higher-order interactions in the UIG may have completely different meanings from the latent semantic representations of the original items, introducing noise into the embeddings.

[0013] Collaborative Knowledge-aware Attentive Network (CKAN) combines collaborative filtering with knowledge graphs, proposing collaborative signals to explicitly encode user-item information for propagation within the knowledge graph. It also employs a knowledge-aware attention mechanism to assess the importance of different neighbors. CKAN simultaneously performs high-order modeling of both user and item features, enriching the feature representations of users and items. However, it fails to consider the impact of potential relationships between entities on the modeling of user and item features.

[0014] Knowledge Graph-based Intent Networks (KGINs) can identify users' latent intentions and recognize the relationship between users and items at a fine-grained level to achieve better recommendation results. However, they ignore the impact of latent relationships on recommendations, resulting in noisy user intentions being learned.

[0015] Existing research mostly focuses on mining information from only one end of the knowledge graph, considering only the relationship between the user or project end and entities in the knowledge graph. It lacks full utilization of knowledge graph information and ignores the importance of potential relationships for personalized recommendations. Summary of the Invention

[0016] In view of this, the purpose of this invention is to provide a recommendation method based on knowledge graphs and attention mechanisms.

[0017] To achieve the above objectives, the present invention provides the following technical solution:

[0018] A recommendation method based on knowledge graphs and attention mechanisms includes the following steps:

[0019] S1. Define the knowledge graph perception recommendation problem, and establish a user-item interaction matrix and a knowledge graph as input.

[0020] S2. Set up seed sets for users and projects in the knowledge graph propagation layer, and propagate the seed sets along adjacent entities to capture knowledge-based high-order interaction information of users and projects;

[0021] S3. In the knowledge-aware attention embedding layer, the attention mechanism is used to learn the corresponding latent relation representation for each relation, thereby obtaining a set of potentially related entities;

[0022] S4. Process the entity set through the aggregator to obtain the corresponding user aggregation vector and item aggregation vector, and predict the user's preference rating for the item based on the aggregation vector;

[0023] S5. Design a loss function to train the above process. After training, the loss function is used for user preference recommendation.

[0024] Furthermore, in step S1, a user set of M users is defined in the recommendation scenario. A collection of N projects Then, based on the user's implicit feedback, define the user-project interaction matrix Y∈R. M*N It is represented as:

[0025]

[0026] Among them, y uv =1 indicates that user u and project v have a history of interaction; otherwise, y = 1. uv =1 indicates that user u and project v have no historical interaction;

[0027] Construct a knowledge graph G as input:

[0028] G={(h,r,t)|h,t∈E,r∈R}

[0029] Where (h,r,t) represents a large number of entity triples, h∈E, r∈R, t∈E represent the head, relation, and tail of the knowledge triples respectively, and E and R represent the sets of entities and relations in the knowledge graph respectively;

[0030] Let set A = {(v,e)|v∈V,e∈E} represent the alignment relationship between items and entities, where (v,e) means that item v can be aligned with entity e in the knowledge graph;

[0031] Given a user-item interaction matrix Y and a knowledge graph G, establish a prediction function:

[0032] y uv =F(u,v|Θ,G)

[0033] Among them, y uv Let F represent the predicted probability, and Θ represent the model parameters of function F.

[0034] Furthermore, in step S2, the following steps are implemented in the knowledge graph propagation layer:

[0035] S21. Define the user seed set for user u and the project seed set for project v:

[0036]

[0037] in, Represents the user seed set, Represents the project seed set;

[0038] S22. Propagate the seed set along links in the knowledge graph to obtain an expanded entity set and triples at different distances from the initial entity set, where the entity sets of user u and item v are represented as:

[0039]

[0040] Where k represents the distance from the initial set of entities, and the subscript o is a placeholder for the symbols u or v;

[0041] The set of the k-th triplet for user u and item v is represented as:

[0042]

[0043] Where H represents the number of hops in the propagation.

[0044] Furthermore, in step S3, the following steps are performed in the knowledge-aware attention embedding layer:

[0045] S31. Based on the attention mechanism, for query Q and key K, information is aggregated using relation and tail entity node, while value V is the relation itself. Therefore:

[0046]

[0047] S32, Establishing potential relationships between entities: Attention embedding α i :

[0048] α i =π1(Q i ,K i V i )

[0049] in, π1(Q i ,K i V i For generating networks with potential associative attention:

[0050] π1(Q i ,K i V i ) = V i softmax(Q i ,K i )

[0051]

[0052] Where softmax(·) is the normalization function;

[0053] S33. Use optimized triples to learn the relationship between users or items and the associated vector α. i Based on the degree of preference, find the most valuable information in the entity set of each hop, and the node e in each hop. i Represented as:

[0054] e i =π2(α) i ,o)

[0055] Where, α i For the associated vector, o is a placeholder for user u or item v; π2(α i (,o) is a aggregation network, represented as:

[0056] π2(α i ,o)=softmax(α i ,o)t i

[0057] Among them, t i Embedding of the tail entity in the triple;

[0058] S34. Obtain the representation of the k-th level triplet for user u or item v:

[0059]

[0060] In this context, the subscript and superscript 'o' are consistent placeholders for the symbols 'u' or 'v'. It is a set The number of triplet pairs;

[0061] S35. Establish a set of potential associations between user u and project v:

[0062]

[0063] in, and These represent the seed sets added for users and projects, respectively.

[0064] Furthermore, in step S4, the entity representations in each hop set during the diffusion of the user item representation entity set are aggregated by an aggregator. That is, multiple representations in the entity sets of user u and item v are aggregated into a single vector representation, and then the inner product is used to predict the user's preference rating for the item.

[0065]

[0066] in, ε represents the predicted preference rating of user u for item v. u ε represents the aggregate vector of users. v This represents the aggregate vector of the project.

[0067] Furthermore, the aggregator includes at least a summation aggregator, a concat aggregator, and a pooling aggregator. The summation aggregator sums the representation vectors of the multi-hop set before performing the nonlinear transformation; the concat aggregator concatenates the representation vectors in the representation set and performs the nonlinear transformation; the pooling aggregator takes the maximum value from multiple vectors and then performs the nonlinear transformation.

[0068] The operation process of the summation aggregator is represented as follows:

[0069]

[0070] Where σ is the non-linear function Sigmoid, and W and b are the trainable weights and biases;

[0071] The operation process of the Concat aggregator is represented as follows:

[0072]

[0073] in || is a splicing operation;

[0074] The operation process of the pooling aggregator is represented as follows:

[0075]

[0076] Among them, pool max (·) is a pooling aggregator that selects the maximum value in the result set.

[0077] Furthermore, in step S5, a loss function is set to train the above process, wherein the loss function is expressed as:

[0078]

[0079] in, It is cross-entropy loss. This represents a positive set of user-item pairs. This represents a negative set of user-item pairs; For the parameter set; E and R are embedded tables of all entities and relations.

[0080] The L2 regularization term is parameterized for λ.

[0081] The beneficial effects of this invention are as follows:

[0082] This invention proposes a novel knowledge graph recommendation model, KGER, to address the data sparsity and cold-start problems of traditional recommendation systems. KGER makes recommendations by capturing latent relationships between entities and exploring the different weights users and items place on these relationships. Specifically, KGER first iteratively propagates through the knowledge graph to acquire more knowledge. Through knowledge-based deep propagation, it successfully captures high-order interaction information between users and items based on knowledge, effectively improving the model's ability to represent the latent vectors of users and items. It then explores latent relationships between knowledge based on a self-attention mechanism and constructs a knowledge-aware attention module to distinguish the contribution of different neighbors. This invention considers a problem neglected in previous knowledge graph recommendation systems: the connection between the diversity of relationships between different entities and the potential associations within the entity set. It designs a novel relationship-aware personalized attention module to mine the potential relationships between entities and spreads the learning of the most valuable information for users or items within the knowledge graph. Relationship awareness explores potential relationships between entities, and by assigning weights to the diversity of relationships between different entities, it can discover the potential relationships between entities.

[0083] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0084] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0085] Figure 1 This is a schematic diagram illustrating the overall framework of the recommendation method based on knowledge graphs and attention mechanisms of the present invention.

[0086] Figure 2 This is a schematic diagram showing the comparison results between the KGER model of this invention and all baseline models. Detailed Implementation

[0087] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0088] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0089] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0090] Please see Figures 1-2 This is a recommendation method based on knowledge graphs and attention mechanisms.

[0091] Example

[0092] Knowledge graphs have been widely studied and applied as auxiliary information in recommender systems due to their ability to effectively solve the sparsity and cold-start problems of collaborative filtering. However, most existing research methods focus on extracting high-order user or item features to enrich the representation of users and items, neglecting the importance of latent relationships hidden within entity sets for aggregating neighborhood information. This embodiment provides a recommendation method based on knowledge graphs and attention mechanisms. Specifically, it proposes a novel method called Knowledge Graph for Exploring Relations (KGER), which explores latent relationships between entities in a new way to capture implicit information hidden in the graph and proposes a natural way to aggregate high-order user and item features based on latent relationships. Specifically, KGER captures latent relationships hidden between entity sets through a self-attention network mechanism. A novel knowledge-aware attention module is designed to obtain the different importance of users and items to latent relationships to aggregate features of entity neighborhoods, automatically retrieving high-order information from the knowledge graph. KGER captures more hidden information in the knowledge graph to improve the representation of high-order user and item features. This embodiment demonstrates through experiments on two real-world public datasets that KGER outperforms several state-of-the-art baseline models in click-through rate prediction and top-k recommendation scenarios. Figure 1 As shown, recommendation methods based on knowledge graphs and attention mechanisms include at least the following steps:

[0093] S1. Define the knowledge graph perception recommendation problem, and establish a user-item interaction matrix and a knowledge graph as input.

[0094] S2. Set up seed sets for users and projects in the knowledge graph propagation layer, and propagate the seed sets along adjacent entities to capture knowledge-based high-order interaction information of users and projects;

[0095] S3. In the knowledge-aware attention embedding layer, the attention mechanism is used to learn the corresponding latent relation representation for each relation, thereby obtaining a set of potentially related entities;

[0096] S4. Process the entity set through the aggregator to obtain the corresponding user aggregation vector and item aggregation vector, and predict the user's preference rating for the item based on the aggregation vector;

[0097] S5. Design a loss function to train the above process. After training, the loss function is used for user preference recommendation.

[0098] In step S1 of this embodiment, it is assumed that in a typical recommendation scenario, there is a user set of M users. A collection of N projects A user-project interaction matrix Y can be constructed based on the user's interaction history with the project. User-project interaction matrix Y∈R M*N Derived from the implicit feedback of users:

[0099]

[0100] Among them, y uv =1 indicates that user u and project v have a history of interaction; otherwise, y = 1. uv =0. Since the user-project interactions in the dataset used in this embodiment are all explicit feedback, it is necessary to convert the explicit feedback into implicit feedback, that is, to set specific rating thresholds for different datasets and convert the rating dataset into a binary classification dataset.

[0101] In addition to the interaction matrix Y, a knowledge graph G corresponding to the recommendation scenario is also required as input:

[0102] G={(h,r,t)|h,t∈E,r∈R}

[0103] Here, (h,r,t) represents a large number of entity triples, where h∈E, r∈R, and t∈E represent the head, relation, and tail of the knowledge triple, respectively, and E and R represent the sets of entities and relations in the knowledge graph, respectively. For example, the triple (Guo Fan, Director, *The Wandering Earth 2*) states the fact that Guo Fan is the director of the movie *The Wandering Earth 2*. Note that the item set V is contained within the entity set E.

[0104] In addition, the alignment relationship between items and entities is illustrated using the set A = {(v,e)|v∈V,e∈E}, where (v,e) indicates that item v can be aligned with entity e in the knowledge graph.

[0105] The ultimate goal of the recommendation is to determine the degree of preference for item v for user u, given that user u has not interacted with it. For a given user-item interaction matrix Y and knowledge graph G, the aim is to learn a prediction function:

[0106] y uv =F(u,v|Θ,G)

[0107] Among them, y uv Let F represent the predicted probability, and Θ represent the model parameters of function F.

[0108] In step S2 of this embodiment, a knowledge graph propagation layer is established. In the knowledge graph propagation layer, the seed set contains two types of information encoding. For the user seed set, the user's historical interest set is used as the user seed set. For the item seed set, since it is already aligned with the entity nodes in the knowledge graph, the seed node of item v is itself. The two seed sets with the same encoding propagate higher-order information through diffusion in the knowledge graph.

[0109] In the knowledge graph propagation layer, a seed set of users and projects is set up and propagated in the knowledge graph according to a certain pattern to obtain a multi-hop set.

[0110] Knowledge graphs contain a wide variety of entities and complex relationships between them, offering a potential perspective for exploring user interests and aggregating entity information. However, for a given user or project, not all entities are equally important, and many irrelevant entities (referred to as noise) exist.

[0111] Therefore, in order to filter out noise, this invention proposes to merge information in the seed set along the relation, so that each merged entity is related to the seed set.

[0112] For user u's seed set, the alignment of the user's historical interest interaction set with the knowledge graph is used as the initial set for propagation within the knowledge graph. The user seed set for user u is defined as follows:

[0113]

[0114] Meanwhile, the project seed set of project v is its alignment item in the knowledge graph. The project seed set of project v is defined as follows:

[0115]

[0116] In knowledge graphs, adjacent entities often have strong correlations. Propagating along links in a knowledge graph can yield expanded entity sets and triples at different distances from the initial entity set, effectively expanding the latent vector representations of users and items.

[0117] The entity sets of user u and project v are recursively represented as follows:

[0118]

[0119] Here, k represents the distance from the initial entity set, and the subscript o is a placeholder for the symbols u or v. Given the definition of the entity set, the k-th set of triples for user u and item v is defined as follows:

[0120]

[0121] H represents the number of hops in the propagation.

[0122] Using knowledge graphs as edge information to construct the model of this invention is highly meaningful, as nearest-neighbor entities can be seen as an intuitive extension of user preferences and item features. The initial entity set obtained through the user-item interaction matrix, similar to the source of sound waves, propagates layer by layer in the knowledge graph medium, from near to far. Through knowledge-based deep propagation, high-order knowledge-based interaction information between users and items is successfully captured, effectively improving the model's ability to represent the latent vectors of users and items.

[0123] In step S3 of this embodiment, a novel knowledge-aware attention mechanism is proposed, which learns how to select more valuable information in each propagation of an entity set. Potential relationships exist between entities; that is, the importance of each relationship varies greatly across different entity sets. Different users and projects should also have different levels of attention when dealing with relationships.

[0124] The difference between a user seed set and a project seed set is that a user set contains multiple entity nodes, each corresponding to a user's historical interaction records, while a project set has only one node per project.

[0125] Taking users as an example, user interests are propagated in the knowledge graph using a user seed set as the initial entity set. Considering that different users should have different levels of attention to different relationships, a simple approach is to directly calculate the attention score by performing an inner product between the user vector u and the relationship vector. However, this approach ignores the potential associations between entities. To address this, this embodiment proposes an attention network that can capture the potential associations between entities, learning a potential representation 'a' for each relationship.

[0126] In general, attention mechanisms can be described as mapping a query and a set of key-value pairs to an output. The output is computed as a weighted sum of values, where the weight assigned to each value is calculated by a function compatible with the query and the corresponding key.

[0127] When exploring potential relationships between entities, the connection between tail entities and relationships is taken into account. For the Q and K pairs, this embodiment uses the relationship and the tail entity node to aggregate information.

[0128] Because when considering the properties of a relation node, the tail node it points to must be considered, the same relation will exhibit different properties when it is in different node spaces.

[0129] V stands for the relation itself. As shown below:

[0130]

[0131]

[0132] The learning process captures a weight for the relationships between each entity set, automatically exploring the importance of different relationships between different entity sets. Since (h,r,t) is the i-th triple in the k-th layer triple set, a latent relationship attention embedding α between entities is constructed. i as follows:

[0133] α i =π1(Q i ,K i V i )

[0134] in, π1(Q i ,K i V i For generating networks with potential associative attention:

[0135] π1(Q i ,K i V i ) = V i softmax(Q i ,K i )

[0136]

[0137] Where softmax(·) is the normalization function; Q i ,K i After calculation and normalization, the weights and V are obtained. i Multiplication yields a vector representation of the potential correlations, which is the weighted relation.

[0138] Correlation vector α i It can capture the potential associations between entity sets. By exploring the potential associations between entity sets, more attention is paid to the more important relations, and less weight is given to the irrelevant relations, effectively reducing the noise between entity sets.

[0139] Nevertheless, the associated vector α i It cannot be directly used for recommendation yet. Capturing the potential relationships between entities is only the first step. The more crucial work lies in using the optimized triples to learn the relationship between users or items and the association vector α. i Based on the degree of preference, find the most valuable information in the entity set of each hop. In each hop, node e... i As shown below:

[0140] e i=π2(α) i ,o)

[0141] Where, α i This is the association vector, where o is a placeholder for user u or item v. π2(α) i ,o) is a aggregation network.

[0142] π2(α i ,o)=softmax(α i ,o)t i

[0143] Among them, t i This refers to the embedding of the tail entity in the triplet.

[0144] This embodiment uses the first hop as an example:

[0145]

[0146] in, It is the first-level triplet set of users or projects.

[0147] User u or project v and associated vector α i After the inner product is normalized, the weight is multiplied by the tail node in the triple, and finally the entity set used for aggregation in each hop is obtained.

[0148] Finally, we obtain the representation of the k-th level triplet for user u or item v:

[0149]

[0150] In this context, the subscript and superscript 'o' are consistent placeholders for the symbols 'u' or 'v'. It is a set The number of triples in the middle.

[0151] It's important to note that the seed set, as the source point for propagation and diffusion within the knowledge graph, is closest to the original representation. Therefore, the seed set has a strong connection with users or items. Accordingly, we add seed set representations for users and items:

[0152]

[0153] In the knowledge-aware attention embedding layer, a set of entities for obtaining potential associations between user u and item v is constructed as follows:

[0154]

[0155] In step S4 of this embodiment, by exploring the potential associations in the entity set when users and items spread in the knowledge graph, this embodiment can fully explore the impact of higher-order relationships on recommendations. This embodiment proposes three aggregators to aggregate the entity representations in each hop set when the user and item representation entity set spreads, that is, to aggregate multiple representations in the entity sets of user u and item v into a single vector representation.

[0156] The first aggregator is the sum aggregator, which sums the representation vectors of the multi-hop set before performing the nonlinear transformation:

[0157]

[0158] Where σ is the non-linear function Sigmoid, and W and b are the trainable weights and biases. The other two aggregators use the same sign and non-linear function.

[0159] The second aggregator is the Concat aggregator, which concatenates the representation vectors in the representation set and uses the following nonlinear transformation:

[0160]

[0161] in || represents the concatenation operation. Using the Concat aggregator requires ensuring… and The same number of items are represented in the text.

[0162] The third aggregator is the Pooling aggregator, which takes the maximum value of multiple vectors and then performs a nonlinear transformation:

[0163]

[0164] Among them, pool max (·) indicates a pooling aggregator, which selects the maximum value in the result set. Pooling aggregators do not need to be considered. and The problem of whether the quantities are the same is addressed by selecting only the maximum value.

[0165] Using ε u The aggregate vector represents the user's data; the aggregate vector for the same item v is represented by ε. v This is represented as follows, and the inner product is ultimately used to predict user preference ratings for items:

[0166]

[0167] in, This represents the predicted preference rating of user u for item v.

[0168] In step S5 of this embodiment, in order to balance the number of positive and negative samples and ensure the effectiveness of model training, this embodiment extracts the same number of negative samples as the number of positive samples for each user, and then designs a loss function for training. The loss function is expressed as:

[0169]

[0170] in, It is cross-entropy loss. This represents a positive set of user-item pairs. Represents a negative set of user-item pairs. Let E and R be the parameter set, which are embedded tables of all entities and relations. The L2 regularization term is parameterized for λ.

[0171] After training, this embodiment also conducts experimental verification. This embodiment analyzes and evaluates the proposed model in three real-world scenarios.

[0172] Dataset Description: To evaluate the effectiveness of the KGER model, this embodiment selects two publicly available datasets for evaluation in two scenarios: music and books.

[0173] • Last.FM: A large-scale dataset provided by the last.fm online music system, containing information such as listener listening history, music tags, and artist information, often used for music recommendation. The identification of tracks as items is provided by the last.fm online music system.

[0174] The Book-Crossing dataset was collected from the book-crossing community, where users rate books on a scale of 0-10. Since both LastFM and Book-Crossing datasets use explicit feedback, they need to be converted to implicit feedback in the model of this invention, where 1 represents a positive sample and 0 represents a negative sample. If there is interaction between the user and the item, it is considered a positive sample and is set to 1.

[0175] To construct negative samples, this embodiment randomly selects the same number of items as positive samples from the set of items that have not interacted with the user. The completed dataset is shown in Table 1:

[0176] Table 1

[0177] Last.FM Book-Crossing Number of users 1872 17860 Number of projects 3846 14967 Number of interactions 42346 139746 Entity count 9366 77903 Number of relations 60 25 Number of triplet 15518 151500

[0178] Baseline model:

[0179] CKE is a typical embedding-based model that combines a CF module with embedded structural, textual, and visual knowledge of objects within a unified Bayesian framework. In this embodiment, a CF module plus a structured knowledge module is used for implementation.

[0180] PER is a typical path-based approach that treats the knowledge graph as a heterogeneous information network and extracts meta-path-based latent features to represent the connectivity between users and items along different types of relationship paths.

[0181] RippleNet is a typical propagation-based model, similar to ripples on water, spreading outwards from the center. It uses user history interactions as a seed set of user interests to propagate preferences and enrich user representations within the knowledge graph.

[0182] KGCN is a typical propagation-based model that preferentially aggregates domain information of entity nodes in a knowledge graph. It extends non-spectral graph GCN methods to knowledge graphs and enhances item representations by iteratively aggregating domain information in the knowledge graph.

[0183] KGNN-LS is a propagation-based model that transforms heterogeneous knowledge graphs into user-specific weighted graphs and computes personalized item embeddings in graph neural networks. It utilizes label smoothing regularization to identify important user relationships and provides edge weight regularization to reduce induction bias.

[0184] KGAT is a propagation-based model that proposes to combine the user-item mutual graph and the knowledge graph into a single graph. It uses the TransR algorithm and attention mechanism to optimize node embeddings by recursively propagating node embeddings from the embeddings of neighboring nodes.

[0185] CKAN is a propagation-based model, a collaborative knowledge-aware attention network that uses a heterogeneous propagation method to encode two types of information and uses a knowledge-aware attention mechanism to distinguish the contributions of different knowledge neighbors.

[0186] RKGCN leverages knowledge graphs to simultaneously improve the representations of users and items, then uses these enhanced representations to capture users' dynamic interests when encountering different items, thereby making more accurate recommendations. By enriching the representations of users and items using knowledge graphs, RKGCN is able to provide more personalized and relevant recommendations.

[0187] Experimental Setup: In this embodiment, each dataset is divided into a training set, an evaluation set, and a test set in a 6:2:2 ratio. The application scenarios chosen are CTR and top-k recommendation. This embodiment uses the model learned from the training set to predict the probability of each interaction in the test set. For CTR prediction, AUC and F1 are used to evaluate the model's performance; for top-k recommendation, Recall@K is used for performance evaluation. All baseline models are tested using the best data or the best parameters they can provide, meaning all baseline models are optimized.

[0188] Experimental Results: This embodiment presents the comparison results between KGER and all baseline models. The results of all methods in CTR prediction and top-K recommendation are shown in Table 2 and Table 3, respectively. Figure 2 As shown.

[0189] Table 2

[0190]

[0191]

[0192] Intuitively, KGER achieved good results in both recommendation scenarios, and we can observe that:

[0193] (1) In both scenarios across the two datasets, KGER performed best among all methods. Specifically, as shown in Table 2, KGER improved the AUC and F1 scores by 1.19% and 0.65% respectively compared to the best baseline on the Last.FM dataset; and by 2.68% and 1.95% respectively on the Book-Crossing dataset. Figure 2 As shown, KGER has achieved a more significant improvement in Recall@K.

[0194] (2) All models performed worse on the Book-Crossing dataset than on the Last.FM dataset. The main reason is that the data in the Book-Crossing dataset is relatively sparse, and for KGER, sparse data means fewer types of relationships and a lack of sufficient data to mine more accurate potential relationships.

[0195] (3) In the top-k recommendation scenario, KGER performs better when k increases. KGER’s superior performance verifies the effectiveness of exploring potential relationships in improving recommendation performance.

[0196] (4) The experimental results show that in the two recommendation scenarios and the two datasets, the embedding-based method and the path-based method are significantly worse than the propagation-based method. This verifies the effectiveness of aggregating high-order information based on associations in knowledge graphs to improve recommendation performance.

[0197] (5) Path-based methods (PER) perform worse than other methods because it is actually difficult to define the optimal meta-path. Embedding-based methods (CKE) also perform poorly, possibly due to a lack of visual and textual data.

[0198] (6) Experimental results show that propagation-based methods have good performance in recommendation, but other methods ignore the importance of potential relationships and also ignore the fact that users and items should have different attention weights for different relationships when aggregating high-order neighborhood features. The experimental results of this embodiment verify the advancement and effectiveness of the KGER model.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A recommendation method based on knowledge graphs and attention mechanisms, characterized in that: The recommendation method includes the following steps: S1. Define the knowledge graph perception recommendation problem, and establish a user-item interaction matrix and a knowledge graph as input. S2. Set up seed sets for users and projects in the knowledge graph propagation layer, and propagate the seed sets along adjacent entities to capture knowledge-based high-order interaction information of users and projects; In step S2, the following steps are performed in the knowledge graph propagation layer: S21. Define User User seed set and projects Project seed set: in, Represents the user seed set, Represents the project seed set; S22. Propagate the seed set along links in the knowledge graph to obtain an expanded entity set and triples at different distances from the initial entity set, where the user... and projects The entity set is represented as: in, The subscript symbol represents the distance from the initial set of entities. It is a symbol or Uniform placeholders; Then the user and projects The The set of triplets is represented as: in, Indicates the number of hops in the propagation; S3. In the knowledge-aware attention embedding layer, the attention mechanism is used to learn the corresponding latent relation representation for each relation, thereby obtaining a set of potentially related entities; In step S3, the following steps are performed in the knowledge-aware attention embedding layer: S31. Based on the attention mechanism, for queries s and keys Using relationships With tail entity node Information aggregation, and value If it is the relation itself, then: S32. Establishing potential relationships between entities: Attention embedding : in, , For generating networks with potential associative attention: in, This is the normalization function; S33. Utilize optimized triples to learn user or item association vectors. Based on the degree of preference, find the most valuable information in the entity set of each hop, and the nodes in each hop. Represented as: in, This is the association vector, where o represents the user. or project Placeholder; For a convergent network, it is represented as: in, Embedding of the tail entity in the triple; S34. Obtain User or project The Representation of layer triples: Among them, subscript and superscript It is a symbol or Consistent placeholders, It is a set The number of triplet pairs; S35, Create User With the project The set of potentially related entities: in, and These represent the seed sets added for users and projects, respectively. S4. Process the entity set through the aggregator to obtain the corresponding user aggregation vector and item aggregation vector, and predict the user's preference rating for the item based on the aggregation vector; In step S4, the entity representations in each hop set during the diffusion of the user item representation entity set are aggregated by the aggregator, that is, the user... With the project Multiple representations in the entity set are aggregated into a single vector representation, and then the inner product is used to predict user preference ratings for items: in, Indicates the user who made the prediction For the project Preference rating, The aggregate vector representing the user, Represents the aggregate vector of the project; S5. Design a loss function to train steps S1-S4. After training, the loss function is used for user preference recommendation.

2. The recommendation method based on knowledge graphs and attention mechanisms according to claim 1, characterized in that: In step S1, it is set that the recommended scenario contains A user set of users and Project collection of projects Then, define the user-project interaction matrix based on the user's implicit feedback. It is represented as: in, Indicates user With the project There is historical interaction, otherwise Indicates user With the project There is no historical interaction; Building a knowledge graph As input: in, Represents a large number of entity triples. , , Let these represent the head, relation, and tail of a knowledge triple, respectively. and These represent the sets of entities and relations in the knowledge graph, respectively. Set collection This indicates the alignment relationship between projects and entities, where, Indicates project Able to interact with entities in a knowledge graph Alignment; For a given user-item interaction matrix and knowledge graph Establish the prediction function: in, Indicates the predicted probability. Representation function Model parameters.

3. The recommendation method based on knowledge graphs and attention mechanisms according to claim 1, characterized in that: The aggregator includes at least a summation aggregator, a concat aggregator, and a pooling aggregator. The summation aggregator sums the representation vectors of the multi-hop set before performing the nonlinear transformation; the concat aggregator concatenates the representation vectors in the representation set and performs the nonlinear transformation; the pooling aggregator takes the maximum value from multiple vectors and then performs the nonlinear transformation. The operation process of the summation aggregator is represented as follows: in, The sigmoid function is a non-linear function. and For trainable weights and biases; The operation process of the Concat aggregator is represented as follows: in , It's a splicing operation; The operation process of the pooling aggregator is represented as follows: in, It is a pooling aggregator that can select the maximum value in the result set.

4. The recommendation method based on knowledge graphs and attention mechanisms according to claim 1, characterized in that: In step S5, a loss function is set to train steps S1-S4, where the loss function is expressed as: in, It is cross-entropy loss. This represents a positive set of user-item pairs. This represents a negative set of user-item pairs; For parameter set; and An embedded table for all entities and relationships. for Parameterized L2 regularization terms.

Images