Knowledge graph comparison learning recommendation method based on attention mechanism

Abstract

The invention belongs to the technical field of recommendation systems, and provides a knowledge graph comparative learning recommendation method based on an attention mechanism in order to solve the limitation of uniform attention of a knowledge graph on triple application and avoid semantic noise at the same time, which comprises the following steps: constructing original data into an attention-guided bipartite graph structure; calculating message transmission from the user to the article and from the article to the user to obtain an article bipartite graph embedding result; constructing a knowledge graph through a dynamic correlation evaluation mechanism; fusing the attention-enhanced bipartite graph and the knowledge graph; performing comparative learning of semantic perception, replacing a naive enhancement strategy for equally treating all interactions with a learnable enhancement strategy aligned with a semantic correlation score, and generating a more robust and meaningful user-article representation by a comparative learning framework while avoiding semantic noise, better capturing a real preference mode and inhibiting false interactions; and finally, joint loss optimization is carried out, and the overall recommendation precision is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of recommendation systems. BACKGROUND

[0002] As an information filtering tool, recommendation systems have become an indispensable component in digital platforms, helping users navigate through the vast amount of available content and services.

[0003] Existing recommendation systems based on graph neural networks typically encode users and items as single dense vectors when processing user-item interactions, as shown in Figure 1 This approach entangles semantically orthogonal preference dimensions into a single vector. For example, a user's interactions with restaurants can stem from different underlying factors: dietary restrictions, situational context, aesthetic appeal, or location convenience, which represent semantically orthogonal dimensions that should be modeled separately.

[0004] Integrating knowledge graphs as rich information networks for items has gained attention. Knowledge graphs are directed heterogeneous graphs that connect entities and relationships, providing rich factual connections for recommended items. However, current knowledge graph integration applies uniform attention to all triples, failing to distinguish between information crucial to specific recommendation contexts and trivial attributes. As shown in the middle part, Figure 1 When recommending restaurants to users with dietary restrictions, nutritional facts such as calorie content and diabetes suitability are highly relevant, while attributes such as restaurant establishment year or building color provide little value to dietary recommendations. Uniform attention allocation will lead to incorrect restaurant assignments.

[0005] Recent research has introduced contrastive learning knowledge graphs, such as KGCL (Knowledge Graph Contrastive Learning), which captures and strengthens the association between an entity and its domain in the knowledge graph through contrastive learning. The model is trained to make the entity and its correct neighborhood representation closer in the vector space, while making the representation of other unrelated neighborhoods more distant. However, KGCL uses random deletion of some interactions to enhance strong associations, often retaining false interactions while destroying true preference signals, leading to semantic noise. SUMMARY

[0006] To address the limitations of uniform attention applied to triples in knowledge graphs and avoid semantic noise, the present application proposes a knowledge graph contrastive learning recommendation method based on attention mechanisms, including the following steps: Step one, in the original data, in the form of user-item-rating, the original data is constructed into an attention-guided bipartite graph structure, where users and items are different types of nodes, and interactions are connected as edges between corresponding users and items; Step two, for each rating grade r, the user-to-item and item-to-user message passing is calculated using the rating-specific weight matrix, and finally the item embedding and attention-enhanced item embedding are fused to obtain the item bipartite graph embedding result, and the user preference is decomposed into different semantic dimensions through attention-guided feature separation; Step three, build a knowledge graph through a dynamic correlation evaluation mechanism to obtain an item embedding result that fuses user knowledge graph preferences; Step four, fuse the attention-enhanced bipartite graph and the knowledge graph: deeply and intelligently fuse the item bipartite graph embedding result obtained in step two and the knowledge graph propagation result obtained in step three to build a unified multi-modal representation learning framework, and obtain the final fused item embedding result; Step five, semantic-aware contrastive learning: combine the knowledge-enhanced user embedding U obtained in step three and the final fused item embedding result into a unified embedding matrix E, and balance feature enhancement through a prototype filtering network to obtain the final embedding matrix result after feature enhancement; Step six, joint loss optimization, using InfoNCE loss as the optimization target of contrastive learning, maximizing the consistency between positive sample pairs and minimizing the consistency between negative sample pairs, , wherein, BPR loss, contrastive learning loss, knowledge graph embedding loss, is the set of all parameters, 、 、 are the weight coefficients of the contrastive loss, the knowledge graph embedding loss and the L2 regularization, respectively; multiple iterations are trained, and finally the prediction score is calculated through the dot product of the fused user and item embeddings.

[0007] Technical effects: The present application separates the semantic orthogonal user preference dimensions through the introduction of a multi-modal representation learning framework and an attention-guided decomposition mechanism. Unlike existing recommendation systems based on graph neural networks that encode users and items into a single embedding, the present application uses an attention-guided decomposition mechanism in step two to identify and model different preference factors as independent subspaces. This separation prevents orthogonal dimensions from being entangled in a single dense vector, thereby enhancing recommendation specificity and enabling more interpretable modeling of complex user behavior.

[0008] To overcome the limitation of applying uniform attention to all knowledge graph triples, the dynamic relevance assessment mechanism in step three is introduced to adaptively weight knowledge triples according to the semantic importance of the specific recommendation context. The framework adopts multi-hop ripple propagation and learnable attention mechanism to distinguish key attributes and trivial attributes. This selective attention ensures that the most informative knowledge graph connections drive personalized recommendations while filtering context-irrelevant information.

[0009] Unlike existing methods that treat all interactions equally with random augmentation strategies, the semantic-aware contrastive learning method in step five of the present application replaces the naive augmentation strategy that treats all interactions equally with a learnable augmentation strategy aligned with the semantic relevance score. By aligning the augmentation weight with the semantic relevance score, the contrastive learning framework generates more robust and meaningful user-item representations while avoiding semantic noise, better capturing real preference patterns while suppressing false interactions.

[0010] To verify the recommendation effect of the present application, Recall (Recall) and NDCG (Normalized Discounted Cumulative Gain) are used as important indicators for evaluating the model, and three data sets Amazon-book, Yelp2018 and Last-FM are selected as original data, and compared with existing models. The comparison accuracy is shown in the following table.

[0011]

[0012] As can be seen, the method proposed in the present application has the highest numerical value in precision verification, and has practical value. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 The schematic diagram of the limitations of the existing knowledge-aware recommendation method in the background of the present application is shown.

[0014] Figure 2 The overall flowchart of the present application is shown.

[0015] Figure 3 The overall architecture module schematic diagram of the embodiment method of the present application is shown. DETAILED DESCRIPTION

[0016] In order to better understand the technical solutions of the present application, the embodiments provided by the present application will be described in detail below in conjunction with the drawings, but the embodiments of the present application are not limited thereto.

[0017] Step one, process the dataset, use the Amazon-book dataset, which contains 70679 users and 24915 items, and 652514 interaction records. In the original data, it is stored in the form of user-item-rating. The original data in the dataset is constructed into an attention-guided bipartite graph structure, where users and items are different types of nodes, and interactions are edges connecting corresponding users and items, as shown in the left "interaction" part. Figure 2 The left "interaction" part.

[0018] Step two, as shown in the left part, for each rating level r, use the rating-specific weight matrix to calculate the message passing of users to items and items to users, and the specific formula is as follows: Figure 2 , , wherein is the rating-specific weight matrix, d is the embedding dimension, is the degree normalization factor, is the neighbor node set of user u, is the neighbor node set of item i; is the original embedding vector of user u, is the original embedding vector of item i.

[0019] In the message aggregation stage, the incoming messages of each node are aggregated through the nonlinear transformation of the ReLU activation function. The attention mechanism transforms the item embedding through the attention weight matrix to calculate the attention score : , wherein is the attention weight matrix, and then the attention scores are normalized using the softmax function to obtain the attention weight , as shown in the "Softmax" module in Figure 2 : , wherein I is the set of all items. The attention weight is applied through element-level multiplication to generate the attention-enhanced item embedding representation , and finally the item embedding is obtained. The item bipartite graph embedding result fused with the attention-enhanced item embedding , wherein is the weight matrix for processing the attention-enhanced item embedding. This step decomposes user preferences into different semantic dimensions through attention-guided feature separation, preventing orthogonal dimensions from being tangled into a single dense vector.​

[0020] Step three, as Figure 2 shown in the middle part, build the knowledge graph through dynamic correlation evaluation mechanism: Embedding the item As the initial representation of item entities in the knowledge graph, user embedding Build a seed set according to user historical interaction, and expand related entities in the knowledge graph through multi-hop propagation. Define the k-hop related entity set of user as: , Where t represents the tail entity, h represents the head entity, r represents the relationship, G represents the knowledge graph, The k-1 hop related entity set of user u is represented, H represents the maximum propagation hop number, The user historical click item set is used as the seed set; For each knowledge triple in the k-hop ripple set , calculate the correlation probability between the candidate item and the head entity, as shown in the weighted average module in Figure 2 : , Where, v Represents the embedding vector of the candidate item, Represents the embedding matrix of the relationship in the i th triple, Represents the embedding vector of the head entity in the i th triple, Represents the k-hop ripple set of user u, that is, the set of all knowledge triples starting from ; Aggregate the tail entity embedding by weighting the correlation probability to get the k-hop response vector : Sum all triples in the k-hop ripple set of user u , Where, Represents the embedding vector of the tail entity in the i th triple, combine all hop number response vectors to form knowledge-enhanced user embedding U, capture user interest at different propagation levels. Build item embedding results that integrate user knowledge graph preferences through item embedding and knowledge-enhanced user embedding U, as shown in the lower part of Figure 2 : , Where, Represents the item embedding, and U represents the knowledge-enhanced user embedding.

[0021] This dynamic correlation evaluation mechanism adaptively weights knowledge triples according to the semantic importance of specific recommendation contexts, distinguishes between key attributes and trivial attributes, and ensures that the most informative knowledge graph connections drive personalized recommendations.

[0022] Step four, the bipartite graph embedding result obtained in step two is deeply intelligently fused with the knowledge graph propagation result obtained in step three to construct a unified multi-modal representation learning framework. For example, Figure 2 middle part As shown in the fusion module, first, the two kinds of embeddings are connected in the feature dimension through splicing operation, and then the spliced features are linearly transformed using a learnable fusion weight matrix to obtain the final fused item embedding result: , wherein, is the fusion weight matrix, represents the item embedding of the user's knowledge graph preference, is the item bipartite graph embedding result obtained in step two; Step five, as shown in the right part of the process, Figure 2 the knowledge-enhanced user embedding U obtained in step three and the final fused item embedding of step four are combined into a unified embedding matrix E. Specifically, the K-means clustering algorithm is run in each training cycle to extract preference prototypes from the initialized embedding space , , wherein, denotes the initialized user and item embedding matrix; The dot product similarity between the unified embedding matrix E and the prototype is calculated as the preference score, and the sigmoid function is used for normalization, as shown in Figure 2 the "Sigmoid" and "Mask" modules in the middle: , wherein, , M represents the number of users, N represents the number of items, denotes the unified embedding matrix; The multi-modal representation learning framework enhances the user and item embeddings through a prototype filtering network. This prototype filtering network includes two key components: a linear transformation function (·) to enhance the model fitting capability, and a self-gating module (·) to control the degree of feature enhancement. For example, Figure 2As shown in the "Linear Transformation" and "Self-Gated Module" sections, the prototype filtering network balances feature enhancement to obtain the final embedding matrix result after feature enhancement: , in, Linear transformation functions are used to enhance model fitting ability. This indicates that the self-gating module is used to control the degree of feature enhancement.

[0023] Step 6: Joint Loss Optimization; Using InfoNCE loss as the optimization objective for contrastive learning, such as Figure 2 As shown in the "Comparison" module at the bottom right, this maximizes the consistency between positive sample pairs (Augmented Embeddings) and minimizes the consistency between negative sample pairs. Total Loss Combining BPR loss, contrastive loss, knowledge graph embedding loss, and L2 regularization: , in, For BPR loss, To compare learning loss, For knowledge graph embedding loss, For the set of all parameters, , , The weights are defined as contrastive loss, knowledge graph embedding loss, and L2 regularization weights, respectively. After multiple iterations of training, the predicted score is calculated by fusing the dot product of user and item embeddings. A semantically aware enhancement strategy is employed, guiding the weight enhancement based on the consistency of the knowledge graph structure, intelligently preserving genuine preference signals while suppressing false interactions.

[0024] Furthermore, a temperature parameter τ is introduced into the InfoNCE loss function to control the contrast intensity. The role of the temperature parameter τ is to adjust the smoothness of the similarity distribution: when τ is small, the model is more sensitive to differences between samples, and the contrastive learning has stronger discriminative power; when τ is large, the similarity distribution is smoother, and the model training is more stable. By adjusting the temperature parameter, a balance can be achieved between contrast intensity and training stability.

[0025] In the above formulas, the superscript T of some parameters represents matrix transpose. Furthermore, all contents not described in detail in this specification are existing technologies known to those skilled in the art. Moreover, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. An attention mechanism-based knowledge graph contrastive learning recommendation method, characterized in that, Comprising the following steps: Step one, in the original data, in the form of user-item-rating, the original data is constructed into an attention-guided bipartite graph structure, in which users and items are different types of nodes, and interactions are edges connecting corresponding users and items; Step two, for each rating level r, use the rating-specific weight matrix to calculate the user-to-item and item-to-user message passing, and finally get the item embedding and attention-enhanced item embedding fusion item bipartite graph embedding result, and decompose the user preference into different semantic dimensions through attention-guided feature separation; Step three, build a knowledge graph through a dynamic correlation evaluation mechanism to get an item embedding result that integrates user knowledge graph preferences; Step four, fuse the attention-enhanced bipartite graph and the knowledge graph: deeply and intelligently fuse the item bipartite graph embedding result obtained in step two and the knowledge graph propagation result obtained in step three to build a unified multi-modal representation learning framework, and get the final fused item embedding result; Step five, semantic-aware contrastive learning: combine the knowledge-enhanced user embedding and the final fused item embedding result into a unified embedding matrix E, and balance feature enhancement through prototype filtering network to get the final embedding matrix result after feature enhancement; Step six, joint loss optimization: use the InfoNCE loss function as the optimization target of contrastive learning, maximize the consistency between positive sample pairs, and minimize the consistency between negative sample pairs, ， wherein, is the BPR loss, is the contrastive learning loss, is the knowledge graph embedding loss, is the set of all parameters, , , are the weight coefficients of the contrastive loss, the knowledge graph embedding loss, and the L2 regularization, respectively, and the prediction score is calculated by fusing the dot product of the user and item embeddings after multiple iterations of training.

2. The attention mechanism-based knowledge graph contrastive learning recommendation method according to claim 1, characterized in that, The specific formula of step two is as follows: ， ， wherein, is a score-specific weight matrix, d is the embedding dimension, is a degree normalization factor, is the set of neighbor nodes of user u, is the set of neighbor nodes of item i; is the original embedding vector of user u, is the original embedding vector of item i; in the message aggregation phase, the incoming messages of each node are aggregated by a nonlinear transformation through a ReLU activation function, and the attention mechanism is through an attention weight matrix transforms the item embedding to calculate the attention score : ， wherein, is the attention weight matrix, the attention scores are normalized using a softmax function to obtain the attention weights , ， wherein I is a set of all items, the attention enhanced item embedding representation is generated by applying attention weights through element-wise multiplication , resulting in the final item embedding item bipartite graph embedding result fused with the attention enhanced item embedding , ， wherein is the processed attention-enhanced item embedding weight matrix.

3. The attention mechanism based knowledge graph contrastive learning recommendation method according to claim 1, characterized in that, In step three, the item embedding is used as the initial representation of the item entity in the knowledge graph, the user embedding is constructed according to the user historical interaction to form a seed set, and the related entities in the knowledge graph are expanded through multi-hop propagation, and the k-hop related entity set of the user is defined as: ， wherein, t denotes a tail entity, h denotes a head entity, r denotes a relation, G denotes a knowledge graph, denotes a k-1 hop related entity set of a user u, H denotes a maximum propagation hop number, is a user historical click item set, as a seed set; For each knowledge triple in the k-hop set of ripples calculate a probability of relevance between the candidate item and the head entity, ， wherein, v denotes an embedding vector of a candidate item, denotes an embedding matrix of the relation in the i th triple, denotes an embedding vector of the head entity in the i th triple, denotes the k-hop ripple set of user u, i.e. the set of all knowledge triples starting from the k-hop response vector is obtained by aggregating the tail entity embeddings weighted by the relevance probabilities : sum over all triples in the k-hop set of user u ,​ wherein, denotes the embedding vector of the tail entity in the i th triple, and all the response vectors of the number of hops are combined to form a knowledge-enhanced user embedding, which captures the user's interest at different propagation levels; an item embedding result is constructed by fusing the item embedding and the knowledge-enhanced user embedding, ， wherein, represents an item embedding, and U represents a knowledge-augmented user embedding.

4. The attention mechanism based knowledge graph contrastive learning recommendation method according to claim 1, characterized in that, The step five is specifically: running K-means clustering algorithm in each training cycle to extract preference prototypes from the initialized embedding space , ， wherein, denotes the initialized user and item embedding matrices; Calculate the dot product similarity between the unified embedding matrix E and the prototype as the preference score, and use the sigmoid function to normalize, ， wherein, M represents the number of users, and N represents the number of items, represents a unified embedding matrix; The multimodal representation learning framework augments user and item embeddings with features through a prototype filtering network, which comprises two key components: a linear transformation function to enhance model fitting capabilities. (·), and a self-gated module for controlling the degree of feature enhancement. (·); By balancing feature enhancement through a prototype filtering network, the final embedding matrix result after feature enhancement is obtained: ， wherein, is a linear transformation function for enhancing model fitting capability, denotes a self-gating module for controlling the degree of feature enhancement.

5. The attention mechanism based knowledge graph contrastive learning recommendation method according to claim 1, characterized in that, In step six, the temperature parameter τ is introduced into the InfoNCE loss function to control the contrast intensity, when τ is small, the model is more sensitive to the difference between samples, and the contrast learning has stronger discrimination; when τ is large, the similarity distribution is smoother, and the model training is more stable, by adjusting the temperature parameter τ, a balance between contrast intensity and training stability is achieved.

Images