Knowledge graph contrastive learning recommendation method based on attention mechanism

Abstract

The application belongs to the technical field of recommendation systems, in order to solve the uniform attention restriction of the knowledge graph on triplets, and avoid semantic noise, a knowledge graph contrast learning recommendation method based on an attention mechanism is proposed, the original data is constructed into an attention-guided bipartite graph structure; the message passing of users to items and items to users is calculated to obtain the item bipartite graph embedding result; the knowledge graph is constructed through a dynamic correlation evaluation mechanism; the attention-enhanced bipartite graph and the knowledge graph are fused; the contrast learning of semantic perception is carried out, the learnable enhancement strategy aligned with the semantic correlation score is used to replace the naive enhancement strategy that treats all interactions equally, the contrast learning framework generates more robust and meaningful user-item representations, while avoiding semantic noise, better capturing real preference patterns while suppressing false interactions; finally, the overall recommendation accuracy is improved through joint loss optimization.

Description

Technical Field

[0001] This invention belongs to the field of recommendation system technology. Background Technology

[0002] Recommendation systems, as information filtering tools, have become an indispensable component of digital platforms, helping users navigate through a vast amount of available content and services.

[0003] Existing graph neural network-based recommendation systems typically encode users and items as a single dense vector when processing user-item interactions, such as... Figure 1 As shown, this approach entangles semantically orthogonal preference dimensions into a single vector. For example, a user's interaction with a restaurant might stem from different underlying factors: dietary restrictions, contextual information, aesthetic appeal, or location convenience. These factors represent semantically orthogonal dimensions and should be modeled separately.

[0004] Integrating knowledge graphs as rich information networks for items has gained attention. Knowledge graphs are directed heterogeneous graphs connecting entities and relationships, providing rich factual connections for item recommendations. However, current knowledge graph integrations apply uniform attention to all triples, failing to distinguish between information crucial to a specific recommendation context and trivial attributes. For example... Figure 1 As shown in the middle section, when recommending restaurants to users with dietary restrictions, nutritional facts such as calorie content and diabetic suitability are highly correlated, while attributes such as the year the restaurant was established or the color of the building provide little value for dietary recommendations. Even attention distribution will lead to incorrect restaurant allocation.

[0005] Recent research has introduced contrastive learning into knowledge graphs, such as KGCL (Knowledge Graph Contrastive Learning). This approach captures and strengthens the association between an entity and its domain within a knowledge graph, training the model to make the entity closer to its correct neighborhood representation in the vector space, while distancing it from other unrelated neighborhood representations. However, KGCL uses random deletion of interactions to enhance strong associations, often retaining spurious interactions while disrupting genuine preference signals, leading to semantic noise. Summary of the Invention

[0006] To address the limitations of applying uniform attention to triples in knowledge graphs and to avoid semantic noise, this invention proposes a contrastive learning recommendation method for knowledge graphs based on an attention mechanism, comprising the following steps: Step 1: Store the original data in the form of user-item-rating, and construct the original data into an attention-guided bipartite graph structure, where users and items are different types of nodes, and interactions are edges connecting the corresponding users and items; Step 2: For each rating level r, use the rating-specific weight matrix to calculate the message passing from user to item and from item to user. Finally, obtain the item bipartite graph embedding result that is a fusion of item embedding and attention-enhanced item embedding. Decompose user preferences into different semantic dimensions through attention-guided feature separation. Step 3: Construct a knowledge graph through a dynamic relevance evaluation mechanism to obtain item embedding results that integrate user knowledge graph preferences; Step 4: Integrate the attention-enhanced bipartite graph and knowledge graph: Deeply and intelligently integrate the item bipartite graph embedding results obtained in Step 2 with the knowledge graph propagation results obtained in Step 3 to construct a unified multimodal representation learning framework and obtain the final fused item embedding results; Step 5: Perform semantic awareness contrastive learning: Combine the knowledge-enhanced user embedding U obtained in Step 3 with the final fused item embedding result into a unified embedding matrix E. Balance the feature enhancement through a prototype filtering network to obtain the final embedding matrix result after feature enhancement. Step 6: Joint Loss Optimization. InfoNCE loss is used as the optimization objective for contrastive learning, maximizing the consistency between positive sample pairs and minimizing the consistency between negative sample pairs. , in, For BPR loss, To compare learning loss, For knowledge graph embedding loss, For the set of all parameters, , , These are the contrastive loss, knowledge graph embedding loss, and L2 regularization weight coefficients, respectively. After multiple iterations of training, the prediction score is finally calculated by fusing the dot product of the embeddings of users and items.

[0007] Technical effects: This invention introduces a multimodal representation learning framework and an attention-guided decomposition mechanism to reveal the separation of semantically orthogonal user preference dimensions. Unlike existing graph neural network-based recommendation systems that encode users and items as a single embedding, this invention employs an attention-guided decomposition mechanism in step two to identify and model different preference factors as independent subspaces. This separation prevents orthogonal dimensions from becoming entangled in a single dense vector, thereby enhancing recommendation specificity and enabling more interpretable modeling of complex user behaviors.

[0008] To overcome the limitation of applying uniform attention to all knowledge graph triples, a dynamic relevance evaluation mechanism is introduced in step three, which adaptively weights knowledge triples based on the semantic importance of the specific recommendation context. The framework employs multi-hop ripple propagation and a learnable attention mechanism to distinguish between key and trivial attributes. This selective attention ensures that the most informative knowledge graph connections drive personalized recommendations while filtering out context-irrelevant information.

[0009] Unlike existing methods that treat all interactions equally using random augmentation strategies, step five of this invention employs a semantically aware contrastive learning method. It replaces the naive augmentation strategy that treats all interactions equally with a learnable augmentation strategy aligned with semantic relevance scores. By aligning augmentation weights with semantic relevance scores, the contrastive learning framework generates more robust and meaningful user-item representations while avoiding semantic noise, better capturing genuine preference patterns, and suppressing spurious interactions.

[0010] To verify the recommendation effect of this invention, Recall and NDCG (Normalized Discount Cumulative Gain) were used as important indicators for evaluating the model. Three datasets, Amazon-book, Yelp2018, and Last-FM, were selected as the original data and compared with existing models. The comparison accuracy is shown in the table below.

[0011] Therefore, it can be seen that the method proposed in this invention has the highest numerical value in accuracy verification and has practical value. Attached Figure Description

[0012] Figure 1 This is a schematic diagram illustrating the limitations of existing knowledge perception and recommendation methods in the context of this invention.

[0013] Figure 2 This is a flowchart illustrating the overall process of this invention.

[0014] Figure 3 This is a schematic diagram of the overall architecture modules of the method in an embodiment of the present invention. Detailed Implementation

[0015] To better understand the technical solution of this invention, the following is in conjunction with... Figures 2-3 The embodiments provided by the present invention are described in detail, but the implementation of the present invention is not limited thereto.

[0016] Step 1: Process the dataset using the Amazon-book dataset, which contains 70,679 users, 24,915 items, and 652,514 interaction records. The raw data is stored in a user-item-rating format. The raw data is then constructed into an attention-guided bipartite graph structure, where users and items are different types of nodes, and interactions are edges connecting the corresponding users and items, such as... Figure 3 As shown in the "Interactive" section on the left.

[0017] Step 2, as follows Figure 3 As shown on the left, for each rating level r, the message passing from user to item and from item to user is calculated using a rating-specific weight matrix, with the following formula: , , in, It is a rating-specific weight matrix, where d is the embedding dimension. Let be the degree normalization factor, and be the set of neighbor nodes of user u. Let i be the set of neighboring nodes of item i; The original embedding vector of user u, Let be the original embedding vector of item i.

[0018] During the message aggregation phase, the incoming messages from each node are aggregated through a non-linear transformation of the ReLU activation function. The attention mechanism utilizes an attention weight matrix. Transform item embeddings to calculate attention scores : , in, The attention scores are then normalized using the softmax function to obtain the attention weights. ,like Figure 3 As shown in the "Softmax" module: , Where I is the set of all items. Attention-enhanced item embeddings are generated by applying attention weights through element-wise multiplication. Finally, the item is embedded. Item bipartite graph embedding results fused with attention-enhanced item embedding , , in This involves processing the item embedding weight matrix after attention enhancement. This step decomposes user preferences into different semantic dimensions through attention-guided feature separation, preventing orthogonal dimensions from becoming entangled in a single dense vector.

[0019] Step 3, as follows Figure 3 As shown in the middle section, a knowledge graph is constructed through a dynamic relevance evaluation mechanism: Embed the item As the initial representation of item entities in a knowledge graph, user embedding A seed set is constructed based on the user's historical interactions, and related entities are expanded in the knowledge graph through multi-hop propagation. The set of related entities for a user's k hops is defined as follows: , Where t represents the tail entity, h represents the head entity, r represents the relation, and G represents the knowledge graph. Let H represent the set of entities related to user u for k-1 hops, and let H represent the maximum propagation hop count. Use the set of items clicked in the user's history as a seed set; For the k-jump ripple set For each knowledge triple in the dataset, calculate the correlation probability between the candidate item and the head entity, such as... Figure 3 The medium-weighted average module is shown below: , in, v Represents the embedding vector of the candidate item. Indicates the first i Embedding matrix of relations in triplets Indicates the first i The embedding vector of the head entity in each triplet. Describes the set of k-hop ripples for user u, i.e., from The set of all knowledge triples that originate from; The k-hop response vector is obtained by weighting the aggregate tail entity embeddings based on correlation probability. : Summing and traversing the set of k-jump ripples for user u All triples , in, Indicates the first i The embedding vectors of the tail entities in each triplet are combined with the response vectors of all hop counts to form a knowledge-enhanced user embedding U, capturing user interests at different levels of propagation. This is achieved through item embedding. And knowledge-enhanced user embeddings U construct item embedding results that integrate user knowledge graph preferences, such as Figure 3 The middle section is shown below: , in, 'U' represents item embedding, and 'U' represents knowledge-enhanced user embedding.

[0020] This dynamic relevance evaluation mechanism adaptively weights knowledge triples based on the semantic importance of specific recommendation contexts, distinguishes between key attributes and trivial attributes, and ensures that the most informative knowledge graph connections drive personalized recommendations.

[0021] Step 4: Deeply and intelligently fuse the bipartite graph embedding results obtained in Step 2 with the knowledge graph propagation results obtained in Step 3 to construct a unified multimodal representation learning framework. For example... Figure 3 Middle part As shown in the fusion module, the two embeddings are first concatenated along the feature dimension using a concatenation operation. Then, a learnable fusion weight matrix is ​​used to perform a linear transformation on the concatenated features to obtain the final fused item embedding result. , in, To fuse the weight matrix, This indicates the embedding of items that incorporate user knowledge graph preferences. The embedding result of the item bipartite graph obtained in step two; Step 5, as follows Figure 3 As shown in the flowchart on the right, the knowledge obtained in step three is used to enhance the user embedding U, and the final fusion item embedding in step four is used. The components are combined into a unified embedding matrix E. Specifically, in each training epoch, the K-means clustering algorithm is run to extract preference prototypes from the initialized embedding space. , , in, This represents the initial user and item embedding matrix; The dot product similarity between the unified embedding matrix E and the prototype is calculated as the preference score, and normalized using the sigmoid function, such as... Figure 3 The "Sigmoid" and "Mask" modules are shown below: , in, M represents the number of users, and N represents the number of items. Represents a unified embedding matrix; The multimodal representation learning framework augments user and item embeddings with features through a prototype filtering network. This prototype filtering network comprises two key components: a linear transformation function... (·) is used to enhance model fitting ability and self-gated modules. (·) is used to control the degree of feature enhancement. For example... Figure 3As shown in the "Linear Transformation" and "Self-Gated Module" sections, the prototype filtering network balances feature enhancement to obtain the final embedding matrix result after feature enhancement: , in, Linear transformation functions are used to enhance model fitting ability. This indicates that the self-gating module is used to control the degree of feature enhancement.

[0022] Step 6: Joint Loss Optimization; Using InfoNCE loss as the optimization objective for contrastive learning, such as Figure 3 As shown in the "Comparison" module at the bottom right, this maximizes the consistency between positive sample pairs (Augmented Embeddings) and minimizes the consistency between negative sample pairs. Total Loss Combining BPR loss, contrastive loss, knowledge graph embedding loss, and L2 regularization: , in, For BPR loss, To compare learning loss, For knowledge graph embedding loss, For the set of all parameters, , , The weights are defined as contrastive loss, knowledge graph embedding loss, and L2 regularization weights, respectively. After multiple iterations of training, the predicted score is calculated by fusing the dot product of user and item embeddings. A semantically aware enhancement strategy is employed, guiding the weight enhancement based on the consistency of the knowledge graph structure, intelligently preserving genuine preference signals while suppressing false interactions.

[0023] Furthermore, a temperature parameter τ is introduced into the InfoNCE loss function to control the contrast intensity. The role of the temperature parameter τ is to adjust the smoothness of the similarity distribution: when τ is small, the model is more sensitive to differences between samples, and the contrastive learning has stronger discriminative power; when τ is large, the similarity distribution is smoother, and the model training is more stable. By adjusting the temperature parameter, a balance can be achieved between contrast intensity and training stability.

[0024] In the above formulas, the superscript T of some parameters represents matrix transpose. Furthermore, all contents not described in detail in this specification are existing technologies known to those skilled in the art. Moreover, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A knowledge graph comparative learning recommendation method based on attention mechanism, characterized in that, Includes the following steps: Step 1: Store the original data in the form of user-item-rating, and construct the original data into an attention-guided bipartite graph structure, where users and items are different types of nodes, and interactions are edges connecting the corresponding users and items; Step 2: For each rating level r, use the rating-specific weight matrix to calculate the message passing from user to item and from item to user. Finally, obtain the item bipartite graph embedding result that is a fusion of item embedding and attention-enhanced item embedding. User preferences are decomposed into different semantic dimensions through attention-guided feature separation. Step 3: Construct a knowledge graph through a dynamic relevance evaluation mechanism to obtain item embedding results that integrate user knowledge graph preferences; Step 4: Integrate the attention-enhanced bipartite graph and knowledge graph: Deeply and intelligently integrate the item bipartite graph embedding results obtained in Step 2 with the knowledge graph propagation results obtained in Step 3 to construct a unified multimodal representation learning framework and obtain the final fused item embedding results; Step 5: Perform semantic awareness contrastive learning: Combine the knowledge-enhanced user embedding and the final fused item embedding results into a unified embedding matrix E, and balance the feature enhancement through a prototype filtering network to obtain the final embedding matrix result after feature enhancement. Step Six: Joint Loss Optimization: Use the InfoNCE loss function as the optimization objective for contrastive learning, maximizing the consistency between positive sample pairs and minimizing the consistency between negative sample pairs. ， in, For BPR loss, To compare learning loss, For knowledge graph embedding loss, For the set of all parameters, , , The weights are contrastive loss, knowledge graph embedding loss, and L2 regularization, respectively. After multiple iterations of training, the prediction score is finally calculated by fusing the dot product of the embeddings of users and items.

2. The knowledge graph comparative learning recommendation method based on attention mechanism according to claim 1, characterized in that, The specific formula for step two is as follows: ， ， in, It is a rating-specific weight matrix, where d is the embedding dimension. Let be the degree normalization factor, and be the set of neighbor nodes of user u. Let i be the set of neighboring nodes of item i; The original embedding vector of user u, Let be the original embedding vector of item i; during the message aggregation phase, the incoming messages of each node are aggregated through a nonlinear transformation of the ReLU activation function, and the attention mechanism uses the attention weight matrix. Transform item embeddings to calculate attention scores : ， in, For the attention weight matrix, normalize these attention scores using the softmax function to obtain the attention weights. , ， in I For the set of all items, attention-enhanced item embeddings are generated by applying attention weights through element-wise multiplication. Finally, the item is embedded. Item bipartite graph embedding results fused with attention-enhanced item embedding , ， in It is the item embedding weight matrix after attention enhancement.

3. The knowledge graph comparative learning recommendation method based on attention mechanism according to claim 1, characterized in that, In step three, the item embedding is used as the initial representation of the item entity in the knowledge graph. The user embedding constructs a seed set based on the user's historical interactions and expands related entities in the knowledge graph through multi-hop propagation. The user's k-hop related entity set is defined as follows: ， in, t Indicates the tail entity. h Indicates the head entity. r Let G represent a relation, and G represent a knowledge graph. Let H represent the set of entities related to user u for k-1 hops, and let H represent the maximum propagation hop count. Use the set of items clicked in the user's history as a seed set; For the k-jump ripple set For each knowledge triple in the dataset, calculate the correlation probability between the candidate item and the head entity. ， in, v Represents the embedding vector of the candidate item. Indicates the first i Embedding matrix of relations in triplets Indicates the first i The embedding vector of the head entity in each triplet. Describes the set of k-hop ripples for user u, i.e., from The set of all knowledge triples from the origin; the k-hop response vector is obtained by weighting the tail entity embeddings with relevance probability. : Summing and traversing the set of k-jump ripples for user u All triples , in, Indicates the first i The embedding vectors of the tail entities in each triplet are combined with the response vectors of all hop counts to form a knowledge-enhanced user embedding, capturing user interests at different propagation levels. Item embeddings that integrate user knowledge graph preferences are constructed through item embeddings and knowledge-enhanced user embeddings. ， in, 'U' represents item embedding, and 'U' represents knowledge-enhanced user embedding.

4. The knowledge graph comparative learning recommendation method based on attention mechanism according to claim 1, characterized in that, Step five specifically involves running the K-means clustering algorithm in each training cycle to extract preference prototypes from the initialized embedding space. , ， in, This represents the initial user and item embedding matrix; The dot product similarity between the unified embedding matrix E and the prototype is calculated as the preference score, and then normalized using the sigmoid function. ， in, M represents the number of users, and N represents the number of items. Represents a unified embedding matrix; The multimodal representation learning framework augments user and item embeddings with features through a prototype filtering network, which comprises two key components: a linear transformation function to enhance model fitting capabilities. (·), and a self-gated module for controlling the degree of feature enhancement. (·); By balancing feature enhancement through a prototype filtering network, the final embedding matrix result after feature enhancement is obtained: ， in, Linear transformation functions are used to enhance model fitting ability. This indicates that the self-gating module is used to control the degree of feature enhancement.

5. The knowledge graph comparative learning recommendation method based on attention mechanism according to claim 1, characterized in that, Step six introduces a temperature parameter τ into the InfoNCE loss function to control the contrast intensity. When τ is small, the model is more sensitive to the differences between samples, and the contrast learning has a stronger discriminative power. When τ is large, the similarity distribution is smoother, and the model training is more stable. By adjusting the temperature parameter τ, a balance is achieved between contrast intensity and training stability.

Images