A structural and semantic fusion-based contrastive learning knowledge graph embedding method

Abstract

The application relates to a structure and semantic fusion-based comparative learning knowledge graph embedding method, and belongs to the technical field of knowledge graph embedding. The method comprises the following steps: constructing a knowledge graph, calculating the importance scores of each entity node and performing random walk operation to generate a path sequence set containing alternately appearing entity nodes and relationship edges; selecting a path sequence as an anchor sample, and respectively constructing positive samples and negative samples from two angles of a structure view and a semantic view; constructing an extended triple loss, and performing random mask operation on any entity or relationship in the anchor sample to construct a mask entity relationship loss; taking the weighted sum of the extended triple loss and the mask entity relationship loss as a training target, and outputting entity and relationship embedding for link prediction. The method aims to solve the problem that the existing recommendation system can simultaneously consider structure patterns and semantic information when introducing knowledge graph embedding learning, and effectively improve path sampling quality and context utilization efficiency.

Description

Technical Field

[0001] This invention relates to a contrastive learning knowledge graph embedding method based on the fusion of structure and semantics, belonging to the field of knowledge graph embedding technology. Background Technology

[0002] In traditional fields such as intelligent question answering, recommendation systems, and semantic retrieval, the performance of related algorithms is limited by various factors. For example, traditional keyword-matching algorithms in intelligent question answering systems, while quickly finding relevant information, often suffer from low accuracy when faced with queries containing complex semantics, making them unsuitable for practical applications. In recommendation systems, traditional collaborative filtering algorithms, although capable of providing some personalized recommendations, perform poorly and lack accuracy when dealing with the cold start problem (i.e., new users or new items). To address these issues, researchers have begun to introduce knowledge graphs into these fields, hoping to leverage their powerful semantic connections and structured information to improve algorithm performance.

[0003] A knowledge graph (KG) is a structured representation used to describe entities and their semantic relationships. By mapping entities and relationships to a graph structure, knowledge graphs can better express complex semantics in different domains. Knowledge graphs are widely used in tasks such as intelligent question answering, recommendation systems, and semantic retrieval, effectively improving the system's understanding and reasoning capabilities regarding semantics. However, relying solely on knowledge graphs still faces some challenges. For example, traditional knowledge graph embedding (KGE) methods, such as TransE, TransH, and TransR, while effectively embedding entities and relationships in the graph into a continuous low-dimensional vector space, primarily focus on modeling the relative positions between entities and relationships. They struggle to handle complex many-to-many, symmetric, or antisymmetric relationships, leading to suboptimal performance in certain domains.

[0004] As our understanding of knowledge graphs deepens, researchers have proposed more complex bilinear models (such as DistMult, ComplEx, and RotatE) and models based on convolutional neural networks (such as ConvE) in an attempt to improve the models' ability to characterize diverse relationships and contexts. While these methods have made improvements in handling complex relationships, they still face the following problems: limited ability to model path dependencies, insufficient utilization of contextual information, and difficulty in unifying structural and semantic goals for optimization.

[0005] Recently, the Transformer architecture has demonstrated excellent representation learning capabilities in natural language processing and graph structure modeling. Therefore, Transformer-based knowledge graph embedding methods have emerged. These methods attempt to leverage the powerful context modeling capabilities of the Transformer to improve the fusion of structural and semantic information. However, knowledge graphs are inherently non-sequential graph structures, and directly linearizing them into a Transformer may lead to the loss or distortion of graph structure information. Furthermore, existing methods often rely on heuristic strategies for constructing paths or contexts, making it difficult to guarantee the semantic representativeness of the selected paths, which affects the effectiveness of training samples and the final performance of the model. More importantly, most existing works treat structural modeling and semantic modeling separately, lacking a unified optimization objective, resulting in a lack of deep integration between the two during the optimization process, thus limiting the model's performance.

[0006] Therefore, how to simultaneously consider structural patterns and semantic information in knowledge graph embedding learning, and effectively improve path sampling quality and context utilization efficiency, remains a pressing technical problem in this field. Existing technologies have not yet proposed a method that can deeply integrate structural and semantic information while ensuring efficient path modeling and improving model performance. Therefore, this patent proposes a contrastive learning method combining structural and semantic fusion, aiming to solve the aforementioned technical challenges. Through innovative loss function design and a self-supervised learning mechanism, it enhances the representation capabilities of knowledge graphs while ensuring a deep unity of structure and semantics, thereby effectively improving the performance of link prediction and other downstream tasks. Summary of the Invention

[0007] The purpose of this invention is to provide a contrastive learning knowledge graph embedding method based on the fusion of structure and semantics, which aims to solve the problem that existing recommendation systems can simultaneously take into account structural patterns and semantic information when introducing knowledge graph embedding learning, and effectively improve the path sampling quality and context utilization efficiency.

[0008] To achieve the above objectives, the technical solution of the present invention is: a contrastive learning knowledge graph embedding method based on structure and semantic fusion, comprising the following steps:

[0009] Step 1: Construct a knowledge graph containing a set of entities and a set of relations, calculate the importance score of each entity node in the knowledge graph, and perform a random walk operation based on the importance score to generate a set of path sequences containing alternating entity nodes and relation edges;

[0010] Step 2: Select a path sequence from the path sequence set as the anchor sample, and construct positive and negative samples from both the structural view and semantic view perspectives respectively;

[0011] Step 3: Construct an extended triplet loss based on the anchor sample, the positive sample, and the negative sample, and perform a random masking operation on any entity or relation in the anchor sample. Use the Transformer encoder to perform context modeling and predict the masked element, thereby constructing the masked entity relation loss; wherein, the extended triplet loss is used to bring the anchor sample closer to the positive sample and push the anchor sample further away from the negative sample.

[0012] Step 4: Use the weighted sum of the extended triplet loss and the masked entity relation loss as the training target until convergence, and output the entity and relation embeddings to perform link prediction based on the entity and relation embeddings.

[0013] Optionally, Step 1 specifically includes:

[0014] Step 1.1: Construct a collection of entities With relation set knowledge graph ,in, For the dimension of the entity set, For the dimension of the relation set, For the head entity, For the relationship, For each entity node in the knowledge graph, the tail entity is defined. Initialize its importance score ,in, The total number of nodes;

[0015] Step 1.2: Set the damping factor Maximum number of iterations and convergence tolerance ;

[0016] Step 1.3: For each entity node traverse all its neighbor nodes and update the nodes. Importance score, expressed as:

[0017]

[0018] in, This represents the importance score of the current entity node. For nodes The adjacency set, The importance score of neighboring nodes. To represent neighboring nodes The number of neighbors;

[0019] Step 1.4: Calculate the sum of the differences in importance scores for all nodes before and after the current iteration. The expression is:

[0020]

[0021] Step 1.5: If If the convergence condition is met, the iteration stops; otherwise, proceed to the next iteration.

[0022] Step 1.6: When the convergence condition is met or the number of iterations reaches the maximum number of iterations. When the time comes, output the final set of importance scores for all nodes;

[0023] Step 1.7: Perform a random walk operation based on the final importance score set to generate a set of path sequences containing alternating entity nodes and relation edges, expressed as:

[0024]

[0025] in, For a set of sequences, The total number of paths, For entity nodes, Let n be the relation node and n be the total number of entities.

[0026] Optionally, the random walk operation specifically involves: reducing the probability of a node being selected when the number of times a node appears in the path exceeds a preset threshold; and padding the path to a uniform length by filling in placeholders when the path length is less than the target length.

[0027] Optionally, Step 2 specifically includes:

[0028] Step 2.1: From the perspective of the structural view, using the actual path as anchor samples, construct positive and negative samples based on the relationship type of the previous or two hops of the path:

[0029] Among them, if the first jump relationship It is a symmetric relation, and the symmetric relation is for any ( ), always exist ( It also holds true, among which, For the head entity, For tail entities, To establish the relationship between the head and tail entities, positive samples are constructed by reversing the first hop and completing the path using random walk operations. The expression is:

[0030]

[0031] In the formula, These are entity nodes in the original anchor sample. and Represented as relations and entities supplemented during the random walk, where, For the first The relationship of step selection, The entity reached after applying the selected relationship;

[0032] negative samples Constructed by perturbing entities or random replacement, the expression is:

[0033]

[0034] In the formula, or , where the symbol This is represented as a non-equivalence relation;

[0035] Among them, if the first two jumps are related The relationship satisfies a composite relation, wherein the composite relation is an existential relation. , making , where the symbol To represent a relational composition operation, the first two jumps are combined into a single side to construct a positive sample. The expression is:

[0036]

[0037] in, These are entity nodes in the original anchor sample;

[0038] negative samples This is achieved by maintaining the multi-hop structure but perturbing the first or second half of the composite relationship to disrupt the composite logic, and the expression is:

[0039]

[0040] In the formula, or Not part of a compound relationship;

[0041] Among them, if and They are inverse relationships, and the inverse relationship is: ,and Then, a new path is formed by reversing the first hop, thereby constructing a positive sample. The expression is:

[0042]

[0043] negative samples By replacing a certain jump with a non-inverse relation, the expression becomes:

[0044]

[0045] in, ;

[0046] Among them, if the relationship If the relationship is antisymmetric, then negative samples are constructed by reversing the first hop. The expression is:

[0047]

[0048] From the same head entity Starting from this point, while maintaining consistency between path length and relation type, we search for alternative paths to ensure structural consistency, resulting in a positive sample representation as follows:

[0049]

[0050] In the formula, Positive samples;

[0051] Step 2.2: From the perspective of semantic view, semantic view is based on entity text description. Obtain its vector representation through a pre-trained language model. The expression is:

[0052]

[0053] in, For the pre-trained Sentence BERT model, Let be the dimension of the vector space;

[0054] Calculate the vector representation of the text description of all entities along the entire path. The expression is:

[0055]

[0056] The positive samples are selected by calculating the cosine similarity and choosing the sample with the highest similarity to the anchor path. The sample with the lowest similarity is the negative sample. The expression is:

[0057]

[0058]

[0059]

[0060] in, Represents the cosine similarity function. Represents cosine similarity. Represents the sample set, For anchor samples, For the j-th sample in the sample set, Let be the vector representation of the j-th sample in the sample set. It is represented as a transpose matrix.

[0061] Optionally, Step 3 specifically includes:

[0062] Step 3.1: Define the anchor sample Semantically consistent positive sample paths Negative sample paths inconsistent with semantics The training triplet instance set is used to obtain a contrastive objective to impose a relative similarity constraint during training: to encourage anchor paths to be closer to semantically consistent positive sample paths in the vector space, while moving away from semantically inconsistent negative sample paths. The contrastive loss function is obtained based on this contrastive objective. The expression is:

[0063]

[0064] in, The interval hyperparameter is positive. Represents the hinge function. The vector representation of the anchor sample. The vector representation of negative samples. A vector representation of a positive sample;

[0065] A sampling strategy involving multiple positive and multiple negative samples is employed in each training batch to obtain the extended triplet loss function. for:

[0066]

[0067] in, Represents the set of all samples. Indicates the sample The corresponding set of positive samples, Indicates the sample The corresponding set of negative samples;

[0068] Step 3.2: Given the input sequence Randomly select one of the symbols and replace it with Labeling, thereby obtaining new sequences Based on the Transformer encoder generating a contextual representation at position i, the original symbol... or To make predictions, the optimization objective is to maximize the log-likelihood of the true symbol under given context conditions, expressed as:

[0069]

[0070] in, To mask the loss of entity relationships, The model represents the first time. The predicted probability distribution of each masked position. Represents the true value. This indicates the context. This represents the set of mask positions.

[0071] Optionally, Step 4 specifically includes:

[0072] Step 4.1: and Joint optimization is performed to obtain a comprehensive training objective. The expression is:

[0073]

[0074] in, This is a hyperparameter used to control the weight balance between the two types of loss terms;

[0075] Step 4.2: Train the model based on the training objective. After training, the model outputs embedding vectors of entities and relations. The embedding vectors represent the semantic information of each entity and its relation in the knowledge graph.

[0076] Step 4.3: Perform link prediction based on the entity and relation embedding. The goal of link prediction is to predict the most likely tail entity given the head entity and relation.

[0077] The beneficial effects of this invention are as follows: This invention provides a technical solution in the field of knowledge graph embedding, effectively improving the performance of link prediction tasks, and is particularly suitable for downstream tasks such as intelligent recommendation, knowledge graph construction, natural language processing, and drug discovery. By deeply integrating structural modeling and semantic modeling, this invention significantly enhances the model's ability to capture complex entity relationships, significantly impacting the accuracy of recommendation systems, optimizing medical data analysis, and promoting personalized medical services. Furthermore, this technology can also be applied to areas such as social network analysis, intelligent search engines, and automated question-answering systems, further promoting the in-depth development of artificial intelligence and big data applications. Attached Figure Description

[0078] Figure 1 This is a model architecture diagram of the present invention;

[0079] Figure 2These are positive and negative sample generation diagrams of the structural view of this invention;

[0080] Figure 3 This is the semantic view of positive and negative sample generation diagrams of this invention;

[0081] Figure 4 This is a graph showing the experimental results of the present invention under different damping factors;

[0082] Figure 5 This is a graph showing the experimental results of the present invention under different numbers of paths and path lengths. Detailed Implementation

[0083] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0084] Example 1: As Figure 1 The diagram shown illustrates the model architecture of a contrastive learning knowledge graph embedding method based on structural and semantic fusion, as described in this embodiment. It should be understood that the technical solution provided by this invention can be applied to downstream tasks such as intelligent recommendation, knowledge graph construction, natural language processing, and drug discovery. This embodiment uses a drug discovery application scenario as an example to further illustrate the technical solution of this invention. The steps include:

[0085] Step 1: Construct a knowledge graph containing a set of entities and a set of relations, calculate the importance score of each entity node in the knowledge graph, and perform a random walk operation based on the importance score to generate a set of path sequences containing alternating entity nodes and relation edges;

[0086] Step 1.1: Construct a collection of entities With relation set knowledge graph ,in, For the dimension of the entity set, For the dimension of the relation set, For the head entity, For the relationship, For each entity node in the knowledge graph, the tail entity is defined. Initialize its importance score ,in, The total number of nodes;

[0087] Specifically, in drug discovery applications, the entity set can include "hospital", "doctor", "patient", "disease", "drug", "medical device", etc.; the relation set describes the relationships between these entities, such as "doctor works at hospital", "patient has disease", "drug treats disease", "device is used for surgery", etc. Based on the entity set and relation set, a knowledge graph for drug discovery can be constructed.

[0088] Step 1.2: Set the damping factor Maximum number of iterations and convergence tolerance ;

[0089] Step 1.3: For each entity node traverse all its neighbor nodes and update the nodes. Importance score, expressed as:

[0090]

[0091] in, This represents the importance score of the current entity node. For nodes The adjacency set, The importance score of neighboring nodes. To represent neighboring nodes The number of neighbors;

[0092] Step 1.4: Calculate the sum of the differences in importance scores for all nodes before and after the current iteration. The expression is:

[0093]

[0094] Step 1.5: If If the convergence condition is met, the iteration stops; otherwise, proceed to the next iteration.

[0095] Step 1.6: When the convergence condition is met or the number of iterations reaches the maximum number of iterations. When the time comes, output the final set of importance scores for all nodes;

[0096] Step 1.7: Perform a random walk operation based on the final importance score set to generate a set of path sequences containing alternating entity nodes and relation edges, expressed as:

[0097]

[0098] in, For a set of sequences, The total number of paths, For entity nodes, Let n be the relation node and n be the total number of entities.

[0099] Optionally, the random walk operation specifically involves: reducing the probability of a node being selected when the number of times a node appears in the path exceeds a preset threshold; and padding the path to a uniform length by filling in placeholders when the path length is less than the target length.

[0100] Step 2: Select a path sequence from the path sequence set as the anchor sample, and construct positive and negative samples from both the structural view and semantic view perspectives respectively;

[0101] Step 2.1: As Figure 2 As shown, from the perspective of the structural view, using the actual path as the anchor sample, positive and negative samples are constructed based on the relationship type of the previous or two hops of the path:

[0102] Among them, if the first jump relationship It is a symmetric relation, and the symmetric relation is for any ( ), always exist ( It also holds true, among which, For the head entity, For tail entities, To establish the relationship between the head and tail entities, positive samples are constructed by reversing the first hop and completing the path using random walk operations. The expression is:

[0103]

[0104] In the formula, These are entity nodes in the original anchor sample. and Represented as relations and entities supplemented during the random walk, where, For the first The relationship of step selection, The entity reached after applying the selected relationship;

[0105] Because symmetric relationships are bidirectional, reversing the path direction does not lead to semantic errors. Therefore, negative samples Constructed by perturbing entities or random replacement, the expression is:

[0106]

[0107] In the formula, or , where the symbol This is represented as a non-equivalence relation;

[0108] Among them, if the first two jumps are related The relationship satisfies a composite relation, wherein the composite relation is an existential relation. , making , where the symbol To represent a relational composition operation, the first two jumps are combined into a single side to construct a positive sample. The expression is:

[0109]

[0110] in, These are entity nodes in the original anchor sample;

[0111] negative samples This is achieved by maintaining the multi-hop structure but perturbing the first or second half of the composite relationship to disrupt the composite logic, and the expression is:

[0112]

[0113] In the formula, or Not part of a compound relationship;

[0114] Among them, if and They are inverse relationships, and the inverse relationship is: ,and Then, a new path is formed by reversing the first hop, thereby constructing a positive sample. The expression is:

[0115]

[0116] Since an inverse relationship implies a complete reversal, negative samples... By replacing a certain jump with a non-inverse relation, the expression becomes:

[0117]

[0118] in, ;

[0119] Among them, if the relationship If the relationship is antisymmetric, then negative samples are constructed by reversing the first hop. The expression is:

[0120]

[0121] To construct semantically plausible positive samples under antisymmetric relations, this embodiment performs a structure-based search within the local neighborhood of the knowledge graph anchor path. Specifically, it searches from the same head entity... Starting from this point, while maintaining consistency between path length and relation type, we search for alternative paths to ensure structural consistency, resulting in a positive sample representation as follows:

[0122]

[0123] In the formula, Positive samples;

[0124] It is important to understand that in this embodiment, random walks prioritize visiting nodes with high centrality in the local graph, thereby achieving a balance between exploratory nature and semantic consistency.

[0125] Step 2.2: From the perspective of semantic view, semantic view is based on entity text description. Obtain its vector representation through a pre-trained language model. The expression is:

[0126]

[0127] in, For the pre-trained Sentence BERT model, Let be the dimension of the vector space;

[0128] Calculate the vector representation of the text description of all entities along the entire path. The expression is:

[0129]

[0130] like Figure 3 As shown, the positive samples are calculated using cosine similarity and selected based on the highest similarity to the anchor path. The sample with the lowest similarity is the negative sample. The expression is:

[0131]

[0132]

[0133]

[0134] in, Represents the cosine similarity function. Represents cosine similarity. Represents the sample set, For anchor samples, For the j-th sample in the sample set, Let be the vector representation of the j-th sample in the sample set. It is represented as a transpose matrix.

[0135] Step 3: Construct an extended triplet loss based on the anchor sample, the positive sample, and the negative sample, and perform a random masking operation on any entity or relation in the anchor sample. Use the Transformer encoder to perform context modeling and predict the masked element, thereby constructing the masked entity relation loss; wherein, the extended triplet loss is used to bring the anchor sample closer to the positive sample and push the anchor sample further away from the negative sample.

[0136] Step 3.1: To effectively learn path-level semantic representations in knowledge graphs, this embodiment constructs an extended triplet loss function. This loss function aims to fully utilize the positive and negative samples obtained through both relation type-based construction and semantic similarity enhancement. Specifically, it defines the loss function based on anchor samples. Semantically consistent positive sample paths Negative sample paths inconsistent with semantics The training triplet instance set is used to obtain a contrastive objective to impose a relative similarity constraint during training: to encourage anchor paths to be closer to semantically consistent positive sample paths in the vector space, while moving away from semantically inconsistent negative sample paths. The contrastive loss function is obtained based on this contrastive objective. The expression is:

[0137]

[0138] in, The interval hyperparameter is positive. Represents the hinge function. The vector representation of the anchor sample. The vector representation of negative samples. A vector representation of a positive sample;

[0139] To improve training efficiency and robustness, this embodiment employs a sampling strategy of multiple positive and multiple negative samples in each training batch, thereby obtaining the extended triplet loss function. for:

[0140]

[0141] in, Represents the set of all samples. Indicates the sample The corresponding set of positive samples, Indicates the sample The corresponding set of negative samples;

[0142] It is understood that this embodiment, through the extended contrastive learning objective, enables the model to better capture subtle semantic differences between relational paths and to achieve stronger generalization ability for new inference patterns in downstream tasks.

[0143] Step 3.2: Given the input sequence Randomly select one of the symbols and replace it with Labeling, thereby obtaining new sequences Based on the Transformer encoder generating a contextual representation at position i, the original symbol... or The prediction objective is to maximize the log-likelihood of the true symbol given its context. This optimization method enables the model to learn a context-aware representation that can simultaneously utilize structural and semantic cues to accurately reconstruct the masked element. The expression is as follows:

[0144]

[0145] in, To mask the loss of entity relationships, The model represents the first time. The predicted probability distribution of each masked position. Represents the true value. This indicates the context. This represents the set of mask positions.

[0146] It is understood that this embodiment aims to isolate the contextual information on which the masked element depends by randomly masking a symbol (entity or relation) in the anchor sample, thereby forcing the model to focus on fine-grained relational semantic modeling.

[0147] Step 4: Use the weighted sum of the extended triplet loss and the masked entity relation loss as the training target until convergence, and output the entity and relation embeddings to perform link prediction based on the entity and relation embeddings.

[0148] Step 4.1: and Joint optimization is performed to obtain a comprehensive training objective. The expression is:

[0149]

[0150] in, This is a hyperparameter used to control the weight balance between the two types of loss terms;

[0151] Step 4.2: Train the model based on the training objective. After training, the model outputs embedding vectors of entities and relations. The embedding vectors represent the semantic information of each entity and its relation in the knowledge graph.

[0152] Step 4.3: Perform link prediction based on the entity and relation embedding. The goal of link prediction is to predict the most likely tail entity given the head entity and relation.

[0153] The following example, a specific application scenario in drug discovery, illustrates the implementation process of this invention: Given a query triple "(Patient A, Recommended Treatment, ?)," an anchor path is obtained from a random walk: Patient A — suffers from → XX disease — available drugs → XXX — belongs to → XXX drug. Based on the structural view, positive samples of composite relationships (merging "suffers from → available drugs → belongs to" into a direct association) and negative samples of perturbation relationships are constructed. Simultaneously, under the semantic view, similarity is calculated using the text vectors of the drug instructions and disease descriptions to select semantically positive and negative samples. After joint optimization using extended triple loss and masked entity relationship loss, candidate drug recommendation schemes are scored, ultimately predicting the tail entity as "XXX drug," which is superior to other drug schemes, thus achieving accurate link prediction for "(Patient A, Recommended Treatment, XXX drug)".

[0154] Based on the specific implementation details, the effectiveness of the technical solution of the present invention will be demonstrated through experiments.

[0155] Experiment 1: This experiment used three common knowledge graph datasets: FB15K237, WN18RR, and UMLS. The performance of the model was evaluated using Hit@N and MRR metrics. The goal of the experiment was to analyze the advantages of the present invention compared to existing baseline models in link prediction tasks, to verify its effectiveness in practical applications. Baseline models were categorized into four types based on the methods used. The table shows that methods based on Transformer and attention generally outperformed methods based on translation distance models, semantic matching models, and graph neural networks, consistent with the outstanding performance of Transformer in various tasks in recent years. The experimental results are shown in Table 1 below.

[0156] Table 1. Experimental results comparing the present invention with existing baseline models in the link prediction task.

[0157]

[0158] On the FB15K-237 dataset, which is specifically designed to eliminate inverse relation leakage, models with dependency symmetry are particularly challenging. On this dataset, our invention achieved the best mean reciprocal rank (MRR=0.395) and Hits@10 (0.581), representing improvements of 3.9% and 2.3% respectively compared to the strongest baseline model, TracKGE. Notably, our invention achieved a Hits@1 metric of 0.311, roughly on par with TracKGE's 0.313, but achieved higher values ​​for Hits@3 and Hits@10. This indicates that our method is not only competitive in top-1 predictions but also performs better in overall ranking quality, avoiding over-reliance on extremely high-confidence Top-1 predictions. This demonstrates that our invention does not merely rely on memorizing direct entity-relation patterns but is able to capture deeper contextual information and structural features. Compared with classic semantic models (such as ComplEx with an MRR of 0.323 and ConvE with an MRR of 0.325), this invention achieves a relative improvement of over 22% in MRR, fully demonstrating the advantages of fusing structural modeling with semantic priors.

[0159] On the WN18RR dataset, which emphasizes logical reasoning rather than lexical hierarchy (such as hyponyms and synonyms), this invention achieves MRR=0.520 and Hits@10=0.610, both improvements over the previous best results TCKGE (MRR=0.504) and HittER (MRR=0.503). Particularly noteworthy is the 4.2% improvement over TCKGE in Hits@1, indicating that this method is more accurate in distinguishing fine-grained hierarchical relationships. Given the relatively limited number of relationship types but highly coupled entity semantics in WN18RR, this result demonstrates that this invention effectively models entity-level interactions through semantic optimization. Furthermore, it consistently maintains an advantage in each Hits@k metric, showing its ability to maintain stable performance even when facing semantically similar candidate entities—a common challenge in lexical knowledge graphs.

[0160] On the UMLS biomedical dataset, which is densely relational and possesses strong ontological features, this invention achieved the lowest average ranking (MR=1.41) and reached 99.0 on the Hits@10 metric, matching the best-performing ConvE but significantly outperforming methods such as TracKGE (97.42) and ComplEx (96.7). Compared to the baseline models DistMult (MR=5.52) and ComplEx (MR=3.59), the significant decrease in average ranking indicates that this method can prioritize highly relevant triples earlier. This result demonstrates that in scenarios with highly dense relations and involving hierarchical medical concepts, the advantages of this invention in deeply fusing latent semantic features with structure-aware capabilities are particularly significant.

[0161] In summary, this invention achieves significant performance improvements and can be used in fields such as intelligent recommendation systems, drug discovery, and medical data analysis. It can improve the accuracy and efficiency of entity relationship prediction, thereby enhancing the system's decision support capabilities.

[0162] Experiment 2: To evaluate the contributions of each component in this invention, systematic ablation experiments were conducted using two widely used benchmark datasets, FB15K-237 and WN18RR. Specifically, the experiments designed the following three ablation scenarios: removal of the importance score-based sampling strategy to examine the role of node importance ranking in the path generation process; disabling the semantically aware sample augmentation mechanism, i.e., no longer utilizing semantic similarity constraints in the contrastive loss, to analyze the impact of semantic augmentation on model performance; and masking only entities, without simultaneously masking entities and relations, to compare the impact of different masking strategies on representation capabilities during the self-supervised learning stage. The experimental results are shown in Table 2 below.

[0163] Table 2 Ablation Experiment Results

[0164]

[0165] First, looking at the ablation results in the first row of the table, this module has a particularly significant impact on the FB15K-237 dataset. On the WN18RR dataset, the performance degradation after removing importance scores is relatively mild, with MRR decreasing by only 0.089 (from 0.520 to 0.431), and Hit@10 remaining almost unchanged (from 0.610 to 0.598). This may be related to the relatively regular graph structure and fewer relation types in WN18RR. Under this structure, path construction between entities is not easily affected by changes in node importance, thus its value is relatively limited. However, on the FB15K237 dataset, after removing importance scores, MRR plummeted from 0.395 to 0.278, and Hit@10 decreased from 0.581 to 0.322, indicating that the entity importance-guided random walk strategy plays a crucial role in path sampling in complex relational networks during path construction. Secondly, disabling semantic augmentation has a disastrous impact on both datasets, especially on FB15K-237, where MRR drops from 0.395 to 0.161 and Hit@10 drops to 0.313, almost completely eliminating its predictive power. This result indicates that while structural information provides basic graph topological clues for the model, semantic information plays a crucial role in distinguishing semantic differences between complex paths. A large number of polysemous words and synonymous relationships in FB15K-237 rely on semantic information from entity descriptions for accurate modeling, thus the lack of semantic augmentation has a greater impact. On WN18RR, although there is also a performance decline (MRR drops from 0.520 to 0.332), some inference ability is still retained, suggesting that some path relationships in WN18RR may have stronger structural uniqueness, making semantic modeling slightly less important than in FB15K-237. Finally, in the comparison of masking strategies, masking only entities resulted in varying degrees of performance degradation on both datasets. Specifically, the MRR for FB15K-237 decreased from 0.395 to 0.322, and for WN18, it decreased from 0.520 to 0.386. This indicates that in Transformer path modeling, masking only entities weakens the model's ability to capture relational information, especially in datasets containing many complex relation types (such as reverse, multi-hop, and combined relations) (like FB15K-237). This result further validates the effectiveness of co-modeling of structural and semantic information for path prediction.

[0166] Experiment 3: In this experiment, the sensitivity of the proposed method to several key hyperparameters was investigated. These hyperparameters directly affect different components in the model learning process. Specifically, the hyperparameters examined include: the damping factor in the invention, which is used to balance the weight allocation of local and global structures during path sampling; and the path length in the sequence construction module based on random walks, which determines the structural depth covered by the path representation. The above experiments were conducted on the FB15K-237 and WN18RR datasets to evaluate the robustness and generalization ability of the proposed method under different configurations.

[0167] First, we verified the effect of different damping factors on the model performance. The experimental results are as follows: Figure 4 As shown, at lower damping factor values, the limited structural context included in the sampling paths leads to decreased model performance. In this case, the random walk process is prone to frequent restarts, thus limiting the exploration of multi-hop semantic dependencies in the graph. With the damping factor, the model can acquire more informative long-distance paths, thereby improving representation quality and outperforming in metrics such as MRR and Hits@k.

[0168] This parameter setting achieves an effective balance between preserving local structure and exploring global context, enabling the model to capture multi-hop relationships without excessively weakening the semantic relevance of the sampled paths. Notably, this result is consistent with the damping factor commonly used in PageRank-like algorithms (typically set to 0.85), further validating the applicability of this parameter in graph structure learning.

[0169] While higher damping factor values ​​can encourage deeper graph traversal, excessively long paths can introduce noisy or semantically irrelevant neighbor nodes, especially in dense or heterogeneous knowledge graphs like FB15K-237. In such cases, the model may experience decreased returns or even performance degradation due to oversmoothing and increased uncertainty in node representations. On the WN18RR dataset, the performance degradation is relatively mild, mainly because this dataset has a relatively simple lexical hierarchy and fewer spurious connections in multi-hop paths.

[0170] Then, the impact of path length and sequence number on model performance was verified, and the experimental results are as follows: Figure 5As shown in the diagram, on the FB15k-237 dataset, we observed that as the path length increased from 4 to 32, both Hit@3 and MRR metrics showed stable and significant improvements across all path count levels. This indicates that longer paths provide more contextual and relational information, which is beneficial for contrastive learning and masked representation learning. Performance peaked at approximately 32 path lengths; further increases to 64 resulted in limited improvement and even a slight decrease at high sampling rates. Similarly, increasing the number of paths per triple from 5000 to 20000 continuously improved performance, reaching its optimum around 20000 to 25000 paths. However, beyond 20000 paths, the performance increase gradually weakened, suggesting potential redundancy or noise accumulation at this point. Overall, this trend indicates that deep and dense path sampling is beneficial for the FB15k-237 dataset, which exhibits higher relational diversity and semantic complexity, but its benefits have an upper limit, requiring a balance between computational overhead and diminishing returns.

[0171] In contrast, WN18RR exhibits a more subtle pattern. Moderately increasing path length and number (e.g., from length 4 to 16, number of samples 5000 to 15000) can improve performance, but further increases lead to stagnation or even a decline. Specifically, Hit@3 typically performs best in the range of path length 8 to 16 and sample count 15000 to 20000; while excessively long paths (e.g., L=64) consistently perform poorly even with high sample counts. The MRR metric shows a similar trend, peaking around L=8 or 16 and N=20000, but performance significantly decreases when any hyperparameter is pushed to its extreme. This suggests that for WN18RR, which has a stronger hierarchical structure and higher lexical regularity, shorter and semantically concentrated paths are more valuable; excessively long or numerous paths may introduce trivial or repetitive relationships, increasing noise and weakening learning effectiveness.

[0172] Overall, the experimental results of the three experiments confirm that path length and the number of paths have a crucial impact on representation quality, but their effect is non-linear and dataset-dependent. Choosing an appropriate hyperparameter range—i.e., moderate length and a relatively high but not excessive sampling rate—can significantly improve model performance while avoiding unnecessary computational overhead.

[0173] In summary, this invention first guides random walks to generate path subgraphs containing both structural and semantic information using importance scores. Based on this, a contrastive learning framework combining structural and semantic views is constructed. Positive and negative sample pairs are generated in two ways: first, based on relation type sampling to distinguish paths with different relation patterns; and second, based on semantic similarity sampling to compare semantic differences in entity description vectors. An extended triplet loss function unifies these two strategies to achieve robust and discriminative knowledge representation learning. Furthermore, this invention introduces a Transformer-based self-supervised mask modeling module to perform mask prediction on entities or relations in the path sequence, thereby further improving the quality of the embedded representation. Therefore, this invention can be widely applied to scenarios requiring knowledge-based reasoning, such as question-answering systems and recommendation systems.

[0174] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A contrastive learning knowledge graph embedding method based on structural and semantic fusion, characterized in that, The method includes the following steps: Step 1: Construct a knowledge graph containing a set of entities and a set of relations, calculate the importance score of each entity node in the knowledge graph, and perform a random walk operation based on the importance score to generate a set of path sequences containing alternating entity nodes and relation edges; Step 2: Select a path sequence from the path sequence set as the anchor sample, and construct positive and negative samples from both the structural view and semantic view perspectives respectively; Step 3: Construct an extended triplet loss based on the anchor sample, the positive sample, and the negative sample, and perform a random masking operation on any entity or relation in the anchor sample. Use the Transformer encoder to perform context modeling and predict the masked element, thereby constructing the masked entity relation loss; wherein, the extended triplet loss is used to bring the anchor sample closer to the positive sample and push the anchor sample further away from the negative sample. Step 4: Use the weighted sum of the extended triplet loss and the masked entity relation loss as the training target until convergence, and output the entity and relation embeddings to perform link prediction based on the entity and relation embeddings; Step 2 includes: Step 2.2: From the perspective of semantic view, semantic view is based on entity text description. Obtain its vector representation through a pre-trained language model. The expression is: ； in, For the pre-trained Sentence BERT model, Let be the dimension of the vector space; Calculate the vector representation of the text description of all entities along the entire path. The expression is: ； in, n The total number of entities; The positive samples are selected by calculating the cosine similarity and choosing the sample with the highest similarity to the anchor path. The sample with the lowest similarity is the negative sample. The expression is: ； ； ； in, Represents the similarity function. Represents cosine similarity. Represents the sample set, For anchor samples, For the j-th sample in the sample set, Let be the vector representation of the j-th sample in the sample set. Represented as a transpose matrix; Step 3 includes: Defined by anchor sample Semantically consistent positive sample paths Negative sample paths inconsistent with semantics The training triplet instance set is used to obtain a contrastive objective to impose a relative similarity constraint during training: to encourage anchor paths to be closer to semantically consistent positive sample paths in the vector space, while moving away from semantically inconsistent negative sample paths. The contrastive loss function is obtained based on this contrastive objective. The expression is: ； in, The interval hyperparameter is positive. Represents the hinge function. The vector representation of the anchor sample. The vector representation of a negative sample. A vector representation of a positive sample; A sampling strategy involving multiple positive and multiple negative samples is employed in each training batch to obtain the extended triplet loss function. for: ； in, Represents the set of all samples. Indicates the sample The corresponding set of positive samples, Indicates the sample The corresponding set of negative samples.

2. The contrastive learning knowledge graph embedding method based on structural and semantic fusion according to claim 1, characterized in that, Step 1 specifically refers to: Step 1.1: Construct a collection of entities With relation set knowledge graph ,in, For the dimension of the entity set, For the dimensions of the relation set, For the head entity, For the relationship, For each entity node in the knowledge graph, the tail entity is defined. Initialize its importance score ,in, The total number of nodes; Step 1.2: Set the damping factor Maximum number of iterations and convergence tolerance ; Step 1.3: For each entity node traverse all its neighbor nodes and update the nodes. Importance score, expressed as: ； in, This represents the importance score of the current entity node. For nodes The adjacency set, The importance score of neighboring nodes. To represent neighboring nodes The number of neighbors; Step 1.4: Calculate the sum of the differences in importance scores for all nodes before and after the current iteration. The expression is: ； Step 1.5: If If the convergence condition is met, the iteration stops; otherwise, proceed to the next iteration. Step 1.6: When the convergence condition is met or the number of iterations reaches the maximum number of iterations. When the time comes, output the final set of importance scores for all nodes; Step 1.7: Perform a random walk operation based on the final importance score set to generate a set of path sequences containing alternating entity nodes and relation edges, expressed as: ； in, For a set of sequences, The total number of paths, For entity nodes, Let n be the relation node and n be the total number of entities.

3. The contrastive learning knowledge graph embedding method based on structure and semantic fusion according to claim 2, characterized in that, The random walk operation specifically involves: reducing the probability of a node being selected when it appears more than a preset threshold number of times in the path; and padding the path to a uniform length by filling in placeholders when the path length is less than the target length.

4. The contrastive learning knowledge graph embedding method based on structure and semantic fusion according to claim 3, characterized in that, Step 2 also includes: From a structural viewpoint, using the actual path as anchor samples, positive and negative samples are constructed based on the relationship type of the previous or two hops of the path: Among them, if the first jump relationship It is a symmetric relation, and the symmetric relation is for any ( ), always exist ( It also holds true, among which, For the head entity, For tail entities, To establish the relationship between the head and tail entities, positive samples are constructed by reversing the first hop and completing the path using random walk operations. The expression is: ； In the formula, These are entity nodes in the original anchor sample. and Represented as relations and entities supplemented during the random walk, where, For the first The relationship of step selection, The entity reached after applying the selected relationship; negative samples Constructed by perturbing entities or random replacement, the expression is: ； In the formula, or , where the symbol This is represented as a non-equivalence relation; Among them, if the first two jumps are related The relationship satisfies a composite relation, wherein the composite relation is an existential relation. , making , where the symbol To represent a relational composition operation, the first two jumps are combined into a single side to construct a positive sample. The expression is: ； in, These are entity nodes in the original anchor sample; negative samples This is obtained by maintaining the multi-hop structure but perturbing the first or second half of the composite relationship to disrupt the composite logic, and the expression is: ； In the formula, or Not part of a compound relationship; Among them, if and They are inverse relationships, and the inverse relationship is: ,and Then, a new path is formed by reversing the first hop, thereby constructing a positive sample. The expression is: ； negative samples By replacing a certain jump with a non-inverse relation, the expression becomes: ； in, ; Among them, if the relationship If the relationship is antisymmetric, then negative samples are constructed by reversing the first hop. The expression is: ； From the same head entity Starting from this point, while maintaining consistency between path length and relation type, we search for alternative paths to ensure structural consistency, resulting in a positive sample representation as follows: ； In the formula, This is a positive sample.

5. The contrastive learning knowledge graph embedding method based on structure and semantic fusion according to claim 4, characterized in that, Step 3 also includes: Given an input sequence Randomly select one of the symbols and replace it with Labeling, thereby obtaining new sequences Based on the Transformer encoder generating a contextual representation at position i, the original symbol... or To make predictions, the optimization objective is to maximize the log-likelihood of the true symbol under given context conditions, expressed as: ； in, To mask the loss of entity relationships, The model represents the first time. The predicted probability distribution of each masked position. Represents the actual value. This indicates the context. This represents the set of mask positions.

6. The contrastive learning knowledge graph embedding method based on structure and semantic fusion according to claim 5, characterized in that, Step 4 specifically refers to: Step 4.1: and Joint optimization is performed to obtain a comprehensive training objective. The expression is: ； in, This is a hyperparameter used to control the weight balance between the two types of loss terms; Step 4.2: Train the model based on the training objective. After training, the model outputs embedding vectors of entities and relations. The embedding vectors represent the semantic information of each entity and its relation in the knowledge graph. Step 4.3: Perform link prediction based on the entity and relation embedding. The goal of link prediction is to predict the most likely tail entity given the head entity and relation.

Images