Knowledge graph embedding method based on negative sampling and weighting

Abstract

This invention discloses a knowledge graph embedding method based on negative sampling and weighting. The steps include: acquiring a medical dataset; sampling high-quality candidate negative sample sets for all positive triples based on pre-trained embedding vectors and a designed high-quality candidate negative sample set construction strategy; calculating the scoring function values ​​of triples and their corresponding negative triples according to a translation-based knowledge graph embedding model; statistically analyzing the relevant frequency data in the dataset and calculating the frequency-based weights for each triple; calculating the loss L of the triple set based on the scoring function values ​​and weights to obtain the final loss function of the training framework; optimizing the loss function of the training model using stochastic gradient descent; and outputting the potential association between drugs and diseases / symptoms. This application improves the negative sampling quality of the model and mitigates the adverse effects of long-tail entities and relationships on model training by weighting the loss function, thereby improving model training performance.

Description

Technical Field

[0001] This invention belongs to the field of medical information technology, and in particular relates to a knowledge graph embedding method based on negative sampling and weighting. Background Technology

[0002] Currently, knowledge graphs (KGs) are crucial for storing and processing structured data, and as advanced knowledge interconnection solutions, they provide a wide range of intelligent system power for many applications, attracting widespread attention from academia and industry. Knowledge graph embedding (KGE) embeds entities and relations from a KG into a continuous vector space, enabling efficient computation of semantic information while preserving the inherent structure of the knowledge graph, thus making the large-scale application of knowledge graphs a reality. Therefore, in recent years, knowledge graph embedding has been widely used in the biomedical field, helping to discover potential associations between drugs and diseases, genes, and symptoms, accelerating new drug development and the design of personalized treatment plans.

[0003] Currently, most knowledge graph embedding models are trained using the traditional negative sampling approach. First, negative samples are sampled from the positive samples in the dataset according to a certain strategy. Then, the loss functions for both positive and negative samples are calculated based on the model's scoring function for optimization training, resulting in the embedded vector representations of entities and relations in the knowledge graph. Traditional knowledge graph embedding models largely rely on negative sampling, which involves constructing a candidate negative sample set relative to the positive sample set based on a certain strategy, and then selecting a portion of the negative samples from this set for training. However, knowledge graphs face the data sparsity problem. Traditional negative sampling-based model training methods further exacerbate the sparsity of the actual training data by only sampling a subset of negative samples. This leads to insufficient training samples and may cause the model to overlook some important negative samples, thus affecting its training performance.

[0004] In recent years, researchers have proposed various KGE models, among which classic models include TransE, DistMult, SimpleE, and ComplEx. To better extract useful information from datasets and obtain better knowledge embeddings, most knowledge graph embedding models employ traditional negative sampling-based training methods. First, negative samples are sampled from the positive samples in the dataset according to a certain strategy. Then, the loss functions of positive and negative samples are calculated based on the model's scoring function for optimization training, thereby obtaining the embedding vector representations of entities and relations in the knowledge graph. Traditional knowledge graph embedding models largely rely on negative sampling, that is, constructing a candidate negative sample set relative to the positive sample set based on a certain strategy, and selecting a portion of the negative samples from this set for training. Common negative sampling strategies include random negative sampling, adversarial negative sampling, and negative sampling that introduces additional information. However, knowledge graphs face the data sparsity dilemma. Traditional negative sampling-based model training methods further exacerbate the sparsity of the actual training data by only sampling a portion of the negative samples for training. This results in insufficient training samples and may ignore some important negative samples, thus affecting the performance of model training.

[0005] Researchers explored a non-sampling-based training method, where all non-positive triples consisting of entities and relations in the knowledge graph are used as negative samples for training. This addresses the problems inherent in negative sampling-based training by skipping the negative sampling process. However, this method has several drawbacks. First, using all unobserved triples as negative samples introduces false negatives, meaning fact triples not included in the knowledge graph dataset due to data sparsity are used as negative samples, affecting model training accuracy. Second, this method introduces a large number of easily separable negative samples far from the model's decision boundary. Studies on the distribution of negative sample rating function values ​​show that most negative triples are easily separable, meaning they contribute little to the gradient update direction during model training, thus affecting the model's convergence speed and accuracy.

[0006] In summary, through the analysis of the two traditional training methods mentioned above, it can be found that both methods face a common problem, namely, they cannot guarantee the participation of high-quality negative samples in training. The definition of a high-quality negative sample given in reference

[36] is a sample that is difficult to judge as true or false without referring to the true value. The sample is only meaningful in semantics, but is actually incorrect. That is to say, a high-quality negative sample is neither a false negative sample nor an easily distinguishable negative sample. The traditional training method based on negative sampling randomly samples unobserved samples as negative samples for training, while the training method based on non-sampling uses all unobserved samples as negative samples for training. Neither of these methods can guarantee the quality of the negative samples participating in training, thus limiting the improvement of model training performance.

[0007] Furthermore, existing knowledge graphs face the problem of data imbalance, meaning that entities and relations follow a long-tail distribution. Only a small number of entities and relations frequently appear in the knowledge graph, while the vast majority appear only in a limited number of triples. Current knowledge graph embedding training does not distinguish the frequency of different entities and relations, causing the model training to focus more on frequently occurring entities and relations, while paying relatively less attention to infrequent long-tail entities and relations. This results in long-tail entities and relations not being adequately trained, which also affects the improvement of model training performance. In the link prediction task experiment of the TransE model on the benchmark dataset FB15k-237, the high-frequency entity "United Kingdom" appearing 1953 times in the dataset achieved a MRR (Mean Inverse Rank, the higher the value, the better the embedding performance) score of 0.408, while the low-frequency entity "Pizza" appearing only 8 times received an MRR score of only 0.194. This experiment illustrates, from one perspective, the adverse impact of long-tail entities and relations on model training performance. Summary of the Invention

[0008] Traditional negative sampling-based model training methods use randomly selected negative samples for training, which can easily miss some important negative samples and negatively impact the training of knowledge graph embedding models. Some researchers have explored non-sampling model training frameworks, using all unobserved triples as negative samples to improve model training performance. However, both of these model training methods inevitably introduce spurious negative samples and easily separable negative samples far from the model's decision boundary into the training process. Furthermore, they do not consider the adverse effects of long-tail entities and relationships on model training, thus limiting the improvement of model training performance.

[0009] In view of this, this application proposes a general knowledge graph embedding framework, HNSW-KGE, based on high-quality negative sampling and weighting. First, pre-training is performed using the NS-KGE non-sampling training framework to quickly obtain an initial, relatively high-quality embedding vector representation for all entities and relations. Second, a candidate negative sample set construction strategy is designed. Based on the pre-trained embedding vectors, a certain number of negative samples that are neither spurious nor easily separable are sampled from all positive triples to form a candidate negative sample set, providing high-quality negative samples for model training. Finally, the loss function is weighted based on the frequency of entities and relations appearing in the triples to mitigate the adverse effects of long-tail entities and relations on model training.

[0010] To achieve the above objectives, this application discloses a knowledge graph embedding method based on negative sampling and weighting, the method comprising the following steps:

[0011] Obtain a medical dataset, in which entities include drugs, diseases, and symptoms;

[0012] Based on the pre-trained embedding vectors and the designed high-quality candidate negative sample set construction strategy, a high-quality candidate negative sample set is sampled for all positive triples.

[0013] Based on a translation-based knowledge graph embedding model, calculate the scoring function values ​​of triples and their corresponding negative triples;

[0014] Collect relevant frequency data from the dataset and calculate the frequency-based weight W for each triple. f ;

[0015] Based on the scoring function value and W f Calculate the loss L of the triple set to obtain the final loss function of the training framework;

[0016] Stochastic gradient descent (SGD) is used to optimize the loss function of the training model and learn the optimal parameters of the translation-based knowledge graph embedding model, including the optimal embedding vectors h,r,t, where h,t,r are the embedding vectors of the head entity, tail entity, and relation in the triple, respectively.

[0017] Output the potential association between drugs and diseases and symptoms.

[0018] Preferably, during the training process, the candidate negative sample set is dynamically updated at a certain frequency based on the knowledge graph embedding vector obtained at that time, thereby improving the quality of the candidate negative samples.

[0019] Preferably, to avoid overfitting, during model training, the second norm of all entities and relations in the dataset is forced to be less than or equal to 1, i.e., ||h||2≤1, ||r||2≤1, ||t||2≤1.

[0020] Preferably, the non-sampled knowledge graph embedding framework NS-KGE is selected as the pre-trained model, thereby quickly obtaining an initial knowledge graph embedding vector representation.

[0021] Preferably, the high-quality candidate negative sample set construction strategy constructs a candidate negative sample set for all positive triples, serving as a candidate library for negative samples, so as to sample high-quality negative samples from it to participate in model training, thereby improving the performance of model training; the high-quality candidate negative sample set construction strategy specifically includes:

[0022] The candidate negative sample set includes the candidate head entity set, which is the candidate negative sample set obtained by replacing the head entities, and the candidate tail entity set, which is the candidate negative sample set obtained by replacing the tail entities.

[0023] The sampling size for each triplet is N = N h +N t The candidate negative sample set, wherein the size of the candidate head entity set is N.h The size of the candidate tail entity set is N t ;

[0024] Set N h N t When considering the one-to-many or many-to-one relation types corresponding to the triples, specifically, for triples of the one-to-many relation type, more candidate head entities are set, thus the size of the candidate head entity set is larger than the size of the candidate head and tail entity sets; while for triples of the many-to-one relation type, more candidate tail entities are set, thus the size of the candidate tail entity set is larger than the size of the candidate head entity set. The size of the candidate head entity set is defined as N. h The size of the candidate tail entity set is N t ,Right now

[0025]

[0026] Among them, t rh h refers to the average number of tail entities connected to the head entity of all triples with relation r in a knowledge graph. rt This refers to the average number of head entities connected to the tail entity in all triples of relation r;

[0027] Construct candidate head entity sets and candidate tail entity sets for the triple (h,r,t), ultimately obtaining a triple (h,r,t) of size N = N h +N t The candidate negative sample set;

[0028] A high-quality candidate negative sample set construction process based on pre-training is executed. All triples in the dataset are used to construct their corresponding candidate negative sample sets based on this process, resulting in a high-quality candidate negative sample set for the dataset. In addition, to avoid low-quality candidate negative sample sets due to the performance limitations of the pre-trained model itself, the negative sample set is dynamically updated at a certain frequency during training based on the knowledge graph embedding vector obtained at that time, thereby providing a high-quality negative sample guarantee for the next step of optimization training based on the loss function.

[0029] Preferably, the loss function L for model training is defined as:

[0030]

[0031] Where (h,r,t) and (h',r,t') represent positive triples and negative triples, respectively, and S and S' represent the set of positive triples and the set of negative triples, respectively;

[0032] A triplet frequency-based weighting factor is added to the loss function. Different weights are assigned to the triplet loss function based on the frequency of entities and relations appearing in the dataset within the training triplet. Specifically, lower weights are assigned to more frequent entities and relations, while higher weights are assigned to less frequent entities and relations. This increases the training focus on long-tail entities and relations, aiming to learn better embeddings. A triplet frequency-based weighting factor W is defined. f for:

[0033]

[0034] Among them, f h ,f t ,f r Let h, t, and r represent the frequencies of entities h, t, and relation r in the dataset, respectively. Let w represent the median frequency of entities and relations in the dataset, respectively. Following the Pareto principle (80 / 20 rule), the median frequency is defined as the frequency of the top 20% of entities and relations ranked by their frequency of occurrence, from highest to lowest. h w t w r These are the frequency weighting factors for entities h, t, and relation r, respectively.

[0035] Weighting factor W based on triple frequency f For low-frequency triples, where the entities and relations in the triples are low-frequency entities and relations that appear less than the median frequency, and the weight of the low-frequency entity or relation is greater than 1, the W in the loss function of the low-frequency triples... f Weights greater than 1 are used to increase attention to low-frequency entities and relations during training; similarly, for high-frequency triples (triples containing entities and relations that appear more frequently than the median), the weight of the high-frequency entity or relation is less than 1, and the weight of W in the loss function of the high-frequency triple is less than 1. f The weights will be less than 1, thus reducing the focus on these high-frequency entities and relationships during training;

[0036] The weighting factor W based on the frequency of triples f The loss function applied to model training yields:

[0037]

[0038] Preferably, the translation-based knowledge graph embedding model is the TransE model;

[0039] For TransE, to facilitate weighting of the loss functions for positive and negative triples, the loss function based on marginal hyperparameters is modified as follows:

[0040]

[0041] Where γ is the marginal parameter, σ is the sigmoid activation function, (h i ',r,t i ') is the negative triplet corresponding to (h,r,t) participating in training. The loss function L of the TransE model is:

[0042]

[0043] The beneficial effects of this invention are as follows:

[0044] A novel knowledge graph embedding training framework, HNSW-KGE, is proposed. Based on the pre-training of the NS-KGE model and a high-quality negative sampling strategy, the negative sampling quality of the model is improved. By weighting the loss function, the adverse effects of long-tail entities and relations on model training are mitigated, thereby improving the model training performance.

[0045] The proposed HNSW-KGE training framework is universal for negative sampling-based KGE models and can be applied to various types of knowledge graph embedding models. Taking the TransE model as an example, the application of this training framework in the model is demonstrated.

[0046] This application proposes a general knowledge graph embedding framework, HNSW-KGE, based on high-quality negative sampling and weighting. First, a non-sampling-based NS-TransE model is pre-trained to quickly learn a relatively high-quality embedding vector for entities and relations in the knowledge graph. Second, a high-quality candidate negative sample set construction strategy is designed. Based on the pre-trained embedding vectors, a high-quality candidate negative sample set is constructed for positive triples, thus providing high-quality negative samples for training the negative sampling-based model. Then, a weighting factor based on triple frequency is used to weight the loss function, mitigating the adverse effects of long-tail entities and relations on model training and improving the model's training performance. Attached Figure Description

[0047] Figure 1 A general training process for knowledge graph embedding models based on negative sampling;

[0048] Figure 2 A schematic diagram of the present invention. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0050] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0051] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language people use in daily life—and thus it has a close relationship with linguistic research. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0052] The technical solutions provided in this application involve technologies such as machine learning and natural language processing in artificial intelligence, which are specifically described and illustrated through the following embodiments.

[0053] Before introducing the embodiments of this application, some terms involved in this application will be explained.

[0054] 1. Traditional negative sampling methods: Geometric transformation-based methods model semantic relationships between entities and relations through geometric transformations. Tensor decomposition-based methods construct scoring functions to measure the probability of triples being true; among them, the widely used DistMult models the scoring function as the inner product of the head and tail entity vectors and the diagonal matrix of the relation. Neural network-based methods utilize the nonlinear transformation characteristics of deep neural networks such as CNN, RNN, and GNN to model complex relationships between entities and relations. Among them, ConvE and ConvKB models use CNN to model the latent semantics between entities and relations; Gradner et al. use RNN to model relation paths to capture longer relation dependencies in the KG; R-GCN, RGHAT, and CompGCN use GNN to capture structured information in the KG.

[0055] 2. Negative Sampling Method for Knowledge Graph Embedding

[0056] Negative sampling methods for embedding knowledge into models can be broadly categorized into three types: random negative sampling, adversarial negative sampling, and additional data-enhanced sampling (ADS). These will be discussed in detail below. Random negative sampling is the most basic and widely used method. However, it often samples overly simplistic negative samples, i.e., easily identifiable negative samples, which fail to provide useful information for the model's optimization and training, leading the model to get stuck in local optima.

[0057] Adversarial negative sampling methods are typically based on the principles of Generative Adversarial Networks (GANs), utilizing the semantic information of samples in the embedding space to select negative samples in order to sample better, harder-to-classify negative samples. However, they suffer from drawbacks such as complex frameworks, unstable training results, and long training times. To address these issues, RotatE proposes a self-adversarial negative sampling strategy that dynamically adjusts the sampling probability of negative samples during model training to sample harder-to-classify negative samples. Negative sampling methods that introduce additional information introduce external information during the negative sampling process to increase the sampling probability of harder-to-classify negative samples.

[0058] However, in general, regardless of the negative sampling method, the different negative samples sampled in different rounds will have a certain impact on the stability of model training, and will inevitably introduce false negative samples, which will also affect the accuracy of model training.

[0059] 3. Knowledge Graph Embedding Model Based on Non-Sampling

[0060] To overcome the high training complexity and prediction bias caused by the over-reliance on negative sample selection in negative sampling, a regularization term is added to the loss function to prevent the model from generating high scores for all triples, thus eliminating the need for negative sampling. However, the calculation of the regularization term still implicitly uses all possible triples in the knowledge graph, including positive and negative triples. The non-sampling knowledge graph embedding framework NS-KGE treats all non-positive triples composed of entities and relations as negative triples for training. This method addresses the shortcomings of negative sampling training by skipping negative sampling. However, this method introduces spurious negative samples and a large number of easily identifiable negative samples, which will affect the accuracy and convergence speed of model training.

[0061] refer to Figure 1 , Figure 2 This application proposes a knowledge graph embedding method, HNSW-KGE (Universal Knowledge Graph Embedding), based on high-quality negative sampling and weighting. K nowledge G raph E mbedding Framework Based On High-quality N egative S ampling And W First, the non-sampling framework NS-KGE model (Zhao Zhang, Weighted Knowledge Graph Embedding. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'23), July 23–27, 2023, China. ACM, New York, NY, USA, 11 pages. https: / / doi.org / 10.1145 / 3539618.3591784) is selected for pre-training. Based on the global features in the dataset, a relatively high-quality initial embedding vector is quickly learned for entities and relations in the knowledge graph, providing support for the subsequent construction of a high-quality candidate negative sample set. Second, a strategy for constructing a high-quality candidate negative sample set is designed. To overcome the problem of both traditional negative sampling-based and non-sampling-based training methods failing to exclude low-quality negative samples from training, this application, based on pre-trained embedding vectors, employs a designed high-quality candidate negative sample set construction strategy. This strategy samples a certain number of negative samples—neither spurious nor easily separable—from all positive triples to form a candidate negative sample set, ensuring high-quality negative samples for subsequent model training. Finally, a loss function weighting strategy based on triple frequency is adopted. Weighting the loss function has proven to be an effective method to mitigate the impact of long-tail entities on training. Therefore, a weighting factor based on triple frequency is used to weight the loss function to adjust for the adverse effects of long-tail entities and relationships on model training. To evaluate the performance of the HNSW-KGE training framework, experiments were conducted on four KGE models—DistMult, SimpleE, ComplEx, and TransE—verifying the effectiveness of the HNSW-KGE framework.

[0062] Table 1 Basic Symbols in this Application

[0063]

[0064]

[0065] The problem under study will be formally described. Table 1 lists the basic symbols used.

[0066] The general training process for knowledge graph embedding models based on negative sampling is as follows: Figure 1 As shown. Given a knowledge graph G = (E, R, S), triples (h, r, t) ∈ S, h, t ∈ E, r ∈ R.

[0067] First, negative sampling. A common approach to constructing the candidate negative sample set needed for model training is to randomly replace the head or tail entity of the triple (h,r,t) with entities from the entity set E to obtain the corresponding negative triple (h',r,t'), thus obtaining the candidate negative sample set required for model training. Generally, the candidate negative sample set S′ can be formally described as... Based on certain strategies, such as random negative sampling or difficult-to-distinguish negative sample sampling, a portion of negative triples are sampled from the candidate negative sample set to participate in training.

[0068] Then, the loss is calculated. Based on the model's scoring function, the scoring function values ​​f(h,r,t) and f(h',r,t') for the positive triple (h,r,t) and its corresponding negative triple (h',r,t') are calculated respectively. The loss of the triple is then calculated based on the scoring function values, thus obtaining the model's loss function L. The model's loss functions are broadly classified into two categories: loss functions based on marginal hyperparameters and loss functions based on cross-entropy. The loss function based on marginal hyperparameters is defined as follows:

[0069]

[0070] γ is a marginal parameter. Translation-based models, such as TransE and TransH, mostly employ loss functions based on marginal hyperparameters. The loss function based on cross-entropy is defined as:

[0071]

[0072] Semantic matching-based models, such as DistMult, SimpleE, and ComplEx, mostly employ a loss function based on cross-entropy.

[0073] Finally, the training is optimized. Optimization algorithms such as Stochastic Gradient Descent (SGD) and Adaptive Gradient Descent (AdaGrad) are used to optimize the loss function L to obtain the optimal parameters of the model, including the embedding vectors h,t and r of entities h,t and relation r.

[0074] The following sections will detail the HNSW-KGE training framework of this application. First, we will introduce the NS-KGE model used for pre-training. Next, we will describe the strategy for constructing a high-quality candidate negative sample set. Then, we will introduce the weighting strategy based on the loss function of triple frequencies. Finally, we will discuss the application of the training framework to several different knowledge graph embedding models.

[0075] 1. Pre-training based on NS-KGE

[0076] To provide a basis for constructing a high-quality candidate negative sample set, we select the non-sampling knowledge graph embedding framework NS-KGE as a pre-trained model, thereby quickly obtaining an initial knowledge graph embedding vector representation. The NS-KGE model will be introduced in detail below.

[0077] The non-sampled knowledge graph embedding framework NS-KGE can be applied to knowledge graph embedding models based on squared loss functions or models where the loss function can be converted to squared loss. Compared with traditional knowledge graph embedding models, NS-KGE has relatively high training efficiency and also improves training accuracy. Its basic idea is based on the closed-world hypothesis, treating all triples consisting of entities and relations in the dataset but not in the dataset as negative samples for model training. It uses a least-squares loss function between the true label values ​​of positive and negative triples and the predicted values ​​based on a scoring function to define its optimized training loss function. To address the problem of excessive model training complexity caused by massive amounts of data, the model reduces the complexity of the loss function through mathematical derivation. The NS-KGE framework uses a least-squares loss function to define its optimized training loss function, i.e.

[0078]

[0079] Among them, C hrt f represents the weight of the corresponding triple (h,r,t). r (h,t) represents the true label value of the triple (h,r,t). For a positive triple (h,r,t), define f r (h,t)=1, for a negative triple (h,r,t), define f r (h,t)=0, This represents the predicted label value based on the scoring function.

[0080] NS-DistMult is a sub-model obtained by applying the NS-KGE model framework to the DistMult model. The prediction function of the DistMult model is defined as follows:

[0081]

[0082] To reduce the time and space complexity of the model, after mathematically deriving formula (4) in conjunction with formula (3), the least squares-based loss function of NS-DistMult is obtained as follows:

[0083]

[0084] in,

[0085]

[0086] Similarly, NS-TransE is a sub-model obtained by applying the NS-KGE framework to the TransE model. Based on the TransE scoring function:

[0087] f(h,r,t)=||h+rt|| (7)

[0088] The transformation yields the prediction function of the TransE model. for:

[0089]

[0090] After mathematical derivation and simplification, the NS-TransE loss function based on least squares is obtained as follows:

[0091]

[0092] Among them, L P The calculation is shown in formula (6).

[0093] In terms of training performance, NS-KGE achieves better performance than the original baseline model in most cases. For example, on the TCGA dataset, Hit@10 performance is improved from 0.316 for TransE to 0.447 for the NS-TransE model. This is because the original baseline model uses a negative sampling-based method that only randomly samples a portion of the negative samples in the dataset, potentially ignoring some important negative samples, while the NS-KGE framework uses all negative samples in the dataset for training. Regarding training efficiency, NS-KGE also shows a significant improvement in training speed compared to the original baseline model. For example, compared to TransE, NS-TransE's training speed is approximately 10 times faster on the TCGA dataset. This is because the original negative sampling-based model spends most of its training time on negative sampling, while the NS-KGE framework skips the negative sampling process.

[0094] However, NS-KGE has certain limitations. First, its scalability is limited. The model uses mathematical derivation to reduce the complexity of the loss function, but the derivation process is too complex, making it applicable only to a limited number of models and restricting its application scope. Second, it easily introduces spurious negative samples. The model treats all unobserved triples in the dataset as negative samples for training, while many fact triples, due to data sparsity, are not included in the dataset and are treated as negative samples, affecting the model's training accuracy. Third, its non-sampling approach introduces a large number of easily identifiable negative samples into the training, affecting the model's convergence speed and accuracy. Therefore, we only use NS-KGE for pre-training, serving as the starting point for all model training to quickly obtain a relatively high-quality initial embedding vector, providing a reference for constructing a high-quality candidate negative sample set. Furthermore, considering both training effect and efficiency, we choose the NS-TransE sub-model from NS-KGE as the pre-training model.

[0095] 2. Construction of a high-quality candidate negative sample set

[0096] To provide high-quality negative samples for negative sampling-based training, after obtaining an initial, relatively high-quality embedding vector for entities and relations in the dataset, we will next construct a candidate negative sample set for all positive triples according to our designed high-quality candidate negative sample set construction strategy. This candidate negative sample library will be used to sample high-quality negative samples for model training, thereby improving the performance of model training. The high-quality candidate negative sample set construction strategy will be described in detail below.

[0097] The candidate negative sample set includes the candidate head entity set (i.e., the candidate negative sample set obtained by replacing the head entities) and the candidate tail entity set (i.e., the candidate negative sample set obtained by replacing the tail entities). The sampling size for each triplet is N = N-1. h +N t The candidate negative sample set, wherein the size of the candidate head entity set is N. h The size of the candidate tail entity set is N t Set N h N t When considering triples, we will take into account the one-to-many or many-to-one relation types they correspond to. Specifically, for triples of the one-to-many relation type, due to the uniqueness of the head entity, negative samples constructed by replacing the head entity are unlikely to be spurious negative samples. Therefore, we can consider setting more candidate head entities, and thus, the size of the candidate head entity set should be larger than the size of the candidate head and tail entity set. For triples of the many-to-one relation type, we can consider setting more candidate tail entities, and the size of the candidate tail entity set should be larger than the size of the candidate head entity set. We define the size of the candidate head entity set as N. h The size of the candidate tail entity set is N t,Right now

[0098]

[0099] Among them, t rh h refers to the average number of tail entities connected to the head entity of all triples with relation r in a knowledge graph. rt It refers to the average number of head entities connected to the tail entity in all triples of relation r.

[0100] Next, taking TransE as an example, we will introduce the construction process of the candidate header entity set of triples (h,r,t) in detail:

[0101] Step 0, Initialization. Construct a data structure as {(r,t),{(h1,f(h'1,r,t)),...(h' Nh ,f(h Nh ,r,t))}}, with a size of N h The array, h1,...,h Nh Initialize it to empty, f(h',r,t) represents its corresponding scoring function value. The scoring function of the TransE model is shown in formula (7). Here it is initialized to a large value, such as 100.

[0102] Step 1: Construct candidate negative triples. Randomly select an entity h' from the entity set and replace h in (h,r,t) to obtain the negative triple (h',r,t).

[0103] Step two: Determine if (h', r, t) exists in the dataset. If it does, restart step one. Otherwise, continue to step three. This step is to exclude the candidate negative triplet from being a known positive sample.

[0104] Step 3: Determine if (h', r, t) is a false negative sample.

[0105] f(h',r,t) <f(h,r,t)+u (12)

[0106] Check if the condition is true. If it is true, then (h',r,t) may be a false negative sample, and restart step one. Otherwise, continue to step four. Here, u≥0 is a hyperparameter. This step is to exclude the candidate negative sample from being a false negative sample.

[0107] Step 4: Filter candidate head entities. Iteratively check if the score function value f(h',r,t) of (h',r,t) is smaller than the score function values ​​of all negative triples constructed from candidate head entities. If it is smaller than a certain negative triple (h',r,t) in the candidate head entity set... i f(h',r,t), i.e., f(h',r,t). <f(h iThe expression (h',r,t) indicates that (h',r,t) is greater than (h). i If the negative triples (h', r, t) are more difficult to distinguish, then replace (h', f(h', r, t)) with (h', f(h', r, t)). i ,f(h i Otherwise, restart step one. This step is to filter out N with the highest degree of difficulty in differentiation. h The entities are ranked N from smallest to largest according to their scoring function values. h The entity, will this N h One entity is selected as a candidate head entity.

[0108] Step 5, Output. After looping through all entities in the entity set, output the triple (h, r, t) of size N. h The candidate head entity set.

[0109] The method for constructing the candidate tail entity set of the triple (h,r,t) is similar. The final triple (h,r,t) has a size of N = N h +N t The process involves constructing a high-quality candidate negative sample set based on pre-training. This process constructs a corresponding candidate negative sample set for each triple in the dataset, resulting in a high-quality candidate negative sample set for the dataset. Furthermore, to avoid low-quality candidate negative sample sets due to performance limitations of the pre-trained NS-TransE model, the negative sample set is dynamically updated periodically during training based on the knowledge graph embedding vectors obtained at that time, following the steps described above. This ensures high-quality negative samples for the subsequent loss function-based optimization training.

[0110] The specific process for constructing a high-quality candidate negative sample set based on pre-training is as follows:

[0111] Input the knowledge graph dataset S = {(h,r,t)};

[0112] The dataset was pre-trained using the NS-KGE non-sampling training framework to obtain initial embedding vectors for entities and relations;

[0113] A triple (h,r,t) is sampled from the training set;

[0114] Execute 1) and 2) respectively, where 1) includes: selecting an entity h' from the entity set to replace h in (h,r,t) to obtain (h',r,t), and calculating its scoring function value f(h',r,t) based on the pre-trained embedding vector;

[0115] Determine if (h',r,t) is not in the dataset. If not, check if f(h',r,t) > f(h,r,t) + u. If so, check if the function value of f(h',r,t) is among the top N in ascending order of the function values ​​of the negative triples obtained from all replaced head entities. h If yes, then h' will be included in the candidate head entity set of (h,r,t); otherwise, step 1 will be repeated.

[0116] 2) Includes: Select an entity t' from the entity set to replace t in (h,r,t) to obtain (h,r,t'), and calculate its scoring function value f(h,r,t') based on the pre-trained embedding vector;

[0117] Determine if (h,r,t') is not in the dataset. If not, check if f(h,r,t') > f(h,r,t) + u. If so, check if the function value of f(h,r,t') is among the top N in ascending order of the function values ​​of the negative triples obtained from all replaced head entities. t If yes, then t' will be included in the candidate head entity set of (h,r,t); otherwise, step 2 will be repeated.

[0118] Check if the number of entities in the candidate head entity set and the tail entity set has reached Nh and Nt respectively. Otherwise, continue to execute 1) and 2). If yes, check if all triples in the training set have completed the construction of the candidate negative sample set. If yes, output the candidate negative sample S'(h,r,t) of all triples in the training set.

[0119] 3. Weighted loss function based on triple frequency

[0120] The loss function L for model training is defined as:

[0121]

[0122] Where (h,r,t) and (h',r,t') represent positive triples and negative triples, respectively, and S and S' represent the set of positive triples and the set of negative triples, respectively.

[0123] To mitigate the impact of long-tail entities and relations on model training, it's necessary to increase attention to them, thereby improving the training performance of knowledge graph embedding models. Therefore, we add a triple frequency-based weighting factor to the loss function. Based on the frequency of entities and relations appearing in the dataset within the training triples, we assign different weights to the triples in the loss function. Specifically, we assign lower weights to more frequent entities and relations, and higher weights to less frequent ones. This increases the training focus on long-tail entities and relations, aiming to learn better embeddings. We define the triple frequency-based weighting factor W.f for:

[0124]

[0125] Among them, f h ,f t ,f r Let h, t, and r represent the frequencies of entities h, t, and relation r in the dataset, respectively. Let w represent the median frequency of entities and relations in the dataset, respectively. According to the Pareto principle, we define the median frequency as the frequency of the top 20% of entities and relations ranked from highest to lowest frequency. In formula (15), w... h In the calculation formula, the denominator is f. h +1 is added to avoid situations where some entities have a frequency of 0 in the dataset when calculating the frequency weights of entities in negative triplets. t ,w r Similar situations exist.

[0126] Weighting factor W based on triple frequency f This can be understood as follows: for low-frequency triples, that is, when the entities and relations in the triples are low-frequency entities and relations that appear less frequently than the median frequency, according to formula (15), the weight of the low-frequency entity or relation will be greater than 1. Then, according to formula (14), the W in the loss function of the low-frequency triples... f The weight value will be greater than 1, thus increasing the attention to low-frequency entities and relations during training; similarly, for high-frequency triples, that is, triples containing high-frequency entities and relations that appear more frequently than the median frequency, according to formula (15), the weight of the high-frequency entity or relation will be less than 1, and according to formula (14), the W in the loss function of the high-frequency triple will be less than 1. f The weights will be less than 1, thus reducing the focus on these high-frequency entities and relationships during training. Here, we can use the analogy of student learning and exams. For knowledge points that frequently appear in practice books, similar to high-frequency triples, focusing on these points won't significantly improve grades due to their high practice frequency; less time can be spent on them. Conversely, for knowledge points that don't frequently appear in practice books, similar to low-frequency triples, more attention is needed due to less practice. Strengthening focus on these knowledge points is the key to improving grades.

[0127] The weighting factor W based on the frequency of triples f Applying this to formula (13), we get:

[0128]

[0129] Among them, W f The calculation is shown in formula (14).

[0130] 4. Application of HNSW-KGE in different models

[0131] In this section, we will use four typical knowledge graph embedding models—TransE, DistMult, SimplE, and ComplEx—as examples to demonstrate the application of our proposed HNSW-KGE training framework on different knowledge graph embedding models.

[0132] For TransE, the scoring function f(h,r,t) of the triple is defined as shown in Equation (7).

[0133] The TransE model is a translation-based model, and most of these models use the marginal hyperparameter-based loss function shown in formula (1). To facilitate weighting the loss functions for positive and negative triples, the marginal hyperparameter-based loss function is modified as follows:

[0134]

[0135] Where γ is the marginal parameter, σ is the sigmoid activation function, (h i ',r,t i ') is the negative triplet corresponding to (h,r,t) that participated in the training. Combining formula (16) with weighted loss function, the loss function L for this type of model can be obtained as:

[0136]

[0137] The overall flow of the HNSW-KGE framework proposed in this application is as follows: Figure 2 As shown. First, based on the pre-trained embedding vectors and the designed high-quality candidate negative sample set construction strategy, a high-quality candidate negative sample set is sampled for all positive triples; second, according to different models, the scoring function values ​​of the triples and their corresponding negative triples are calculated; then, the relevant frequency data in the dataset are statistically analyzed and substituted into formulas (14) and (15) to obtain the frequency-based weight W of each triple. f Then, based on the scoring function value and W f The loss L of the triplet set is calculated, where the loss of the translation-based model is calculated according to formula (18), and the loss of the semantic matching-based model is calculated according to formula (22), thus obtaining the final loss function of the training framework. Finally, stochastic gradient descent (SGD) is used to optimize the loss function of the training model and learn the optimal parameters of the model, including the optimal embedding vectors h, r, t. Note that during the training process, the candidate negative sample set will be dynamically updated at a certain frequency according to the knowledge graph embedding vectors obtained at that time, so as to avoid the low quality of candidate negative samples due to the performance limitations of the pre-trained model itself.

[0138] To avoid overfitting, we enforce a constraint during model training that the second norm of all entities and relations in the dataset is less than or equal to 1, i.e., ||h||2≤1, ||r||2≤1, and ||t||2≤1.

[0139] We evaluate the performance of the HNSW-KGE framework using a link prediction task. This section first introduces the dataset used in the experiments, then the experimental design, followed by the evaluation metrics, then the comparison model, and finally the parameter settings.

[0140] This application will use two benchmark datasets, TCGA and ClinicalTrials.gov, for experiments to evaluate the training performance of the HNSW-KGE framework. TCGA (The Cancer Genome Atlas) provides genomic, transcriptomic, epigenetic, and clinical data for various cancer types; ClinicalTrials.gov is a global registry of clinical trials maintained by the U.S. National Library of Medicine, containing information on ongoing and completed clinical trials. Basic statistical information for these two datasets is shown in Table 2.

[0141] Table 2 Statistical information of the dataset

[0142] Dataset TCGA ClinicalTrials.gov #entity 14,541 40,943 #relation 237 11 #Triplet 292,581 89,969

[0143] Following the general dataset setup method for link prediction tasks, the triplet set is divided into training, validation, and test sets, accounting for approximately 85%, 5%, and 10%, respectively. The specific experimental steps for link prediction are as follows:

[0144] First, the optimal parameters of the model are obtained by training the training set and the validation set, including the optimal embedding vectors for all entities and relations.

[0145] Then, a link prediction task is performed on the test set to evaluate the model's ability to predict the missing head or tail entity in the triplet, i.e., the ability to predict t given (h,r) or h given (r,t). Taking the evaluation model's prediction of the tail entity as an example, for the test triplet (h,r,t), the tail entity t is replaced by each entity x in the entity set to obtain a set of candidate triplets (h,r,x). According to the model's definition of the scoring function f(h,r,x), for example, the TransE model uses formula (7), the score of this set of candidate triplets is calculated and sorted in ascending order to obtain the ranking of the test triplet (h,r,t) among all candidate triplets, denoted as rank(t). Similarly, to evaluate the model's ability to predict the head entity, the ranking of the entity test triplet (h,r,t) among all candidate triplets (x,r,t) is obtained, denoted as rank(h).

[0146] Finally, based on the values ​​of rank(h) and rank(t) of the test triple (h,r,t), the values ​​of each evaluation index in the test set are calculated according to the calculation formula of the evaluation index.

[0147] In addition, a "Filter" setting was employed. Specifically, before ranking each test triplet, triples present in the training, validation, and test sets were removed from the candidate triplet set. This is because such candidate triplets are actually positive triplets, and it is reasonable for their ranking based on the scoring function to be higher than that of the original test triplets. Therefore, it is necessary to remove these candidate triplets before obtaining the ranking of each test triplet, thus eliminating the influence of "interference" factors.

[0148] In the link prediction task, the following evaluation metrics are used to assess the quality of the knowledge graph embeddings learned by the model, in order to evaluate the training performance of the model.

[0149] (1) MR value, which is the average of the rankings of all test triples, is defined as follows:

[0150]

[0151] Where S t Represents the test triple set, |S t | indicates the size of the test triple set.

[0152] (2) MRR value, which is the average of the reciprocals of the rankings of all test triples, is defined as follows:

[0153]

[0154] (3) Hits@K, which is the ratio of the number of test triples with a ranking no greater than K to the total number of test triples. This experiment uses indicators such as Hits@1, Hits@3 and Hits@10.

[0155] A larger MRR and Hits@N values, and a smaller MR value, indicate better model training performance; conversely, a smaller MRR and Hits@N values ​​indicate worse model performance.

[0156] This application categorizes the comparative models into three types:

[0157] (1) The original baseline model, namely the knowledge graph embedding model based on traditional negative sampling. The TransE model was selected as the comparison model. The experimental results of these models in link prediction were obtained directly from published papers.

[0158] TransE: A translation-based model that models relation vectors as a "translation" operation between head and tail entity vectors embedded in a vector space, i.e., a translation transformation. It measures the authenticity of triples by measuring the difference between the head and tail entity vectors after translation.

[0159] (2) Models using the NS-KGE framework, i.e., versions of the original baseline models trained using the NS-KGE framework, including four models: NS-TransE, NS-DisMult, NS-SimplE, and NS-ComplEx. The experimental results of these models are directly derived from the results of published papers.

[0160] (3) Models using the HNSW-KGE framework, i.e., versions of the original baseline models trained using the HNSW-KGE framework, including the HNSW-TransE model.

[0161] The learning rate λ ∈ {0.0001, 0.001, 0.01, 0.1}, embedding dimension d ∈ {100, 300, 500, 1000}, hyperparameter u ∈ {0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3}, hyperparameter N ∈ {10, 30, 50, 80, 100}, and γ ∈ {0.1, 0.2, 0.5, 1, 2, 4, 8, 10} are used. The batch size is 128. The model is trained iteratively for 1000 epochs on each dataset. The optimal parameters are determined by Hits@10 on the validation set. The optimal parameter configurations for the model on both datasets are shown in Table 3.

[0162] Table 3 Optimal Hyperparameter Settings

[0163]

[0164] To verify the effectiveness of the model, we first introduce and analyze the comparative experimental results of HNSW-KGE with the contrasting model on two public datasets.

[0165] Table 4 shows the results of the comparative experiment between the HNSW-KGE model and two contrasting models on the link prediction task. In Table 4, NS-X refers to the version of model X using the NS-KGE framework, and HNSW-X refers to the version of model X using the HNSW-KGE framework. ↑ indicates that a larger evaluation index value is better, and ↓ indicates that a smaller evaluation index value is better. In the table, bold data represents the best experimental results among the contrasting models, and underlined data represents the second best experimental results.

[0166] Table 4. Link Prediction Experiment Results

[0167]

[0168]

[0169] First, based on the table as a whole, among the three comparative models, the HNSW-KGE model proposed in this application achieved the best or second-best evaluation results in most cases, and the experimental results verified the effectiveness of the HNSW-KGE framework.

[0170] Secondly, compared to the original TransE baseline model, the model applying the HNSW-KGE framework showed an average performance improvement of 58.8% on the evaluation metric MRR, 3.2% on MR, 47.2% on Hit@10, 75.1% on Hit@3, and 285.6% on Hit@1 on both datasets. This indicates that the candidate negative sample set constructed by our proposed HNSW-KGE framework using the initial embedding vectors obtained from pre-training and a high-quality candidate negative sample set construction strategy can provide relatively high-quality negative samples during model training, thereby improving the training effect of negative sampling-based models. This results in the model applying the HNSW-KGE framework outperforming these traditional negative sampling-based knowledge graph embedding models.

[0171] Furthermore, compared to the knowledge graph embedding model NS-TransE applied to the NS-KGE framework, the model applying the HNSW-KGE framework showed an average performance improvement of 16.9% on the evaluation metric MRR, 9.9% on MR, 9.2% on Hit@10, 16.6% on Hit@3, and 90.1% on Hit@1 in experiments on both datasets. This indicates that applying the HNSW-KGE framework of this application can better improve the training performance of the model compared to applying the NS-KGE framework. This is because training models based on the NS-KGE framework treats all unobserved samples in the dataset as negative samples, inevitably introducing a large number of false negative samples and easily separable negative samples far from the decision boundary. In contrast, the HNSW-KGE framework proposed in this application uses a pre-trained, relatively high-quality entity and relation embedding vector, along with a high-quality candidate negative sample set construction strategy, to construct a candidate negative sample set. This effectively excludes false negative samples and a large number of easily separable negative samples from the training process, allowing the model to achieve better training results based on these relatively high-quality negative samples. Furthermore, the HNSW-KGE framework incorporates a triple frequency-based weighting factor into the loss function, giving more attention to long-tail entities and relations in the dataset during training, which can mitigate their adverse effects on model training. Therefore, experimental results demonstrate the effectiveness of the high-quality candidate negative sample set construction strategy and the triple frequency-based weighting strategy in the HNSW-KGE framework for model training.

[0172] Then, compared to the original baseline model and the NS-KGE framework, the model applying the HNSW-KGE framework consistently outperformed the MRR evaluation metric, but performed relatively poorly on MR, especially on the CLINICALTRIALS.GOV dataset. This is because MR differs from MRR in that the former is more sensitive to prediction errors; if some real entities rank low in their predicted candidate entity list, it will have a significant impact on MR, while MRR, due to the reciprocal operation, has a smaller impact. The HNSW-KGE framework performs better on MRR, indicating that it can rank real entities higher in most cases. The relatively poor MR evaluation metric on the CLINICALTRIALS.GOV dataset is because the CLINICALTRIALS.GOV dataset contains a larger number of entities than TCGA, making it more prone to real entities being ranked lower.

[0173] Finally, the experimental results show that the HNSW-KGE framework can improve the training performance of negative sampling-based models for translation-based models (TransE), demonstrating the framework's versatility for training negative sampling-based models and its potential for application to other knowledge graph embedding models.

[0174] In summary, the comparative experimental results on the link prediction task demonstrate that the model applying the HNSW-KGE framework performs well in this task. Further analysis reveals that the HNSW-KGE framework, based on pre-trained embedding vectors and a high-quality candidate negative sample set construction method, obtains relatively higher-quality negative samples for training. The loss function weighting strategy mitigates the adverse effects of long-tail entities and relationships on model training, thus enabling the training of better embeddings for the knowledge graph. Therefore, the model applying this framework benefits from the link prediction task experiments.

[0175] The beneficial effects of this invention are as follows:

[0176] A novel knowledge graph embedding training framework, HNSW-KGE, is proposed. Based on the pre-training of the NS-KGE model and a high-quality negative sampling strategy, the negative sampling quality of the model is improved. By weighting the loss function, the adverse effects of long-tail entities and relations on model training are mitigated, thereby improving the model training performance.

[0177] The proposed HNSW-KGE training framework is universal for negative sampling-based KGE models and can be applied to various types of knowledge graph embedding models. Taking the TransE model as an example, the application of this training framework in the model is demonstrated.

[0178] This application proposes a general knowledge graph embedding framework, HNSW-KGE, based on high-quality negative sampling and weighting. First, a non-sampling-based NS-TransE model is pre-trained to quickly learn a relatively high-quality embedding vector for entities and relations in the knowledge graph. Second, a high-quality candidate negative sample set construction strategy is designed. Based on the pre-trained embedding vectors, a high-quality candidate negative sample set is constructed for positive triples, thus providing high-quality negative samples for training the negative sampling-based model. Then, a weighting factor based on triple frequency is used to weight the loss function, mitigating the adverse effects of long-tail entities and relations on model training and improving the model's training performance.

[0179] As used herein, the term "preferred" is meant as an example, illustration, or illustration. Any aspect or design described herein as "preferred" need not be construed as being more advantageous than other aspects or designs. Rather, the use of the term "preferred" is intended to present the concept in a specific manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusionary "or." That is, unless otherwise specified or clear from the context, "X uses A or B" naturally includes either of the permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.

[0180] Furthermore, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the aforementioned components (e.g., elements, etc.), the terminology used to describe such components is intended to correspond to any component (unless otherwise indicated) that performs the specified function of said component (e.g., is functionally equivalent to it), even if structurally not equivalent to the disclosed structure performing the functions in the exemplary implementations of this disclosure shown herein. Moreover, although specific features of this disclosure have been disclosed with respect to only one of several implementations, such features may be combined with one or more features of other implementations that may be desirable and advantageous for a given or particular application. Furthermore, with regard to the use of the terms “comprising,” “having,” “containing,” or variations thereof in the Detailed Description or claims, such terms are intended to be included in a manner similar to the term “including.”

[0181] The functional units in this invention embodiment can be integrated into a processing module, or each unit can exist physically separately, or multiple units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. The aforementioned devices or systems can execute the storage methods in the corresponding method embodiments.

[0182] In summary, the above embodiments are one implementation of the present invention, but the implementation of the present invention is not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made that deviate from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A knowledge graph embedding method based on negative sampling and weighting, characterized in that, Includes the following steps: Obtain a medical dataset, in which entities include drugs, diseases, and symptoms; Based on the pre-trained embedding vectors and the designed high-quality candidate negative sample set construction strategy, a high-quality candidate negative sample set is sampled for all positive triples. Based on a translation-based knowledge graph embedding model, calculate the scoring function values ​​of triples and their corresponding negative triples; Collect relevant frequency data from the dataset and calculate the frequency-based weight W for each triple. f ; Based on the scoring function value and W f Calculate the loss L of the triple set to obtain the final loss function of the training framework; Stochastic gradient descent (SGD) is used to optimize the loss function of the training model and learn the optimal parameters of the translation-based knowledge graph embedding model, including the optimal embedding vectors h, r, t, where h, t, r are the embedding vectors of the head entity, tail entity, and relation in the triple, respectively. Output the potential association between drugs and diseases and symptoms; The loss function L for model training is defined as follows: Where (h, r, t) and (h', r, t') represent positive triples and negative triples, respectively, and S and S' represent the set of positive triples and the set of negative triples, respectively; A triplet frequency-based weighting factor is added to the loss function. Different weights are assigned to the triplet loss function based on the frequency of entities and relations appearing in the dataset within the training triplet. Specifically, lower weights are assigned to more frequent entities and relations, while higher weights are assigned to less frequent entities and relations. This increases the training focus on long-tail entities and relations, aiming to learn better embeddings. A triplet frequency-based weighting factor W is defined. f for: Among them, f h , f t , f r Let represent the frequencies of entities h, t, and relation r in the dataset, respectively. , Let w represent the median frequency of entities and relations in the dataset, respectively. Following the Pareto principle (80 / 20 rule), the median frequency is defined as the frequency of the top 20% of entities and relations ranked by their frequency of occurrence, from highest to lowest. h w t w r These are the frequency weighting factors for entities h, t, and relation r, respectively. Weighting factor W based on triple frequency f For low-frequency triples, where the entities and relations in the triples are low-frequency entities and relations that appear less than the median frequency, and the weight of the low-frequency entity or relation is greater than 1, the W in the loss function of the low-frequency triples... f Weights greater than 1 are used to increase attention to low-frequency entities and relations during training; similarly, for high-frequency triples (triples containing entities and relations that appear more frequently than the median), the weight of the high-frequency entity or relation is less than 1, and the weight of W in the loss function of the high-frequency triple is less than 1. f The weights will be less than 1, thus reducing the focus on these high-frequency entities and relationships during training; The weighting factor W based on the frequency of triples f The loss function applied to model training yields: 。 2. The knowledge graph embedding method based on negative sampling and weighting according to claim 1, characterized in that, During training, the candidate negative sample set is dynamically updated at a certain frequency based on the knowledge graph embedding vector obtained at that time, thereby improving the quality of the candidate negative samples.

3. The knowledge graph embedding method based on negative sampling and weighting according to claim 2, characterized in that, To avoid overfitting, during model training, the second norm of all entities and relations in the dataset is forced to be less than or equal to 1, i.e., ||h||2 ≤ 1, ||r||2 ≤ 1, ||t||2 ≤ 1.

4. The knowledge graph embedding method based on negative sampling and weighting according to claim 3, characterized in that, The non-sampled knowledge graph embedding framework NS-KGE is selected as the pre-trained model to quickly obtain an initial knowledge graph embedding vector representation.

5. The knowledge graph embedding method based on negative sampling and weighting according to claim 4, characterized in that, The high-quality candidate negative sample set construction strategy constructs a candidate negative sample set for all positive triples as a candidate library of negative samples, so as to sample high-quality negative samples from it to participate in model training, thereby improving the performance of model training. The specific strategies for constructing high-quality candidate negative sample sets include: The candidate negative sample set includes the candidate head entity set, which is the candidate negative sample set obtained by replacing the head entities, and the candidate tail entity set, which is the candidate negative sample set obtained by replacing the tail entities. The sampling size for each triplet is N = N h +N t The candidate negative sample set, wherein the size of the candidate head entity set is N. h The size of the candidate tail entity set is N t ; Set N h , N t When considering the one-to-many or many-to-one relation types corresponding to the triples, specifically, for triples of the one-to-many relation type, more candidate head entities are set, thus the size of the candidate head entity set is larger than the size of the candidate head and tail entity sets; while for triples of the many-to-one relation type, more candidate tail entities are set, thus the size of the candidate tail entity set is larger than the size of the candidate head entity set. The size of the candidate head entity set is defined as N. h The size of the candidate tail entity set is N t ,Right now Among them, t rh h refers to the average number of tail entities connected to the head entity of all triples with relation r in a knowledge graph. rt This refers to the average number of head entities connected to the tail entity in all triples of relation r; Construct candidate head entity sets and candidate tail entity sets for the triple (h, r, t), ultimately obtaining a triple (h, r, t) of size N = N h +N t The candidate negative sample set; A high-quality candidate negative sample set construction process based on pre-training is executed. All triples in the dataset are used to construct their corresponding candidate negative sample sets based on this process, resulting in a high-quality candidate negative sample set for the dataset. In addition, to avoid low-quality candidate negative sample sets due to the performance limitations of the pre-trained model itself, the negative sample set is dynamically updated at a certain frequency during training based on the knowledge graph embedding vector obtained at that time, thereby providing a high-quality negative sample guarantee for the next step of optimization training based on the loss function.

6. The knowledge graph embedding method based on negative sampling and weighting according to claim 5, characterized in that, The translation-based knowledge graph embedding model is the TransE model; For TransE, to facilitate weighting of the loss functions for positive and negative triples, the loss function based on marginal hyperparameters is modified as follows: Where γ is the marginal parameter, σ is the sigmoid activation function, (h i ', r, t i ') is the negative triplet corresponding to (h, r, t) participating in training. The loss function L of the TransE model is: 。

Images