Link prediction method based on knowledge graph

Abstract

The invention discloses a link prediction method based on a knowledge graph. The method comprises the following steps: acquiring a medicine data set; based on an embedded vector obtained by pre-training and a designed high-quality candidate negative sample set construction strategy, sampling a high-quality candidate negative sample set for all positive triads; according to different knowledge graph embedding models, calculating score function values of the triad and the corresponding negative triad; carrying out statistics on related frequency data in the data set, and calculating a frequency-based weight of each triple; according to the scoring function value and the weight, calculating the loss L of the triple set to obtain a final loss function of the training framework; optimizing a loss function of the training model by adopting a stochastic gradient descent method; and outputting a potential association relationship between the medicine and the disease and the symptom. According to the method, the negative sampling quality of the model is improved, the adverse effects of long-tail entities and relationships on model training are relieved by weighting the loss function, and the model training performance is improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical information technology, and in particular relates to a link prediction method based on knowledge graph. Background Art

[0002] Link prediction is an important problem in the field of network science and data mining, which aims to predict links that do not yet exist in the network or new links that may appear in the future. Link prediction is of great significance in a variety of application scenarios. By predicting potential drug-target interactions, drug side effects, and drug repositioning (new uses of old drugs), researchers can accelerate the drug discovery and development process and reduce R&D costs and risks. For example: by predicting which drug molecules may interact with which target proteins, potential candidate drugs can be screened out; or by predicting potential side effects, the safety of drugs can be evaluated in advance to reduce the risks of clinical trials; or known drugs can be used for new therapeutic purposes, and by predicting potential drug-disease associations, new uses of existing drugs can be discovered, saving time and costs in the development of new drugs.

[0003] Currently, knowledge graph (KG) is an important carrier for storing and processing structured data, and as an advanced knowledge interconnection solution, it provides a wide range of intelligent system power for many applications and has received extensive attention from academia and industry. Knowledge graph embedding (KGE) embeds entities and relationships in KG into a continuous vector space, which can achieve efficient calculation of semantic information while retaining the inherent structure of the knowledge graph, thus making the large-scale application of knowledge graph a reality. Therefore, in recent years, knowledge graph embedding has been widely used in the field of biomedicine. Knowledge graph embedding helps to discover the potential association between drugs and diseases, genes, and symptoms, and accelerate the development of new drugs and the design of personalized treatment plans.

[0004] At present, most knowledge graph embedding models are trained using the traditional negative sampling-based method. First, negative samples are sampled based on a certain strategy according to the positive samples in the data set, and then the loss functions of the positive and negative samples are calculated according to the model's scoring function for optimization training, thereby obtaining the embedding vector representation of entities and relationships in the knowledge graph. Most traditional knowledge graph embedding models rely on negative sampling, that is, a candidate negative sample set relative to the positive sample set is constructed based on a certain strategy, and a part of the negative samples are selected from the candidate negative sample set to participate in the training process. However, knowledge graphs face the dilemma of data sparsity. The traditional model training method based on negative sampling further aggravates the sparsity of the data actually involved in the training because only some negative samples are sampled for training, which makes the model training insufficient due to insufficient training samples and may ignore some important negative samples, thus affecting the performance of model training.

[0005] In recent years, researchers have proposed various KGE models, among which classic models include TransE, DistMult, SimplE and ComplEx. In order to better mine useful information in the data set to obtain better knowledge embedding, most knowledge graph embedding models are trained using the traditional negative sampling method. First, negative samples are sampled based on a certain strategy according to the positive samples in the data set, and then the loss function of the positive and negative samples is calculated according to the model's scoring function for optimization training, so as to obtain the embedding vector representation of entities and relationships in the knowledge graph. Most traditional knowledge graph embedding models rely on negative sampling, that is, a candidate negative sample set relative to the positive sample set is constructed based on a certain strategy, and a part of the negative samples are selected from the candidate negative sample set to participate in the training process. Common negative sampling strategies include random negative sampling, adversarial negative sampling, and negative sampling with additional information. However, knowledge graphs face the dilemma of data sparsity. The traditional model training method based on negative sampling further aggravates the sparsity of the data actually involved in the training because only some negative samples are sampled for training, which makes the model training insufficient due to insufficient training samples and may ignore some important negative samples, thus affecting the performance of model training.

[0006] The researchers explored a non-sampling-based training method, which is to use all non-positive triplets composed of entities and relationships in the knowledge graph as negative samples for training, and solve the problems of the negative sampling-based training method by skipping the negative sampling process. However, this training method has several shortcomings. First, all unobserved triplets are used as negative samples for training, which is easy to introduce false negative samples, that is, fact triplets that are not included in the knowledge graph dataset due to data sparsity are used as negative samples for training, which will affect the accuracy of model training; second, this training method will introduce a large number of easy-to-separate negative samples far away from the model decision boundary to participate in the training. The study of the distribution of the scoring function values ​​of negative samples shows that most negative triplets are easy-to-separate negative samples, that is, negative samples that contribute less to the gradient update direction of model training, thereby affecting the convergence speed and accuracy of model training.

[0007] In summary, through the analysis of the above two traditional training methods, it can be found that these two methods face a common problem, that is, they cannot guarantee high-quality negative samples to participate in training. The definition of high-quality negative samples given in the literature

[36] is a sample that is difficult to judge whether it is true or false without referring to the true value. The sample is only semantically meaningful, but in fact it is an incorrect sample. In other words, high-quality negative samples are neither false negative samples nor easy-to-classify negative samples. The traditional training method based on negative sampling randomly samples unobserved samples as negative samples to participate in training, while the training method based on non-sampling treats all unobserved samples as negative samples to participate in training. Both methods cannot guarantee the quality of negative samples participating in training, thereby limiting the improvement of model training performance.

[0008] In addition, the existing knowledge graph faces the problem of data imbalance, that is, entities and relations conform to the long-tail distribution, only a small number of entities and relations appear frequently in the knowledge graph, and the vast majority of entities and relations only appear in a limited number of triples. In the current knowledge graph embedding training, the frequency of occurrence of different entities and relations is not distinguished, so that the model training pays more attention to entities and relations with high frequency, while paying relatively less attention to long-tail entities and relations with low frequency, resulting in the long-tail entities and relations not being fully trained during the training process, which will also affect the improvement of model training performance. In the link prediction task experiment of the TransE model conducted on the benchmark dataset FB15k-237, the high-frequency entity "Britain" appeared 1953 times in the dataset, and its evaluation indicator MRR (mean reciprocal rank, the larger the value, the better the embedding performance) on the link prediction task scored 0.408, while the low-frequency entity "pizza" that only appeared 8 times got an MRR score of only 0.194. This experiment illustrates from one side the adverse effects of long-tail entities and relations on model training performance. Summary of the invention

[0009] Traditional model training methods based on negative sampling randomly sample some negative samples for training, which can easily miss some important negative samples and cause adverse effects on knowledge graph embedding model training. Some researchers have explored non-sampling model training frameworks, using all unobserved triples as negative samples for training to improve model training performance. However, both model training methods will inevitably introduce false negative samples and easy-to-separate negative samples far from the model decision boundary to participate in training, and the adverse effects of long-tail entities and relationships on model training are not considered during training, thus limiting the improvement of model training performance.

[0010] In view of this, the present application proposes a general knowledge graph embedding framework HNSW-KGE based on high-quality negative sampling and weighting. First, pre-training is performed based on the NS-KGE non-sampling training framework to quickly obtain an initial relatively high-quality embedding vector representation of all entities and relationships; secondly, a candidate negative sample set construction strategy is designed. Based on the embedding vector obtained by pre-training, a certain number of negative samples that are neither false negative samples nor easily distinguishable negative samples are sampled for all positive triples to form a candidate negative sample set, providing high-quality negative sample guarantees for model training. Finally, the loss function is weighted based on the frequency of entities and relationships in the triples to alleviate the adverse effects of long-tail entities and relationships on model training.

[0011] To achieve the above objectives, the present application discloses a link prediction method based on knowledge graph, which includes the following steps:

[0012] Obtain a medical data set, where entities include drugs, diseases, and symptoms;

[0013] Based on the embedding vector obtained by pre-training and the designed high-quality candidate negative sample set construction strategy, high-quality candidate negative sample sets are sampled for all positive triples;

[0014] According to the knowledge graph embedding model based on semantic matching, the scoring function value of the triple and its corresponding negative triple is calculated;

[0015] Count the relevant frequency data in the data set and calculate the frequency-based weight W of each triple f ;

[0016] According to the scoring function value and W f , calculate the loss L of the triple set and get the final loss function of the training framework;

[0017] The stochastic gradient descent method SGD is used to optimize the loss function of the training model and learn the optimal parameters of the knowledge graph embedding model based on semantic matching, including the optimal embedding vectors h, r, t, where h, t, r are the embedding vectors of the head entity, tail entity, and relationship in the triplet, respectively. Link prediction is performed on drugs, diseases, and symptoms to obtain the potential correlation between drugs and diseases and symptoms.

[0018] Preferably, during the training process, the candidate negative sample set is dynamically updated at a certain frequency according to the knowledge graph embedding vector obtained at that time to improve the quality of the candidate negative samples.

[0019] Preferably, to avoid overfitting, during model training, the second-order norms of all entities and relations in the dataset are constrained to be less than or equal to 1, that is, ||h||2≤1, ||r||2≤1, ||t||2≤1.

[0020] Preferably, the non-sampling knowledge graph embedding framework NS-KGE is selected as the pre-training model to quickly obtain an initial knowledge graph embedding vector representation.

[0021] Preferably, the high-quality candidate negative sample set construction strategy constructs a candidate negative sample set for all positive triples as a candidate library of negative samples, so as to sample high-quality negative samples from the library to participate in model training, thereby improving the performance of model training; the high-quality candidate negative sample set construction strategy specifically includes:

[0022] The candidate negative sample set includes a candidate head entity set, i.e., a candidate negative sample set obtained by replacing the head entity, and a candidate tail entity set, i.e., a candidate negative sample set obtained by replacing the tail entity;

[0023] The sampling size for each triple is N=N h +N t The candidate negative sample set, where the size of the candidate head entity set is Nh , the size of the candidate tail entity set is N t ;

[0024] Setting N h ,N t When considering the one-to-many or many-to-one relationship types corresponding to the triples, specifically, for the triples of the one-to-many relationship type, more candidate head entities are set, so the size of the candidate head entity set is larger than the size of the candidate head and tail entity sets; and for the triples of the many-to-one relationship type, more candidate tail entities are set, and the size of the candidate tail entity set is larger than the size of the candidate head entity set; the size of the candidate head entity set is defined as N h And the size of the candidate tail entity set is N t ,Right now

[0025]

[0026] Among them, t rh It refers to the average number of tail entities connected to all triple head entities with relation r in the knowledge graph, h rt It refers to the average number of head entities connected to the tail entity in all triples of relation r;

[0027] Construct the candidate head entity set and candidate tail entity set of the triple (h, r, t), and finally obtain the size of the triple (h, r, t) to be N = N h +N t Candidate negative sample set;

[0028] Execute the process of building a high-quality candidate negative sample set based on pre-training, and build the corresponding candidate negative sample set for all triples in the data set based on this process to obtain the high-quality candidate negative sample set corresponding to the data set; in addition, in order to avoid the low quality of the candidate negative sample set due to the performance limitations of the pre-training model itself, during the training process, according to the knowledge graph embedding vector obtained at that time, the negative sample set is dynamically updated at a certain frequency according to the above steps, so as to provide high-quality negative samples for the next step of optimization training based on the loss function.

[0029] Preferably, the loss function L for model training is defined as:

[0030]

[0031] Among them, (h, r, t) and (h', r, t') represent positive triples and negative triples respectively, S and S' represent the set of positive triples and the set of negative triples respectively;

[0032] Add a weight factor based on triple frequency to the loss function, and set different weights for the triple loss function according to the frequency of entities and relations in the training triples in the dataset; specifically, assign lower weights to entities and relations with higher frequencies, and assign higher weights to entities and relations with lower frequencies. In this way, increase the training attention to long-tail entities and relations in order to learn better embeddings; define the weight factor W of the triple based on the triple frequency f for:

[0033]

[0034] Among them, f h ,f t ,f r Respectively represent the frequency of entity h, t and relation r in the data set, Respectively represent the median frequency of entities and relations in the data set. According to the Pareto Principle, the median frequency is defined as the frequency of the 20% entities and relations ranked from high to low in terms of frequency of occurrence. h 、w t 、w r They are the frequency weight factors of entity h,t and relation r respectively;

[0035] Weight factor W based on triple frequency f For low-frequency triples, that is, the entities and relations in the triples are low-frequency entities and relations with a frequency lower than the median frequency, the weight of the low-frequency entity or relation is greater than 1, and W in the loss function of the low-frequency triples is f The weight value is greater than 1, thereby increasing the attention to low-frequency entities and relations in training; similarly, for high-frequency triples, that is, the triples contain high-frequency entities and relations with a frequency higher than the median frequency, the weight of the high-frequency entity or relationship is less than 1, and W in the loss function of the high-frequency triples f The weight is less than 1, thus reducing the attention paid to these high-frequency entities and relationships during training;

[0036] The weight factor W based on the triple frequency f The loss function applied to model training is:

[0037]

[0038] Preferably, the semantic matching-based knowledge graph embedding model includes one of DistMult, SimplE and ComplEx;

[0039] For DistMult, the scoring function f(h,r,t) for a triple is defined as

[0040] f(h,r,t)=hT ·diag(r)·t

[0041] For SimplE, the scoring function f(h,r,t) for a triple is defined as:

[0042]

[0043] For ComplEx, the scoring function f(h,r,t) for a triple is defined as:

[0044]

[0045] are the transpose of the real and imaginary parts of the complex embedding vector h of entity h, respectively, and t re ,t im ,r re ,r im are the real and imaginary parts of the complex embedding vectors t and r of entity t and relation r, respectively.

[0046] The loss function L of the knowledge graph embedding model based on semantic matching is:

[0047]

[0048] The beneficial effects of the present invention are as follows:

[0049] A new knowledge graph embedding training framework HNSW-KGE is proposed. The negative sampling quality of the model is improved based on the pre-training of the NS-KGE model and the high-quality negative sampling strategy. The adverse effects of long-tail entities and relations on model training are alleviated by weighting the loss function to improve the model training performance.

[0050] The proposed HNSW-KGE training framework is universal for negative sampling-based KGE models and can be applied to a variety of different types of knowledge graph embedding models, demonstrating the application of this training framework in different types of models.

[0051] This application proposes a general knowledge graph embedding framework HNSW-KGE based on high-quality negative sampling and weighting. First, a non-sampling-based NS-TransE model is used for pre-training to quickly learn a relatively high-quality embedding vector for entities and relations in the knowledge graph; secondly, a high-quality candidate negative sample set construction strategy is designed, and a high-quality candidate negative sample set is constructed for positive triples based on the embedding vector obtained by pre-training, thereby providing high-quality negative samples for model training based on negative sampling; then, a weight factor based on the frequency of triples is used to weight the loss function to alleviate the adverse effects of long-tail entities and relations on model training and improve the training effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 General training process for knowledge graph embedding models based on negative sampling;

[0053] Figure 2 Framework diagram of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0055] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0056] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0057] The technical solutions provided in the embodiments of the present application involve technologies such as machine learning and natural language processing of artificial intelligence, which are specifically introduced and explained through the following embodiments.

[0058] Before introducing the embodiments of the present application, some terms involved in the present application are first explained.

[0059] 1. Traditional negative sampling method: Geometric transformation-based method uses geometric transformation to model the semantic relationship between entities and relations. The method based on tensor decomposition uses tensor decomposition to construct a scoring function to measure the probability of a triple being established; among them, the widely used DistMult models the scoring function as the inner product of the two vectors of the head and tail entities and the diagonal matrix of the relationship. The neural network-based method uses the characteristics of deep neural networks such as CNN, RNN, and GNN to model nonlinear transformations to model complex relationships between entities and relations. Among them, models such as ConvE and ConvKB use CNN to model the potential semantics between entities and relations; Gradner et al. use RNN to model relationship paths to capture longer relationship dependencies in KG; R-GCN, RGHAT, and CompGCN use GNN to capture structured information in KG.

[0060] 2. Negative Sampling Method for Knowledge Graph Embedding

[0061] The negative sampling methods in the knowledge embedding model can be roughly divided into three categories: random negative sampling, adversarial negative sampling, and negative sampling with additional information (Additional Data Enhanced Sampling), which are introduced below. Random negative sampling is the most basic and widely used negative sampling method. Random negative sampling often samples overly simple negative samples, that is, easy-to-classify negative samples, which cannot provide useful information for the optimization training of the model, causing the model to fall into a local optimal solution.

[0062] Adversarial Sampling is usually based on the principle of GAN, using the semantic information of samples in the embedding space to select negative samples in order to sample better hard-to-distinguish negative samples. However, it has disadvantages such as complex framework, unstable training results and long training time. To this end, RotatE proposes a self-adversarial negative sampling strategy, which dynamically adjusts the sampling probability of negative samples during the model training process to sample hard-to-distinguish negative samples for training. The negative sampling method that introduces additional information introduces external information in the negative sampling process to increase the sampling probability of hard-to-distinguish negative samples.

[0063] However, in general, no matter which negative sampling method is used, since the negative samples sampled in different rounds are different, it will have a certain impact on the stability of model training, and will inevitably introduce false negative samples, which will also affect the accuracy of model training.

[0064] 3. Knowledge graph embedding model based on non-sampling

[0065] In order to overcome the problems of high training complexity and prediction bias caused by negative sampling's over-reliance on negative sample selection, a regularization term is added to the loss function to prevent the model from generating high scores for all triplets, thereby eliminating the need for negative sampling. However, the calculation of the regularization term still implicitly uses all possible triplets in the knowledge graph, including positive and negative triplets. The non-sampled knowledge graph embedding framework NS-KGE uses all non-positive triplets composed of entities and relations as negative triplets for training. This method solves the shortcomings of the negative sampling training method by skipping negative sampling. However, this method will introduce false negative samples and a large number of easy-to-classify negative samples, which will affect the accuracy and convergence speed of model training.

[0066] refer to Figure 1 and Figure 2 This application proposes a link prediction method HNSW-KGE (Universal K nowledge G raph E mbedding Framework Based On H igh-quality N egative S ampling And Weighting). First, the non-sampling framework NS-KGE model (Zhao Zhang, Weighted Knowledge Graph Embedding. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR'23), July 23–27, 2023, ACM, New York, NY, USA, 11 pages. https: / / doi.org / 10.1145 / 3539618.3591784) is selected for pre-training. Based on the global features in the dataset, a relatively high-quality initial embedding vector is quickly learned for the entities and relations in the knowledge graph, providing support for the subsequent construction of a high-quality candidate negative sample set. Secondly, a high-quality candidate negative sample set construction strategy is designed. In order to overcome the problem that low-quality negative samples cannot be excluded from participating in training in both traditional negative sampling-based and non-sampling-based training methods, this application is based on the embedding vector obtained by pre-training. Through the designed high-quality candidate negative sample set construction strategy, a certain number of negative samples that are neither false negative samples nor easy-to-separate negative samples are sampled for all positive triples to form a candidate negative sample set, providing high-quality negative sample guarantees for subsequent model training. Finally, a loss function weighting strategy based on triple frequency is adopted. Weighting the loss function has been proven to be an effective way to alleviate the impact of long-tail entities on training. To this end, a weight factor based on triple frequency is used to weight the loss function to adjust the adverse effects of long-tail entities and relationships on model training. In order to evaluate the performance of the HNSW-KGE training framework, the training framework was applied to four KGE models, including DistMult, SimplE, ComplEx and TransE, and experiments were carried out to verify the effectiveness of the HNSW-KGE framework.

[0067] Table 1 Basic symbols in this application

[0068]

[0069] The problem under study will be formally described. Table 1 lists the basic symbols used.

[0070] The general training process of the knowledge graph embedding model based on negative sampling is as follows Figure 1 As shown. Given a knowledge graph G = (E, R, S), triples (h, r, t) ∈ S, h, t ∈ E, r ∈ R.

[0071] First, negative sampling. To construct the candidate negative sample set required for model training, a common approach is to randomly replace the head entity or tail entity of the triple (h, r, t) with the entity in the entity set E to obtain its corresponding negative triple (h', r, t'), thereby obtaining the candidate negative sample set required for model training. In general, the candidate negative sample set S' can be formally described as Based on certain strategies, such as random negative sampling, hard-to-separate negative sample sampling, etc., some negative triplets are sampled from the candidate negative sample set to participate in training.

[0072] Then, calculate the loss. According to the scoring function of the model, calculate the scoring function values ​​f(h,r,t) and f(h',r,t') of the positive triple (h,r,t) and its corresponding negative triple (h',r,t') respectively, and calculate the loss of the triple based on the scoring function value, so as to obtain the loss function L of the model. The loss function of the model can be roughly divided into two categories, the loss function based on marginal hyperparameters and the loss function based on cross entropy. Among them, the loss function based on marginal hyperparameters is defined as:

[0073]

[0074] γ is the marginal parameter. Most translation-based models, such as TransE, TransH, etc., use loss functions based on marginal hyperparameters. The loss function based on cross entropy is defined as:

[0075]

[0076] Models based on semantic matching, such as DistMult, SimplE, ComplEx, etc., mostly use loss functions based on cross entropy.

[0077] Finally, optimization training is performed. The loss function L is optimized and trained using optimization algorithms such as stochastic gradient descent SGD and adaptive gradient descent AdaGrad to obtain the optimal parameters of the model, including the embedding vectors h, t and r of the entity h, t and the relationship r.

[0078] The following is a detailed introduction to the HNSW-KGE training framework of this application. First, the NS-KGE model used for pre-training is introduced, followed by the designed high-quality candidate negative sample set construction strategy, and then the weighting strategy of the loss function based on the triple frequency, and finally the application of the training framework on several different knowledge graph embedding models.

[0079] 1. Pre-training based on NS-KGE

[0080] In order to provide a basis for the construction of high-quality candidate negative sample sets, we select the non-sampling knowledge graph embedding framework NS-KGE as a pre-training model to quickly obtain an initial knowledge graph embedding vector representation. Next, we will introduce the NS-KGE model in detail.

[0081] The non-sampling knowledge graph embedding framework NS-KGE can be applied to knowledge graph embedding models based on square loss functions or models that can convert loss functions into square losses. Compared with traditional knowledge graph embedding models, NS-KGE has relatively high training efficiency and has certain improvements in training accuracy. Its basic idea is based on the closed world hypothesis. All triples composed of entities and relationships in the dataset that are not in the dataset are used as negative samples to participate in model training. The least squares loss function between the true label values ​​of positive and negative triples and the predicted values ​​based on the scoring function is used to define the loss function for its optimized training. In order to solve the problem of high model training complexity caused by the huge amount of data, the model reduces the complexity of the loss function through mathematical derivation. The NS-KGE framework uses a least squares-based loss function to define the loss function for its optimized training, that is

[0082]

[0083] Among them, C hrt represents the weight of the corresponding triple (h, r, t), f r (h, t) represents the true label value of the triple (h, r, t). For the positive triple (h, r, t), define f r (h, t) = 1, for the negative triple (h, r, t), define f r (h,t)=0, Represents the predicted label value based on the scoring function.

[0084] NS-DistMult is a sub-model obtained by applying the NS-KGE model framework to the DistMult model. The prediction function of the DistMult model is defined as:

[0085]

[0086] In order to reduce the time and space complexity of the model, after mathematical derivation of formula (4) and formula (3), the loss function of NS-DistMult based on least squares is obtained as follows:

[0087]

[0088] in,

[0089]

[0090] Similarly, NS-TransE is a sub-model obtained by applying the NS-KGE framework to the TransE model. According to the scoring function of TransE:

[0091] f(h,r,t)=||h+rt|| (7)

[0092] Transform to obtain the prediction function of the TransE model for:

[0093]

[0094] After mathematical derivation and simplification, the loss function of NS-TransE based on least squares is obtained as follows:

[0095]

[0096] Among them, L P The calculation of is shown in formula (6).

[0097] In terms of training effect, NS-KGE has achieved better performance in most cases compared with the original baseline model. For example, on the TCGA dataset, Hit@10 is improved from 0.316 of TransE to 0.447 of the NS-TransE model. This is because the original baseline model uses a negative sampling-based method to randomly sample only part of the negative sample information in the dataset, which may ignore some important negative samples, while the NS-KGE framework uses all negative samples in the dataset for training. In terms of training efficiency, the training speed of NS-KGE is also significantly improved compared with the original baseline model. For example, compared with TransE, on the TCGA dataset, the training speed of NS-TransE is increased by about 10 times. This is because the original negative sampling-based model training spends most of the time on negative sampling, while the NS-KGE framework skips the negative sampling process.

[0098] However, NS-KGE has certain limitations. First, its scalability is limited. The model uses mathematical derivation to reduce the complexity of the loss function. Since the derivation process is too complicated, it is only applicable to a limited number of models, which limits the scope of application of the model. Second, it is easy to introduce false negative samples. The model uses all unobserved triplets in the data set as negative samples for training, while many fact triplets are not included in the data set due to data sparsity and are treated as negative samples for training, which affects the accuracy of model training. Third, the non-sampling method will introduce a large number of easy-to-separate negative samples for training, which affects the convergence speed and accuracy of model training. For this reason, we only use NS-KGE for pre-training as the starting point for all model training, so as to quickly obtain an initial embedding vector with relatively good quality, which provides a reference for the construction of the subsequent high-quality candidate negative sample set. In addition, considering the comprehensive training effect and training efficiency, we choose the NS-TransE sub-model in NS-KGE as the pre-training model.

[0099] 2. Construction of high-quality candidate negative sample sets

[0100] In order to provide high-quality negative samples for negative sampling-based training, after obtaining an initial high-quality embedding vector of entities and relations in the dataset, we will build a candidate negative sample set for all positive triples according to the high-quality candidate negative sample set construction strategy we designed, as a candidate library of negative samples, so as to sample high-quality negative samples from it to participate in model training, thereby improving the performance of model training. Next, we will specifically introduce the high-quality candidate negative sample set construction strategy.

[0101] The candidate negative sample set includes the candidate head entity set, that is, the candidate negative sample set obtained by replacing the head entity, and the candidate tail entity set, that is, the candidate negative sample set obtained by replacing the tail entity. The sampling size for each triple is N=N h +N t The candidate negative sample set, where the size of the candidate head entity set is N h , the size of the candidate tail entity set is N t . Set N h ,N t When , the one-to-many or many-to-one relationship types corresponding to the triples will be considered. Specifically, for triples of one-to-many relationship types, due to the uniqueness of the head entity, the negative samples constructed by replacing the head entity are unlikely to be false negative samples. We can consider setting more candidate head entities. Therefore, the size of the candidate head entity set should be larger than the size of the candidate head and tail entity sets; and for triples of many-to-one relationship types, we can consider setting more candidate tail entities. The size of the candidate tail entity set should be larger than the size of the candidate head entity set. We define the size of the candidate head entity set as N h And the size of the candidate tail entity set is N t,Right now

[0102]

[0103] Among them, t rh It refers to the average number of tail entities connected to all triple head entities with relation r in the knowledge graph, h rt It refers to the average number of head entities connected to the tail entity in all triples of relation r.

[0104] Next, taking TransE as an example, the construction process of the candidate header entity set of triples (h, r, t) is introduced in detail:

[0105] Step 0, initialization. Construct a data structure of {(r,t),{(h1,f(h'1,r,t)),...(h' Nh ,f(h Nh ,r,t))}}, size is N h Array of h1,...,h Nh Initialized to empty, f(h',r,t) represents the corresponding scoring function value. The scoring function of model TransE is shown in formula (7), which is initialized to a larger value, such as 100.

[0106] Step 1: Construct candidate negative triples. Randomly select an entity h' from the entity set to replace h in (h, r, t) to obtain a negative triple (h', r, t).

[0107] Step 2: Determine whether (h', r, t) is in the data set. If so, restart step 1. Otherwise, continue to step 3. This step is to exclude the candidate negative triplet as a known positive sample.

[0108] Step 3: Determine whether (h',r,t) is a false negative sample.

[0109] f(h',r,t) <f(h,r,t)+u(12)

[0110] Is it true? If true, it means that (h', r, t) may be a false negative sample, and restart step 1. Otherwise, continue to step 4. Among them, u ≥ 0 is a hyperparameter. This step is to exclude the candidate negative sample as a false negative sample.

[0111] Step 4: Filter candidate head entities. Loop to determine whether the score function value f(h',r,t) of (h',r,t) is smaller than the score function value of the negative triples constructed by all candidate head entities. If it is smaller than a negative triple (h',r,t) in the candidate head entity set, i ',r,t), that is, f(h',r,t) <f(h i',r,t), indicating that (h',r,t) is better than (h i ',r,t) is more difficult to distinguish, then (h',f(h',r,t)) is used to replace (h' i ,f(h i ',r,t)). Otherwise, restart step 1. This step is to screen out the N with the greatest degree of difficulty. h entities, that is, the top N entities are ranked from small to large according to the scoring function value h Entity, this N h entities as candidate head entities.

[0112] Step 5: Output. After looping through all entities in the entity set, output a triple (h, r, t) of size N. h The candidate header entity set.

[0113] The method of constructing the candidate tail entity set of the triple (h, r, t) is similar. The size of the final triple (h, r, t) is N = N h +N t Based on the pre-trained high-quality candidate negative sample set construction process, all triples in the data set are used to construct their corresponding candidate negative sample sets based on this process to obtain the high-quality candidate negative sample set corresponding to the data set. In addition, in order to avoid the low quality of the candidate negative sample set due to the performance limitations of the pre-trained model NS-TransE itself, during the training process, the negative sample set will be dynamically updated at a certain frequency according to the above steps based on the knowledge graph embedding vector obtained at that time. This provides high-quality negative sample guarantee for the next step of optimization training based on the loss function.

[0114] The process of constructing a high-quality candidate negative sample set based on pre-training is as follows:

[0115] Input knowledge graph dataset S = {(h, r, t)};

[0116] The dataset is pre-trained based on the NS-KGE non-sampling training framework to obtain the initial embedding vectors of entities and relations;

[0117] Sample a triplet (h, r, t) in the training set;

[0118] 1) and 2) are performed respectively, where 1) includes: selecting an entity h' in the entity set to replace h in (h, r, t) to obtain (h', r, t), and calculating its scoring function value f(h', r, t) based on the embedding vector obtained by pre-training;

[0119] Determine whether (h',r,t) is not in the data set. If not, determine whether f(h',r,t)>f(h,r,t)+u. If yes, determine whether the function value of f(h',r,t) is in the top N in ascending order among the function values ​​of all negative triples obtained by replacing the head entity. h , if yes, h' is selected into the candidate head entity set of (h,r,t), and in other cases, step 1) is repeated.

[0120] 2) It includes: selecting an entity t' in the entity set to replace t in (h, r, t) to obtain (h, r, t'), and calculating its scoring function value f(h, r, t') based on the embedding vector obtained by pre-training;

[0121] Determine whether (h,r,t') is not in the data set. If not, determine whether f(h,r,t')>f(h,r,t)+u. If yes, determine whether the function value of f(h,r,t') is in the top N in ascending order among the function values ​​of all negative triples obtained by replacing the head entity. t , if yes, then select t' into the candidate head entity set of (h,r,t), and repeat step 2) for other cases.

[0122] Check whether the number of entities in the candidate head entity set and tail entity set reaches Nh and Nt respectively. Otherwise, continue to execute 1) and 2). If yes, determine whether all triples in the training set have completed the construction of the candidate negative sample set. If yes, output the candidate negative samples S'(h, r, t) of all triples in the training set.

[0123] 3. Weighting of loss function based on triple frequency

[0124] The loss function L for model training is defined as:

[0125]

[0126] Among them, (h, r, t) and (h', r, t') represent positive triples and negative triples respectively, and S and S' represent the set of positive triples and the set of negative triples respectively.

[0127] In order to alleviate the impact of long-tail entities and relations on model training, it is necessary to increase the attention paid to long-tail entities and relations, thereby improving the training performance of the knowledge graph embedding model. Therefore, we add a weight factor based on triple frequency to the loss function, and set different weights for the loss function of the triple according to the frequency of entities and relations in the training triples in the dataset. Specifically, lower weights are given to entities and relations with higher frequencies, and higher weights are given to entities and relations with lower frequencies. In this way, the training attention to long-tail entities and relations is increased in order to learn better embeddings. We define the weight factor W of the triple based on the triple frequencyf for:

[0128]

[0129] Among them, f h ,f t ,f r Respectively represent the frequency of entity h, t and relation r in the data set, Respectively represent the median frequency of entities and relations in the data set. According to the Pareto Principle, we define the median frequency as the frequency of the 20% entities and relations ranked from high to low in terms of frequency of occurrence. The w in formula (15) h The denominator in the calculation formula is f h +1 is to avoid the situation where some entities have a frequency of 0 in the dataset when calculating the frequency weight of the entities in the negative triples. t ,w r And similar situations.

[0130] Weight factor W based on triple frequency f It can be understood that for low-frequency triples, that is, the entities and relations in the triples are low-frequency entities and relations whose occurrence frequency is lower than the median frequency, according to formula (15), the weight of the low-frequency entity or relation will be greater than 1. According to formula (14), W in the loss function of the low-frequency triples is f The weight value will be greater than 1, thereby increasing the attention to low-frequency entities and relations during training; similarly, for high-frequency triples, that is, the high-frequency entities and relations in the triples that appear more frequently than the median frequency, according to formula (15), the weight of the high-frequency entity or relationship will be less than 1. According to formula (14), W in the loss function of the high-frequency triples f The weight will be less than 1, thus reducing the attention paid to these high-frequency entities and relationships during training. Here, we also use the example of students studying for exams as an analogy. For knowledge points that often appear in exercise books, they are similar to high-frequency triples. Since these knowledge points are practiced frequently, the effect of paying attention to them on improving grades will not be too obvious, so you can spend less time paying attention to them. For knowledge points that do not often appear in exercise books, they are similar to low-frequency triples. Since they are practiced less frequently, they need more attention. Strengthening attention to these knowledge points is the key to improving grades.

[0131] The weight factor W based on the triple frequency f Applying formula (13) yields:

[0132]

[0133] Among them, W f The calculation of is shown in formula (14).

[0134] 4. Application of HNSW-KGE on different models

[0135] In this section, we will take three typical knowledge graph embedding models, DistMult, SimplE and ComplEx, as examples to demonstrate the application of our proposed HNSW-KGE training framework on different knowledge graph embedding models.

[0136] By weighting the loss function in combination with formula (16), the loss function L of this type of model can be obtained as:

[0137]

[0138] Among them, γ is the marginal parameter, σ is the sigmoid activation function, (h i ',r,t i ') is the negative triplet corresponding to (h, r, t) that participates in the training.

[0139] For DistMult, the scoring function f(h,r,t) for a triple is defined as

[0140] f(h,r,t)=h T ·diag(r)·t(19)

[0141] For SimplE, the scoring function f(h,r,t) for a triple is defined as:

[0142]

[0143] For ComplEx, the scoring function f(h,r,t) for a triple is defined as:

[0144]

[0145] Models such as DistMult, SimplE and ComplEx are semantic matching-based models. The loss function of this type of model based on cross entropy is defined as shown in formula (2). By weighting the loss function with formula (16), the loss function L of this type of model can be obtained as:

[0146]

[0147] The overall process of the HNSW-KGE framework proposed in this application is as follows Figure 2As shown in Figure 1, firstly, based on the embedding vector obtained by pre-training and the designed high-quality candidate negative sample set construction strategy, high-quality candidate negative sample sets are sampled for all positive triples; secondly, the scoring function values ​​of the triples and their corresponding negative triples are calculated according to different models; then, the relevant frequency data in the data set are counted and brought into formulas (14) and (15) to obtain the frequency-based weight W of each triple f ; Then, according to the scoring function value and W f , calculate the loss L of the triple set, and calculate the loss based on the semantic matching model according to formula (22) to obtain the final loss function of the training framework. Finally, the stochastic gradient descent method SGD is used to optimize the loss function of the training model and learn the optimal parameters of the model, including the optimal embedding vector h, r, t. Note that during the training process, the candidate negative sample set will be dynamically updated at a certain frequency according to the knowledge graph embedding vector obtained at that time to avoid the low quality of the candidate negative samples due to the performance limitations of the pre-trained model itself.

[0148] To avoid overfitting, we enforce the constraint that the second-order norm of all entities and relations in the dataset is less than or equal to 1 during model training, that is, ||h||2≤1, ||r||2≤1, ||t||2≤1.

[0149] We evaluate the performance of the HNSW-KGE framework through the link prediction task. In this section, we first introduce the dataset used in the experiment, then the experimental design, then the evaluation indicators, then the comparison model, and finally the parameter settings.

[0150] This application will use two benchmark datasets, TCGA and ClinicalTrials.gov, to conduct experiments and evaluate the training performance of the HNSW-KGE framework. TCGA (The Cancer Genome Atlas) provides genome, transcriptome, epigenetic and clinical data of various cancer types; ClinicalTrials.gov is a global clinical trial registry maintained by the U.S. National Library of Medicine, which contains information on ongoing and completed clinical trials. The basic statistical information of these two datasets is shown in Table 2.

[0151] Table 2 Statistics of the dataset

[0152] Dataset TCGA ClinicalTrials.gov #entity <![CDATA[14 , 541]]> <![CDATA[40 , 943]]> #relation 237 11 #triplet 292,581 89,969

[0153] According to the general setting method of the link prediction task for the data set, the triple set is divided into a training set, a validation set, and a test set, accounting for approximately 85%, 5%, and 10% respectively. The specific experimental steps for link prediction are as follows:

[0154] First, the optimal parameters of the model are obtained through training with training sets and validation sets, including the optimal embedding vectors of all entities and relations.

[0155] Then, the link prediction task is performed on the test set to evaluate the model's ability to predict the missing head entity or tail entity in the triple, that is, the ability to predict t given (h, r) or h given (r, t). Taking the evaluation model's ability to predict the tail entity as an example, for the test triple (h, r, t), the tail entity t is replaced by each entity x in the entity set to obtain a set of candidate triples (h, r, x). According to the model's definition of the scoring function f(h, r, x), the DistMult model uses formula (19) and other formulas to calculate the score of the scoring function of this set of candidate triples and sort them in ascending order. The ranking of the test triple (h, r, t) among all candidate triples can be obtained, which is recorded as rank(t). Similarly, the ability of the evaluation model to predict the head entity is obtained, and the ranking of the entity test triple (h, r, t) among all candidate triples (x, r, t) is obtained, which is recorded as rank(h).

[0156] Finally, according to the rank(h) and rank(t) values ​​of the test triple (h, r, t), the values ​​of each evaluation indicator in the test set are calculated according to the calculation formula of the evaluation indicator.

[0157] In addition, the "Filter" setting is used. Specifically, before evaluating the ranking of each test triple, the triples in the training set, validation set, and test set are removed from the set of candidate triples. This is because the candidate triple is actually a positive triple, and it is reasonable that its score ranking based on the scoring function is higher than the ranking of the original test triple. Therefore, it is necessary to remove the candidate triple before obtaining the score ranking of each test triple so that it does not participate in the ranking, in order to eliminate the influence of the "interference" factor.

[0158] In the link prediction task, the following evaluation indicators are used to evaluate the quality of the knowledge graph embedding learned by the model to evaluate the training performance of the model.

[0159] (1) MR value, which is the average ranking of all test triples, is defined as follows:

[0160]

[0161] Where S t represents the test triple set, |S t | represents the size of the test triple set.

[0162] (2) MRR value, which is the average of the reciprocal rankings of all test triples, is defined as follows:

[0163]

[0164] (3) Hits@K, that is, the ratio of the number of test triplets whose ranking is not greater than K to the number of all test triplets. This experiment uses indicators such as Hits@1, Hits@3, and Hits@10.

[0165] The larger the MRR value and Hits@N value, and the smaller the MR value, the better the training performance of the model; otherwise, the worse the performance of the model.

[0166] This application divides the comparison models into three categories:

[0167] (1) The original baseline model, i.e., the knowledge graph embedding model based on traditional negative sampling. Three classic models, DisMult, SimplE, and ComplEx, are selected as comparison models. The experimental results of these models on link prediction are directly obtained from published papers.

[0168] DistMult: A semantic matching-based model that uses matrix decomposition to learn knowledge graph embedding. It models all triples in the knowledge graph as a three-dimensional tensor, and models entity embedding and relationship embedding through three-dimensional tensor decomposition.

[0169] SimplE: A semantic matching-based model that introduces the inverse relationship of the relationship based on DistMult. It is an interpretable bilinear model.

[0170] ComplEx: A semantic matching based model that uses Hermitian dot products in complex space to capture the interaction information between head and tail entities and relations.

[0171] (2) Models using the NS-KGE framework, that is, the original baseline models trained using the NS-KGE framework, including NS-DisMult, NS-SimplE, and NS-ComplEx. The experimental results of link prediction for these models directly use the results of published papers.

[0172] (3) Models using the HNSW-KGE framework, that is, the original baseline model trained using the HNSW-KGE framework, including three models: HNSW-DisMult, HNSW-SimplE, and HNSW-ComplEx.

[0173] The learning rate of stochastic gradient descent SGD is λ∈{0.0001,0.001,0.01,0.1}, the embedding dimension d∈{100,300,500,1000}, the hyperparameter u∈{0,0.05,0.1,0.15,0.2,0.25,0.3}, the hyperparameter N∈{10,30,50,80,100}, γ∈{0.1,0.2,0.5,1,2,4,8,10}. Batch=128. The model is iteratively trained for 1000 rounds on each dataset. The optimal parameters are determined by Hits@10 on the validation set. The optimal parameter configurations of the models for the two datasets are shown in Table 3.

[0174] Table 3 Optimal hyperparameter settings

[0175]

[0176] In order to verify the effectiveness of the model, the comparative experimental results of HNSW-KGE and the comparison models on two public datasets are first introduced and analyzed.

[0177] The results of the comparison experiment between the HNSW-KGE model and the two comparison models in the link prediction task are shown in Table 4. In Table 4, NS-X refers to the version of the X model using the NS-KGE framework, HNSW-X refers to the version of the X model using the HNSW-KGE framework, ↑ indicates that the larger the evaluation index value, the better, ↓ indicates that the smaller the evaluation index value, the better. In the table, the bold data indicates the best experimental results among the comparison models, and the underlined data indicates the second best experimental results.

[0178] Table 4 Link prediction experimental results

[0179]

[0180] First, from the overall table, among the three types of comparison models, the HNSW-KGE model proposed in this application has achieved the best or second-best evaluation results in most cases, especially on the TCGA dataset. Except for the MR evaluation index on HNSW-ComplEx, HNSW-KGE outperforms other comparison models in all evaluation indicators. Although in some individual cases, some comparison models are slightly better than HNSW-KGE. For example, on the CLINICALTRIALS.GOV dataset, although NS-SimplE is slightly better than HNSW-SimplE in the Hit@10 index, the two results are close, and in the Hit@1 and Hit@3 indicators, HNSW-SimplE is 2.89% and 6.10% higher than NS-SimplE on average, respectively. The experimental results verify the effectiveness of the HNSW-KGE framework.

[0181] Secondly, compared with the three original baseline models of DisMult, SimplE, and ComplEx, the model using the HNSW-KGE framework has an average performance improvement of 58.8% on the evaluation index MRR, 3.2% on MR, 47.2% on Hit@10, 75.1% on Hit@3, and 285.6% on Hit@1 in experiments on two data sets. This shows that the proposed HNSW-KGE framework uses the candidate negative sample set constructed based on the initial embedding vector obtained by pre-training and the high-quality candidate negative sample set construction strategy to provide relatively high-quality negative samples in the model training process, thereby improving the training effect of the negative sampling model, making the model training performance of the HNSW-KGE framework better than these traditional knowledge graph embedding models based on negative sampling.

[0182] Again, compared with the knowledge graph embedding models NS-DisMult, NS-SimplE, NS-ComplEx, etc. applied to the NS-KGE framework, the model using the HNSW-KGE framework has an average performance improvement of 16.9% on the evaluation index MRR, 9.9% on MR, 9.2% on Hit@10, 16.6% on Hit@3, and 90.1% on Hit@1 in experiments on two data sets. This shows that compared with the application to the NS-KGE framework, the application of the HNSW-KGE framework of this application can better improve the training performance of the model. This is because the model training based on the NS-KGE framework uses all unobserved samples in all data sets as negative samples for training, which inevitably introduces a large number of false negative samples and easy-to-classify negative samples far from the decision boundary during the model training; while the HNSW-KGE framework proposed in this application uses a relatively high-quality entity and relationship embedding vector obtained by pre-training, and a high-quality candidate negative sample set construction strategy to construct a candidate negative sample set, which can exclude false negative samples and a large number of easy-to-classify negative samples from participating in the training to a certain extent, so that the model can obtain better training results based on the training of these relatively high-quality negative samples. In addition, the HNSW-KGE framework adds a weight factor based on the frequency of triples to the loss function, so that the long-tail entities and relationships in the data set receive more attention during training, which can alleviate the adverse effects of long-tail entities and relationships on model training to a certain extent. Therefore, the experimental results prove the effectiveness of the high-quality candidate negative sample set construction strategy and the loss function weighting strategy based on the weight factor of the triple frequency in the HNSW-KGE framework for model training.

[0183] Then, compared with the original baseline model and the NS-KGE framework, the model using the HNSW-KGE framework always achieves better performance on the MRR evaluation index, but relatively poor performance on MR, especially on the CLINICALTRIALS.GOV dataset. This is because the difference between MR and MRR is that the former is more sensitive to prediction errors. If some real entities rank low in its predicted candidate entity list, it will have a greater impact on MR, while MRR has less impact on it due to the reciprocal operation. The HNSW-KGE framework performs better on MRR, indicating that the framework can rank real entities to a higher position in most cases. The MR evaluation index on the CLINICALTRIALS.GOV dataset is relatively poor because the CLINICALTRIALS.GOV dataset contains more entities than TCGA, making it easier for real entities to rank low on the CLINICALTRIALS.GOV dataset.

[0184] Finally, the experimental results show that the HNSW-KGE framework can improve the negative sampling-based model training performance of semantic matching-based models (DisMult, SimplE, ComplEx), showing the versatility of the framework for negative sampling-based model training and its potential for application to other knowledge graph embedding models.

[0185] In summary, the comparative experimental results on the link prediction task show that the model using the HNSW-KGE framework performs well in this task. Further analysis of the results shows that the HNSW-KGE framework can obtain relatively higher quality negative samples for training based on the embedding vector obtained by pre-training and the method of constructing a high-quality candidate negative sample set. The weighted strategy based on the loss function can alleviate the adverse effects of long-tail entities and relationships on model training, thereby training better embedding for the knowledge graph, so that the model using this framework can benefit in the experiment of the link prediction task.

[0186] The beneficial effects of the present invention are as follows:

[0187] A new knowledge graph embedding training framework HNSW-KGE is proposed. The negative sampling quality of the model is improved based on the pre-training of the NS-KGE model and the high-quality negative sampling strategy. The adverse effects of long-tail entities and relations on model training are alleviated by weighting the loss function to improve the model training performance.

[0188] The proposed HNSW-KGE training framework is universal for negative sampling-based KGE models and can be applied to a variety of different types of knowledge graph embedding models, demonstrating the application of this training framework in different types of models.

[0189] This application proposes a general knowledge graph embedding framework HNSW-KGE based on high-quality negative sampling and weighting. First, a non-sampling-based NS-TransE model is used for pre-training to quickly learn a relatively high-quality embedding vector for entities and relations in the knowledge graph; secondly, a high-quality candidate negative sample set construction strategy is designed, and a high-quality candidate negative sample set is constructed for positive triples based on the embedding vector obtained by pre-training, thereby providing high-quality negative samples for model training based on negative sampling; then, a weight factor based on the frequency of triples is used to weight the loss function to alleviate the adverse effects of long-tail entities and relations on model training and improve the training effect of the model.

[0190] As used herein, the word "preferred" is intended to be used as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as being more advantageous than other aspects or designs. On the contrary, the use of the word "preferred" is intended to present concepts in a specific way. The term "or" as used in this application is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X uses A or B" means any one of the naturally included permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.

[0191] Moreover, although the present disclosure has been shown and described with respect to one or implementations, those skilled in the art will think of equivalent variations and modifications based on the reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations, and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if the structure is not equivalent to the disclosed structure of the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that may be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0192] The functional units in the embodiments of the present invention may be integrated into a processing module, or each unit may exist physically separately, or multiple or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc. The above-mentioned devices or systems may execute the storage method in the corresponding method embodiment.

[0193] To sum up, the above embodiment is an implementation mode of the present invention, but the implementation mode of the present invention is not limited by the embodiment. Any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A link prediction method based on knowledge graph, characterized in that: The following steps are involved: Obtain a medical data set, where entities include drugs, diseases, and symptoms; Based on the embedding vector obtained by pre-training and the designed high-quality candidate negative sample set construction strategy, high-quality candidate negative sample sets are sampled for all positive triples; According to the knowledge graph embedding model based on semantic matching, the scoring function value of the triple and its corresponding negative triple is calculated; Count the relevant frequency data in the data set and calculate the frequency-based weight W of each triple f ; According to the scoring function value and W f , calculate the loss L of the triple set and get the final loss function of the training framework; The stochastic gradient descent method SGD is used to optimize the loss function of the training model and learn the optimal parameters of the knowledge graph embedding model based on semantic matching, including the optimal embedding vectors h, r, t, where h, t, r are the embedding vectors of the head entity, tail entity, and relationship in the triplet, respectively. Link prediction is performed on drugs, diseases, and symptoms to obtain the potential correlation between drugs and diseases and symptoms.

2. The link prediction method based on knowledge graph according to claim 1, characterized in that: During the training process, the candidate negative sample set is dynamically updated at a certain frequency based on the knowledge graph embedding vector obtained at that time to improve the quality of the candidate negative samples.

3. The link prediction method based on knowledge graph according to claim 1, characterized in that: To avoid overfitting, the second-order norm of all entities and relations in the dataset is forced to be less than or equal to 1 during model training, that is, ||h||2≤1, ||r||2≤1, ||t||2≤1.

4. The link prediction method based on knowledge graph according to claim 1, characterized in that: The non-sampling knowledge graph embedding framework NS-KGE is selected as the pre-training model to quickly obtain an initial knowledge graph embedding vector representation.

5. The link prediction method based on knowledge graph according to claim 1, characterized in that: The high-quality candidate negative sample set construction strategy constructs a candidate negative sample set for all positive triples as a candidate library of negative samples, so as to sample high-quality negative samples from the library to participate in model training, thereby improving the performance of model training; The strategy for constructing a high-quality candidate negative sample set specifically includes: The candidate negative sample set includes a candidate head entity set, i.e., a candidate negative sample set obtained by replacing the head entity, and a candidate tail entity set, i.e., a candidate negative sample set obtained by replacing the tail entity; The sampling size for each triple is N=N h +N t The candidate negative sample set, where the size of the candidate head entity set is N h , the size of the candidate tail entity set is N t ; Setting N h ,N t When considering the one-to-many or many-to-one relationship types corresponding to the triples, specifically, for the triples of the one-to-many relationship type, more candidate head entities are set, so the size of the candidate head entity set is larger than the size of the candidate head and tail entity sets; and for the triples of the many-to-one relationship type, more candidate tail entities are set, and the size of the candidate tail entity set is larger than the size of the candidate head entity set; the size of the candidate head entity set is defined as N h And the size of the candidate tail entity set is N t ,Right now Among them, t rh It refers to the average number of tail entities connected to all triple head entities with relation r in the knowledge graph, h rt It refers to the average number of head entities connected to the tail entity in all triples of relation r; Construct the candidate head entity set and candidate tail entity set of the triple (h, r, t), and finally obtain the size of the triple (h, r, t) to be N = N h +N t Candidate negative sample set; Execute the process of building a high-quality candidate negative sample set based on pre-training, and build the corresponding candidate negative sample set for all triples in the data set based on this process to obtain the high-quality candidate negative sample set corresponding to the data set; in addition, in order to avoid the low quality of the candidate negative sample set due to the performance limitations of the pre-training model itself, during the training process, according to the knowledge graph embedding vector obtained at that time, the negative sample set is dynamically updated at a certain frequency according to the above steps, so as to provide high-quality negative samples for the next step of optimization training based on the loss function.

6. The link prediction method based on knowledge graph according to claim 1, characterized in that: The loss function L for model training is defined as: Among them, (h, r, t) and (h', r, t') represent positive triples and negative triples respectively, S and S' represent the set of positive triples and the set of negative triples respectively; Add a weight factor based on triple frequency to the loss function, and set different weights for the triple loss function according to the frequency of entities and relations in the training triples in the dataset; specifically, assign lower weights to entities and relations with higher frequencies, and assign higher weights to entities and relations with lower frequencies. In this way, increase the training attention to long-tail entities and relations in order to learn better embeddings; define the weight factor W of the triple based on the triple frequency f for: Among them, f h ,f t ,f r Respectively represent the frequency of entity h, t and relation r in the data set, Respectively represent the median frequency of entities and relations in the data set. According to the Pareto Principle, the median frequency is defined as the frequency of the 20% entities and relations ranked from high to low in terms of frequency of occurrence. h 、w t 、w r They are the frequency weight factors of entity h,t and relation r respectively; Weight factor W based on triple frequency f For low-frequency triples, that is, the entities and relations in the triples are low-frequency entities and relations with a frequency lower than the median frequency, the weight of the low-frequency entity or relation is greater than 1, and W in the loss function of the low-frequency triples is f The weight value is greater than 1, thereby increasing the attention to low-frequency entities and relations in training; similarly, for high-frequency triples, that is, the triples contain high-frequency entities and relations with a frequency higher than the median frequency, the weight of the high-frequency entity or relationship is less than 1, and W in the loss function of the high-frequency triples f The weight is less than 1, thus reducing the attention paid to these high-frequency entities and relationships during training; The weight factor W based on the triple frequency f The loss function applied to model training yields:

7. The link prediction method based on knowledge graph according to claim 1, characterized in that: The semantic matching-based knowledge graph embedding model includes one of DistMult, SimplE and ComplEx; for DistMult, the scoring function f(h, r, t) of the triple is defined as f(h,r,t)=h T ·diag(r)·t For SimplE, the scoring function f(h,r,t) for a triple is defined as: For ComplEx, the scoring function f(h,r,t) for a triple is defined as: are the transpose of the real and imaginary parts of the complex embedding vector h of entity h, respectively, and t re ,t im ,r re ,r im are the real and imaginary parts of the complex embedding vectors t and r of entity t and relation r, respectively; The loss function L of the knowledge graph embedding model based on semantic matching is:

Images