Reasoning Method and Device for Knowledge Graph

Abstract

The present invention provides a reasoning method and apparatus for a knowledge graph. The method includes: obtaining an entity word vector corresponding to an entity text in a target knowledge graph and a relationship word vector corresponding to a relationship text in the target knowledge graph; inputting the entity word vector and the relationship word vector into a contrastive learning model to obtain an entity contrastive learning vector of the entity word vector and a relationship contrastive learning vector of the relationship word vector output by the contrastive learning model; and determining an inference result of the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector. For the reasoning method and apparatus for a knowledge graph provided by the present invention, the entity contrastive learning vector and the relationship contrastive learning vector obtained through the contrastive learning model are used for subsequent reasoning of the knowledge graph. Since the difference between similar word vectors is larger and the difference between dissimilar word vectors is smaller, the accuracy of knowledge graph reasoning is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method and apparatus for reasoning of a knowledge graph. Background Art

[0002] The reasoning of a knowledge graph generally includes the normalization of the knowledge graph and the link prediction of the knowledge graph. The normalization of the knowledge graph is to find entities and relationships with the same semantics in the knowledge graph, and the link prediction of the knowledge graph can infer new knowledge from the knowledge graph.

[0003] The existing reasoning of the knowledge graph directly performs word vector initialization on the relationship text and entity text in the knowledge graph and then conducts the reasoning of the knowledge graph. The word vectors obtained by the existing method often have certain defects. For example, they cannot well distinguish text problems such as synonyms and near-synonyms, which will greatly reduce the quality of the downstream reasoning task when applied to the downstream reasoning task, resulting in low accuracy of the reasoning result. Summary of the Invention

[0004] The present invention provides a method and apparatus for reasoning of a knowledge graph to solve the technical problem in the prior art that directly performing word vector initialization on the relationship text and entity text in the knowledge graph and conducting the reasoning of the knowledge graph results in low accuracy of the reasoning result.

[0005] The present invention provides a method for reasoning of a knowledge graph, including:

[0006] Obtaining an entity word vector corresponding to an entity text in a target knowledge graph and a relationship word vector corresponding to a relationship text in the target knowledge graph;

[0007] Inputting the entity word vector and the relationship word vector into a contrastive learning model to obtain an entity contrastive learning vector of the entity word vector and a relationship contrastive learning vector of the relationship word vector output by the contrastive learning model;

[0008] Determining a reasoning result of the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector;

[0009] The contrastive learning model is trained from an initial contrastive learning model based on a set of paired examples in the target knowledge graph, positive example pairs semantically related in the set of examples, and negative example pairs semantically unrelated in the set of examples.

[0010] According to the method for reasoning of a knowledge graph provided by the present invention, it further includes:

[0011] Determining the negative cross-entropy of the contrastive learning model according to the similarity of the positive example pairs and the similarity of the negative example pairs;

[0012] Based on the convergence of the negative cross-entropy, it is determined that the contrastive learning model converges.

[0013] According to an inference method of a knowledge graph provided by the present invention, after the negative cross-entropy converges, it further includes:

[0014] Obtain the auxiliary information of entity pairs in the example set, and the auxiliary information of relationship pairs in the example set;

[0015] According to the auxiliary information of the entity pairs and the auxiliary information of the relationship pairs, determine the pairwise ranking loss value of the contrastive learning model;

[0016] According to the similarity of entities of the same type and the similarity of relationships of the same type in the example set, and the similarity of entities of different types and the similarity of relationships of different types, determine the normalization loss value of the contrastive learning model;

[0017] Add the pairwise ranking loss value and the normalization loss value to obtain an added loss value, and determine that the added loss value converges.

[0018] According to an inference method of a knowledge graph provided by the present invention, the obtaining the auxiliary information of entity pairs in the example set, and the auxiliary information of relationship pairs in the example set includes:

[0019] Obtain the similarity score of entity pairs in the example set, and use the similarity score of the entity pairs as the auxiliary information of the entity pairs;

[0020] Obtain the similarity score of relationship pairs in the example set, and use the similarity score of the relationship pairs as the auxiliary information of the relationship pairs.

[0021] According to an inference method of a knowledge graph provided by the present invention, when the inference of the target knowledge graph includes the normalization of the knowledge graph, the determining the inference result of the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector includes:

[0022] Cluster the entity contrastive learning vectors to obtain entity clusters of the same entities;

[0023] Cluster the relationship contrastive learning vectors to obtain relationship clusters of the same relationships, and use the entity clusters of the same entities and the relationship clusters of the same relationships as the inference result of the target knowledge graph.

[0024] According to an inference method of a knowledge graph provided by the present invention, when the inference of the target knowledge graph includes link prediction of the knowledge graph, determining the inference result of the target knowledge graph according to the entity contrast learning vector and the relationship contrast learning vector includes:

[0025] Predicting entities and relationships in the target knowledge graph according to the entity contrast learning vector and the relationship contrast learning vector to obtain prediction vectors;

[0026] Selecting the word vector with the highest similarity to the prediction vector from the entity word vectors and relationship word vectors of the target knowledge graph, and using the entity or relationship corresponding to the word vector with the highest similarity as the inference result of the target knowledge graph.

[0027] The present invention also provides an inference device for a knowledge graph, including:

[0028] A word vector acquisition module, configured to acquire entity word vectors corresponding to entity texts in the target knowledge graph, and relationship word vectors corresponding to relationship texts in the target knowledge graph;

[0029] A contrast learning module, configured to input the entity word vectors and the relationship word vectors into a contrast learning model, and obtain an entity contrast learning vector of the entity word vectors and a relationship contrast learning vector of the relationship word vectors output by the contrast learning model;

[0030] A knowledge graph inference module, configured to determine the inference result of the target knowledge graph according to the entity contrast learning vector and the relationship contrast learning vector;

[0031] The contrast learning model is obtained by training an initial contrast learning model based on a sample set of paired samples in the target knowledge graph, positive example pairs semantically related in the sample set, and negative example pairs semantically unrelated in the sample set.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the inference method of the knowledge graph as described in any one of the above is implemented.

[0033] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the inference method of the knowledge graph as described in any one of the above is implemented.

[0034] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the inference method of the knowledge graph as described in any one of the above is implemented.

[0035] The inference method and device for the knowledge graph provided by the present invention obtain an entity contrast learning vector and a relationship contrast learning vector of the original word vectors through a contrast learning model. The gap between similar word vectors in the original word vectors is reduced, and the distance between dissimilar word vectors in the original word vectors is increased. The obtained entity contrast learning vector and relationship contrast learning vector are used for subsequent inference of the knowledge graph. Since the difference between similar word vectors is larger and the difference between dissimilar word vectors is smaller, the accuracy of knowledge graph inference is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly describe the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.

[0037] Figure 1 is a schematic flowchart of the inference method for the knowledge graph provided by the present invention;

[0038] Figure 2 is a schematic flowchart of the inference process of the knowledge graph provided by the embodiments of the present invention;

[0039] Figure 3 is a schematic diagram of auxiliary information as a loss term provided by the embodiments of the present invention;

[0040] Figure 4 is a schematic structural diagram of the inference device for the knowledge graph provided by the present invention;

[0041] Figure 5 is a schematic structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention fall within the scope of protection of the present invention.

[0043] Figure 1 is a schematic flowchart of the inference method for the knowledge graph provided by the present invention. Referring to Figure 1 , the inference method for the knowledge graph provided by the present invention may include:

[0044] Step 110: Obtain the entity word vectors corresponding to the entity texts in the target knowledge graph, and the relation word vectors corresponding to the relation texts in the target knowledge graph;

[0045] Step 120: Input the entity word vectors and the relation word vectors into the contrastive learning model to obtain the entity contrastive learning vectors of the entity word vectors and the relation contrastive learning vectors of the relation word vectors output by the contrastive learning model;

[0046] Step 130: Determine the inference result of the target knowledge graph according to the entity contrastive learning vector and the relation contrastive learning vector;

[0047] The contrastive learning model is trained from an initial contrastive learning model based on a set of paired examples in the target knowledge graph, positive example pairs with semantic relevance in the set of examples, and negative example pairs with no semantic relevance in the set of examples.

[0048] It should be noted that a knowledge graph: is a modern theory that combines the theories and methods of disciplines such as applied mathematics, graphics, information visualization technology, and information science with methods such as bibliometric citation analysis and co-occurrence analysis, and uses a visual graph to vividly display the core structure, development history, frontier fields, and overall knowledge architecture of a discipline to achieve the purpose of multi-disciplinary integration. The main goal of a knowledge graph is to describe various entities and concepts existing in the real world, and the relationships between them, by using relationships to describe the association between two entities. From the perspective of the Web (World Wide Web, global wide area network), a knowledge graph is like a hyperlink between simple texts, supporting semantic search by establishing semantic links between data. From the perspective of natural language processing, a knowledge graph is to extract semantic and structured data from text. From the perspective of artificial intelligence, a knowledge graph is a tool that uses a knowledge base to assist in understanding human language. From the perspective of a database, a knowledge graph is a method of storing knowledge in the form of a graph. A knowledge graph is a relatively general formal description framework for semantic knowledge, using nodes to represent semantic symbols and edges to represent the relationships between semantics. A knowledge graph aims to describe various entities or concepts existing in the real world and their relationships, forming a huge semantic network graph, with nodes representing entities or concepts and edges consisting of attributes or relationships.

[0049] Entity: Refers to something that is distinguishable and exists independently. Such as a certain person, a certain city, a certain plant, a certain commodity, etc. All things in the world are composed of specific things, and this refers to entities. Entities are the most basic elements in a knowledge graph, and there are different relationships between different entities.

[0050] Relationship: There is a certain mutual relationship between entities, between different concepts, and between concepts and entities.

[0051] Triple: A triple is a general representation of a knowledge graph; the basic form of a triple mainly includes (Entity 1 - Relationship - Entity 2), etc. Each entity can be identified by a globally unique ID, and the relationship can be used to connect two entities and describe the association between them. For example, in an example of a knowledge graph, Location A is an entity, Location B is an entity, and Location A - Capital - Location B is a triple example of (Entity - Relationship - Entity).

[0052] Self-supervised learning: Self-supervised learning mainly uses auxiliary tasks to mine its own supervision information from large-scale unsupervised data, and trains the network through this constructed supervision information, so as to learn valuable representations for downstream tasks. That is to say, the supervision information of self-supervised learning is not manually labeled, but the algorithm automatically constructs supervision information in large-scale unsupervised data to perform supervised learning or training.

[0053] Contrastive learning is a type of self-supervised learning that does not rely on manually labeled class label information and directly uses the data itself as supervision information. Contrastive learning is a method for the task of describing similar and different things for deep learning models. The core point of contrastive learning is: by automatically constructing similar instances and dissimilar instances, that is, positive samples and negative samples, learning to contrast positive samples and negative samples in the feature space, so that similar instances are close in the feature space, while dissimilar instances are far apart in the feature space and the difference becomes larger. The model representation obtained through such a learning process can be used to perform downstream tasks.

[0054] The execution subject of the knowledge graph reasoning method provided by the present invention can be an electronic device, a component in the electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a Network Attached Storage (NAS), or a personal computer (PC), etc. The present invention does not make specific limitations.

[0055] Taking the computer's execution of the inference method of the knowledge graph provided by the present invention as an example, the technical solution of the present invention will be described in detail below.

[0056] In step 110, the target knowledge graph is composed of triples of relation texts and entity texts. After obtaining the entity texts and relation texts in the target knowledge graph, through vectorizing the texts, the entity word vectors corresponding to the entity texts and the relation word vectors corresponding to the relation texts are obtained.

[0057] Optionally, after obtaining the relation texts and entity texts in the target knowledge graph, the corresponding word vectors can be obtained through the Word2Vec method based on text data. Word2Vec represents each word with a vector through a shallow neural network language model. By constructing an input layer, a mapping layer, and an output layer, the neural network is used to learn the word with the highest probability of occurrence in the context of the word. Through the training of the text word library, the text is transformed into a vector in an n-dimensional vector space, and the cosine similarity in the space represents the semantic proximity of the words, thereby obtaining the word vectors corresponding to the text.

[0058] Optionally, the corresponding word vectors of the relation texts and entity texts can be obtained through the glove (Global Vectors for Word Representation) method. Through glove calculation, a word text can be represented as a vector composed of real numbers. The calculated word vectors capture some semantic characteristics between words, such as similarity and analogy. By calculating the vectors, such as Euclidean distance or cosine similarity, the semantic similarity between words is calculated, thereby determining the word vectors corresponding to the text.

[0059] In step 120, after obtaining the entity word vectors and relation word vectors in step 110, the entity word vectors and relation word vectors are input into the contrast learning model to obtain the entity contrast learning vectors of the entity word vectors and the relation contrast learning vectors of the relation word vectors output by the contrast learning model.

[0060] Among them, the contrast learning model is used to reduce the gap between similar word vectors and increase the distance between dissimilar word vectors.

[0061] After obtaining the entity word vectors and relation word vectors, by inputting the obtained entity word vectors and relation word vectors into the contrast learning model, the entity contrast learning vectors of the entity word vectors and the relation contrast learning vectors of the relation word vectors output by the contrast model are obtained.

[0062] The obtained entity contrast learning vectors and relation contrast learning vectors reduce the gap between similar word vectors in the original entity word vectors and original relation word vectors, and increase the distance between dissimilar word vectors in the original entity word vectors and original relation word vectors.

[0063] Through the contrast learning model, similar entities and relations are brought closer in the feature space, while dissimilar instances are pulled farther apart in the feature space, with the difference becoming larger. This realizes the optimization of the original word vectors of relations and entities.

[0064] In step 130, after determining the entity contrast learning vectors and relation contrast learning vectors in step 120, the determined entity contrast learning vectors and relation contrast learning vectors are used for the subsequent reasoning of the target knowledge graph, and the reasoning result of the knowledge graph is determined.

[0065] The reasoning of the knowledge graph generally refers to the normalization of the knowledge graph and the link prediction of the knowledge graph. The normalization of the knowledge graph means finding entities and relations with the same semantics in the knowledge graph. Link prediction refers to inferring new knowledge from the knowledge graph, that is, inferring new triples based on the existing entities and relations in the knowledge graph.

[0066] Generally, word vectors semantically close to the text are obtained according to the method of text similarity. According to the similarity method, after obtaining the word vectors of entity texts and relation texts, there are often certain defects, such as the inability to well distinguish text problems such as synonyms and near-synonyms. If the word vectors of entity texts and relation texts obtained directly are used for the reasoning of the knowledge graph, the accuracy of the reasoning result will not be high.

[0067] After inputting the word vectors of entity texts and relation texts into the contrast learning model, entity contrast learning vectors and relation contrast learning vectors output by the contrast learning model are obtained. Among them, the obtained entity contrast learning vectors and relation contrast learning vectors reduce the gap between similar word vectors in the original entity word vectors and original relation word vectors, and increase the distance between dissimilar word vectors in the original entity word vectors and original relation word vectors.

[0068] Using the obtained entity contrast learning vectors and relation contrast learning vectors for the subsequent reasoning of the target knowledge graph, since the difference between similar word vectors is larger and the difference between dissimilar word vectors is smaller, during the process of knowledge graph reasoning, it is better to determine the entities and relations with the same semantics in the target knowledge graph, as well as the differences between entities and relations in the target knowledge graph, which can improve the accuracy of knowledge graph reasoning.

[0069] The contrastive learning model is obtained by training an initial contrastive learning model based on a set of examples of paired examples in a target knowledge graph, as well as positive example pairs that are semantically related and negative example pairs that are semantically unrelated in the set of examples.

[0070] The contrastive learning model is constructed based on a self-supervised pairwise learning method. Therefore, directly according to the entity text and relationship text in the target knowledge graph, paired entity examples and paired relationship examples are selected to construct a paired example set, which includes entity text pairs and relationship text pairs. The entity text pairs include positive example pairs that are semantically related and negative example pairs that are semantically unrelated. The relationship text pairs include positive example pairs that are semantically related and negative example pairs that are semantically unrelated. According to the constructed paired example set, the initial contrastive learning model is trained to obtain the trained contrastive learning model.

[0071] The reasoning method of the knowledge graph provided by the embodiments of the present invention obtains an entity contrastive learning vector and a relationship contrastive learning vector of the original word vector through the contrastive learning model. The gap between similar word vectors in the original word vector is reduced, and the distance between dissimilar word vectors in the original word vector is increased. The obtained entity contrastive learning vector and relationship contrastive learning vector are used for subsequent reasoning of the knowledge graph. Since the difference between similar word vectors is larger and the difference between dissimilar word vectors is smaller, the accuracy of knowledge graph reasoning is improved.

[0072] In one embodiment, it further includes: determining the negative cross-entropy of the contrastive learning model according to the similarity of the positive example pairs and the similarity of the negative example pairs; determining that the contrastive learning model converges based on the convergence of the negative cross-entropy.

[0073] Construct a paired example set, which includes a set of positive example pairs that are semantically related and negative example pairs that are semantically unrelated and a set of negative example pairs that are semantically unrelated where x i and are semantically related, and x i and are semantically unrelated. Determine the negative cross-entropy of the contrastive learning model as:

[0074]

[0075] where l i is the negative cross-entropy of the contrastive learning model, h i and are the vector representations of the texts of x i and , and h i and are the vector representations of the texts of x i and . For the positive example pair h i and the cosine similarity of For the negative example pair h i and the cosine similarity of, τ is a pre-set temperature hyperparameter, j is the example index in the example set, and N is the total number of examples in the example set.

[0076] Optionally, the word vectors obtained from the entity text or relation text in the target knowledge graph can be input into the GRU encoder, and the output representation of the GRU encoder is discarded through two Dropout operations. Two new representations can be obtained for each entity text and relation text, namely positive example pairs. Negative example pairs are obtained by pairwise combination of the representations between different entity texts and relation texts.

[0077] Train the contrastive learning model according to the example set. When the negative cross-entropy of the contrastive learning model converges, the contrastive learning model converges.

[0078] The inference method of the knowledge graph provided by the embodiments of the present invention determines the negative cross-entropy of the contrastive learning model through the similarity of the positive example pair and the similarity of the negative example pair. When the negative cross-entropy converges, it is determined that the contrastive learning model converges, providing a basis for obtaining entity contrastive learning vectors and relation contrastive learning vectors through the contrastive learning model in the future.

[0079] In one embodiment, after the negative cross-entropy converges, it further includes: obtaining the auxiliary information of the entity pairs in the example set and the auxiliary information of the relation pairs in the example set; determining the pairwise ranking loss value of the contrastive learning model according to the auxiliary information of the entity pairs and the auxiliary information of the relation pairs; determining the normalization loss value of the contrastive learning model according to the similarity of the same-type entities and the similarity of the same-type relations in the example set, and the similarity of different-type entities and the similarity of different-type relations; adding the pairwise ranking loss value and the normalization loss value to obtain an added loss value, and determining that the added loss value converges.

[0080] For the entity pairs and relation pairs in the example set, obtain the auxiliary information of the entity pairs and the auxiliary information of the relation pairs. Among them, the auxiliary information of the entity pairs and the auxiliary information of the relation pairs are used to determine the pairwise ranking loss value of the contrastive learning model. Train the contrastive learning model through the auxiliary information of the entity pairs and the auxiliary information of the relation pairs. Among them, the auxiliary information can be the scoring value of the entity pair similarity or the relation pair similarity.

[0081] According to the obtained auxiliary information of the entity pairs and the auxiliary information of the relation pairs, the pairwise ranking loss value of the contrastive learning model is determined as:

[0082]

[0083] Among them, L C is the pairwise ranking loss value, and λ str , λ reg , λ ent,θ , λ ent,θ are set constant factors; D + is the set of positive example triples in the example set, and i is the index of D + ; D - is the set of negative example triples in the example set, and j is the index of D_; γ is the margin width for triple representation learning in the example set, and the σ function is the logistic function; η i and η j represent the scores for triples i and j, which can be obtained by scoring the triples with the TransE scoring function; e v is one entity in the entity pair, and e v′ is the other entity in the entity pair; is the set of scores for the auxiliary information of the entity pair, is the set of scores for the auxiliary information of the relation pair; Z ent,θ is the auxiliary information of the entity pair, and Z rel,θ is the auxiliary information of the relation pair; V is the set of all entity pairs, and v is the index of the entity pair; R is the set of all relation pairs, and r is the index of the relation pair.

[0084] Classify the entities and relations in the example set according to the entity types and relation types in the example set. Determine the normalization loss value of the contrastive learning model according to the similarity between entities of the same type and the similarity between entities of different types in the example set as:

[0085]

[0086] Among them, L CC is the normalization loss value, |x| is the size of the example set, and refer to the text representations of the same type, is the representation of different types of text, is the similarity between entities or relations of the same type, is the similarity between entities or relations of different types. ρ ∈ [-1, 1] is the predefined margin width, and i and j are the entity indices or relation indices in the example set.

[0087] Add up to the pairwise ranking loss value L C and the normalization loss value L CC to obtain the additive loss value Determine that the obtained additive loss value converges.

[0088] By determining that the obtained additive loss value converges, the contrastive learning model is trained based on the loss of the auxiliary information and the loss of the knowledge graph scoring, so that the entity contrastive learning vector and the relationship contrastive learning vector output by the trained contrastive learning model during use are more discriminative and isotropic. As Figure 2 As shown in the schematic diagram of the reasoning process of the knowledge graph provided by the embodiment of the present invention, the contrastive learning model trained by the auxiliary information is used to obtain the entity contrastive learning vector and the relationship contrastive learning vector in the target knowledge graph, and the obtained entity contrastive learning vector and relationship contrastive learning vector are used for link prediction and normalization of the target knowledge graph, realizing the reasoning process of the target knowledge graph and improving the accuracy of the target knowledge graph reasoning.

[0089] The reasoning method of the knowledge graph provided by the embodiment of the present invention trains the contrastive learning model by combining the pairwise ranking loss value and the normalization loss value of the contrastive learning model, so that the entity contrastive learning vector and the relationship contrastive learning vector output by the trained contrastive learning model during use are more discriminative and isotropic, and further improves the accuracy of subsequent knowledge graph reasoning.

[0090] In one embodiment, obtaining the auxiliary information of the entity pairs in the example set and the auxiliary information of the relationship pairs in the example set includes: obtaining the similarity score of the entity pairs in the example set, and using the similarity score of the entity pairs as the auxiliary information of the entity pairs; obtaining the similarity score of the relationship pairs in the example set, and using the similarity score of the relationship pairs as the auxiliary information of the relationship pairs.

[0091] The auxiliary information of the entity pairs in the example set can be the score of the entity pair similarity. The auxiliary information of the relationship pairs in the example set can be the score of the relationship pair similarity. As Figure 3 As shown in the schematic diagram of the auxiliary information as a loss term provided by the embodiment of the present invention, the similarity scores of the text data "apple" and the text data "red apple" are obtained. The text data "apple", the text data "red apple" and the similarity scores are used as the constraint loss of the contrastive learning model to train the contrastive learning model. So that for the trained model, the entity contrastive learning vector and the relationship contrastive learning vector output during use are more discriminative and isotropic.

[0092] Optionally, the similarity score of the entity pairs or relationship pairs in the example set can be determined by one or more of entity linking technology, the synonym dictionary PPDB tool, IDF word overlap, lemmatization method, and the AMIE tool.

[0093] Entity Linking Technology: Using the entity linking algorithm, entity texts linked to the same entity are combined in pairs to obtain entity pairs. The similarity score for those pairs is 1, and the similarity scores for the remaining entity pairs are determined based on similarity.

[0094] Thesaurus PPDB: If two entity phrases have the same interpretation, they are regarded as an entity pair, and the similarity score for this pair is 1. The similarity scores for the remaining entity pairs are determined based on similarity.

[0095] IDF (Inverse Document Frequency) Word Overlap: All entity / relationship Winnipegs are combined in pairs, and the scores for each pair of phrases are determined according to the IDF calculation formula.

[0096] Lemmatization: Remove the tenses, plurals, capitalizations and other deformations of the phrases, find the same phrases after removal, and use them as entity or relationship pairs. The similarity score for these pairs is 1, and the similarity scores for the remaining entity or relationship pairs are determined based on similarity.

[0097] AMIE: Use the AMIE rule mining tool to mine rules of length 2 in the knowledge base, and use the rule head and the relationships in the rules as relationship phrase pairs, and then determine the similarity scores.

[0098] The inference method of the knowledge graph provided by the embodiments of the present invention provides a basis for calculating the pairwise ranking loss value of the contrast learning model by using the similarity scores of entity pairs as the auxiliary information of the entity pairs and the similarity scores of relationship pairs as the auxiliary information of the relationship pairs.

[0099] In one embodiment, when the inference of the target knowledge graph includes the normalization of the knowledge graph, determining the inference result of the target knowledge graph according to the entity contrast learning vector and the relationship contrast learning vector includes: clustering the entity contrast learning vectors to obtain entity clusters of the same entity; clustering the relationship contrast learning vectors to obtain relationship clusters of the same relationship, and using the entity clusters of the same entity and the relationship clusters of the same relationship as the inference result of the target knowledge graph.

[0100] The normalization of the knowledge graph means finding entities and relationships with the same semantics in the knowledge graph. When the inference of the target knowledge graph includes the normalization of the knowledge graph, according to the entity contrast learning vector and the relationship contrast learning vector of the target knowledge graph obtained by the contrast learning model, cluster the entity contrast learning vectors to obtain entity clusters of the same entity. Cluster the relationship contrast learning vectors to obtain relationship clusters of the same relationship. The obtained entity clusters of the same entity and relationship clusters of the same relationship are the inference results of the target knowledge graph.

[0101] To verify the accuracy of the normalization in this embodiment, the trained contrastive learning model is tested on the test set and compared with other related methods, as shown in Table 1 specifically.

[0102] Among them, the comparison methods include:

[0103] ConvE: A two-dimensional convolution method proposed on the knowledge graph to achieve the normalization of the knowledge graph.

[0104] CaRe: Based on the CaRe model, adding BERT implicit constraints on entity types to achieve the normalization of the knowledge graph.

[0105] GRU_TUCKER: A combined model of GRU encoder and TUCKER scoring function to achieve the normalization of the knowledge graph.

[0106] As shown in Table 1. The normalization method in this embodiment has better performance indicators than the current mainstream normalization models on the datasets ReVerb20K and ReVerb45K.

[0107] Table 1 Link prediction performance of different prediction methods on different datasets

[0108]

[0109] Among them, MRR is the mean reciprocal rank, MR is the mean rank, and Hit@10 refers to the top 10 answer accuracy.

[0110] The inference method of the knowledge graph provided by the embodiment of the present invention realizes the inference for the target knowledge graph by clustering the entity contrast learning vectors to obtain entity clusters of the same entities and clustering the relationship contrast learning vectors to obtain relationship clusters of the same relationships.

[0111] In one embodiment, when the inference of the target knowledge graph includes link prediction of the knowledge graph, determining the inference result of the target knowledge graph according to the entity contrast learning vector and the relationship contrast learning vector includes: predicting the entities and relationships in the knowledge graph according to the entity contrast learning vector and the relationship contrast learning vector to obtain a prediction vector; selecting the word vector with the highest similarity from the entity word vectors and relationship word vectors of the target knowledge graph, and using the entity or relationship corresponding to the word vector with the highest similarity as the inference result of the target knowledge graph.

[0112] Link prediction refers to inferring new knowledge from a knowledge graph, that is, based on the existing entities and relationships in the knowledge graph, inferring new entities or relationships to form new triples. When the inference of the target knowledge graph includes link prediction of the knowledge graph, entity contrast learning vectors and relationship contrast learning vectors of the target knowledge graph obtained according to the contrast learning model are used to predict entities or relationships in the target knowledge graph.

[0113] The specific inference process is as follows: Based on the determined entity contrast learning vectors and relationship contrast learning vectors in the target knowledge graph, entities and relationships in the target knowledge graph are predicted. For example, based on known entities and relationships, another entity corresponding to the entity and relationship is predicted. Or based on two known entities, the relationship corresponding to the two known entities is predicted. Based on the known entities or relationships and the predicted entities or relationships, new triples are formed, that is, the results of link prediction are obtained.

[0114] Entities and relationships in the target knowledge graph are predicted to obtain prediction vectors, and the word vectors with the highest similarity to the prediction vectors are selected from the entity word vectors and relationship word vectors of the target knowledge graph. The entity or relationship corresponding to the word vector with the highest similarity is used as the result of link prediction, that is, the inference result of the target knowledge graph.

[0115] To verify the accuracy of link prediction in this embodiment, the trained contrast learning model is tested on a test set and compared with other related methods, as shown in Table 2 specifically.

[0116] Among them, the comparison methods include:

[0117] Galárraga-IDF: Use inverse document frequency word example overlap for agglomerative hierarchical clustering to achieve link prediction.

[0118] GloVe+HAC: Use GloVe word vectors for agglomerative hierarchical clustering to achieve link prediction.

[0119] GloVe+HAC+SI: Use GloVe word vectors and auxiliary information for agglomerative hierarchical clustering to achieve link prediction.

[0120] HolE(GloVe): Based on path reasoning to achieve link prediction.

[0121] As shown in Table 2. The performance indicators of the link prediction method in this embodiment on the datasets Base, Ambiguous, and ReVerb45K are all better than those of the current mainstream link prediction models.

[0122] Table 2 Link prediction performance of different prediction methods on different datasets

[0123]

[0124] Among them, Macro is the macro F1 value of the model (macro precision * macro recall * 2 / (macro precision + macro recall)); Micro is the micro F1 value of the model (micro precision * micro recall * 2 / (micro precision + micro recall)); Pair is the pairwise F1 value of the model. For entity text pairs, pairwise precision and pairwise recall are calculated.

[0125] The inference method of the knowledge graph provided by the embodiments of the present invention predicts entities and relationships in the target knowledge graph through entity contrast learning vectors and relationship contrast learning vectors, realizing inference for the target knowledge graph.

[0126] Figure 4 It is a schematic structural diagram of the inference device of the knowledge graph provided by the present invention, as Figure 4 shown. The device includes:

[0127] A word vector acquisition module 410, configured to acquire entity word vectors corresponding to entity texts in the target knowledge graph, and relationship word vectors corresponding to relationship texts in the target knowledge graph;

[0128] A contrast learning module 420, configured to input the entity word vectors and the relationship word vectors into a contrast learning model, and obtain an entity contrast learning vector of the entity word vectors and a relationship contrast learning vector of the relationship word vectors output by the contrast learning model;

[0129] A knowledge graph inference module 430, configured to determine an inference result of the target knowledge graph according to the entity contrast learning vector and the relationship contrast learning vector;

[0130] The contrast learning model is trained from an initial contrast learning model based on a sample set of pairwise examples in the target knowledge graph, positive example pairs semantically related in the sample set, and negative example pairs semantically unrelated in the sample set.

[0131] The inference device of the knowledge graph provided by the embodiments of the present invention obtains an entity contrast learning vector and a relationship contrast learning vector of the original word vectors through a contrast learning model, reducing the gap between similar word vectors in the original word vectors and increasing the distance between dissimilar word vectors in the original word vectors. Using the obtained entity contrast learning vector and relationship contrast learning vector for subsequent knowledge graph inference, since the difference between similar word vectors is larger and the difference between dissimilar word vectors is smaller, the accuracy of knowledge graph inference is improved.

[0132] In one embodiment, the contrast learning module 420 is specifically configured to:

[0133] Determine the negative cross-entropy of the contrastive learning model according to the similarity of the positive example pairs and the similarity of the negative example pairs;

[0134] Based on the convergence of the negative cross-entropy, determine the convergence of the contrastive learning model.

[0135] In one embodiment, the contrastive learning module 420 is further specifically configured to:

[0136] After the negative cross-entropy converges, it further includes:

[0137] Obtain the auxiliary information of the entity pairs in the example set and the auxiliary information of the relationship pairs in the example set;

[0138] Determine the pairwise ranking loss value of the contrastive learning model according to the auxiliary information of the entity pairs and the auxiliary information of the relationship pairs;

[0139] Determine the normalization loss value of the contrastive learning model according to the similarity of the same-type entities and the similarity of the same-type relationships in the example set, and the similarity of different-type entities and the similarity of different-type relationships;

[0140] Add the pairwise ranking loss value and the normalization loss value to obtain an added loss value, and determine the convergence of the added loss value.

[0141] In one embodiment, the contrastive learning module 420 is further specifically configured to:

[0142] The obtaining the auxiliary information of the entity pairs in the example set and the auxiliary information of the relationship pairs in the example set includes:

[0143] Obtain the similarity scores of the entity pairs in the example set, and use the similarity scores of the entity pairs as the auxiliary information of the entity pairs;

[0144] Obtain the similarity scores of the relationship pairs in the example set, and use the similarity scores of the relationship pairs as the auxiliary information of the relationship pairs.

[0145] In one embodiment, the knowledge graph reasoning module 430 is specifically configured to:

[0146] When the reasoning of the target knowledge graph includes the normalization of the knowledge graph, the determining the reasoning result of the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector includes:

[0147] Cluster the entity contrastive learning vectors to obtain entity clusters of the same entities;

[0148] Cluster the relation contrast learning vectors to obtain relation clusters with the same relations, and use the entity clusters of the same entities and the relation clusters of the same relations as the inference results of the target knowledge graph.

[0149] In one embodiment, the knowledge graph inference module 430 is further specifically configured to:

[0150] When the inference of the target knowledge graph includes link prediction of the knowledge graph, determining the inference result of the target knowledge graph according to the entity contrast learning vector and the relation contrast learning vector includes:

[0151] Predict entities and relations in the target knowledge graph according to the entity contrast learning vector and the relation contrast learning vector to obtain prediction vectors;

[0152] Select the word vectors with the highest similarity to the prediction vectors from the entity word vectors and relation word vectors of the target knowledge graph, and use the entities or relations corresponding to the word vectors with the highest similarity as the inference results of the target knowledge graph.

[0153] Figure 5 An example of a schematic diagram of the entity structure of an electronic device is shown as Figure 5 shown. The electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540. Among them, the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logical instructions in the memory 530 to execute the inference method of the knowledge graph, and the method includes:

[0154] Obtain entity word vectors corresponding to entity texts in the target knowledge graph, and relation word vectors corresponding to relation texts in the target knowledge graph;

[0155] Input the entity word vectors and the relation word vectors into a contrast learning model to obtain the entity contrast learning vectors of the entity word vectors and the relation contrast learning vectors of the relation word vectors output by the contrast learning model;

[0156] Determine the inference result of the target knowledge graph according to the entity contrast learning vector and the relation contrast learning vector;

[0157] The contrast learning model is obtained by training an initial contrast learning model based on a sample set of paired examples in the target knowledge graph, positive example pairs semantically related in the sample set, and negative example pairs semantically unrelated in the sample set.

[0158] In addition, when the logical instructions in the above-mentioned memory 530 can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0159] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the reasoning method of the knowledge graph provided by the above-mentioned various methods. The method includes:

[0160] Obtain an entity word vector corresponding to the entity text in the target knowledge graph, and a relationship word vector corresponding to the relationship text in the target knowledge graph;

[0161] Input the entity word vector and the relationship word vector into a contrastive learning model to obtain an entity contrastive learning vector of the entity word vector and a relationship contrastive learning vector of the relationship word vector output by the contrastive learning model;

[0162] Determine the reasoning result of the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector;

[0163] The contrastive learning model is trained from an initial contrastive learning model based on a set of examples of paired examples in the target knowledge graph, positive example pairs semantically related in the set of examples, and negative example pairs semantically unrelated in the set of examples.

[0164] On yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the reasoning method of the knowledge graph provided above. The method includes:

[0165] Obtain an entity word vector corresponding to the entity text in the target knowledge graph, and a relationship word vector corresponding to the relationship text in the target knowledge graph;

[0166] Input the entity word vector and the relation word vector into a contrastive learning model to obtain an entity contrastive learning vector of the entity word vector output by the contrastive learning model and a relation contrastive learning vector of the relation word vector.

[0167] Determine the inference result of the target knowledge graph according to the entity contrastive learning vector and the relation contrastive learning vector.

[0168] The contrastive learning model is obtained by training an initial contrastive learning model based on a set of paired examples in the target knowledge graph, positive example pairs semantically related in the set of examples, and negative example pairs semantically unrelated in the set of examples.

[0169] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0170] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A reasoning method for a knowledge graph, characterized in that Including: Obtain the entity word vectors corresponding to the entity texts in the target knowledge graph, and the relationship word vectors corresponding to the relationship texts in the target knowledge graph; Input the entity word vectors and the relationship word vectors into a contrastive learning model to obtain the entity contrastive learning vectors of the entity word vectors output by the contrastive learning model and the relationship contrastive learning vectors of the relationship word vectors; Determine the inference result of the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector; The contrastive learning model is trained from an initial contrastive learning model based on a set of paired examples in the target knowledge graph, positive example pairs semantically related in the set of examples, and negative example pairs semantically unrelated in the set of examples; When the inference of the target knowledge graph includes the normalization of the knowledge graph, the determining the inference result of the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector includes: Cluster the entity contrastive learning vectors to obtain entity clusters of the same entity; Cluster the relationship contrastive learning vectors to obtain relationship clusters of the same relationship, and use the entity clusters of the same entity and the relationship clusters of the same relationship as the inference result of the target knowledge graph.

2. The inference method of the knowledge graph according to claim 1, wherein Also including: Determine the negative cross-entropy of the contrastive learning model according to the similarity of the positive example pairs and the similarity of the negative example pairs; Based on the convergence of the negative cross-entropy, determine the convergence of the contrastive learning model.

3. The inference method of the knowledge graph according to claim 2, characterized in that After the negative cross-entropy converges, it also includes: Obtain the auxiliary information of the entity pairs in the set of examples, and the auxiliary information of the relationship pairs in the set of examples; Determine the pairwise ranking loss value of the contrastive learning model according to the auxiliary information of the entity pairs and the auxiliary information of the relationship pairs; Determine the normalization loss value of the contrastive learning model according to the similarity of the same-type entities and the similarity of the same-type relationships in the set of examples, and the similarity of different-type entities and the similarity of different-type relationships; Add the pairwise ranking loss value and the normalization loss value to obtain an added loss value, and determine the convergence of the added loss value.

4. The reasoning method of the knowledge graph according to claim 3, characterized in that, The obtaining the auxiliary information of the entity pairs in the set of examples, and the auxiliary information of the relationship pairs in the set of examples includes: Obtain the similarity score of the entity pairs in the set of examples, and use the similarity score of the entity pairs as the auxiliary information of the entity pairs; Obtain the similarity score of the relationship pairs in the set of examples, and use the similarity score of the relationship pairs as the auxiliary information of the relationship pairs.

5. The reasoning method of the knowledge graph according to claim 1, wherein When the inference of the target knowledge graph includes link prediction of the knowledge graph, the determining the inference result of the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector includes: Predict the entities and relationships in the target knowledge graph according to the entity contrastive learning vector and the relationship contrastive learning vector to obtain prediction vectors; Select the word vector with the highest similarity to the prediction vector from the entity word vectors and relation word vectors of the target knowledge graph, and use the entity or relation corresponding to the word vector with the highest similarity as the reasoning result of the target knowledge graph.

6. An inference device for a knowledge graph, characterized in that, Including: A word vector acquisition module, configured to acquire entity word vectors corresponding to entity texts in the target knowledge graph, and relation word vectors corresponding to relation texts in the target knowledge graph; A contrast learning module, configured to input the entity word vectors and the relation word vectors into a contrast learning model, and obtain an entity contrast learning vector of the entity word vectors and a relation contrast learning vector of the relation word vectors output by the contrast learning model; A knowledge graph reasoning module, configured to determine the reasoning result of the target knowledge graph according to the entity contrast learning vector and the relation contrast learning vector; The contrast learning model is obtained by training an initial contrast learning model based on a sample set of paired examples in the target knowledge graph, positive example pairs with semantic relevance in the sample set, and negative example pairs with semantic irrelevance in the sample set; When the reasoning of the target knowledge graph includes the normalization of the knowledge graph, the determining the reasoning result of the target knowledge graph according to the entity contrast learning vector and the relation contrast learning vector includes: Clustering the entity contrast learning vectors to obtain entity clusters of the same entity; Clustering the relation contrast learning vectors to obtain relation clusters of the same relation, and using the entity clusters of the same entity and the relation clusters of the same relation as the reasoning result of the target knowledge graph.

7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the knowledge graph reasoning method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a processor, it implements the knowledge graph reasoning method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the knowledge graph reasoning method according to any one of claims 1 to 5.

Images